You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by eh...@apache.org on 2019/02/15 16:09:10 UTC
[hadoop] branch HDFS-12090 updated (6657cca -> 64da704)
This is an automated email from the ASF dual-hosted git repository.
ehiggs pushed a change to branch HDFS-12090
in repository https://gitbox.apache.org/repos/asf/hadoop.git.
omit 6657cca HDFS-13794. [PROVIDED Phase 2] Teach BlockAliasMap.Writer remove method. Contributed by Ewan Higgs
omit 06477ab HDFS-13421. [PROVIDED Phase 2] Implement DNA_BACKUP command in Datanode. Contributed by Ewan Higgs.
omit 8cdd033 HDFS-13310. The DatanodeProtocol should have a DNA_BACKUP to backup blocks. Original patch contributed by Ewan Higgs. Followup work and fixed contributed by Virajith Jalaparthi.
add 567b555 HDDS-343. Containers are stuck in closing state in scm. Contributed by Elek, Marton.
add 6e5ffb7 HDFS-13885. Add debug logs in dfsclient around decrypting EDEK. Contributed by Kitti Nanasi.
add 9e96ac6 HADOOP-10219. ipc.Client.setupIOstreams() needs to check for ClientCache.stopClient requested shutdowns. Contributed by Kihwal Lee and Lukas Majercak.
add b993216 HDDS-75. Support for CopyContainer. Contributed by Elek, Marton.
add 6bbd249 HDDS-98. Adding Ozone Manager Audit Log. Contributed by Dinesh Chitlangia.
add 54f2044 HDFS-13857. RBF: Choose to enable the default nameservice to read/write files. Contributed by yanghuafeng.
add 9964e33 HDDS-369. Remove the containers of a dead node from the container state map. Contributed by Elek, Marton
add 6e4c731 HDDS-396. Remove openContainers.db from SCM. Contributed by Dinesh Chitlangia.
add 6883fe8 HDDS-383. Ozone Client should discard preallocated blocks from closed containers. Contributed by Shashikant Banerjee
add 6ccb809 HDFS-13812. Fix the inconsistent default refresh interval on Caching documentation. Contributed by Hrishikesh Gadre.
add 85c3fe3 HDDS-268. Add SCM close container watcher. Contributed by Ajay Kumar.
add c7403a4 HDDS-315. ozoneShell infoKey does not work for directories created as key and throws 'KEY_NOT_FOUND' error. Contributed by Dinesh Chitlangia.
add 045270a HDDS-333. Create an Ozone Logo. Contributed by Priyanka Nagwekar.
add dffb7bf YARN-8638. Allow linux container runtimes to be pluggable. Contributed by Craig Condit
add df0d61e HDDS-358. Use DBStore and TableStore for DeleteKeyService. Contributed by Anu Engineer
add 9315db5 HDFS-13815. RBF: Add check to order command. Contributed by Ranith Sardar.
add e780556 HADOOP-15696. KMS performance regression due to too many open file descriptors after Jetty migration. Contributed by Wei-Chiu Chuang.
add 9af96d4 HADOOP-15707. Add IsActiveServlet to be used for Load Balancers. Contributed by Lukas Majercak.
add 8286bf2 HDDS-303. Removing logic to identify containers to be closed from SCM. Contributed by Nanda kumar.
add 962089a HADOOP-15657 Registering MutableQuantiles via Metric annotation. Contributed by Sushil Ks
add b6c543f HDDS-404. Implement toString() in OmKeyLocationInfo. Contributed by Dinesh Chitlangia.
add eb0b5a8 MAPREDUCE-7131. Job History Server has race condition where it moves files from intermediate to finished but thinks file is in intermediate. Contributed by Anthony Hsu
add fa2945e HDDS-405. User/volume mapping is not cleaned up during the deletion of the last volume. Contributed by Elek, Marton.
add b3161c4 HDDS-297. Add pipeline actions in Ozone. Contributed by Mukul Kumar Singh and Shashikant Banerjee
add eca1a4b HDFS-13695. Move logging to slf4j in HDFS package. Contributed by Ian Pickering.
add c5bf43a HDDS-406. Enable acceptace test of the putKey for rpc protocol. Contributed by Elek, Marton.
add 527288e HDFS-13836. RBF: Handle mount table znode with null value. Contributed by yanghuafeng.
add 396ce7b HDDS-397. Handle deletion for keys with no blocks. Contributed by Lokesh Jain.
add be1ec00 HDDS-408. Read (getKey) operation is very slow. Contributed by Nandakumar.
add 94ed5cf HDDS-214. Bump ozone/hdds version after the release branch cut
add 36c7c78 HADOOP-15709 Move S3Guard LocalMetadataStore constants to org.apache.hadoop.fs.s3a.Constants (Contributed by Gabor Bota)
add ff64d35 HDDS-351. Add chill mode state to SCM. Contributed by Ajay Kumar.
add ab90248 HDDS-400. Check global replication state for containers of dead node. Contributed by Elek, Marton.
add 3dc2988 YARN-8699. Add Yarnclient#yarnclusterMetrics API implementation in router. Contributed by Bibin A Chundatt.
add 410dd3f HDFS-13744. OIV tool should better handle control characters present in file or directory names. Contributed by Zsolt Venczel.
add 73fcbdd HDDS-190. Improve shell error message for unrecognized option. Contributed by Elek, Marton.
add ca3adf5 HDDS-398. Support multiple tests in freon. Contributed by Elek, Marton.
add 1633378 HADOOP-15731. TestDistributedShell fails on Windows. Contributed by Botong Huang.
add 335a813 HDFS-13820. Add an ability to disable CacheReplicationMonitor. Contributed by Hrishikesh Gadre.
add 7d62334 YARN-8751. Reduce conditions that mark node manager as unhealthy. Contributed by Craig Condit
add bf8a175 YARN-8706. Updated docker container stop logic to avoid double kill. Contributed by Chandni Singh
add d924ca2 HDDS-410. ozone scmcli list is not working properly. Contributed by Hanisha Koneru.
add 8944407 HDFS-13862. RBF: Router logs are not capturing few of the dfsrouteradmin commands. Contributed by Ayush Saxena
add de85f84 YARN-8698. [Submarine] Failed to reset Hadoop home environment when submitting a submarine job. (Zac Zhou via wangda)
add 30eceec HDFS-13806. EC: No error message for unsetting EC policy of the directory inherits the erasure coding policy from an ancestor directory. Contributed by Ayush Saxena.
add eef3baf HDFS-13895. EC: Fix Intermittent Failure in TestDFSStripedOutputStreamWithFailureWithRandomECPolicy. Contributed by Ayush Saxena.
add 0da4964 HDFS-13884. Improve the description of the setting dfs.image.compress. Contributed by Ryan Wu.
add 585a4f9 HDDS-417. Ambiguous error message when using genconf tool. Contributed by Dinesh Chitlangia.
add 1eef0ac HADOOP-15677. WASB: Add support for StreamCapabilities. Contributed by Thomas Marquardt.
add 8fe4062 HDDS-403. Fix createdOn and modifiedOn timestamp for volume, bucket, key. Contributed by Dinesh Chitlangia.
add 317f317 HDDS-421. Resilient DNS resolution in datanode-service. Contributed by Elek, Marton.
add 987d819 YARN-8709: CS preemption monitor always fails since one under-served queue was deleted. Contributed by Tao Yang.
add 96892c4 HDFS-13237. [Documentation] RBF: Mount points across multiple subclusters. Contributed Íñigo Goiri
add 8e54da1 HDFS-13051. Fix dead lock during async editlog rolling if edit queue is full. Contributed by Daryn Sharp.
add 4b8c2b1 HADOOP-14671. Upgrade Apache Yetus to 0.8.0
add 7ce997a MAPREDUCE-7135. TestTaskAttemptContainerRequest should reset UserGroupInformation. Contributed by Oleksandr Shevchenko
add 7b5886b HDDS-222. Remove hdfs command line from ozone distribution. Contributed by Elek, Marton.
add 8ffbbf5 HDDS-431. LeaseManager of CommandWatcher is not started. Contributed by Elek, Marton.
add 598380a MAPREDUCE-7136. TestMRAppMetrics should shutdown DefaultMetricsSystem after completion. Contributed by Oleksandr Shevchenko
add 1d567c2 HDDS-416. Remove currentPosition from ChunkInputStreamEntry. Contributed by Lokesh Jain.
add a406f6f HDDS-424. Consolidate ozone oz parameters to use GNU convention. Contributed by Elek, Marton.
add 9c238ff HDDS-432. Replication of closed containers is not working. Contributed by Elek, Marton.
add 87f63b6 HADOOP-14833. Remove s3a user:secret authentication. Contributed by Steve Loughran
add b2432d2 HDFS-13902. Add JMX, conf and stacks menus to the datanode page. Contributed by fengchuang.
add 76d0365 HADOOP-15584. Move httpcomponents versions in pom.xml. Contributed by Brandon Scheller.
add 9011567 HDDS-425. Move unit test of the genconf tool to hadoop-ozone/tools module. Contributed by Dinesh Chitlangia.
add 1f42ce9 YARN-6855. [YARN-3409] CLI Proto Modifications to support Node Attributes. Contributed by Naganarasimha G R.
add 2475fb0 YARN-6856. [YARN-3409] Support CLI for Node Attributes Mapping. Contributed by Naganarasimha G R.
add d9d93e3 YARN-7842. PB changes to carry node-attributes in NM heartbeat. Contributed by Weiwei Yang.
add 4458b27 YARN-7840. Update PB for prefix support of node attributes. Contributed by Naganarasimha G R.
add d312b5c YARN-7757. Refactor NodeLabelsProvider to be more generic and reusable for node attributes providers. Contributed by Weiwei Yang.
add 2f7712b YARN-6858. Attribute Manager to store and provide node attributes in RM. Contributed by Naganarasimha G R.
add ffcabd2 YARN-7856. Validate Node Attributes from NM. Contributed by Weiwei Yang.
add 86d024e YARN-7965. NodeAttributeManager add/get API is not working properly. Contributed by Weiwei Yang.
add 3b3b6ef YARN-7871. Node attributes reporting from NM to RM. Contributed by Weiwei Yang.
add 6f4bc49 YARN-7988. Refactor FSNodeLabelStore code for Node Attributes store support. Contributed by Bibin A Chundatt.
add 440ff7f YARN-8094. Support configuration based Node Attribute provider. Contributed by Weiwei Yang.
add 89b3ebd YARN-8092. Expose Node Attributes info via RM nodes REST API. Contributed by Weiwei Yang.
add 901e852 YARN-8033. CLI Integration with NodeAttributesManagerImpl. Contributed by Naganarasimha G R.
add a6590c1 YARN-8117. Fix TestRMWebServicesNodes test failure. Contributed by Bibin A Chundatt.
add b9890d1 YARN-7875. Node Attribute store for storing and recovering attributes. Contributed by Bibin A Chundatt.
add 0a01b13 YARN-8100. Support API interface to query cluster attributes and attribute to nodes. Contributed by Bibin A Chundatt.
add 5dc7d6e YARN-8104. Add API to fetch node to attribute mapping. Contributed by Bibin A Chundatt.
add 8cf6a9a YARN-7892. Revisit NodeAttribute class structure. Contributed by Naganarasimha G R.
add 7618342 YARN-8351. Node attribute manager logs are flooding RM logs. Contributed by Weiwei Yang.
add eb08543 YARN-8103. Add CLI interface to query node attributes. Contributed by Bibin A Chundatt.
add 8c94739 YARN-8574. Allow dot in attribute values. Contributed by Bibin A Chundatt.
add 67ae81f YARN-7863. Modify placement constraints to support node attributes. Contributed by Sunil Govindan.
add 5219435 YARN-8721. Relax NE node-attribute check when attribute doesn't exist on a node. Contributed by Sunil Govindan.
add 1e7d6e5 YARN-7865. Node attributes documentation. Contributed by Naganarasimha G R.
add c44088a YARN-8739. Fix jenkins issues for Node Attributes branch. Contributed by Sunil Govindan.
add 5e64e62 YARN-8740. Clear node attribute path after each test run. Contributed by Bibin A Chundatt.
add 47299ce HDDS-433. ContainerStateMachine#readStateMachineData should properly build LogEntryProto. Contributed by Lokesh Jain.
add 47b72c8 HADOOP-15635. s3guard set-capacity command to fail fast if bucket is unguarded. Contributed by Gabor Bota.
add 6e2129c HDDS-428. OzoneManager lock optimization. Contributed by Nanda Kumar.
add 1f6c454 HADOOP-15750. Remove obsolete S3A test ITestS3ACredentialsInURL. Contributed by Steve Loughran
add 78bd3b1 HDFS-13846. Safe blocks counter is not decremented correctly if the block is striped (Contributed by Kitti Nanasi via Daniel Templeton)
add 64c7a12 HDDS-436. Allow SCM chill mode to be disabled by configuration. Contributed by Ajay Kumar.
add 02b9bfd YARN-8658. [AMRMProxy] Metrics for AMRMClientRelayer inside FederationInterceptor. Contributed by Young Chen.
add 8e9afbf YARN-8768. Javadoc error in node attributes. Contributed by Sunil Govindan.
add c18eb97 HDDS-395. TestOzoneRestWithMiniCluster fails with "Unable to read ROCKDB config". Contributed by Dinesh Chitlangia.
add d32a8d5 HADOOP-14734 add option to tag DDB table(s) created. (Contributed by Gabor Bota and Abe Fine)
add d7c0a08 HADOOP-15426 Make S3guard client resilient to DDB throttle events and network failures (Contributed by Steve Loughran)
add c6e19db HDDS-420. putKey failing with KEY_ALLOCATION_ERROR. Contributed by Shashikant Banerjee.
add 5d084d7 HADOOP-12760. sun.misc.Cleaner has moved to a new location in OpenJDK 9. Contributed by Akira Ajisaka.
add e084627 HDDS-415. 'ozone om' with incorrect argument first logs all the STARTUP_MSG. Contributed by Namit Maheshwari.
add f4bda5e YARN-8630. ATSv2 REST APIs should honor filter-entity-list-by-user in non-secure cluster when ACls are enabled. Contributed by Rohith Sharma K S.
add c9e0b69 HDDS-233. Update ozone to latest ratis snapshot build(0.3.0-50588bd-SNAPSHOT). Contributed by Shashikant Banerjee.
add 39c1ea1 YARN-8729. Node status updater thread could be lost after it is restarted. Contributed by Tao Yang.
add e1b242a HDFS-13914. Fix DN UI logs link broken when https is enabled after HDFS-13902. Contributed by Jianfei Jiang.
add 250b500 YARN-8680. YARN NM: Implement Iterable Abstraction for LocalResourceTracker state. Contributed by Pradeep Ambati
add 2886024 MAPREDUCE-7133. History Server task attempts REST API returns invalid data. Contributed by Oleksandr Shevchenko
add 76a0fdf HDDS-456. TestOzoneShell#init is breaking due to Null Pointer Exception. Contributed by Dinesh Chitlangia.
add 4441fe9 HDDS-414. Fix sbin/stop-ozone.sh to stop Ozone daemons. Contributed by Elek, Marton.
add a1de8cb HDFS-13838. WebHdfsFileSystem.getFileStatus() won't return correct "snapshot enabled" status. Contributed by Siyao Meng.
add 291dcf2 HDDS-423. Introduce an ozone specific log4j.properties. Contributed by Elek, Marton.
add ef5c776 HADOOP-15733. Correct the log when Invalid emptier Interval configured. Contributed by Ayush Saxena
add 568ebec YARN-8772. Annotation javax.annotation.Generated has moved
add f1a893f YARN-8720. CapacityScheduler does not enforce max resource allocation check at queue level. Contributed by Tarun Parimi.
add a12f12f HDDS-438. 'ozone oz' should print usage when command or sub-command is missing. Contributed by Dinesh Chitlangia.
add 9923760 YARN-8706. Allow additional flag in docker inspect call. Contributed by Chandni Singh
add 78902f0 YARN-8748. Javadoc warnings within the nodemanager package. Contributed by Craig Condit
add 144a55f YARN-8045. Reduce log output from container status calls. Contributed by Craig Condit
add 0c8a43b HDDS-429. StorageContainerManager lock optimization. Contributed by Nanda Kumar.
add 446cb83 HDDS-452. 'ozone scm' with incorrect argument first logs all the STARTUP_MSG. Contributed by Namit Maheshwari.
add c1df308 HDDS-389. Remove XceiverServer and XceiverClient and related classes. Contributed by chencan.
add 5470de4 HDDS-454. TestChunkStreams#testErrorReadGroupInputStream & TestChunkStreams#testReadGroupInputStream are failing. Contributed by chencan.
add 488806b MAPREDUCE-7140. Refactoring TaskAttemptInfo to separate Map and Reduce tasks. Contributed by Oleksandr Shevchenko
add 6f03746 HDDS-419. ChunkInputStream bulk read api does not read from all the chunks. Contributed by Lokesh Jain and Mukul Kumar.
add b0a659c HDDS-466. Handle null in argv of StorageContainerManager#createSCM. Contributed by Nanda kumar
add b95aa56 HDDS-446. Provide shaded artifact to start ozone service as a datanode plugin. Contributed by Elek Marton
add 985f3bf HDDS-409. Ozone acceptance-test and integration-test packages have undefined hadoop component. Contributed by Dinesh Chitlangia.
add a65c3ea HDDS-449. Add a NULL check to protect DeadNodeHandler#onMessage. Contributed by LiXin Ge.
add c9fa081 HDDS-462. Optimize ContainerStateMap#getMatchingContainerIDs in SCM. Contributed by Nanda kumar.
add 87e2c0f HDDS-465. Suppress group mapping lookup warnings for ozone. Contributed by Xiaoyu Yao.
add 82fbbd5 HDDS-469. Rename 'ozone oz' to 'ozone sh'. Contributed by Arpit Agarwal.
add 07385f8 HDDS-470. Ozone acceptance tests are failing. Contributed by Elek, Marton.
add 95231f1 HDDS-362. Modify functions impacted by SCM chill mode in ScmBlockLocationProtocol. Contributed by Ajay Kumar.
add 33d8327 YARN-8715. Make allocation tags in the placement spec optional for node-attributes. Contributed by Weiwei Yang.
add 9a265fa YARN-8782. Fix exception message in Resource.throwExceptionWhenArrayOutOfBound. Contributed by Gergely Pollak.
add 8469366 HDDS-399. Persist open pipeline information across SCM restart. Contributed by Mukul Kumar Singh.
add fdf5a3f HDFS-13919. Documentation: Improper formatting in Disk Balancer for Settings. Contributed by Ayush Saxena.
add 8af8453 HDDS-435. Enhance the existing ozone documentation. Contributed by Elek, Marton.
add 3d89c3e HDDS-463. Fix the release packaging of the ozone distribution. Contributed by Elek Marton.
add 26d0c63 HADOOP-15754. s3guard: testDynamoTableTagging should clear existing config. Contributed by Gabor Bota.
add 23a6137 HDDS-475. Block Allocation returns same BlockID on different keys creation. Contributed by Nanda Kumar.
add d154193 HDFS-13844. Fix the fmt_bytes function in the dfs-dust.js. Contributed by yanghuafeng.
add 0a26c52 HDDS-487. Doc files are missing ASF license headers. Contributed by Namit Maheshwari.
add 8b2f5e6 HDDS-352. Separate install and testing phases in acceptance tests. Contributed by Elek Marton.
add 281c192 [JDK10] Upgrade Maven Javadoc Plugin from 3.0.0-M1 to 3.0.1.
add ee051ef HADOOP-15742. Log if ipc backoff is enabled in CallQueueManager. Contributed by Ryan Wu.
add 51fda2d HDDS-491. Minor typos in README.md in smoketest. Contributed by chencan.
add 0cc6e03 YARN-8726. [UI2] YARN UI2 is not accessible when config.env file failed to load. Contributed by Akhil PB.
add bbeca01 YARN-8652. [UI2] YARN UI2 breaks if getUserInfo REST API is not available in older versions. Contributed by Akhil PB.
add 001611c HDDS-481. Classes are missing from the shaded ozonefs jar. Contributed by Bharat Viswanadham.
add b6ad84e HDDS-468. Add version number to datanode plugin and ozone file system jar. Contributed by Bharat Viswanadham.
add 78a0d17 YARN-8787. Fix broken list items in PlacementConstraints documentation. Contributed by Masahiro Tanaka.
add f796cfd HADOOP-15304. [JDK10] Migrate from com.sun.tools.doclets to the replacement. Contributed by Akira Ajisaka.
add e4fca6a HADOOP-15759. AliyunOSS: Update oss-sdk version to 3.0.0. Contributed by Jinhu Wu.
add 48319d6 HDDS-483. Update ozone Documentation to fix below issues. Contributed by Namit Maheshwari.
add 6ff509c HDFS-11719. Arrays.fill() wrong index in BlockSender.readChecksum() exception handling. Contributed by Tao Zhang
add f938925 HDDS-464. Fix TestCloseContainerHandlingByClient. Contributed by Lokesh Jain.
add 2cf8927 MAPREDUCE-3801: org.apache.hadoop.mapreduce.v2.app.TestRuntimeEstimators.testExponentialEstimator fails intermittently. Contributed by Jason Lowe
add 295cce3 HDDS-366. Update functions impacted by SCM chill mode in StorageContainerLocationProtocol. Contributed by Ajay Kumar.
add 2df0a8d YARN-8648. Container cgroups are leaked when using docker. Contributed by Jim Brennan
add 5896372 HDDS-495. Ozone docs and ozonefs packages have undefined hadoop component. Contributed by Dinesh Chitlangia.
add e71f61e HADOOP-15755. StringUtils#createStartupShutdownMessage throws NPE when args is null. Contributed by Lokesh Jain and Dinesh Chitlangia
add a968ea4 HDDS-440. Datanode loops forever if it cannot create directories. Contributed by Bharat Viswanadham.
add 5c2ae7e HDDS-496. Ozone tools module is incorrectly classified as 'hdds' component. Contributed by Dinesh Chitlangia.
add 34b2237 MAPREDUCE-7137. MRAppBenchmark.benchmark1() fails with NullPointerException. Contributed by Oleksandr Shevchenko
add 8382b86 MAPREDUCE-7138. ThrottledContainerAllocator in MRAppBenchmark should implement RMHeartbeatHandler. Contributed by Oleksandr Shevchenko
add 44857476 HDFS-13886. HttpFSFileSystem.getFileStatus() doesn't return "snapshot enabled" bit. Contributed by Siyao Meng.
add f176e8a3 HDDS-501. AllocateBlockResponse.keyLocation must be an optional field. Contributed by Arpit Agarwal.
add 7ff00f5 HDDS-500. TestErrorCode.java has wrong package name. Contributed by Anu Engineer.
add 17f5651 HDDS-503. Suppress ShellBasedUnixGroupsMapping exception in tests. Contributed by Arpit Agarwal.
add 3929653 HDDS-488. Handle chill mode exception from SCM in OzoneManager. Contributed by Ajay Kumar.
add 27978bc HDFS-13833. Improve BlockPlacementPolicyDefault's consider load logic. Contributed by Shweta.
add fb85351 HDDS-497. Suppress license warnings for error log files. Contributed by Arpit Agarwal.
add e435e12 HDDS-506. Fields in AllocateScmBlockResponseProto should be optional. Contributed by Arpit Agarwal.
add 28ceb34 HDFS-13868. WebHDFS: GETSNAPSHOTDIFF API NPE when param "snapshotname" is given but "oldsnapshotname" is not. Contributed by Pranay Singh.
add 0712537e YARN-8771. CapacityScheduler fails to unreserve when cluster resource contains empty resource type. Contributed by Tao Yang.
add c0956ee HDDS-476. Add Pipeline reports to make pipeline active on SCM restart. Contributed by Mukul Kumar Singh.
add 15ed74f HDDS-502. Exception in OM startup when running unit tests. Contributed by Arpit Agarwal.
add b3c5221 HDDS-458. numberofKeys is 0 for all containers even when keys are present. Contributed by LiXin Ge.
add 61a4b07 HDDS-461. Container remains in CLOSING state in SCM forever. Contributed by Shashikant Banerjee.
add 56e0d63 HDFS-13908. TestDataNodeMultipleRegistrations is flaky. Contributed by Ayush Saxena.
add 1824d5d YARN-8757. [Submarine] Add Tensorboard component when --tensorboard is specified. Contributed by Wangda Tan.
add efdea85 YARN-8791. Trim docker inspect output for line feed for STOPSIGNAL parsing. Contributed by Chandni Singh
add 042bf74 HDDS-460. Replication manager failed to import container data. Contributed by Elek, Marton.
add 236d16e HDDS-507. EventQueue should be shutdown on SCM shutdown. Contributed by Xiaoyu Yao.
add 090272d HADOOP-15684. triggerActiveLogRoll stuck on dead name node, when ConnectTimeoutException happens. Contributed by Rong Tang.
add 98c9bc4 HDDS-509. TestStorageContainerManager is flaky. Contributed by Xiaoyu Yao and Ajay Kumar.
add a30b4f9 HADOOP-15726. Create utility to limit frequency of log statements. Contributed by Erik Krogen.
add 2eb597b HADOOP-15772. Remove the 'Path ... should be specified as a URI' warnings on startup. Contributed by Ayush Saxena.
add f6bb1ca HDDS-513. Check if the EventQueue is not closed before executing handlers. Contributed by Nanda Kumar.
add 6b5838e YARN-8784. DockerLinuxContainerRuntime prevents access to distributed cache entries on a full disk. Contributed by Eric Badger
add 6fc293f HDFS-13892. Disk Balancer: Make execute command documentation better. Contributed by Ranith Sardar.
add 7ad27e9 HADOOP-15736. Trash : Negative Value For Deletion Interval Leads To Abnormal Behaviour. Contributed by Ayush Saxena.
add 646874c HADOOP-15748. S3 listing inconsistency can raise NPE in globber. Contributed by Steve Loughran.
add 3da94a3 HADOOP-15756. [JDK10] Migrate from sun.net.util.IPAddressUtil to the replacement. Contributed by Akira Ajisaka.
add 429a07e HADOOP-15764. [JDK10] Migrate from sun.net.dns.ResolverConfiguration to the replacement. Contributed by Akira Ajisaka.
add aa4bd49 YARN-8801. Fixed header comments for docker utility functions. Contributed by Zian Chen
add 096a716 HDDS-394. Rename *Key Apis in DatanodeContainerProtocol to *Block apis. Contributed Dinesh Chitlangia.
add 524f7cd HDDS-514. Clean Unregister JMX upon SCMConnectionManager#close. Contributed by Xiaoyu Yao.
add a275277 YARN-8628. [UI2] Few duplicated or inconsistent information displayed in UI2. Contributed by Akhil PB.
add 0cd6346 YARN-8769. [Submarine] Allow user to specify customized quicklink(s) when submit Submarine job. Contributed by Wangda Tan.
add 4758b4b HDFS-13927. Improve TestDataNodeMultipleRegistrations#testDNWithInvalidStorageWithHA wait. Contributed by Ayush Saxena.
add f044dee HADOOP-15407. HADOOP-15540. Support Windows Azure Storage - Blob file system "ABFS" in Hadoop: Core Commit.
add a271fd0 HADOOP-15560. ABFS: removed dependency injection and unnecessary dependencies. Contributed by Da Zhou.
add ce03a93 HADOOP-15446. ABFS: tune imports & javadocs; stabilise tests. Contributed by Steve Loughran and Da Zhou.
add b54b0c1 HADOOP-15659. Code changes for bug fix and new tests. Contributed by Da Zhou.
add cc5cc60 Fixing issue due to commit 2b2399d6 after rebase onto trunk.
add d6a4f39 HADOOP-15669. ABFS: Improve HTTPS Performance. Contributed by Vishwajeet Dusane.
add 9149b97 HADOOP-15660. ABFS: Add support for OAuth Contributed by Da Zhou, Rajeev Bansal, and Junhua Gu.
add 9c1e4e8 HADOOP-15661. ABFS: Add support for ACL. Contributed by Junhua Gu and Da Zhou.
add 6b6f8cc2 HADOOP 15688. ABFS: InputStream wrapped in FSDataInputStream twice. Contributed by Sean Mackrory.
add dd2b22f HADOOP-15682. ABFS: Add support for StreamCapabilities. Fix javadoc and checkstyle. Contributed by Thomas Marquardt.
add df57c6c HADOOP-15692. ABFS: extensible support for custom oauth. Contributed by Junhua Gu and Rajeev Bansal.
add 81dc4a9 HADOOP-15663. ABFS: Simplify configuration. Contributed by Da Zhou.
add 4410eac HADOOP-15664. ABFS: Reduce test run time via parallelization and grouping. Contributed by Da Zhou.
add 97f06b3 HADOOP-15703. ABFS - Implement client-side throttling. Contributed by Sneha Varma and Thomas Marquardt.
add 347a52a Fixing findbugs and license issues related to:
add 6801b30 HADOOP-15728. ABFS: Add backward compatibility to handle Unsupported Operation for storage account with no namespace feature.
add 9475fd9 HADOOP-15745. ABFS: Add ABFS configuration to ConfigRedactor.
add 13c70e9 HADOOP-15740. ABFS: Check variable names during initialization of AbfsClientThrottlingIntercept. Contributed by Sneha Varma.
add e5593cb HADOOP-15694. ABFS: Allow OAuth credentials to not be tied to accounts. Contributed by Sean Mackrory.
add 2621101 HADOOP-15753. ABFS: support path "abfs://mycluster/file/path" Contributed by Da Zhou.
add b4c2304 HADOOP-15757. ABFS: remove dependency on common-codec Base64. Contributed by Da Zhou.
add 1cf38a3 HADOOP-15744. AbstractContractAppendTest fails against HDFS on HADOOP-15407 branch. Contributed by Steve Loughran.
add 5247766 HADOOP-15715. ITestAzureBlobFileSystemE2E timing out with non-scale timeout of 10 min. Contributed by Da Zhou
add 51d3689 HADOOP-15714. Tune abfs/wasb parallel and sequential test execution. Contributed by Da Zhou.
add df2166a HADOOP-15719. Fail-fast when using OAuth over http. Contributed by Da Zhou.
add a55d26b2 HADOOP-15769. ABFS: distcp tests are always skipped. Contributed by Steve Loughran
add 8e831ba HADOOP-15773. Fixing checkstyle and other issues raised by Yetus.
add a5692c2 HADOOP-15704. Mark ABFS extension package and interfaces as LimitedPrivate/Unstable. Contributed by Steve Loughran.
add d0b4624 HADOOP-15778. ABFS: Fix client side throttling for read. Contributed by Sneha Varma.
add 0def614 Merge branch 'HADOOP-15407' into trunk
add 32a35dc YARN-8742. [UI2] Container logs on Application / Service pages on UI2 are not available many case, improve error messages in such cases. Contributed by Akhil PB.
add d060cbe HDFS-13937. Multipart Uploader APIs to be marked as private/unstable in 3.2.0. Contributed by Steve Loughran.
add 62f817d HDDS-447. Separate ozone-dist and hadoop-dist projects with real classpath separation. Contributed by Elek Marton.
add 8de5c92 HDFS-13876. HttpFS: Implement ALLOWSNAPSHOT, DISALLOWSNAPSHOT. Contributed by Siyao Meng.
add 3090922 YARN-8696. [AMRMProxy] FederationInterceptor upgrade: home sub-cluster heartbeat async. Contributed by Botong Huang.
add c07715e HADOOP-15781 S3A assumed role tests failing due to changed error text in AWS exceptions. Contributed by Steve Loughran
add 67173b4 HDDS-441. Create new s3gateway daemon. Contributed by Elek Marton.
add 230d90c MAPREDUCE-7125. JobResourceUploader creates LocalFileSystem when it's not necessary. (Peter Cseh via wangda)
add cf62ff9 YARN-6338. Typos in Docker docs: contains => containers. (Contributed by Zoltan Siegl)
add 29dad7d YARN-8616. systemClock should be used in RMAppImpl instead of System.currentTimeMills(), to be consistent. (Contributed by Szilard Nemeth)
add 9c3fbbc HDDS-444. Add rest service to the s3gateway. Contributed by Elek Marton.
add d0bc6a2 YARN-8817. [Submarine] In cases when user doesn't ask HDFS path while submitting job but framework requires user to set HDFS related environments. Contributed by Wangda Tan.
add 50bc774 YARN-8815. RM fails to recover finished unmanaged AM. Contributed by Bibin A Chundatt.
add 93b0f54 HADOOP-15783. [JDK10] TestSFTPFileSystem.testGetModifyTime fails.
add 2b63512 HADOOP-15671. AliyunOSS: Support Assume Roles in AliyunOSS. Contributed by Jinhu Wu.
add 26c94a0 YARN-8623. Update Docker examples to use image which exists. Contributed by Craig Condit
add d5da992 HADOOP-15723. ABFS: Ranger Support. Contributed by Yuan Gao.
add a4abf02 HADOOP-15739. ABFS: remove unused maven dependencies and add used undeclared dependencies. Contributed by Da Zhou.
add 3928af9 HDDS-370. Add and implement following functions in SCMClientProtocolServer. Contributed by Ajay Kumar.
add 94b5c54 YARN-8752. yarn-registry.md has wrong word ong-lived, it should be long-lived. Contributed by leiqiang.
add 44edcdf YARN-8745. Misplaced the TestRMWebServicesFairScheduler.java file. Contributed by Y. SREENIVASULU REDDY.
add c62b440 HADOOP-15693. Typo in Secure Mode documentation. Contributed by Thomas Levine.
add 3f61950 HDDS-554. In XceiverClientSpi, implement sendCommand(..) using sendCommandAsync(..). Contributed by Tsz Wo Nicholas Sze.
add e5287a4 YARN-8824. App Nodelabel missed after RM restart for finished apps. Contributed by Bibin A Chundatt.
add 6275b42 HDFS-13927. Addendum: Improve TestDataNodeMultipleRegistrations#testDNWithInvalidStorageWithHA wait. Contributed by Ayush Saxena.
add e0ff8e2 HDFS-13840. RBW Blocks which are having less GS should be added to Corrupt. Contributed by Brahma Reddy Battula
add 913f87d YARN-8665. Added Yarn service cancel upgrade option. Contributed by Chandni Singh
add 6b988d8 YARN-8804. resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues. Contributed by Tao Yang
add f9c0221 HDFS-13932. [JDK 11] Casts to BlockStoragePolicy[] in unit tests raise ClassCastExceptions. Contributed by Andrew Purtell.
add 184544e HDDS-401. Update storage statistics on dead node. Contributed by LiXin Ge.
add b237a0d YARN-6456. Added config to set default container runtimes. Contributed by Craig Condit
add 90e2e49 YARN-8270 Adding JMX Metrics for Timeline Collector and Reader. Contributed by Sushil Ks.
add 2a5d431 HDDS-551. Fix the close container status check in CloseContainerCommandHandler. Contributed by Shashikant Banerjee.
add 5c8d907 HADOOP-15764. Addendum patch: Fix NPE in SecurityUtil.
add 284aca8 HDFS-13938. Add a missing break in BaseTestHttpFSWith. Contributed by Siyao Meng.
add 7093afd YARN-8829. Cluster metrics can fail with IndexOutOfBoundsException. Contributed by Akshay Agarwal.
add 8598b49 YARN-8774. Memory leak when CapacityScheduler allocates from reserved container with non-default label. Contributed by Tao Yang.
add e211350 HDFS-8196. Post enabled Erasure Coding Policies on NameNode UI. Contributed by Kitti Nanasi and Kai Sasaki.
add d1c1dde YARN-8734. Readiness check for remote service belonging to the same user. Contributed by Eric Yang
add 3c798c1 HDDS-448. Move NodeStat to NodeStatemanager from SCMNodeManager. Contributed by LiXin Ge.
add 72891fc HDDS-289. Volume and Bucket name should not contain a delimiter ('/'). Contributed by chencan.
add 19ad5be YARN-8800. Updated documentation of Submarine with latest examples. Contributed by Wangda Tan.
add 35c7351 HDDS-455. Ozone genconf tool must use picocli. Contributed by Dinesh Chitlangia.
add 0da03f8 HDDS-391. Simplify Audit Framework to make audit logging easier to use. Contributed by Dinesh Chitlangia.
add fd6be58 YARN-8468. Enable the use of queue based maximum container allocation limit and implement it in FairScheduler. Contributed by Antal Bálint Steinbach.
add f7ff8c0 HDDS-325. Add event watcher for delete blocks command. Contributed by Lokesh Jain.
add cc80ac2 HDDS-557. DeadNodeHandler should handle exception from removeContainerHandler api. Contributed by Ajay Kumar.
add 59d5af2 YARN-8760. [AMRMProxy] Fix concurrent re-register due to YarnRM failover in AMRMClientRelayer. Contributed by Botong Huang.
add 4eff629 HDDS-525. Support virtual-hosted style URLs. Contributed by Bharat Viswanadham
add d0ee6fb YARN-8621. Add test coverage of custom Resource Types for the apps/<appId> REST API endpoint. (Contributed by Szilard Nemeth)
add 7d08219 HDDS-562. Create acceptance test to test aws cli with the s3 gateway. Contributed by Elek Marton.
add f6c5ef9 HDFS-13943. [JDK10] Fix javadoc errors in hadoop-hdfs-client module. Contributed by Akira Ajisaka.
add 5689355 HDFS-13768. Adding replicas to volume map makes DataNode start slowly. Contributed by Surendra Singh Lilhore.
add 43bc984 HADOOP-15793. ABFS: Skip unsupported test cases when non namespace enabled in ITestAzureBlobFileSystemAuthorization Contributed by Yuan Gao.
add a383ac4 HADOOP-15801. ABFS: Fixing skipUserGroupMetadata in AzureBlobFileSystemStore. Contributed by Da Zhou
add 6fab688 HDFS-13936. Multipart upload to HDFS to support 0 byte upload. Contributed by Ewan Higgs.
add 5edb9d3 YARN-8785. Improve the error message when a bind mount is not whitelisted. Contributed by Simon Prewo
add 81072d5 HDDS-560. Create Generic exception class to be used by S3 rest services. Contributed by Bharat Viswanadham.
add a392962 HDDS-561. Move Node2ContainerMap and Node2PipelineMap to NodeManager. Contributed by Lokesh Jain.
add 58fa96b Changed version in trunk to 3.3.0-SNAPSHOT.
add e8b8604 HADOOP-15792. typo in AzureBlobFileSystem.getIsNamespaceEnabeld. Contributed by Abhishek Modi.
add 96ae4ac Changed version in trunk to 3.3.0-SNAPSHOT addendum
add 31fa9bb HADOOP-15767. [JDK10] Building native package on JDK10 fails due to missing javah. Contributed by Takanobu Asanuma.
add ec07579 HDDS-520. Implement HeadBucket REST endpoint. Contributed by Bharat Viswanadham.
add fa7f707 HDFS-13944. [JDK10] Fix javadoc errors in hadoop-hdfs-rbf module. Contributed by Inigo Goiri.
add 046b876 HADOOP-15621 S3Guard: Implement time-based (TTL) expiry for Authoritative Directory Listing. Contributed by Gabor Bota
add 4f752d4 HADOOP-15621 2/2 S3Guard: Implement time-based (TTL) expiry for Authoritative Directory Listing. Contributed by Gabor Bota
add 2626f46 HADOOP-15814. Maven 3.3.3 unable to parse pom file. Contributed by Wei-Chiu Chuang.
add 7b37448 HDDS-526. Clean previous chill mode code from NodeManager. Contributed by Ajay Kumar.
add 095c269 HDDS-567. Rename Mapping to ContainerManager in SCM. Contributed by Nanda kumar.
add 7051bd7 HADOOP-15795. Make HTTPS the default protocol for ABFS. Contributed by Da Zhou.
add 1dc0adf HDFS-13947. Review of DirectoryScanner Class. Contributed by BELUGA BEHR.
add 39b3503 YARN-8840. Add missing cleanupSSLConfig() call for TestTimelineClient test. Contributed by Aki Tanaka.
add 81f635f HADOOP-15817. Reuse Object Mapper in KMSJSONReader. Contributed by Jonathan Eagles.
add 6926fd0 YARN-8758. Support getting PreemptionMessage when using AMRMClientAsyn. (Zian Chen via wangda)
add 2e9913c YARN-8844. TestNMProxy unit test is failing. (Eric Yang via wangda)
add b6d5d84 YARN-8732. Add unit tests of min/max allocation for custom resource types in FairScheduler. (Contributed by Szilard Nemeth)
add e60b797 YARN-8750. Refactor TestQueueMetrics. (Contributed by Szilard Nemeth)
add 396ce0d HDFS-13877. HttpFS: Implement GETSNAPSHOTDIFF. Contributed by Siyao Meng.
add cc2babc HDFS-13950. ACL documentation update to indicate that ACL entries are capped by 32. Contributed by Adam Antal.
add 619e490 HDFS-13957. Fix incorrect option used in description of InMemoryAliasMap.
add 7fb91b8 HDDS-565. TestContainerPersistence fails regularly in Jenkins. Contributed by Dinesh Chitlangia.
add e6b77ad HDDS-572. Support S3 buckets as first class objects in Ozone Manager - 1. Contributed by Anu Engineer.
add 2a07617 HDDS-354. VolumeInfo.getScmUsed throws NPE. Contributed by Hanisha Koneru.
add 153941b HDDS-479. Add more ozone fs tests in the robot integration framework. Contributed by Nilotpal Nandi.
add 241cbec HADOOP-15816. Upgrade Apache Zookeeper version due to security concerns. Contributed by Akira Ajisaka.
add 751f626 YARN-7957. [UI2] YARN service delete option disappears after stopping application. Contributed by Akhil PB.
add 9bc8b1c HDDS-578. om-audit-log4j2.properties must be packaged in ozone-dist . Contributed by Dinesh Chitlangia.
add 94d82f4 YARN-8797. [UI2] Improve error pages in new YARN UI. Contributed by Akhil PB.
add 273cc2d HADOOP-15809. ABFS: better exception handling when making getAccessToken call. Contributed by Da Zhou
add f362037 HDDS-575. LoadExistingContainers should be moved to SCMContainerManager from ContainerStateManager. Contributed by Mukul Kumar Singh.
add 0e28ef3 HDDS-512. update test.sh to remove robot framework & python-pip installation. Contributed by Dinesh Chitlangia.
add 25b1e8a HDDS-512. Addendum: update test.sh to remove robot framework & python-pip installation. Contributed by Dinesh Chitlangia.
add f13e231 HADOOP-15820. ZStandardDecompressor native code sets an integer field as a long. Contributed by Jason Lowe
add 73c660b HDFS-13958. Miscellaneous Improvements for FsVolumeSpi. Contributed by BELUGA BEHR.
add c968365 YARN-8644. Improve unit test for RMAppImpl.FinalTransition. (Contributed by Szilard Nemeth)
add 9bb2801 YARN-4254. ApplicationAttempt stuck for ever due to UnknownHostException. Contributed by Bibin A Chundatt
add cdf5d58 HDDS-571. Update SCM chill mode exit criteria to optionally wait for n datanodes. Contributed by Ajay Kumar.
add b85ee0d YARN-8819. Fix findbugs warnings in YarnServiceUtils
add 3bfff0f HDFS-13964. RBF: TestRouterWebHDFSContractAppend fails with No Active Namenode under nameservice. Contributed by Ayush Saxena.
add 907ef6c YARN-8788. mvn package -Pyarn-ui fails on JDK9
add 228dc19 YARN-7825. [UI2] Maintain constant horizontal application info bar for Application Attempt page. Contributed by Akhil PB.
add d7c7f68 HDDS-521. Implement DeleteBucket REST endpoint. Contributed by Bharat Viswanadham.
add 7c13872 YARN-8659. RMWebServices returns only RUNNING apps when filtered with queue. (Contributed by Szilard Nemeth)
add 745f640 YARN-8843. updateNodeResource does not support units for memory. Contributed by Manikandan R.
add 347ea38 YARN-8763. Added node manager websocket API for accessing containers. Contributed by Zian Chen
add 1043795 HDFS-13962. Add null check for add-replica pool to avoid lock acquiring. Contributed by Surendra Singh Lilhore.
add 08bb6c49 HDFS-13926. ThreadLocal aggregations for FileSystem.Statistics are incorrect with striped reads. Contributed by Xiao Chen, Hrishikesh Gadre.
add a30b1d1 HADOOP-15818. Fix deprecated maven-surefire-plugin configuration in hadoop-kms module
add 9bbeb52 HADOOP-15775. [JDK9] Add missing javax.activation-api dependency. Contributed by Akira Ajisaka.
add d4626b4 HDFS-13967. HDFS Router Quota Class Review. Contributed by BELUGA BEHR.
add 7ba1cfd HADOOP-15827. NPE in DynamoDBMetadataStore.lambda$listChildren for root + auth S3Guard. Contributed by Gabor Bota
add 5b1cfca MAPREDUCE-7035. Skip javadoc build for auto-generated sources in hadoop-mapreduce-client. Contributed by Mukul Kumar Singh.
add 7367ff3 HDDS-450. Generate BlockCommitSequenceId in ContainerStateMachine for every commit operation in Ratis. Contributed by Shashikant Banerjee.
add 9e9915d HDDS-599. Fix TestOzoneConfiguration TestOzoneConfigurationFields. Contributed by Sandeep Nemuri.
add 6f11919 YARN-8853. [UI2] Application Attempts tab is not shown correctly when there are no attempts. Contributed by Akhil PB.
add 600438b HDDS-602. Bump Ozone version to 0.4.0-SNAPSHOT after ozone-0.3 branch cut.
add a23ea68 YARN-8852. Add documentation for submarine installation details. (Zac Zhou via wangda)
add c3d22d3 YARN-7652. Handle AM register requests asynchronously in FederationInterceptor. Contributed by Botong Huang.
add 03d66b1 HDDS-590. Add unit test for HDDS-583. Contributed by Namit Maheshwari.
add 4b5b1ac MAPREDUCE-7130. Rumen crashes trying to handle MRAppMaster recovery events. Contributed by Peter Bacsko
add a05bd12 HDFS-13979. Review StateStoreFileSystemImpl Class. Contributed by BELUGA BEHR.
add bd50fa9 HADOOP-15825. ABFS: Enable some tests for namespace not enabled account using OAuth. Contributed by Da Zhou.
add d5dd6f3 HDDS-585. Handle common request identifiers in a transparent way. Contributed by Elek Marton.
add bf04f19 YARN-8845. Removed unused hadoop.registry.rm reference. Contributed by Íñigo Goiri
add 5b7ba48 HDDS-577. Support S3 buckets as first class objects in Ozone Manager - 2. Contributed by Bharat Viswanadham.
add 6a39739 HDDS-583. SCM returns zero as the return code, even when invalid options are passed. Contributed by Namit Maheshwari.
add c1fe657 HDDS-478. Log files related to each daemon doesn't have proper startup and shutdown logs. Contributed by Dinesh Chitlangia.
add 6a06bc3 HDDS-559. fs.default.name is deprecated. Contributed by Dinesh Chitlangia.
add 605622c HDFS-11396. TestNameNodeMetadataConsistency#testGenerationStampInFuture timed out. Contributed by Ayush Saxena.
add 4de2dc2 HDDS-568. ozone sh volume info, update, delete operations fail when volume name is not prefixed by /. Contributed by Dinesh Chitlangia.
add 794c045 HDDS-604. Correct Ozone getOzoneConf description. Contributed by Dinesh Chitlangia.
add 6fa3feb HADOOP-15832. Upgrade BouncyCastle to 1.60. Contributed by Robert Kanter.
add f068296 HDDS-443. Create reusable ProgressBar utility for freon tests. Contributed by Zsolt Horvath.
add edce866 YARN-8858. CapacityScheduler should respect maximum node resource when per-queue maximum-allocation is being used. Contributed by Wangda Tan.
add b39b802 HDFS-13949. Correct the description of dfs.datanode.disk.check.timeout in hdfs-default.xml. Contributed by Toshihiro Suzuki.
add cd28051 HDDS-517. Implement HeadObject REST endpoint. Contributed by LiXin Ge.
add 5ce70e1 YARN-7644. NM gets backed up deleting docker containers. Contributed by Chandni Singh
add 90552b1 HDDS-608. Access denied exception for user hdfs while making getContainerWithPipeline call. Contributed by Mukul Kumar Singh.
add 10185d9 HDFS-13882. Set a maximum delay for retrying locateFollowingBlock. Contributed by Kitti Nanasi.
add cdc4350 HADOOP-15839. Review + update cloud store sensitive keys in hadoop.security.sensitive-config-keys. Contributed by Steve Loughran.
add 3ead525 HDFS-12459. Fix revert: Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang.
add bf3d591 HDFS-13956. iNotify should include information to identify a file as either replicated or erasure coded. Contributed by Hrishikesh Gadre.
add 045069e YARN-6989 Ensure timeline service v2 codebase gets UGI from HttpServletRequest in a consistent way. Contributed by Abhishek Modi
add 2bd000c HDDS-619. hdds.db.profile should not be tagged as a required setting & should default to DISK. Contributed by Dinesh Chitlangia.
add f261c31 HADOOP-15708. Reading values from Configuration before adding deprecations make it impossible to read value with deprecated key (zsiegl via rkanter)
add 7b57f2f HADOOP-15785. [JDK10] Javadoc build fails on JDK 10 in hadoop-common. Contributed by Dinesh Chitlangia.
add d3da741 YARN-8666. [UI2] Remove application tab from YARN Queue Page. Contributed by Yesha Vora.
add 8a37983 YARN-8753. [UI2] Lost nodes representation missing from Nodemanagers Chart. Contributed by Yesha Vora.
add ee816f1 HADOOP-15837. DynamoDB table Update can fail S3A FS init. Contributed by Steve Loughran.
add e787d65 YARN-8861. executorLock is misleading in ContainerLaunch. Contributed by Chandni Singh
add aeeb038 YARN-8710. Service AM should set a finite limit on NM container max retries. Contributed by Suma Shivaprasad
add 47eeb3e HDDS-630. Rename KSM to OM in Hdds.md. Contributed by Takanobu Asanuma.
add 96d28b4 YARN-8777. Container Executor C binary change to execute interactive docker command. Contributed by Eric Yang
add 735a36a MAPREDUCE-7149. Javadocs for FileInputFormat and OutputFormat to mention DT collection. Contributed by Steve Loughran.
add 2addebb HDFS-13951. HDFS DelegationTokenFetcher can't print non-HDFS tokens in a tokenfile. Contributed by Steve Loughran.
add eb34b5f HDDS-601. On restart, SCM throws 'No such datanode' exception.
add 3532aa3 HDFS-13968. BlockReceiver Array-Based Queue. Contributed by BELUGA BEHR.
add 6dcfef7 HDFS-13878. HttpFS: Implement GETSNAPSHOTTABLEDIRECTORYLIST. Contributed by Siyao Meng.
add c05b260 HDDS-627. OzoneFS read from an MR Job throws java.lang.ArrayIndexOutOfBoundsException. Contributed by Mukul Kumar Singh.
add 64f2b32 HADOOP-15676. Cleanup TestSSLHttpServer. Contributed by Szilard Nemeth.
add d91d47b HADOOP-15717. TGT renewal thread does not log IOException (snemeth via rkanter)
add 8d19818 YARN-5742 Serve aggregated logs of historical apps from timeline service. Contributed by Rohith Sharma KS
add 0bc6d04 HDDS-550. Serialize ApplyTransaction calls per Container in ContainerStateMachine. Contributed by Shashikant Banerjee.
add 5055247 HDDS-636. Turn off ozone.metastore.rocksdb.statistics for now. Contributed by Arpit Agarwal.
add 604c27c HDDS-634. OzoneFS: support basic user group and permission for file/dir. Contributed by Xiaoyu Yao.
add fb18cc5 HDDS-625. putKey hangs for a long time after completion, sometimes forever. Contributed by Arpit Agarwal
add a3edfdd YARN-8834 Provide Java client for fetching Yarn specific entities from TimelineReader. Contributed by Abhishek Modi
add c011810 HDFS-13973. getErasureCodingPolicy should log path in audit event. Contributed by Shweta.
add 74db993 HDDS-609. On restart, SCM does not exit chill mode as it expects DNs to report containers in ALLOCATED state. Contributed by Hanisha Koneru.
add bca928d YARN-3879 [Storage implementation] Create HDFS backing storage implementation for ATS reads. Contributed by Abhishek Modi.
add 5da0422 HDDS-628. Fix outdated names used in HDDS documentations.
add e36ae96 HADOOP-15831. Include modificationTime in the toString method of CopyListingFileStatus. Contributed by Ted Yu.
add de44e10 HDFS-13156. HDFS Block Placement Policy - Client Local Rack. Contributed by Ayush Saxena.
add 6e0e6da HDFS-13945. TestDataNodeVolumeFailure is Flaky. Contributed by Ayush Saxena.
add 5ec86b4 HADOOP-14445. Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances. Contributed by Daryn Sharp, Xiao Chen, Rushabh S Shah.
add f1342cd HADOOP-15832. Addendum: Upgrade BouncyCastle to 1.60. Contributed by Robert Kanter
add 38fe447 HDDS-524. Implement PutBucket REST endpoint. Contributed by Bharat Viswanadham.
add f63ee08 HDFS-13802. RBF: Remove FSCK from Router Web UI. Contributed by Fei Hui.
add e12edb3 HDDS-555. RandomKeyGenerator runs not closing the XceiverClient properly. Contributed by Mukul Kumar Singh.
add 07d5529 HDDS-555. RandomKeyGenerator runs not closing the XceiverClient properly(addendum). Contributed by Mukul Kumar Singh.
add 85ccab7 HDDS-641. Fix ozone filesystem robot test. Contributed by Mukul Kumar Singh.
add 56b18b9 HDDS-639. ChunkGroupInputStream gets into infinite loop after reading a block. Contributed by Mukul Kumar Singh.
add 8ae8a50 HDDS-606. Create delete s3Bucket. Contributed by Bharat Viswanadham.
add 02e1ef5 HDDS-624.PutBlock fails with Unexpected Storage Container Exception. Contributed by Arpit Agarwal.
add 3a684a2 HDDS-644. Rename dfs.container.ratis.num.container.op.threads. Contributed by Bharat Viswanadham.
add c07b95b HDDS-645. Enable OzoneFS contract tests by default. Contributed by Arpit Agarwal.
add 5c8e023 HDDS-587. Add new classes for pipeline management. Contributed by Lokesh Jain.
add ddc9649 HDDS-646. TestChunkStreams.testErrorReadGroupInputStream fails. Contributed by Nanda kumar.
add 3c1fe07 HDDS-445. Create a logger to print out all of the incoming requests. Contributed by Bharat Viswanadham.
add 28ca5c9 HDDS-613. Update HeadBucket, DeleteBucket to not to have volume in path. Contributed by Bharat Viswanadham.
add cdad91c HDDS-616. Collect all the robot test outputs and return with the right exit code. Contributed by Elek, Marton.
add ae268b4 HDDS-518. Implement PutObject Rest endpoint. Contributed by chencan.
add addb846 HDDS-654. Add ServiceLoader resource for OzoneFileSystem. Contributed by Jitendra Nath Pandey.
add 9227f3d HDDS-653. TestMetadataStore#testIterator fails on Windows. Contributed by Yiqun Lin.
add 0473b68 HDDS-629. Make ApplyTransaction calls in ContainerStateMachine idempotent. Contributed by Shashikant Banerjee.
add 22f37af Revert "HDDS-629. Make ApplyTransaction calls in ContainerStateMachine idempotent. Contributed by Shashikant Banerjee."
add 5209c75 HDDS-603. Add BlockCommitSequenceId field per Container and expose it in Container Reports. Contributed by Shashikant Banerjee.
add 603649d HDDS-579. ContainerStateMachine should fail subsequent transactions per container in case one fails. Contributed by Shashikant Banerjee.
add 8853fc8 HADOOP-15849. Upgrade netty version to 3.10.6.
add 5033deb HDDS-519. Implement ListBucket REST endpoint. Contributed by LiXin Ge.
add 8e5365e YARN-8836. Add tags and attributes in resource definition. Contributed by Weiwei Yang.
add b60ca37 Fix potential FSImage corruption. Contributed by Daryn Sharp.
add b4a38e7 YARN-8830. SLS tool fix node addition. Contributed by Bibin A Chundatt.
add fa94d37 YARN-8869. YARN Service Client might not work correctly with RM REST API for Kerberos authentication. Contributed by Eric Yang.
add f880ff4 YARN-8775. TestDiskFailures.testLocalDirsFailures sometimes can fail on concurrent File modifications. (Contributed by Antal Bálint Steinbach)
add ee1c80e HDFS-13987. RBF: Review of RandomResolver Class. Contributed by BELUGA BEHR.
add b6fc72a HADOOP-14445. Addendum: Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances.
add e13a38f HDDS-629. Make ApplyTransaction calls in ContainerStateMachine idempotent. Contributed by Shashikant Banerjee.
add ef9dc6c HADOOP-15851. Disable wildfly logs to the console. Contributed by Vishwajeet Dusane.
add 7fe1a40 HADOOP-15853. TestConfigurationDeprecation leaves behind a temp file, resulting in a license issue (ayushtkn via rkanter)
add 2614078 HDDS-490. Improve om and scm start up options . Contributed by Namit Maheshwari.
add 0bf8a11 HDDS-439. 'ozone oz volume create' should default to current logged in user. Contributed by Dinesh Chitlangia.
add 0c2914e HDDS-657. Remove {volume} path segments from all the remaining rest endpoints. Contributed by Elek, Marton.
add 753f149 YARN-8879. Kerberos principal is needed when submitting a submarine job. Contributed by Zac Zhou.
add 25f8fcb MAPREDUCE-7132. JobSplitWriter prints unnecessary warnings if EC(RS10,4) is used. Contributed by Peter Bacsko.
add 53e5173 HDDS-667. Fix TestOzoneFileInterfaces. Contributed by Mukul Kumar Singh.
add 5f0b43f HDDS-378. Remove dependencies between hdds/ozone and hdfs proto files. Contributed by Elek, Marton.
add f90c64e HDFS-13993. TestDataNodeVolumeFailure#testTolerateVolumeFailuresAfterAddingMoreVolumes is flaky. Contributed by Ayush Saxena.
add d59ca43 HADOOP-15826. @Retries annotation of putObject() call & uses wrong. Contributed by Steve Loughran and Ewan Higgs.
add c2288ac YARN-8448. AM HTTPS Support for AM communication with RMWeb proxy. (Contributed by Robert Kanter)
add babd144 MAPREDUCE-7150. Optimize collections used by MR JHS to reduce its memory. (Contributed by Misha Dmitriev)
add ed08dd3 YARN-8875. [Submarine] Add documentation for submarine installation script details. (Xun Liu via wangda)
add 46d6e00 YARN-8870. [Submarine] Add submarine installation scripts. (Xun Liu via wangda)
add 143d747 YARN-8798. [Submarine] Job should not be submitted if --input_path option is missing. (Zhankun Tang via wangda)
add 538250d YARN-8892. YARN UI2 doc changes to update security status (verified under security environment). (Sunil G via wangda)
add 84e22a6 YARN-8842. Expose metrics for custom resource types in QueueMetrics. (Contributed by Szilard Nemeth)
add 657032f HDDS-665. Add hdds.datanode.dir to docker-config. Contributed by Bharat Viswanadham.
add 3bfd214 YARN-8810. Fixed a YARN service bug in comparing ConfigFile object. Contributed by Chandni Singh
add 22f85f2 YARN-8582. Document YARN support for HTTPS in AM Web server. (Contributed by Robert Kanter)
add e3342a1 HADOOP-15802. start-build-env.sh creates an invalid /etc/sudoers.d/hadoop-build-${USER_ID} file entry. Contributed by Jon Boone.
add 5331387 HDFS-13662. TestBlockReaderLocal#testStatisticsForErasureCodingRead is flaky
add 64a43c9 HDDS-656. Add logic for pipeline report and action processing in new pipeline code. Contributed by Lokesh Jain.
add b738cb1 HADOOP-15854. AuthToken Use StringBuilder instead of StringBuffer. Contributed by Beluga Behr.
add 41b3603 HADOOP-15861. Move DelegationTokenIssuer to the right path. Contributed by Wei-Chiu Chuang.
add 5085e5f YARN-8759. Copy of resource-types.xml is not deleted if test fails, causes other test failures. Contributed by Antal Bálint Steinbach.
add 9df1c84 HDDS-563. Support hybrid VirtualHost style URL. Contributed by Bharat Viswanadham.
add a9a63ae HDDS-527. Show SCM chill mode status in SCM UI. Contributed by Yiqun Lin.
add 50715c0 HDDS-662. Introduce ContainerReplicaState in StorageContainerManager. Contributed by Nanda kumar.
add d54f559 HDDS-661. When a volume fails in datanode, VersionEndpointTask#call ends up in dead lock. Contributed by Hanisha Koneru.
add 24dc068 HADOOP-11100. Support to configure ftpClient.setControlKeepAliveTimeout. Contributed by Adam Antal.
add 9abda83 HADOOP-15859. ZStandardDecompressor.c mistakes a class for an instance. Contributed by Jason Lowe
add d93d515 HDDS-651. Rename o3 to o3fs for Filesystem.
add 7347fa2 HDFS-14000. RBF: Documentation should reflect right scripts for v3.0 and above. Contributed by CR Hota.
add 4c4364d HDDS-683. Add a shell command to provide ozone mapping for a S3Bucket. Contributed by Bharat Viswanadham.
add 9146d33 HDDS-670. Fix OzoneFS directory rename.
add 3ed7163 HDDS-678. Format of Last-Modified header is invalid in HEAD Object call. Contributed by Elek Marton.
add 2202e00 YARN-8868. Set HTTPOnly attribute to Cookie. Contributed by Chandni Singh.
add 32fe351 YARN-8864. NM incorrectly logs container user as the user who sent a start/stop container request in its audit log. (Contributed by Wilfred Spiegelenburg)
add ba7e816 YARN-8687. Update YARN service file type in documentation. Contributed by Suma Shivaprasad
add 1e78dfc HDFS-13990. Synchronization Issue With HashResolver. Contributed by BELUGA BEHR.
add 0e56c88 HDFS-14003. Fix findbugs warning in trunk for FSImageFormatPBINode. Contributed by Yiqun Lin.
add bfb88b1 YARN-6098. Add documentation for Delete Queue. (Suma Shivaprasad via wangda)
add a457a89 YARN-8456. Fix a configuration handling bug when user leave FPGA discover executable path configuration default but set OpenCL SDK path environment variable. (Zhankun Tang via wangda)
add 780be14 YARN-8896. Limit the maximum number of container assignments per heartbeat. (Zhankun Tang via wangda)
add c640180 HDDS-673. Suppress "Key not found" exception log with stack trace in OM. Contributed by Arpit Agarwal.
add f8d61b9 HDDS-690. Javadoc build fails in hadoop-ozone. Contributed by Takanobu Asanuma.
add beb850d YARN-8899. Fixed minicluster dependency on yarn-server-web-proxy. Contributed by Robert Kanter
add 098d0b8 HDFS-14005. RBF: Web UI update to bootstrap-3.3.7. Contributed by Inigo Goiri.
add be1cffb HDFS-14002. TestLayoutVersion#testNameNodeFeatureMinimumCompatibleLayoutVersions fails. Contributed by Takanobu Asanuma.
add 66f059e YARN-8618. Added detection for non-upgradable service. Contributed by Chandni Singh
add cc09b2b HDFS-14007. Incompatible layout when generating FSImage. Contributed by Inigo Goiri.
add cd21584 HADOOP-15418. Hadoop KMSAuthenticationFilter needs to use getPropsByPrefix instead of iterator to avoid ConcurrentModificationException. Contributed by lqjack and Suma Shivaprasad
add 13cc0f5 YARN-8907. Fix incorrect logging message in TestCapacityScheduler. Contributed by Zhankun Tang.
add 285d2c0 YARN-8449. RM HA for AM web server HTTPS Support. (Contributed by Robert Kanter)
add 9bd1832 HADOOP-15804. upgrade to commons-compress 1.18. Contributed by Akira Ajisaka.
add b22651e HDDS-680. Provide web based bucket browser. Contributed by Elek, Marton.
add cd05791 HDDS-672. Spark shell throws OzoneFileSystem not found. Contributed by Anu Engineer.
add 071b9e1 YARN-8908. Fix errors in yarn-default.xml related to GPU/FPGA. (Zhankun Tang via wangda)
add 5e02b49 YARN-8916. Define a constant docker string in ContainerRuntimeConstants.java for better maintainability. (Zhankun Tang via wangda)
add 9aebafd HDDS-701. Support key multi-delete. Contributed by Elek, Marton.
add 8b64fba HDFS-13994. Improve DataNode BlockSender waitForMinLength. Contributed by BELUGA BEHR.
add 88cce32 HDFS-9872. HDFS bytes-default configurations should accept multiple size units. Contributed by Yiqun Lin.
add 47ad98b YARN-8910. Fixed misleading log statement when container max retries is infinite. Contributed by Chandni Singh
add dc27408 HDDS-612. Even after setting hdds.scm.chillmode.enabled to false, SCM allocateblock fails with ChillModePrecheck exception. Contributed by Hanisha Koneru.
add c456d6b HDDS-621. ozone genconf improvements. Contributed by Dinesh Chitlangia.
add d7b012e HDDS-679. Add query parameter to the constructed query in VirtualHostStyleFilter. Contributed by Bharat Viswanadham.
add e2cecb6 HADOOP-15850. CopyCommitter#concatFileChunks should check that the blocks per chunk is not 0. Contributed by Ted Yu.
add 00254d7 HADOOP-15836. Review of AccessControlList. Contributed by BELUGA BEHR.
add e2a9fa8 HADOOP-15821. Move YARN Registry to Hadoop Registry. Contributed by Íñigo Goiri
add f069d38 HDFS-13983. TestOfflineImageViewer crashes in windows. Contributed by Vinayakumar B.
add a043dfa HADOOP-15821. Addendum patch to fix Hadoop Registry parent project. Contributed by Íñigo Goiri
add 977c6f6 HDDS-705. OS3Exception resource name should be the actual resource name. Contributed by Bharat Viswanadham.
add c9077a9 HDDS-707. Allow registering MBeans without additional jmx properties. Contributed by Arpit Agarwal.
add 7db3bb3 HDDS-544. Unconditional wait findbug warning from ReplicationSupervisor. Contributed by Arpit Agarwal.
add e503345 YARN-7502. Nodemanager restart docs should describe nodemanager supervised property. Contributed by Suma Shivaprasad.
add c696419 Revert "HDDS-705. OS3Exception resource name should be the actual resource name."
add 97a41b3 HDDS-705. OS3Exception resource name should be the actual resource name. Contributed by Bharat Viswanadham.
add 52cb766 HADOOP-15866. Renamed HADOOP_SECURITY_GROUP_SHELL_COMMAND_TIMEOUT keys break compatibility. Contributed by Wei-Chiu Chuang.
add 82919a1 HDDS-638. Enable ratis snapshots for HDDS datanodes. Contributed by Mukul Kumar Singh.
add f6498af HDDS-705. addendum patch to fix find bug issue. Contributed by Bharat Viswanadham.
add 66e7a2c YARN-8922. Fixed test-container-executor test setup and clean up. Contributed by Robert Kanter
add 292c9e0 YARN-8542. Added YARN service REST API to list containers. Contributed by Chandni Singh
add c58811c HDFS-13941. make storageId in BlockPoolTokenSecretManager.checkAccess optional. Contributed by Ajay Kumar.
add e98a506 YARN-8923. Cleanup references to ENV file type in YARN service code. Contributed by Suma Shivaprasad
add e3cca12 HDDS-676. Enable Read from open Containers via Standalone Protocol. Contributed by Shashikant Banerjee.
add 0efddd8 YARN-8873. [YARN-8811] Add CSI java-based client library. Contributed by Weiwei Yang.
add 0b62983 YARN-8826. Fix lingering timeline collector after serviceStop in TimelineCollectorManager. Contributed by Prabha Manepalli.
add b618463 HDDS-708. Validate BCSID while reading blocks from containers in datanodes. Contributed by Shashikant Banerjee.
add 296818a HADOOP-15873. Add JavaBeans Activation Framework API to LICENSE.txt. Contributed by Akira Ajisaka.
add dd268a6 Revert "HADOOP-15836. Review of AccessControlList. Contributed by BELUGA BEHR."
add 93fb3b4 YARN-8904. TestRMDelegationTokens can fail in testRMDTMasterKeyStateOnRollingMasterKey. Contributed by Wilfred Spiegelenburg
add 5e294e9 YARN-8919. Some tests fail due to NoClassDefFoundError for OperatorCreationException (tasanuma0829 via rkanter)
add efdfe67 HDFS-14004. TestLeaseRecovery2#testCloseWhileRecoverLease fails intermittently in trunk. Contributed by Ayush Saxena.
add 635786a HDFS-13566. Add configurable additional RPC listener to NameNode. Contributed by Chen Liang.
add 823bb5d MAPREDUCE-4669. MRAM web UI does not work with HTTPS. (Contributed by Robert Kanter)
add a0c0b79 YARN-8814. Yarn Service Upgrade: Update the swagger definition. Contributed by Chandni Singh
add 8f97d6f HADOOP-15822. zstd compressor can fail with a small output buffer. Contributed by Jason Lowe.
add 0ca5064 HADOOP-15856. Trunk build fails to compile native on Windows. Contributed by Vinayakumar B.
add 1c0aae6 HADOOP-14775. Change junit dependency in parent pom file to junit 5 while maintaining backward compatibility to junit4. Contributed by Akira Ajisaka.
add 5530448 YARN-8939. Javadoc build fails in hadoop-yarn-csi. Contributed by Weiwei Yang.
add 0891cdd HDDS-716. Update ozone to latest ratis snapshot build(0.3.0-aa38160-SNAPSHOT). Contributed by Mukul Kumar Singh.
add 766b78e YARN-8911. ContainerScheduler incorrectly uses percentage number as the cpu resource utlization.
add bbc6dcd YARN-8938. Updated YARN service upgrade document. Contributed by Chandni Singh
add c187404 HDFS-14021. TestReconstructStripedBlocksWithRackAwareness#testReconstructForNotEnoughRacks fails intermittently. Contributed by Xiao Chen.
add 74a5e68 MAPREDUCE-7151. RMContainerAllocator#handleJobPriorityChange expects application_priority always. Contributed by Bilwa S T.
add ebf8e17 HDDS-693. Support multi-chunk signatures in s3g PUT object endpoint. Contributed by Elek Marton.
add 69b3289 YARN-8929. DefaultOOMHandler should only pick running containers to kill upon oom events (haibochen via rkanter)
add e374584 HADOOP-15823. ABFS: Stop requiring client ID and tenant ID for MSI (Contributed by Da Zhou via Daniel Templeton)
add c16c49b YARN-8587. Added retries for fetching docker exit code. Contributed by Charo Zhang
add 021caaa HDDS-516. Implement CopyObject REST endpoint. Contributed by Bharat Viswanadham.
add 244afab HDDS-719. Remove Ozone dependencies on Apache Hadoop 3.2.0. Contributed by Arpit Agarwal.
add 936fc3f HDFS-14025. TestPendingReconstruction.testPendingAndInvalidate fails. Contributed by Ayush Saxena.
add ddc1e0b YARN-8941. Fix findbugs warnings in hadoop-yarn-csi. Contributed by Weiwei Yang.
add ace06a9 HDDS-714. Bump protobuf version to 3.5.1. Contributed by Mukul Kumar Singh.
add 97bd49f HDFS-14026. Overload BlockPoolTokenSecretManager.checkAccess to make storageId and storageType optional. Contributed by Arpit Agarwal.
add 2388712 HDDS-643. Parse Authorization header in a separate filter. Contributed by Bharat Viswanadham.
add c7f349b HDDS-703. Ozone docs does not render correctly on a Mobile Device. Contributed by Elek, Marton.
add fb2b72e HADOOP-15864. Job submitter / executor fail when SBN domain name can not resolved. Contributed by He Xiaoqiao.
add f76e3c3 YARN-8930. CGroup-based strict container memory enforcement does not work with CGroupElasticMemoryController (haibochen via rkanter)
add 34b2521 HADOOP-15882. Upgrade maven-shade-plugin from 2.4.3 to 3.2.0. Contributed by Takanobu Asanuma.
add 9b899f1 HDFS-14029. Sleep in TestLazyPersistFiles should be put into a loop. Contributed by Adam Antal.
add a4b9b7c HDDS-702:Used fixed/external version from hadoop jars in hdds/ozone projects. Contributed by Elek Marton.
add 38a65e3 HDDS-682. Unified o3 address parsing for ozonen sh. Contributed by Elek, Marton.
add 7574d18 HADOOP-15868. AliyunOSS: update document for properties of multiple part download, multiple part upload and directory copy. Contributed by Jinhu Wu.
add e28c00c HADOOP-15815. Upgrade Eclipse Jetty version to 9.3.24. Cotributed by Boris Vulikh.
add dce4ebe HDDS-694. Plugin new Pipeline management code in SCM. Contributed by Lokesh Jain.
add 1851d06 HDFS-13959. TestUpgradeDomainBlockPlacementPolicy is flaky. Contributed by Ayush Saxena.
add 4f10d7e HDFS-14028. HDFS OIV temporary dir deletes folder. Contributed by Adam Antal.
add d07e873 YARN-8569. Create an interface to provide cluster information to application. Contributed by Eric Yang
add 2fa01f8 HDDS-744. Fix ASF license warning in PipelineNotFoundException class. Contributed by Lokesh Jain.
add bfb9adc HADOOP-9567. Provide auto-renewal for keytab based logins. Contributed by Hrishikesh Gadre, Gary Helmling and Harsh J.
add 199703f HDDS-739. Support MultiDeleteRequest without XML namespace. Contributed by Elek Marton.
add 63e7134 Revert "HADOOP-15864. Job submitter / executor fail when SBN domain name can not resolved. Contributed by He Xiaoqiao."
add 1d90a0d YARN-8944. TestContainerAllocation.testUserLimitAllocationMultipleContainers failure after YARN-8896. Contributed by Wilfred Spiegelenburg.
add 78ea897 YARN-8915. Update the doc about the default value of maximum-container-assignments for capacity scheduler. Contributed by Zhankun Tang.
add bfb720e HDDS-573. Make VirtualHostStyleFilter port agnostic. Contributed by Danilo Perez.
add 902345d HDDS-728. Datanodes should use different ContainerStateMachine for each pipeline. Contributed by Mukul Kumar Singh.
add a58048e HDDS-727. ozone.log is not getting created in logs directory. Contributed by Nilotpal Nandi.
add 3655e57 HDDS-743. S3 multi delete request should return XML header in quiet mode. Contributed by Elek Marton.
add 496f0ff HDDS-620. ozone.scm.client.address should be an optional setting. Contributed by chencan and Arpit Agarwal.
add db7e636 HDFS-14027. DFSStripedOutputStream should implement both hsync methods.
add 4ec4ec6 YARN-8950. Fix compilation issue due to dependency convergence error for hbase.profile=2.0.
add a283da2 YARN-8871. Document ATSv2 integrated LogWebService. Contributed by Suma Shivaprasad.
add 486b9a4 YARN-7754. [Atsv2] Update document for running v1 and v2 TS. Contributed by Suma Shivaprasad.
add 7757331 HDDS-749. Restructure BlockId class in Ozone. Contributed by Shashikant Banerjee.
add 62d98ca HADOOP-15855. Review hadoop credential doc, including object store details. Contributed by Steve Loughran.
add d36012b YARN-8854. Upgrade jquery datatable version references to v1.10.19. Contributed by Akhil PB.
add 277a3d8 YARN-6729. Clarify documentation on how to enable cgroup support. Contributed by Zhankun Tang
add f747f5b HADOOP-15886. Fix findbugs warnings in RegistryDNS.java.
add e33b61f HDDS-762. Fix unit test failure for TestContainerSQLCli & TestSCMMetrics. Contributed by Mukul Kumar Singh.
add 773f0d1 HDDS-754. VolumeInfo#getScmUsed throws NPE. Contributed by Hanisha Koneru.
add e4f22b0 HDDS-755. ContainerInfo and ContainerReplica protobuf changes. Contributed by Nanda kumar.
add fac9f91 HDFS-13942. [JDK10] Fix javadoc errors in hadoop-hdfs module. Contributed by Dinesh Chitlangia.
add 9c438ab HDFS-14033. [libhdfs++] Disable libhdfs++ build on systems that do not support thread_local. Contributed by Anatoli Shein.
add ecac351 HDDS-712. Use x-amz-storage-class to specify replication type and replication factor. Contributed by Bharat Viswanadham.
add b519f3f HDDS-659. Implement pagination in GET bucket (object list) endpoint. Contributed by Bharat Viswanadham.
add 478b2cb HDDS-773. Loading ozone s3 bucket browser could be failed. Contributed by Elek Marton.
add 08bb036 HDDS-759. Create config settings for SCM and OM DB directories. Contributed by Arpit Agarwal.
add 6668c19 HDDS-677. Create documentation for s3 gateway to the docs. Contributed by Elek Marton.
add c5eb237 HDDS-777. Fix missing jenkins issue in s3gateway module. Contributed by Bharat Viswanadham.
add b13c567 HDDS-697. update and validate the BCSID for PutSmallFile/GetSmallFile command. Contributed by Shashikant Banerjee.
add 2e8ac14 HDDS-786. Fix the findbugs for SCMClientProtocolServer#getContainerWithPipeline.
add e0ac308 HDDS-771. ChunkGroupOutputStream stream entries need to be properly updated on closed container exception. Contributed by Lokesh Jain.
add d174b91 HADOOP-15895. [JDK9+] Add missing javax.annotation-api dependency to hadoop-yarn-csi.
add 2ab611d YARN-7225. Add queue and partition info to RM audit log. Contributed by Eric Payne
add d98b881 HDFS-14008. NN should log snapshotdiff report. Contributed by Pranay Singh.
add 8fe85af HDFS-13996. Make HttpFS' ACLs RegEx configurable. Contributed by Siyao Meng.
add d16d5f7 HDDS-751. Replace usage of Guava Optional with Java Optional.
add 6d9c18c HDDS-788. Change title page of bucket browser in S3gateway. Contributed by Bharat Viswanadham.
add 44e37b4 HADOOP-15885. Add base64 (urlString) support to DTUtil. Contributed by Inigo Goiri.
add babc946 YARN-8954. Reservations list field in ReservationListInfo is not accessible. Contributed by Oleksandr Shevchenko.
add aed836e YARN-8897. LoadBasedRouterPolicy throws NPE in case of sub cluster unavailability. Contributed by Bilwa S T.
add 989715e YARN-8893. [AMRMProxy] Fix thread leak in AMRMClientRelayer and UAM client. Contributed by Botong Huang.
add f84a278 YARN-8905. [Router] Add JvmMetricsInfo and pause monitor. Contributed by Bilwa S T.
add cb8d679 HADOOP-15687. Credentials class should allow access to aliases.
add 4e3df75 HDFS-14049. TestHttpFSServerWebServer fails on Windows because of missing winutils.exe. Contributed by Inigo Goiri.
add a5e21cd HADOOP-15899. Update AWS Java SDK versions in NOTICE.txt.
add d43cc5d HADOOP-15900. Update JSch versions in LICENSE.txt.
add 15df2e7 HDDS-796. Fix failed test TestStorageContainerManagerHttpServer#testHttpPolicy.
add c8ca174 HDDS-797. If DN is started before SCM, it does not register. Contributed by Hanisha Koneru.
add 942693b HDDS-799. Avoid ByteString to byte array conversion cost by using ByteBuffer in Datanode. Contributed by Mukul Kumar Singh.
add 5ddefdd HDDS-794. Add configs to set StateMachineData write timeout in ContainerStateMachine. Contributed by Shashikant Banerjee.
add 50f40e0 HDDS-794. addendum patch to fix compilation failure. Contributed by Shashikant Banerjee.
add f3f5e7a HDFS-14042. Fix NPE when PROVIDED storage is missing. Contributed by Virajith Jalaparti.
add f329650 HDFS-14043. Tolerate corrupted seen_txid file. Contributed by Lukas Majercak.
add c7fcca0 YARN-8969. AbstractYarnScheduler#getNodeTracker should return generic type to avoid type casting. Contributed by Wanqiang Ji.
add ffc9c50 HDFS-14053. Provide ability for NN to re-replicate based on topology changes. Contributed by Hrishikesh Gadre.
add 5d6554c YARN-8970. Improve the debug message in CS#allocateContainerOnSingleNode. Contributed by Zhankun Tang.
add 6430c98 HADOOP-15904. [JDK 11] Javadoc build failed due to bad use of '>' in hadoop-hdfs.
add 78e43bc HADOOP-15902. [JDK 11] Specify the HTML version of Javadoc to 4.01. Contributed by Takanobu Asanuma.
add 49412a1 HDDS-811. [JDK 11] Fix compilation failures with jdk 11. Contributed by Lokesh Jain.
add ab6aa4c YARN-8865. RMStateStore contains large number of expired RMDelegationToken. Contributed by Wilfred Spiegelenburg
add 349168c HDFS-14047. [libhdfs++] Fix hdfsGetLastExceptionRootCause bug in test_libhdfs_threaded.c (Contributed by Antoli Shein via Daniel Templeton)
add 3d5cc11 HDDS-798. Storage-class is showing incorrectly. Contributed by Bharat Viswanadham.
add f0ce072 HDFS-14051. Refactor NameNodeHttpServer#initWebHdfs to specify local keytab. Contributed by CR Hota.
add 00a67f7 HDDS-810. Move the "\r\n" at the chunk end in TestSignedChunksInputStream#singlechunkwithoutend. Contributed by Chencan.
add 887244d HDFS-14048. DFSOutputStream close() throws exception on subsequent call after DataNode restart. Contributed by Erik Krogen.
add 08d69d9 YARN-8957. Add Serializable interface to ComponentContainers. Contributed by Zhankun Tang
add ba1f9d6 MAPREDUCE-7156. NullPointerException when reaching max shuffle connections. Contributed by Peter Bacsko
add 951c98f YARN-8233. NPE in CapacityScheduler#tryCommit when handling allocate/reserve proposal whose allocatedOrReservedContainer is null. Contributed by Tao Yang.
add 482716e YARN-8976. Remove redundant modifiers in interface ApplicationConstants. Contributed by Zhankun Tang.
add addec29 HDDS-809. Refactor SCMChillModeManager.
add e6444f1 HADOOP-15907. Add missing maven modules in BUILDING.txt. Contributed Wanqiang Ji.
add 8dc1f6d YARN-8866. Fix a parsing error for crossdomain.xml.
add 0b6625a MAPREDUCE-7148. Fast fail jobs when exceeds dfs quota limitation. Contributed by Wang Yan
add c96cbe8 YARN-8977. Remove unnecessary type casting when calling AbstractYarnScheduler#getSchedulerNode. Contributed by Wanqiang Ji.
add f8c72d7 YARN-8880. Add configurations for pluggable plugin framework. Contributed by Zhankun Tang.
add e1bbf7d YARN-8988. Reduce the verbose log on RM heartbeat path when distributed node-attributes is enabled. Contributed by Tao Yang.
add c80f753 HDDS-737. Introduce Incremental Container Report. Contributed by Nanda kumar.
add 9317a61 HDDS-802. Container State Manager should get open pipelines for allocating container. Contributed by Lokesh Jain.
add 724c150 HADOOP-15846. ABFS: fix mask related bugs in setAcl, modifyAclEntries and removeAclEntries.
add 8d99648 HDFS-14039. ec -listPolicies doesn't show correct state for the default policy when the default is not RS(6,3). Contributed by Kitti Nanasi.
add 31614bc HDDS-806. Update Ratis to latest snapshot version in ozone. Contributed by Tsz Wo Nicholas Sze and Mukul Kumar Singh.
add 89b4916 HADOOP-15903. Allow HttpServer2 to discover resources in /static when symlinks are used. Contributed by Inigo Goiri.
add 524a752 YARN-8990. Fix fair scheduler race condition in app submit and queue cleanup. (Contributed by Wilfred Spiegelenburg)
add 66bf624 HDDS-812. TestEndPoint#testCheckVersionResponse is failing. Contributed by Hanisha Koneru.
add 47194fe HDDS-823. OzoneRestClient is failing with NPE on getKeyDetails call. Contributed by Bharat Viswanadham.
add 9dbb2b6 HADOOP-15812. ABFS: Improve AbfsRestOperationException format to ensure full msg can be displayed on console.
add a736b5d HADOOP-15916. Upgrade Maven Surefire plugin to 3.0.0-M1.
add 9fe50b4 HDDS-733. Create container if not exist, as part of chunk write. Contributed by Lokesh Jain.
add 298d250 HDDS-826. Update Ratis to 0.3.0-6f3419a-SNAPSHOT.
add 2664248 YARN-9002. Improve keytab loading for YARN Service. Contributed by Gour Saha
add b5ec85d YARN-8933. [AMRMProxy] Fix potential empty fields in allocation response, move SubClusterTimeout to FederationInterceptor. Contributed by Botong Huang.
add 4e72844 YARN-8902. [CSI] Add volume manager that manages CSI volume lifecycle. Contributed by Weiwei Yang.
add 9c32b50 HDDS-767. OM should not search for STDOUT root logger for audit logging. Contributed by Dinesh Chitlangia.
add c741109 YARN-8987. Usability improvements node-attributes CLI. Contributed by Bibin A Chundatt.
add 3c9d97b HADOOP-15110. Gauges are getting logged in exceptions from AutoRenewalThreadForUserCreds.
add 42f3a70 YARN-8877. [CSI] Extend service spec to allow setting resource attributes. Contributed by Weiwei Yang.
add 18fe65d HDDS-576. Move ContainerWithPipeline creation to RPC endpoint. Contributed by Nanda kumar.
add 1f9c4f3 YARN-8776. Implement Container Exec feature in LinuxContainerExecutor. Contributed by Eric Yang
add f944f33 HDDS-709. Modify Close Container handling sequence on datanodes. Contributed by Shashikant Banerjee.
add e269c3f YARN-8997. [Submarine] Small refactors of modifier, condition check and redundant local variables. Contributed by Zhankun Tang.
add b6d4e19 HDFS-14065. Failed Storage Locations shows nothing in the Datanode Volume Failures. Contributed by Ayush Saxena.
add 4c465f5 YARN-8761. Service AM support for decommissioning component instances. Contributed by Billie Rinaldi
add f8713f8 HDDS-831. TestOzoneShell in integration-test is flaky. Contributed by Nanda kumar.
add 703b286 HADOOP-15923. create-release script should set max-cache-ttl as well as default-cache-ttl for gpg-agent.
add a67642c HADOOP-15912. start-build-env.sh still creates an invalid /etc/sudoers.d/hadoop-build-${USER_ID} file entry after HADOOP-15802. Contributed by Akira Ajisaka.
add e7b63ba HDFS-14070. Refactor NameNodeWebHdfsMethods to allow better extensibility. Contributed by CR Hota
add 75291e6 HADOOP-15869. BlockDecompressorStream#decompress should not return -1 in case of IOException. Contributed by Surendra Singh Lilhore
add 671fd65 HDDS-675. Add blocking buffer and use watchApi for flush/close in OzoneClient. Contributed by Shashikant Banerjee.
add 076b795 YARN-8918. [Submarine] Correct method usage of str.subString in CliUtils. (Zhankun Tang via wangda)
add 762a56c MAPREDUCE-7158. Inefficient Flush Logic in JobHistory EventWriter. (Zichen Sun via wangda)
add 9da6054 Revert "HDFS-13732. ECAdmin should print the policy name when an EC policy is set. Contributed by Zsolt Venczel."
add fcbd205 YARN-9001. [Submarine] Use AppAdminClient instead of ServiceClient to sumbit jobs. (Zac Zhou via wangda)
add a13be20 HADOOP-15876. Use keySet().removeAll() to remove multiple keys from Map in AzureBlobFileSystemStore
add 3fade86 HADOOP-15917. AliyunOSS: fix incorrect ReadOps and WriteOps in statistics. Contributed by Jinhu Wu.
add a948281 HDDS-834. Datanode goes OOM based because of segment size. Contributed by Mukul Kumar Singh.
add b57cc73 HDDS-774. Remove OpenContainerBlockMap from datanode. Contributed by Shashikant Banerjee.
add 21ec4bd YARN-8672. Improve token filename management for localization. Contributed by Chandni Singh
add bac8807 HDDS-819. Match OzoneFileSystem behavior with S3AFileSystem. Contributed by Hanisha Koneru.
add df5e863 HADOOP-15930. Exclude MD5 checksum files from release artifact.
add 66b1335 HADOOP-15926. Document upgrading the section in NOTICE.txt when upgrading the version of AWS SDK. Contributed by Dinesh Chitlangia.
add 9b01f03 HDDS-832. Docs folder is missing from the Ozone distribution package. Contributed by Elek, Marton.
add b12e694 HDDS-658. Implement s3 bucket list backend call and use it from rest endpoint. Contributed by Bharat Viswanadham.
add fb1e8bb HDDS-528. add cli command to checkChill mode status and exit chill mode. Contributed by chencan.
add 2718f75 HDDS-828. Fix deprecation log generated by getting value of the setting ozone.scm.heartbeat.rpc-timeout. Contributed by Yiqun Lin.
add 0a8b1ef HDDS-827. TestStorageContainerManagerHttpServer should use dynamic port. Contributed by Sandeep Nemuri.
add d40912b HDDS-223. Create acceptance test for using datanode plugin. Contributed by Sandeep Nemuri.
add eeaf8ed YARN-8856. TestTimelineReaderWebServicesHBaseStorage tests failing with NoClassDefFoundError. Contributed by Sushil Ks.
add 993c214 HDFS-14054. TestLeaseRecovery2: testHardLeaseRecoveryAfterNameNodeRestart2 and testHardLeaseRecoveryWithRenameAfterNameNodeRestart are flaky. Contributed by Zsolt Venczel.
add 460a94a HDFS-14045. Use different metrics in DataNode to better measure latency of heartbeat/blockReports/incrementalBlockReports of Active/Standby NN. Contributed by Jiandan Yang.
add 1e15c7e HADOOP-15936. [JDK 11] MiniDFSClusterManager & MiniHadoopClusterManager compilation fails due to the usage of '_' as identifier. Contributed by Zsolt Venczel.
add 914b0cf HADOOP-12558. distcp documentation is woefully out of date. Contributed by Dinesh Chitlangia.
add 57866b3 HDDS-821. Handle empty x-amz-storage-class header in Ozone S3 gateway. Contributed by Elek Marton.
add d8ec017 HDDS-843. [JDK11] Fix Javadoc errors in hadoop-hdds-server-scm module. Contributed by Dinesh Chitlangia.
add fcd94ee HDDS-842. [JDK11] Fix Javadoc errors in hadoop-hdds-common module. Contributed by Dinesh Chitlangia.
add a16aa2f HDDS-825. Code cleanup based on messages from ErrorProne. Contributed by Anu Engineer.
add af5958f HADOOP-15925. The config and log of gpg-agent are removed in create-release script. Contributed by Dinesh Chitlangia.
add 8571507 HDFS-13963. NN UI is broken with IE11. Contributed by Ayush Saxena.
add ee3355b YARN-8303. YarnClient should contact TimelineReader for application/attempt/container report.
add 68d5dfd HDFS-14056. Fix error messages in HDFS-12716. Contributed by Ayush Saxena.
add ad5256e HDFS-14063. Support noredirect param for CREATE/APPEND/OPEN/GETFILECHECKSUM in HttpFS. Contributed by Íñigo Goiri.
add c4d9640 HDDS-801. Quasi close the container when close is not executed via Ratis. Contributed by Nanda kumar.
add 3420ddd HDDS-840. [JDK11] Fix Javadoc errors in hadoop-hdds-client module. Contributed by Dinesh Chitlangia.
add abd6d48 YARN-8860. Federation client intercepter class contains unwanted character. Contributed by Abhishek Modi.
add 2937499 HDDS-120. Adding HDDS datanode Audit Log. Contributed by Dinesh Chitlangia.
add e56d9f2 HDFS-14015. Improve error handling in hdfsThreadDestructor in native thread local storage
add d027a24 YARN-8833. Avoid potential integer overflow when computing fair shares. Contributed by liyakun.
add be0708c HADOOP-15919. AliyunOSS: Enable Yarn to use OSS. Contributed by wujinhu.
add cfb915f HDDS-844. Add logic for pipeline teardown after timeout. Contributed by Lokesh Jain.
add 10cf577 HDDS-845. Create a new raftClient instance for every watch request for Ratis. Contributed by Shashikant Banerjee.
add 5a7ca6a HDDS-837. Persist originNodeId as part of .container file in datanode. Contributed by Nanda kumar.
add 9366608 HDDS-847. TestBlockDeletion is failing. Contributed by Lokesh Jain.
add 6357803 YARN-8881. [YARN-8851] Add basic pluggable device plugin framework. (Zhankun Tang via wangda)
add 8b23814 YARN-8960. [Submarine] Can't get submarine service status using the command of "yarn app -status" under security environment. (Zac Zhou via wangda)
add e7438a1 HDDS-849. Fix NPE in TestKeyValueHandler because of audit log write. Contributed by Dinesh Chitlangia.
add b5d7b29 HDDS-718. Introduce new SCM Commands to list and close Pipelines. Contributed by Lokesh Jain.
add 397f523 HADOOP-15939. Filter overlapping objenesis class in hadoop-client-minicluster. Contributed by Xiaoyu Yao.
add 27ffec7 YARN-8778. Add command line interface to invoke interactive docker shell. Contributed by Eric Yang
add 5fb14e0 YARN-8953. [CSI] CSI driver adaptor module support in NodeManager. Contributed by Weiwei Yang.
add 1a00b4e Add 2.9.2 release notes and changes documents.
add d0cc679 HDDS-817. Create SCM metrics for disk from node report. Contributed by Bharat Viswanadham.
add 2a7f485 HDDS-692. Use the ProgressBar class in the RandomKeyGenerator freon test. Contributed by Zsolt Horvath.
add c946f1b Revert "HDDS-692. Use the ProgressBar class in the RandomKeyGenerator freon test. Contributed by Zsolt Horvath."
add 4ecdcc9 HDDS-692. Use the ProgressBar class in the RandomKeyGenerator freon test. Contributed by Zsolt Horvath.
add 0123e51 HDDS-164. Add unit test for HddsDatanodeService. Contributed by Yiqun Lin.
add 10b5da8 HADOOP-14739. Update start-build-env.sh and build instruction for docker for Mac instead of docker toolbox. Contributed by Dinesh Chitlangia.
add c747830 HDDS-835. Use storageSize instead of Long for buffer size configs in Ozone Client. Contributed by Shashikant Banerjee.
add 49824ed YARN-8838. Check that container user is same as websocket user for interactive shell. Contributed by Eric Yang
add 1734ace Revert "HDDS-835. Use storageSize instead of Long for buffer size configs in Ozone Client. Contributed by Shashikant Banerjee."
add a41b648 YARN-8992. Fair scheduler can delete a dynamic queue while an application attempt is being added to the queue. (Contributed by Wilfred Spiegelenburg)
add f994b52 HDDS-855. Move OMMetadataManager from hadoop-ozone/ozone-manager to hadoop-ozone/common. Contributed by Ajay Kumar.
add 14e1a0a HDDS-835. Use storageSize instead of Long for buffer size configs in Ozone Client. Contributed by Shashikant Banerjee.
add c8b3dfa HDDS-860. Fix TestDataValidate unit tests. Contributed by Shashikant Banerjee.
add f63e4e4 YARN-8936. Bump up Atsv2 hbase versions. Contributed by Vrushali C.
add 22867de HDDS-791. Support Range header for ozone s3 object download. Contributed by Bharat Viswanadham.
add 34b6fa7 HDDS-732. Add read method which takes offset and length in SignedChunkInputStream. Contributed by chencan.
add f42b2df HDDS-865. GrpcXceiverService is added twice to GRPC netty server. Contributed by Xiaoyu Yao.
add c4d0ef6 HDDS-816. Create OM metrics for bucket, volume, keys. Contributed by Bharat Viswanadham.
add 892b33e HDFS-14064. WEBHDFS: Support Enable/Disable EC Policy. Contributed by Ayush Saxena.
add ebb9245 HDDS-853. Option to force close a container in Datanode. Contributed by Nanda kumar.
add 919a6e4 HDDS-861. SCMNodeManager unit tests are broken. Contributed by Xiaoyu Yao.
add 8d2789c HDDS-795. RocksDb specific classes leak from DBStore/Table interfaces. Contributed by Márton Elek.
add 5fe1dbf YARN-8964. [UI2] YARN ui2 should use clusters/{cluster name} for all ATSv2 REST APIs. Contributed by Akhil PB.
add f207e30 HDDS-814. dfs.ratis.leader.election.minimum.timeout.duration should not be read by client. Contributed by chencan.
add 176bb3f YARN-8984. AMRMClient#OutstandingSchedRequests leaks when AllocationTags is null or empty. Contributed by Yang Wang.
add ee44b06 HDDS-866. Handle RaftRetryFailureException in OzoneClient. Contributed by Shashikant Banerjee.
add ff24849 HDDS-841. [JDK11] Fix Javadoc errors in hadoop-hdds-container-service module. Contributed by Dinesh Chitlangia.
add 95d526a HDDS-869. Fix log message in XceiverClientRatis#sendCommandAsync. Contributed by Lokesh Jain.
add e223a79 YARN-9042. Fix javadoc error in deviceplugin package. Contributed by Zhankun Tang.
add 5ff0cf8 HADOOP-15943. AliyunOSS: add missing owner & group attributes for oss FileStatus. Contributed by wujinhu.
add 6934a65 HADOOP-15358. SFTPConnectionPool connections leakage. Contributed by Mikhail Pryakhin.
add 753f421 HDDS-820. Use more strict data format for the Last-Modified headers of s3 gateway. Contributed by Elek, Marton.
add a8302e3 HADOOP-15872. ABFS: Update to target 2018-11-09 REST version for ADLS Gen 2.
add 23b441c HADOOP-15948. Inconsistency in get and put syntax if filename/dirname contains space. Contributed by Ayush Saxena.
add b098281 HDFS-13816. dfs.getQuotaUsage() throws NPE on non-existent dir instead of FileNotFoundException. Contributed by Vinayakumar B.
add a9d9694 HADOOP-15922. Fixed doAsUser decoding for DelegationTokenAuthenticationFilter. Contributed by He Xiaoqiao
add 0d84061 YARN-9054. Fix FederationStateStoreFacade#buildGetSubClustersCacheRequest. Contributed by Bibin A Chundatt.
add 33e0df4 YARN-8986. Added port publish for Docker container running with bridge network. Contributed by Charo Zhang
add 099e723 HDDS-868. Handle quasi closed container replicas in SCM. Contributed by Nanda kumar.
add 2730ead YARN-9058. [CSI] YARN service fail to launch due to CSI changes. Contributed by Weiwei Yang.
add 1aad99a MAPREDUCE-7162. TestEvents#testEvents fails. Contributed by Zhaohui Xin.
add 96c104d HDDS-766. Ozone shell create volume fails if volume name does not have a leading slash. Contributed by chencan.
add 300f772 HADOOP-15940. ABFS: For HNS account, avoid unnecessary get call when doing Rename.
add 4c106fc HADOOP-15932. Oozie unable to create sharelib in s3a filesystem.
add f657a2a YARN-9056. Improved YARN service upgrade state logic for readiness check. Contributed by Chandni Singh
add 34a914b HDDS-846. Exports ozone metrics to prometheus. Contributed by Elek, Marton.
add 7dc2721 YARN-9034. ApplicationCLI should have option to take clusterId. Contributed by Rohith Sharma K S.
add b3a052d YARN-9044. LogsCLI should contact ATSv2 for -am option. Contributed by Rohith Sharma K S
add 13a21f6 MAPREDUCE-6190. If a task stucks before its first heartbeat, it never timeouts and the MR job becomes stuck. Contributed by Zhaohui Xin.
add 3ce99e3 YARN-8812. Containers fail during creating a symlink which started with hyphen for a resource file. Contributed by Oleksandr Shevchenko
add 5d96b74 HADOOP-15370. S3A log message on rm s3a://bucket/ not intuitive. Contributed by Gabor Bota.
add e148c3f HADOOP-15798. LocalMetadataStore put() does not retain isDeleted in parent listing. Contributed by Gabor Bota.
add 085f10e HADOOP-15947. Fix ITestDynamoDBMetadataStore test error issues. Contributed by Gabor Bota.
add 64a4b6b HDDS-284. Checksum for ChunksData.
add 4ca3a6b HDFS-14108. Performance improvement in BlockManager Data Structures. Contributed by Beluga Behr.
add 300f560 HDFS-14102. Performance improvement in BlockPlacementPolicyDefault. Contributed by Beluga Behr.
add 4d8de7a MAPREDUCE-7164. FileOutputCommitter does not report progress while merging paths. Contributed by Kuhu Shukla
add 9de8e8d YARN-9030. Log aggregation changes to handle filesystems which do not support setting permissions. (Suma Shivaprasad via wangda)
add 579ef4b YARN-8882. [YARN-8851] Add a shared device mapping manager (scheduler) for device plugins. (Zhankun Tang via wangda)
add 9ed8756 YARN-9061. Improve the GPU/FPGA module log message of container-executor. (Zhankun Tang via wangda)
add 8ebeda9 YARN-8974. Improve the assertion message in TestGPUResourceHandler. (Zhankun Tang via wangda)
add 8976439 YARN-8975. [Submarine] Use predefined Charset object StandardCharsets.UTF_8 instead of String UTF-8. (Zhankun Tang via wangda)
add fe7dab8 YARN-8989. [YARN-8851] Move DockerCommandPlugin volume related APIs' invocation from DockerLinuxContainerRuntime#prepareContainer to #launchContainer. (Zhankun Tang via wangda)
add efc4d91 YARN-9067. YARN Resource Manager is running OOM because of leak of Configuration Object. Contributed by Eric Yang.
add b71cc7f HDDS-642. Add chill mode exit condition for pipeline availability. Contributed by Yiqun Lin.
add c1d24f8 HDFS-13713. Add specification of Multipart Upload API to FS specification, with contract tests.
add a68d766 YARN-8948. PlacementRule interface should be for all YarnSchedulers. Contributed by Bibin A Chundatt.
add 07142f5 YARN-9069. Fix SchedulerInfo#getSchedulerType for custom schedulers. Contributed by Bilwa S T.
add 184cced HDDS-808. Simplify OMAction and DNAction classes used for AuditLogging. Contributed by Dinesh Chitlangia.
add 7eb0d3a HADOOP-14927. ITestS3GuardTool failures in testDestroyNoBucket(). Contributed by Gabor Bota.
add 5e102f9 HDDS-850. ReadStateMachineData hits OverlappingFileLockException in ContainerStateMachine. Contributed by Shashikant Banerjee.
add d0edd37 HADOOP-15959. Revert "HADOOP-12751. While using kerberos Hadoop incorrectly assumes names with '@' to be non-simple"
add f534736 HDFS-14095. EC: Track Erasure Coding commands in DFS statistics. Contributed by Ayush Saxena.
add ae5fbdd HDDS-877. Ensure correct surefire version for Ozone test. Contributed by Xiaoyu Yao.
add 0081b02 HDFS-14112. Avoid recursive call to external authorizer for getContentSummary.
add bad1203 YARN-9010. Fix the incorrect trailing slash deletion in constructor method of CGroupsHandlerImpl. (Zhankun Tang via wangda)
add 0e36e93 HDFS-13870. WebHDFS: Document ALLOWSNAPSHOT and DISALLOWSNAPSHOT API doc. Contributed by Siyao Meng.
add c9bfca2 MAPREDUCE-7165. mapred-site.xml is misformatted in single node setup document. Contributed by Zhaohui Xin.
add 7ccb640 HADOOP-15957. WASB: Add asterisk wildcard support for PageBlobDirSet.
add 62f8211 HDDS-886. Unnecessary buffer copy in HddsDispatcher#dispatch. Contributed by Lokesh Jain.
add 38ea381 HADOOP-15922. Revert patch 004.
add 6d7b44c YARN-9036. Escape newlines in health report in YARN UI. Contributed by Keqiu Hu
add b09cfad HDFS-14106. Refactor NamenodeFsck#copyBlock. Contributed by Beluga Behr.
add 99e201d HDFS-14105. Replace TreeSet in NamenodeFsck with HashSet. Contributed by Beluga Behr.
add d15dc43 HDDS-748. Use strongly typed metadata Table implementation. Contributed by Elek Marton.
add 5a3c771 HDDS-887. Add DispatcherContext info to Dispatcher from containerStateMachine. Contributed by Shashikant Banerjee.
add 8f3e12f HDDS-882. Provide a config to optionally turn on/off the sync flag during chunk writes. Contributed by Shashikant Banerjee.
add 042c8ef HDFS-14075. Terminate the namenode when failed to start log segment. Contributed by Ayush Saxena.
add 3044b78 HDDS-848. Create SCM metrics related to container state. Contributed by Bharat Viswanadham.
add dd5e7c6 HDFS-12946. Add a tool to check rack configuration against EC policies. Contributed by Kitti Nanasi.
add fb10803 HDFS-13818. Extend OIV to detect FSImage corruption. Contributed by Adam Antal.
add ef3b03b HDDS-885. Fix test failures due to ChecksumData. Contributed by Hanisha Koneru.
add c9a3aa6 HADOOP-15950. Failover for LdapGroupsMapping. Contributed by Lukas Majercak.
add 69489ff HDFS-14119. Improve GreedyPlanner Parameter Logging. Contributed by Beluga Behr.
add fb9deed HADOOP-15852. Refactor QuotaUsage. Contributed by Beluga Behr.
add de42555 HDDS-894. Content-length should be set for ozone s3 ranged download. Contributed by Elek Marton.
add a8bbd81 HADOOP-15968. ABFS: add try catch for UGI failure when initializing ABFS.
add ff31313 HADOOP-15970. Upgrade plexus-utils from 2.0.5 to 3.1.0.
add 7274115 HDDS-890. Handle OverlappingFileLockException during writeStateMachineData in ContainerStateMachine. Contributed by Shashikant Banerjee.
add db2d8b0 HADOOP-15966. Hadoop Kerberos broken on macos as java.security.krb5.realm is reset.
add e89941f YARN-9041. Performance Optimization of method FSPreemptionThread#identifyContainersToPreempt. Contributed by Wanqiang Ji.
add aa89492 Revert "HADOOP-15852. Refactor QuotaUsage. Contributed by Beluga Behr."
add 9287ab3 HADOOP-15974. Upgrade Curator version to 2.13.0 to fix ZK tests. Contributed by Akira Ajisaka
add 228156c Revert "YARN-8870. [Submarine] Add submarine installation scripts. (Xun Liu via wangda)"
add 1dabb31 YARN-9057. Removed third party class bundle from CSI jar file. Contributed by Weiwei Yang
add 1b790f4 YARN-9071. Improved status update for reinitialized containers. Contributed by Chandni Singh
add 2bba329 YARN-8994. Fix race condition between move app and queue cleanup in Fair Scheduler. (Contributed by Wilfred Spiegelenburg)
add b3c75c1 YARN-8985. Improve debug log in FSParentQueue when assigning container. (Contributed by Wilfred Spiegelenburg)
add 912b1f9 YARN-9019. Ratio calculation of ResourceCalculator implementations could return NaN. (Contributed by Szilard Nemeth)
add 5d4a432 YARN-9025. TestFairScheduler#testChildMaxResources is flaky. (Contributed by Szilard Nemeth)
add c03024a HDFS-14113. EC : Add Configuration to restrict UserDefined Policies. Contributed by Ayush Saxena.
add 343aaea HDDS-864. Use strongly typed codec implementations for the tables of the OmMetadataManager. Contributed by Elek Marton.
add 9886c27 MAPREDUCE-7159. FrameworkUploader: ensure proper permissions of generated framework tar.gz if restrictive umask is used. Contributed by Peter Bacsko
add 8d882c3 HDDS-880. Create api for ACL handling in Ozone. (Contributed by Ajay Kumar)
add 019836b HDDS-858. Start a Standalone Ratis Server on OM
add 6c852f2 HDDS-892. Parse aws v2 headers without spaces in Ozone s3 gateway. Contributed by Elek Marton.
add cc51607 YARN-9085. Add Guaranteed and MaxCapacity to CSQueueMetrics
add 154449f YARN-8914. Add xtermjs to YARN UI2. Contributed by Eric Yang and Akhil PB
add 8fc0d04 HDFS-14001. [PROVIDED Storage] bootstrapStandby should manage the InMemoryAliasMap. Contributed by Virajith Jalaparti.
add 5e773ef HDDS-909 : Default implementation for Ozone acls. Contributed by Ajay Kumar.
add 1afba83 HDDS-870. Avoid creating block sized buffer in ChunkGroupOutputStream. Contributed by Shashikant Banerjee.
add 1c09a10 YARN-9009. Fix flaky test TestEntityGroupFSTimelineStore.testCleanLogs. Contributed by OrDTesters.
add 17a8708 HDDS-879. MultipartUpload: Add InitiateMultipartUpload in ozone. Contributed by Bharat Viswanadham.
add 64411a6 YARN-8738. FairScheduler should not parse negative maxResources or minResources values as positive. (Contributed by Szilard Nemeth)
add ac578c0 YARN-9087. Improve logging for initialization of Resource plugins. (Contributed by Szilard Nemeth)
add 80e59e7 HDFS-14101. Fixing underflow error in test. Contributed by Zsolt Venczel.
add 3ff8580 HADOOP-15428. s3guard bucket-info will create s3guard table if FS is set to do this automatically. (Contributed by Gabor Bota)
add 39dc734 HDFS-14124. EC : Support EC Commands (set/get/unset EcPolicy) via WebHdfs. Contributed by Ayush Saxena.
add c35de95 HADOOP-15987. ITestDynamoDBMetadataStore should check if table configured properly. Contributed by Gabor Bota.
add 21c9080 HDDS-872. Add Dockerfile and skaffold config to deploy ozone dev build to k8s. Contributed by Elek, Marton.
add 1a25bbe HADOOP-15845. Require explicit URI on CLI for s3guard init and destroy. Contributed by Gabor Bota.
add 202926a HADOOP-15808. Harden Token service loader use.
add 60af851 HDFS-14137. TestMaintenanceState fails with ArrayIndexOutOfBound Exception. Contributed by Ayush Saxena.
add 881230d YARN-9051. Integrate multiple CustomResourceTypesConfigurationProvider implementations into one. (Contributed by Szilard Nemeth)
add fb55e52 YARN-9008. Extend YARN distributed shell with file localization feature. (Contributed by Peter Bacsko)
add 5a03c8b YARN-9114. [UI2] log service redirect url need to support user name. Contributed by Akhil PB
add d44b37d HDDS-889. MultipartUpload: Support uploading a part file in ozone. Contributed by Bharat Viswanadham.
add 82b7985 HADOOP-15988. DynamoDBMetadataStore#innerGet should support empty directory flag when using authoritative listings. Contributed by Gabor Bota.
add 76efeac HADOOP-15995. Add ldap.bind.password.alias in LdapGroupsMapping to distinguish aliases when using multiple providers through CompositeGroupsMapping. Contributed by Lukas Majercak.
add 561b61c HDFS-14144. TestPred fails in Trunk. Contributed by Ayush Saxena.
add 8a53611 YARN-9078. [Submarine] Clean up the code of CliUtils#parseResourcesString. (Zhankun Tang via wangda)
add c771fe6 YARN-8714. [Submarine] Support files/tarballs to be localized for a training job. (Zhankun Tang via wangda)
add 61bdcb7 YARN-9015. [DevicePlugin] Add an interface for device plugin to provide customized scheduler. (Zhankun Tang via wangda)
add 9fba6cc YARN-9112. [Submarine] Support polling applicationId when it's not ready in cluster. (Zhankun Tang via wangda)
add 37eb919 YARN-8885. [DevicePlugin] Support NM APIs to query device resource allocation. (Zhankun Tang via wangda)
add 72e7c6a YARN-8962. Add ability to use interactive shell with normal yarn container. Contributed by Eric Yang
add e0d7508 MAPREDUCE-7170. Doc typo in PluggableShuffleAndPluggableSort.md. Contributed by Zhaohui Xin.
add 92d44b2 HDDS-876. Add blockade tests for flaky network. Contributed by Nilotpal Nandi.
add 4aa0609 HDFS-13985. Clearer error message for ReplicaNotFoundException (Contributed by Adam Antal via Daniel Templeton)
add ca379e1 HDFS-13970. Use MultiMap for CacheManager Directives to simplify the code. Contributed by BELUGA BEHR.
add aa12859 HDFS-14121. Log message about the old hosts file format is misleading (Contributed by Zsolt Venczel via Daniel Templeton)
add cb3382f HDFS-14145. TestBlockStorageMovementAttemptedItems#testNoBlockMovementAttemptFinishedReportAdded fails sporadically in Trunk. Contributed by Ayush Saxena.
add f858f18 HDFS-14088. RequestHedgingProxyProvider can throw NullPointerException when failover due to no lock on currentUsedProxy. Contributed by Yuxuan Wang.
add 00d5e63 HDFS-14006. Refactor name node to allow different token verification implementations. Contributed by CR Hota.
add ca90b24 YARN-9032. Support sh shell for interactive container shell at command line. Contributed by Eric Yang
add b2d7204 YARN-9125. Fixed Carriage Return detection in Docker container launch command. Contributed by Billie Rinaldi
add 26e55d4 YARN-9122. Add table of contents to YARN Service API document. Contributed by Zhankun Tang.
add 1ea29b7 HADOOP-16000. Remove TLSv1 and SSLv2Hello from the default value of hadoop.ssl.enabled.protocols
add aab3109 YARN-9089. Add Terminal Link to Service component instance page for UI2. Contributed by Eric Yang
add 42297f7 YARN-8963. Add flag to disable interactive shell. Contributed by Eric Yang
add 4f62233 YARN-9091. Improve terminal message when connection is refused. Contributed by Eric Yang
add 04c0347 YARN-9095. Removed Unused field from Resource: NUM_MANDATORY_RESOURCES
add 788e747 HDFS-14096. [SPS] : Add Support for Storage Policy Satisfier in ViewFs. Contributed by Ayush Saxena.
add b2523d8 HADOOP-15969. ABFS: getNamespaceEnabled can fail blocking user access thru ACLs.
add 62df607 HADOOP-15972 ABFS: reduce list page size to to 500.
add 346c0c8 HADOOP-16004. ABFS: Convert 404 error response in AbfsInputStream and AbfsOutPutStream to FileNotFoundException.
add 71e0b0d YARN-9040. Fixed memory leak in LevelDBCacheTimelineStore and DBIterator. Contributed by Tarun Parimi
add 5426653 HDDS-908: NPE in TestOzoneRpcClient. Contributed by Ajay Kumar.
add 94b368f HDDS-99. Adding SCM Audit log. Contributed by Dinesh Chitlangia.
add ee10ba2 HDDS-539. Ozone datanode command ignores the invalid options. Contributed by Vinicius Higa Murakami.
add 4ff1c46 HDDS-925. Rename ChunkGroupOutputStream to KeyOutputStream and ChunkOutputStream to BlockOutputStream. Contributed by Shashikant Banerjee.
add b51e9e4 HDDS-912. Update ozone to latest ratis snapshot build (0.4.0-3b0be02-SNAPSHOT). Contributed by Lokesh Jain.
add 99b5061 HDDS-914. Add Grafana support to ozoneperf docker container. Contributed by Dinesh Chitlangia.
add c5c7318 YARN-9072. Send exit command to terminate docker exec on connection close. Contributed by Eric Yang
add 2f6bc25 HDDS-911. Make TestOzoneManager unit tests independent. Contributed by Elek Marton.
add 12c139d HDDS-933. Add documentation for genconf tool under Tools section. Contributed by Dinesh Chitlangia.
add a6e9d27 YARN-9117. Add a check for insecure setup for container terminal. Contributed by Eric Yang
add c7a5a44 YARN-9075. Add ability to register/remove auxiliary service without restart node manager. Contributed by Billie Rinaldi
add ccdd982 YARN-9084. Reset container state and defer readiness check for upgrade. Contributed by Chandni Singh
add b1ce9aa HDDS-940. Remove dead store to local variable in OmMetadataManagerImpl. Contributed by Dinesh Chitlangia.
add cf57113 HDDS-893. pipeline status is ALLOCATED in scmcli listPipelines command. Contributed by Lokesh Jain.
add e815fd9 YARN-9126. Fix container clean up for reinitialization. Contributed by Chandni Singh
add 2e544dc YARN-9129. Ensure flush after printing to log plus additional cleanup. Contributed by Eric Yang
add d963575 HDFS-13661. Ls command with e option fails when the filesystem is not HDFS.
add 499c70e MAPREDUCE-7166. map-only job should ignore node lost event when task is already succeeded. Contributed by Lei Li.
add 5df9fb1 YARN-9130. Add Bind_HOST configuration for Yarn Web Proxy. Contributed by Rong Tang.
add 2af90cc HADOOP-15935. [JDK 11] Update maven.plugin-tools.version to 3.6.0. Contributed by Dinesh Chitlangia.
add ea621fa HDDS-934. Freon run hung on non-functional pipeline. Contributed by Nanda kumar.
add 3961690 HADOOP-15973. Configuration: Included properties are not cached if resource is a stream. Contributed by Eric Payne
add 13d3f99 HDDS-881.009. Encapsulate all client to OM requests into one request message. Contributed by Hanisha Koneru.
add 784ea3a HADOOP-15965. Upgrade to ADLS SDK which has major performance improvement for ingress/egress.
add a668f8e HADOOP-16015. Add bouncycastle jars to hadoop-aws as test dependencies.
add 7affa30 YARN-9131. Updated document usage for dynamic auxiliary service. Contributed by Billie Rinaldi
add a80d321 YARN-9152. Improved AuxServices REST API output. Contributed by Billie Rinaldi
add f659485 YARN-8925. Updating distributed node attributes only when necessary. Contributed by Tao Yang.
add ea72418 YARN-9132. Added file permission check for auxiliary services manifest file. Contributed by Billie Rinaldi
add 2499435 HADOOP-16014. Fix test, checkstyle and javadoc issues in TestKerberosAuthenticationHandler. Contributed by Dinesh Chitlangia.
add f82922d YARN-5168. Added exposed port information for Docker container. Contributed by Xun Liu
add e98e963 YARN-9154. Fix itemization in YARN service quickstart document. Contributed by Ayush Saxena.
add 763e96e HDFS-14166. Ls with -e option not giving the result in proper format. Contributed by Shubham Dewan.
add feb2664 HDFS-13946. Log longest FSN write/read lock held stack trace.
add 657aa43 YARN 9108. Fix FederationIntercepter merge home and secondary allocate response typo. Contributed by Abhishek Modi.
add d944d5e HDFS-14165. In NameNode UI under DataNode tab ,the Capacity column is Non-Aligned. Contributed by Shubham Dewan.
add 26e4be7 HDDS-921. Add JVM pause monitor to Ozone Daemons (OM, SCM and Datanodes). Contributed by Bharat Viswanadham.
add 686fcd4 HDFS-14046. In-Maintenance ICON is missing in datanode info page. Contributed by Ranith Sardar.
add 3a78e5f HDFS-12975. [SBN read] Changes to the NameNode to support reads from standby. Contributed by Chao Sun.
add c9d7343 HDFS-12977. [SBN read] Add stateId to RPC headers. Contributed by Plamen Jeliazkov.
add 9bf0696 HDFS-13331. [SBN read] Add lastSeenStateId to RpcRequestHeader. Contributed by Plamen Jeliazkov.
add 091ad97 HDFS-13286. [SBN read] Add haadmin commands to transition between standby and observer. Contributed by Chao Sun.
add a109f2b HDFS-13578. [SBN read] Add ReadOnly annotation to methods in ClientProtocol. Contributed by Chao Sun.
add e880660 HDFS-13399. [SBN read] Make Client field AlignmentContext non-static. Contributed by Plamen Jeliazkov.
add c81ac2f HDFS-13607. [SBN read] Edit Tail Fast Path Part 1: Enhance JournalNode with an in-memory cache of recent edit transactions. Contributed by Erik Krogen.
add 151c8dd HDFS-13608. [SBN read] Edit Tail Fast Path Part 2: Add ability for JournalNode to serve edits via RPC. Contributed by Erik Krogen.
add 00e99c6 HDFS-13609. [SBN read] Edit Tail Fast Path Part 3: NameNode-side changes to support tailing edits via RPC. Contributed by Erik Krogen.
add 0105d7f HDFS-13706. [SBN read] Rename client context to ClientGSIContext. Contributed by Konstantin Shvachko.
add 64b7cf5 HDFS-12976. [SBN read] Introduce ObserverReadProxyProvider. Contributed by Chao Sun.
add e27708c HDFS-13665. [SBN read] Move RPC response serialization into Server.doResponse(). Contributed by Plamen Jeliazkov.
add 1e22f2b HDFS-13610. [SBN read] Edit Tail Fast Path Part 4: Cleanup. Integration test, documentation, remove unnecessary dummy sync, minors fixups. Contributed by Erik Krogen.
add eae0a5d HDFS-13688. [SBN read] Introduce msync API call. Contributed by Chen Liang.
add 4c9baba HDFS-13789. Reduce logging frequency of QuorumJournalManager#selectInputStreams. Contributed by Erik Krogen.
add 66ec3a6 HDFS-13767. Add msync server implementation. Contributed by Chen Liang.
add 25d8e39 HDFS-13851. Remove AlignmentContext from AbstractNNFailoverProxyProvider. Contributed by Konstantin Shvachko.
add f9fc01c HDFS-13782. ObserverReadProxyProvider should work with IPFailoverProxyProvider. Contributed by Konstantin Shvachko.
add aa42fb0 HDFS-13779. [SBN read] Implement proper failover and observer failure handling logic for for ObserverReadProxyProvider. Contributed by Erik Krogen.
add d2e85b0 HDFS-13880. Add mechanism to allow certain RPC calls to bypass sync. Contributed by Chen Liang.
add 28820fb HDFS-13778. [SBN read] TestStateAlignmentContextWithHA should use real ObserverReadProxyProvider instead of AlignmentContextProxyProvider. Contributed by Konstantin Shvachko and Plamen Jeliazkov.
add 25b63e8 HDFS-13749. [SBN read] Use getServiceStatus to discover observer namenodes. Contributed by Chao Sun.
add b74a7db HDFS-13898. [SBN read] Throw retriable exception for getBlockLocations when ObserverNameNode is in safemode. Contributed by Chao Sun.
add a65bb97 HDFS-13791. Limit logging frequency of edit tail related statements. Contributed by Erik Krogen.
add b5b9b77 HDFS-13961. [SBN read] TestObserverNode refactoring. Contributed by Konstantin Shvachko.
add 1c4265d HDFS-13523. Support observer nodes in MiniDFSCluster. Contributed by Konstantin Shvachko.
add b6f20c3 HDFS-13925. Unit Test for transitioning between different states. Contributed by Sherwood Zheng.
add 8c49135 HDFS-13924. [SBN read] Handle BlockMissingException when reading from observer. Contributed by Chao Sun.
add 1a9ba96 HDFS-14016. [SBN read] ObserverReadProxyProvider should enable observer read by default. Contributed by Chen Liang.
add 652b257 HDFS-14035. NN status discovery does not leverage delegation token. Contributed by Chen Liang.
add a3aab48 HDFS-14017. [SBN read] ObserverReadProxyProviderWithIPFailover should work with HA configuration. Contributed by Chen Liang.
add cbc2f93 HDFS-14067. [SBN read] Allow manual failover between standby and observer. Contributed by Chao Sun.
add 3bb92a1 HDFS-14094. [SBN read] Fix the order of logging arguments in ObserverReadProxyProvider. Contributed by Ayush Saxena.
add 71cf66e HDFS-14120. [SBN read] ORFPP should also clone DT for the virtual IP. Contributed by Chen Liang.
add 2904aa3 HDFS-14131. [SBN read] Create user guide for Consistent Reads from Observer feature. Contributed by Chao Sun.
add b8ad6c8 HDFS-14142. Move ipfailover config key out of HdfsClientConfigKeys. Contributed by Chen Liang.
add b73fb70 HDFS-13873. [SBN read] ObserverNode should reject read requests when it is too far behind. Contributed by Konstantin Shvachko.
add cae2796 HDFS-14138. [SBN read] Description errors in the comparison logic of transaction ID. Contributed by xiangheng.
add ef3e192 HDFS-14146. [SBN read] Handle exceptions from and prevent handler threads from blocking within internalQueueCall. Contributed by Chao Sun.
add c3efc29 HDFS-14116. [SBN read] Fix class cast error in NNThroughputBenchmark with ObserverReadProxyProvider. Contributed by Chao Sun.
add fa85503 HDFS-14149. [SBN read] Fix annotations on new interfaces/classes for SBN reads. Contributed by Chao Sun.
add c9d7737 HDFS-14160. [SBN read] ObserverReadInvocationHandler should implement RpcInvocationHandler. Contributed by Konstantin V Shvachko.
add c546b12 HDFS-14154. [SBN read] Document dfs.ha.tail-edits.period in user guide. Contributed by Chao Sun.
add bd2a59e HDFS-14170. [SBN read] Fix checkstyle warnings related to SBN reads. Contributed by Konstantin V Shvachko.
add 47dd00a Merge branch 'HDFS-12943' into trunk
add 742fd97 YARN-9090. [Submarine] Update submarine installation script document. Contributed by Xun Liu.
add dea2c1b HDDS-951. Replace the source url of Ozone logo. Contributed by Takanobu Asanuma.
add d8f670f HADOOP-15819. FileSystem cache misused in S3A integration tests. Contributed by Adam Antal.
add 57198bc HADOOP-15962. The buffer size is small when unpacking tar archives. Contributed by BELUGA BEHR.
add 15fe1b3 YARN-8878. Remove StringBuffer from ManagedParentQueue.java. Contributed by BELUGA BEHR.
add 128f340 YARN-8894. Improve InMemoryPlan#toString. Contributed by BELUGA BEHR.
add e9a005d HDFS-14171. Performance improvement in Tailing EditLog. Contributed by Kenneth Yang.
add f30473f HDFS-14183. [SPS] Remove the -w parameter from the -satisfystoragepolicy usage. Contributed by Ayush Saxena.
add eee29ed HADOOP-16009. Replace the url of the repository in Apache Hadoop source code.
add 9e71967 MAPREDUCE-7174. Mapreduce example wordmedian should handle generic options. Contributed by Fei Hui.
add 4d50ad0 HDDS-949. Update Ratis to 0.4.0-a8c4ca0-SNAPSHOT. Contributed by Xiaoyu Yao.
add 13f0ee2 HADOOP-15860. ABFS: Throw exception when directory / file name ends with a period (.). Contributed by Shweta Yakkali.
add 0cb3316 HDDS-482. NullPointer exception thrown on console when cli operation failed. Contributed by Nandakumar.
add 996ab48 HDDS-393. Audit Parser tool for processing ozone audit logs. Contributed by Dinesh Chitlangia.
add cb26f15 HADOOP-16025. Update the year to 2019.
add 21fe77e HDFS-14184. [SPS] Add support for URI based path in satisfystoragepolicy command. Contributed by Ayush Saxena.
add f660e5e HDFS-14163. Debug Admin Command Should Support Generic Options. Contributed by Ayush Saxena.
add 040a202 HADOOP-15323. AliyunOSS: Improve copy file performance for AliyunOSSFileSystemStore. Contributed wujinhu.
add cfe89e6 YARN-9164. Shutdown NM may cause NPE when opportunistic container scheduling is enabled. Contributed by lujie.
add 14d232c HDDS-957. Replace incorrect use of system property user.name. Contributed by Dinesh Chitlangia.
add ecdeaa7 HDFS-14084. Need for more stats in DFSClient. Contributed by Pranay Singh.
add dfceffa YARN-9147. Rmove auxiliary services when manifest file is removed. Contributed by Billie Rinaldi
add f4906ac YARN-9038. [CSI] Add ability to publish/unpublish volumes on node managers. Contributed by Weiwei Yang.
add 573b158 YARN-8567. Fetching yarn logs fails for long running application if it is not present in timeline store. Contributed by Tarun Parimi.
add 8c6978c YARN-6149. Allow port range to be specified while starting NM Timeline collector manager. Contributed by Abhishek Modi.
add 51427cb HADOOP-15997. KMS client uses wrong UGI after HADOOP-14445. Contributed by Wei-Chiu Chuang.
add ddc0a40 HDDS-896. Handle over replicated containers in SCM. Contributed by Nandakumar.
add f4e1824 HADOOP-16028. Fix NetworkTopology chooseRandom function to support excluded nodes. Contributed by Sihai Ke.
add 6e35f71 YARN-9166. Fix logging for preemption of Opportunistic containers for Guaranteed containers. Contributed by Abhishek Modi.
add d43af8b HADOOP-15996. Improved Kerberos username mapping strategy in Hadoop. Contributed by Bolke de Bruin
add 999da98 HDDS-915. Submit client request to OM Ratis server. Contributed by Hanisha Koneru.
add 1f42527 Revert "HADOOP-15759. AliyunOSS: Update oss-sdk version to 3.0.0. Contributed by Jinhu Wu."
add 650b9cb YARN-9178. TestRMAdminCli#testHelp is failing in trunk. Contributed by Abhishek Modi.
add 8f004fe YARN-9141. [Submarine] JobStatus outputs with system UTC clock, not local clock. (Zac Zhou via wangda)
add 2c02aa6 YARN-9160. [Submarine] Document 'PYTHONPATH' environment variable setting when using -localization options. (Zhankun Tang via wangda)
add d3321fb Revert "YARN-9178. TestRMAdminCli#testHelp is failing in trunk. Contributed by Abhishek Modi."
add f87b3b1 HADOOP-16030. AliyunOSS: bring fixes back from HADOOP-15671. Contributed by wujinhu.
add 944cf87 YARN-9173. FairShare calculation broken for large values after YARN-8833. Contributed by Wilfred Spiegelenburg.
add 5db7c49 YARN-9162. Fix TestRMAdminCLI#testHelp. Contributed by Ayush Saxena.
add 32d5caa HADOOP-15937. [JDK 11] Update maven-shade-plugin.version to 3.2.1. Contributed by Dinesh Chitlangia.
add d14c56d HDDS-916. MultipartUpload: Complete Multipart upload request. Contributed by Bharat Viswanadham.
add 992dd9d HDDS-901. MultipartUpload: S3 API for Initiate multipart upload. Contributed by Bharat Viswanadham.
add d66925a HDDS-902. MultipartUpload: S3 API for uploading a part file. Contributed by Bharat Viswanadham.
add d715233 HADOOP-14556. S3A to support Delegation Tokens.
add 802932c HADOOP-16031. Fixed TestSecureLogins unit test. Contributed by Akira Ajisaka
add cdfbec4 HDDS-930. Multipart Upload: Abort multiupload request. Contributed by Bharat Viswanadham.
add 06279ec HDDS-946. AuditParser - insert audit to database in batches. contributed by Dinesh Chitlangia.
add 0a01d49 YARN-8822. Nvidia-docker v2 support for YARN GPU feature. (Charo Zhang via wangda)
add 0f26b5e HDDS-931. Add documentation for ozone shell command providing ozone mapping for a S3Bucket. Contributed by Bharat Viswanadham.
add 4894115 YARN-9169. Add metrics for queued opportunistic and guaranteed containers. Contributed by Abhishek Modi.
add 7f78397 Revert "HADOOP-14556. S3A to support Delegation Tokens."
add 4297e20 HDDS-926. Use Timeout rule for the the test methods in TestOzoneManager. Contributed by Dinesh Chitlangia.
add 0921b70 YARN-9037. [CSI] Ignore volume resource in resource calculators based on tags. Contributed by Sunil Govindan.
add 188bebb HADOOP-16018. DistCp won't reassemble chunks when blocks per chunk > 0.
add 396ffba HDDS-968. Fix TestObjectPut failures. Contributed by Bharat Viswanadham.
add 695e93c HDDS-969. Fix TestOzoneManagerRatisServer test failure. Contributed by Bharat Viswanadham.
add 999f31f HDDS-924. MultipartUpload: S3 APi for complete Multipart Upload. Contributed by Bharat Viswanadham.
add 1a08302 HDFS-14189. Fix intermittent failure of TestNameNodeMetrics. Contributed by Ayush Saxena.
add 32cf041 HDDS-965. Ozone: checkstyle improvements and code quality scripts. Contributed by Elek, Marton.
add 6a92346 YARN-6523. Optimize system credentials sent in node heartbeat responses. Contributed by Manikandan R
add 4ab5260 HDFS-14132. Add BlockLocation.isStriped() to determine if block is replicated or Striped (Contributed by Shweta Yakkali via Daniel Templeton)
add 709ddb1 HADOOP-15941. [JDK 11] Compilation failure: package com.sun.jndi.ldap is not visible.
add 3420e26 HADOOP-16027. [DOC] Effective use of FS instances during S3A integration tests. Contributed by Gabor Bota.
add 8dd11a1 HDDS-947. Implement OzoneManager State Machine.
add f4617c6 Revert "HDDS-947. Implement OzoneManager State Machine."
add c634589 Revert "HDFS-14084. Need for more stats in DFSClient. Contributed by Pranay Singh."
add 2091d1a HDDS-941. Rename ChunkGroupInputStream to keyInputStream and ChunkInputStream to BlockInputStream. Contributed by Shashikant Banerjee.
add e8d1900 HADOOP-16040. ABFS: Bug fix for tolerateOobAppends configuration.
add 7211269 HADOOP-15662. Better exception handling of DNS errors.
add 852701f HADOOP-16036. WASB: Disable jetty logging configuration announcement.
add 33c009a4 HADOOP-15909. KeyProvider class should implement Closeable. Contributed by Kuhu Shukla.
add d4ca907 HADOOP-16016. TestSSLFactory#testServerWeakCiphers fails on Java 1.8.0_191 or upper
add 9aeaaa0 HDFS-14198. Upload and Create button doesn't get enabled after getting reset. Contributed by Ayush Saxena.
add dddad98 HADOOP-15975. ABFS: remove timeout check for DELETE and RENAME.
add a4eefe5 HDDS-947. Implement OzoneManager State Machine.
add fb8932a HADOOP-16029. Consecutive StringBuilder.append can be reused. Contributed by Ayush Saxena.
add 01cb958 HADOOP-16013. DecayRpcScheduler decay thread should run as a daemon. Contributed by Erik Krogen.
add bf08f4a HADOOP-15481. Emit FairCallQueue stats as metrics. Contributed by Christopher Gregorian.
add 35fa3bd HADOOP-16045. Don't run TestDU on Windows. Contributed by Lukas Majercak.
add 3bb745d HADOOP-15994. Upgrade Jackson2 to 2.9.8. Contributed by lqjacklee.
add 04fcbef HADOOP-16043. NPE in ITestDynamoDBMetadataStore when fs.s3a.s3guard.ddb.table is not set.
add c4a00d1 HADOOP-15843. s3guard bucket-info command to not print a stack trace on bucket-not-found.
add 6d0bffe HADOOP-14556. S3A to support Delegation Tokens.
add 30863c5 HADOOP-16044. ABFS: Better exception handling of DNS errors followup
add 05c84ab HDDS-977. Exclude dependency-reduced-pom.xml from ozone rat check. Contributed by Elek, Marton.
add 614af50 YARN-9179. Fix NPE in AbstractYarnScheduler#updateNewContainerInfo.
add ccc4362 HADOOP-16019. ZKDelegationTokenSecretManager won't log exception message occured in function setJaasConfiguration.
add f280f52 HDDS-978. Fix typo in doc : Client > S3 section. Contributed by Dinesh Chitlangia.
add 01405df HADOOP-15941. Addendum patch. Contributed by Takanobu Asanuma.
add ff61931 HDDS-6. Enable SCM kerberos auth. Contributed by Ajay Kumar.
add d3920ec HDDS-5. Enable OzoneManager kerberos auth. Contributed by Ajay Kumar.
add 8e6743e HDDS-6. Enable SCM kerberos auth. Contributed by Ajay Kumar.
add bfa4929 Revert "Bad merge with 996a627b289947af3894bf83e7b63ec702a665cd"
add 914e93b HDDS-7. Enable kerberos auth for Ozone client in hadoop rpc. Contributed by Ajay Kumar.
add e47135d HDDS-70. Fix config names for secure ksm and scm. Contributed by Ajay Kumar.
add faf53f8 HDDS-100. SCM CA: generate public/private key pair for SCM/OM/DNs. Contributed by Ajay Kumar.
add 570b503 Fix merge conflicts
add 0b034b7 HDDS-546. Resolve bouncy castle dependency for hadoop-hdds-common. Contributed by Ajay Kumar.
add 2d26944 HDDS-548. Create a Self-Signed Certificate. Contributed by Anu Engineer.
add 9920506 HDDS-547. Fix secure docker and configs. Contributed by Xiaoyu Yao.
add d451188 HDDS-566. Move OzoneSecure docker-compose after HDDS-447. Contributed by Xiaoyu Yao.
add 0aab740 HDDS-10. Add kdc docker image for secure ozone cluster. Contributed by Ajay Kumar.
add 8d7c5f4 HDDS-588. SelfSignedCertificate#generateCertificate should sign the certificate the configured security provider. Contributed by Xiaoyu Yao.
add 16e0bb8 HDDS-591. Adding ASF license header to kadm5.acl. Contributed by Ajay Kumar.
add e89c35a HDDS-704. Fix the Dependency convergence issue on HDDS-4. Contributed by Xiaoyu Yao.
add 61e85d7 HDDS-684. Fix HDDS-4 branch after HDDS-490 and HADOOP-15832. Contributed by Xiaoyu Yao.
add c260c19 HDDS-101. SCM CA: generate CSR for SCM CA clients. Contributed by Xiaoyu Yao.
add 33c274e HDDS-103. SCM CA: Add new security protocol for SCM to expose security related functions. Contributed by Ajay Kumar.
add 8b8a3f5 HDDS-760. Add asf license to TestCertificateSignRequest. Contributed by Ajay Kumar.
add a28ad7a HDDS-753. SCM security protocol server is not starting. Contributed by Ajay Kumar.
add 53120e2 HDDS-592. Fix ozone-secure.robot test. Contributed by Ajay Kumar.
add 6ad794b HDDS-778. Add an interface for CA and Clients for Certificate operations Contributed by Anu Engineer.
add 6d6b1a0 HDDS-836. Add TokenIdentifier Ozone for delegation token and block token. Contributed by Ajay Kumar.
add bb4a26c HDDS-8. Add OzoneManager Delegation Token support. Contributed by Ajay Kumar.
add 7e27706 HDDS-9. Add GRPC protocol interceptors for Ozone Block Token. Contributed by Xiaoyu Yao.
add 8253106 HDDS-873. Fix TestSecureOzoneContainer NPE after HDDS-837. Contributed by Xiaoyu Yao.
add 0c8829a HDDS-696. Bootstrap genesis SCM(CA) with self-signed certificate. Contributed by Anu Engineer.
add 6d522dc HDDS-804. Block token: Add secret token manager. Contributed by Ajay Kumar.
add 417951a HDDS-884. Fix merge issue that causes NPE OzoneManager#httpServer. Contributed by Xiaoyu Yao.
add f894d86 HDDS-115. GRPC: Support secure gRPC endpoint with mTLS. Contributed by Xiaoyu Yao.
add 2b11522 HDDS-929. Remove ozone.max.key.len property. Contributed by Ajay Kumar.
add 50c4045 HDDS-805. Block token: Client api changes for block token. Contributed by Ajay Kumar.
add ddaef67 HDDS-937. Create an S3 Auth Table. Contributed by Dinesh Chitlangia.
add 924bea9 HDDS-102. SCM CA: SCM CA server signs certificate for approved CSR. Contributed by Anu Engineer.
add 30bfc9c HDDS-955. SCM CA: Add CA to SCM. Contributed by Anu Engineer.
add 1d5734e HDDS-938. Add Client APIs for using S3 Auth interface. Contributed by Dinesh Chitlangia.
add a5d0fcf HDDS-963. Fix failure in TestOzoneShell due to null check in SecurityConfig. Contributed by Ajay Kumar.
add 8978466 HDDS-945. Fix generics warnings in delegation token. Contributed by Ajay Kumar.
add 0faa570 HDDS-964. Fix test failure in TestOmMetrics. Contributed by Ajay Kumar.
add 0e16cf1 HDDS-970. Fix classnotfound error for bouncy castle classes in OM,SCM init. Contributed by Ajay Kumar.
add 140565f HDDS-967. Fix failures in TestOzoneConfigurationFields. Contributed by Ajay Kumar.
add 01a7f9e HDDS-597. Ratis: Support secure gRPC endpoint with mTLS for Ratis. Contributed by Ajay Kumar.
add c0683ed HDDS-960. Add cli command option for getS3Secret. Contributed by Dinesh Chitlangia.
add 06c83d3 HDDS-984. Fix TestOzoneManagerRatisServer.testIsReadOnlyCapturesAllCmdTypeEnums. Contributed by Xiaoyu Yao.
add 2aaaf12 HDDS-943. Add block token validation in HddsDispatcher/XceiverServer. Contributed by Ajay Kumar.
add 6be3923 YARN-9150 Making TimelineSchemaCreator support different backends for Timeline Schema Creation in ATSv2. Contributed by Sushil Ks
add 713ded6 YARN-9150 Making TimelineSchemaCreator support different backends for Timeline Schema Creation in ATSv2. Contributed by Sushil Ks
add 104ef5d YARN-8747. [UI2] YARN UI2 page loading failed due to js error under some time zone configuration. Contributed by collinma.
add f048512 HDFS-14192. Track missing DFS operations in Statistics and StorageStatistics. Contributed by Ayush Saxena.
add 54b11de HDDS-898. Continue token should contain the previous dir in Ozone s3g object list. Contributed by Elek Marton.
add 96ea464 HDDS-971. ContainerDataConstructor throws exception on QUASI_CLOSED and UNHEALTHY container state. Contributed by Lokesh Jain.
add 0a46bae YARN-9203. Fix typos in yarn-default.xml.
add 6d7eedf YARN-9194. Invalid event: REGISTERED and LAUNCH_FAILED at FAILED, and NullPointerException happens in RM while shutdown a NM. (lujie via wangda)
add 96a84b6 HDFS-14213. Remove Jansson from BUILDING.txt. Contributed by Dinesh Chitlangia.
add dacc1a7 HDFS-14175. EC: Native XOR decoder should reset the output buffer before using it. Contributed by Ayush Saxena.
add 8c7f6b2 YARN-9197. Add safe guard against NPE for component instance failure. Contributed by kyungwan nam
add 4ac0404 HDDS-959. KeyOutputStream should handle retry failures. Contributed by Lokesh Jain.
add c26d354 HDDS-983. Rename S3Utils to avoid conflict with HDFS classes. Contributed by Bharat Viswanadham.
add 751bc62 Merge branch 'HDDS-4' into trunk
add 824dfa3 YARN-8489. Support "dominant" component concept in YARN service. Contributed by Zac Zhou
add 27aa6e8 HADOOP-16046. [JDK 11] Correct the compiler exclusion of org/apache/hadoop/yarn/webapp/hamlet/** classes for >= Java 9. Contributed by Devaraj K.
add abde1e1 YARN-9204. RM fails to start if absolute resource is specified for partition capacity in CS queues. Contributed by Jiandan Yang.
add 2e2508b Make 3.2.0 aware to other branches
add e996224 Make 3.2.0 aware to other branches - jdiff
add a463cf7 HADOOP-15787. [JDK11] TestIPC.testRTEDuringConnectionSetup fails. Contributed by Zsolt Venczel.
add d43df31 YARN-9210. RM nodes web page can not display node info. Contributed by Jiandan Yang.
add de34fc1 HDFS-14207. ZKFC should catch exception when ha configuration missing. Contributed by Fei Hui.
add 1ff658b HDFS-14221. Replace Guava Optional with Java Optional. Contributed by Arpit Agarwal.
add 6f0756f HDFS-14222. Make ThrottledAsyncChecker constructor public. Contributed by Arpit Agarwal.
add 00ad9e2 HADOOP-16048. ABFS: Fix Date format parser.
add 9390a0b HDDS-913. Ozonefs defaultFs example is wrong in the documentation. Contributed by Supratim Deka.
add 0ef54f7 HDDS-992. ozone-default.xml has invalid text from a stale merge. Contributed by Dinesh Chitlangia.
add 2fa9389 YARN-9146. Added REST API to configure auxiliary service. Contributed by Billie Rinaldi
add 0dd35e2 HADOOP-15922. Fixed DelegationTokenAuthenticator URL decoding for doAs user. Contributed by He Xiaoqiao
add 7d6792e HDFS-14218. EC: Ls -e throw NPE when directory ec policy is disabled. Contributed by Ayush Saxena.
add e3e076d YARN-9205. When using custom resource type, application will fail to run due to the CapacityScheduler throws InvalidResourceRequestException(GREATER_THEN_MAX_ALLOCATION). Contributed by Zhankun Tang.
add e72e27e HDDS-932. Add blockade Tests for Network partition. Contributed by Nilotpal Nandi.
add 721d5c2 YARN-8101. Add UT to verify node-attributes in RM nodes rest API. Contributed by Prabhu Joseph.
add 2d69a35 HDDS-982. Fix TestContainerDataYaml#testIncorrectContainerFile. Contributed by Doroszlai, Attila.
add 221e308 HDFS-14153. [SPS] : Add Support for Storage Policy Satisfier in WEBHDFS. Contributed by Ayush Saxena.
add 0b91329 HDDS-764. Run S3 smoke tests with replication STANDARD. (#462)
add 951cdd7 HDFS-14061. Check if the cluster topology supports the EC policy before setting, enabling or adding it. Contributed by Kitti Nanasi.
add dcbc8b8 HDDS-975. Manage ozone security tokens with ozone shell cli. Contributed by Ajay Kumar.
add f3e642d HDFS-14185. Cleanup method calls to static Assert methods in TestAddStripedBlocks (Contributed by Shweta Yakkali via Daniel Templeton)
add e321b91 HDDS-980. Adding getOMCertificate in SCMSecurityProtocol. Contributed by Ajay Kumar.
add c726445 YARN-8961. [UI2] Flow Run End Time shows 'Invalid date'. Contributed by Akhil PB
add a4bd64e YARN-9116. Capacity Scheduler: implements queue level maximum-allocation inheritance. Contributed by Aihua Xu.
add 09a5859 HDDS-993. Update hadoop version to 3.2.0. Contributed by Supratim Deka.
add f3d8265 HDDS-996. Incorrect data length gets updated in OM by client in case it hits exception in multiple successive block writes. Contributed by Shashikant Banerjee.
add 3c7d700 HDDS-1002. ozonesecure compose incompatible with smoke test. Contributed by Doroszlai, Attila.
add 4e0aa2c HDDS-948. MultipartUpload: S3 API for Abort Multipart Upload. Contributed by Bharat Viswanadham.
add a33ef4f YARN-8867. Added resource localization status to YARN service status call. Contributed by Chandni Singh
add 3c60303 HADOOP-16065. -Ddynamodb should be -Ddynamo in AWS SDK testing document.
add c6d901a HDDS-1006. AuditParser assumes incorrect log format. Contributed by Dinesh Chitlangia.
add 8ff9578 HDDS-1007. Add robot test for AuditParser. Contributed by Dinesh Chitlangia.
add 45c4cfe HDDS-906. Display the ozone version on SCM/OM web ui instead of Hadoop version. Contributed by Doroszlai, Attila.
add 2181b18 HDDS-990. Typos in Ozone doc. Contributed by Doroszlai, Attila.
add 5dae1a0 HDDS-973. HDDS/Ozone fail to build on Windows. Contributed by Xiaoyu Yao.
add a448b05 HDDS-1009. TestAbortMultipartUpload is missing the apache license text. Contributed by Dinesh Chitlangia.
add 9fc7df8 HDDS-793. Support custom key/value annotations on volume/bucket/key. Contributed by Elek, Marton.
add 84bb980 YARN-7761. [UI2] Clicking 'master container log' or 'Link' next to 'log' under application's appAttempt goes to Old UI's Log link. Contributed by Akhil PB.
add 45caeee HDFS-14228. Incorrect getSnapshottableDirListing() javadoc. Contributed by Dinesh Chitlangia.
add 1d52327 HDFS-14084. Need for more stats in DFSClient. Contributed by Pranay Singh.
add 2ec296e HDDS-991. Fix failures in TestSecureOzoneCluster. Contributed by Ajay Kumar.
add dc5af4c HDFS-12729. Document special paths in HDFS. Contributed by Masatake Iwasaki.
add 6cace58 YARN-9222. Print launchTime in ApplicationSummary
add fb69519 HDDS-1011. Fix NPE BucketManagerImpl.setBucketProperty. Contributed by Xiaoyu Yao.
add 1ab69a9 YARN-9221. Added flag to disable dynamic auxiliary service feature. Contributed by Billie Rinaldi
add 91649c3 HDDS-1013. NPE while listing directories.
add 47d6b9b HADOOP-16075. Upgrade checkstyle version to 8.16.
add 3b49d7a HDDS-989. Check Hdds Volumes for errors. Contributed by Arpit Agarwal.
add 8326450 HDDS-974. Add getServiceAddress method to ServiceInfo and use it in TestOzoneShell. Contributed by Doroszlai, Attila.
add 2e636dd YARN-9074. Consolidate docker removal logic in ContainerCleanup. Contributed by Zhaohui Xin
add f5a95f7 YARN-8901. Fixed restart policy NEVER/ON_FAILURE with component dependency. Contributed by Suma Shivaprasad
add 4f63ffe YARN-9237. NM should ignore sending finished apps to RM during RM fail-over. Contributed by Jiandan Yang.
add 2d06112 HDDS-1022. Add cmd type in getCommandResponse in SCMDatanodeProtocolServer. Contributed by Bharat Viswanadham.
add 085f0e8 YARN-9086. [CSI] Run csi-driver-adaptor as aux service. Contributed by Weiwei Yang.
add 5d578d0 HDDS-1004. SCMContainerManager#updateContainerStateInternal fails for QUASI_CLOSE and FORCE_CLOSE events. Contributed by Lokesh Jain.
add 04105bb YARN-6616: YARN AHS shows submitTime for jobs same as startTime. Contributed by Prabhu Joseph
add d1714c2 Revert "HDFS-14084. Need for more stats in DFSClient. Contributed by Pranay Singh."
add b3bc94e HDFS-14236. Lazy persist copy/ put fails with ViewFs.
add 02eb918 HADOOP-16041. Include Hadoop version in User-Agent string for ABFS. Contributed by Shweta Yakkali.
add 1129288 HADOOP-14178. Move Mockito up to version 2.23.4. Contributed by Akira Ajisaka and Masatake Iwasaki.
add d583cc4 HDDS-1024. Handle DeleteContainerCommand in the SCMDatanodeProtocolServer. Contributed by Bharat Viswanadham.
add 14441cc HDDS-1032. Package builds are failing with missing org.mockito:mockito-core dependency version. Contributed by Doroszlai, Attila.
add a3a9ae3 YARN-9251. Build failure for -Dhbase.profile=2.0. Contributed by Rohith Sharma K S.
add 0e95ae4 HDDS-1030. Move auditparser robot tests under ozone basic. Contributed by Dinesh Chitlangia.
add 7456fc9 HDDS-1031. Update ratis version to fix a DN restart Bug. Contributed by Bharat Viswanadham.
add c354195 HDDS-1016. Allow marking containers as unhealthy. Contributed by Arpit Agarwal.
add 945a61c HDDS-549. Add support for key rename in Ozone Shell. Contributed by Doroszlai Attila.
add 5372927 HDDS-1035. Intermittent TestRootList failure. Contributed by Doroszlai Attila.
add 71c49fa YARN-9099. GpuResourceAllocator#getReleasingGpus calculates number of GPUs in a wrong way. Contributed by Szilard Nemeth.
add 033d97a HDDS-956. MultipartUpload: List Parts for a Multipart upload key. Contributed by Bharat Viswanadham.
add bcc3a79 HADOOP-16084. Fix the comment for getClass in Configuration. Contributed by Fengnan Li.
add f738b39 YARN-9191. Add cli option in DS to support enforceExecutionType in resource requests. Contributed by Abhishek Modi.
add 0ab7fc9 HDFS-14187. Make warning message more clear when there are not enough data nodes for EC write. Contributed by Kitti Nanasi.
add 16195ea HDDS-1025. Handle replication of closed containers in DeadNodeHanlder. Contributed by Bharat Viswanadham.
add 13aa939 HDDS-997. Add blockade Tests for scm isolation and mixed node isolation. Contributed by Nilotpal Nandi.
add 7f46d13 HADOOP-16079. Token.toString faulting if any token listed can't load.
add 4123353 HDDS-1037. Fix the block discard logic in Ozone client. Contributed by Shashikant Banerjee.
add 2c13513 YARN-8549 Adding a NoOp timeline writer and reader plugin classes for ATSv2. Contributed by Prabha Manepalli.
add 28ad20a YARN-9262. TestRMAppAttemptTransitions is failing with an NPE. Contributed by lujie.
add f20b043 YARN-9263. TestConfigurationNodeAttributesProvider fails after Mockito updated. Contributed by Weiwei Yang.
add 69bcff3 YARN-9231. TestDistributedShell fix timeout. Contributed by Prabhu Joseph.
add b6f90d3 HDDS-1021. Allow ozone datanode to participate in a 1 node as well as 3 node Ratis Pipeline. Contributed by Shashikant Banerjee.
add ec77e95 HDFS-14232. libhdfs is not included in binary tarball. Contributed by Akira Ajisaka.
add 9aa3dc8 HDFS-14158. Checkpointer ignores configured time period > 5 minutes
add c991e2c MAPREDUCE-7177. Disable speculative execution in TestDFSIO. Contributed by Zhaohui Xin.
add 0f9aa5b HADOOP-16089. AliyunOSS: update oss-sdk version to 3.4.1. Contributed by wujinhu.
add 604b248 YARN-9206. RMServerUtils does not count SHUTDOWN as an accepted state. Contributed by Kuhu Shukla.
add 758e9ce HADOOP-16076. SPNEGO+SSL Client Connections with HttpClient Broken.
add 0e79a86 HDFS-14202. dfs.disk.balancer.max.disk.throughputInMBperSec property is not working as per set value. Contributed by Ranith Sardar.
add 5f15a60 HDFS-14125. Use parameterized log format in ECTopologyVerifier. Contributed by Kitti Nanasi.
add 9a19d6d HDDS-1039. OzoneManager fails to connect with secure SCM. Contributed by Ajay Kumar
add 529791c HADOOP-15938. [JDK 11] Remove animal-sniffer-maven-plugin to fix build. Contributed by Dinesh Chitlangia.
add 3efa168 HDDS-1029. Allow option for force in DeleteContainerCommand. Contributed by Bharat Viswanadham.
add 5718389 YARN-9149. yarn container -status misses logUrl when integrated with ATSv2. Contributed by Abhishek Modi.
add aa7ce50 YARN-9275. Add link to NodeAttributes doc in PlacementConstraints document. Contributed by Masatake Iwasaki.
add ba38db4 YARN-9257. Distributed Shell client throws a NPE for a non-existent queue. Contributed by Charan Hebri.
add e3ec18b YARN-6735. Have a way to turn off container metrics from NMs. Contributed by Abhishek Modi.
add f365957 HADOOP-15229. Add FileSystem builder-based openFile() API to match createFile(); S3A to implement S3 Select through this API.
add 9ace37b HDDS-987. MultipartUpload: S3API for list parts of a object. Contributed by Bharat Viswanadham.
add ba9efe0 HADOOP-16074. WASB: Update container not found error code.
add 9f2da01 HDDS-776. Make OM initialization resilient to dns failures. Contributed by Doroszlai, Attila.
add 194f0b4 HDDS-631. Ozone classpath shell command is not working. Contributed by Elek, Marton.
add 2044967 YARN-9246 NPE when executing a command yarn node -status or -states without additional arguments. Contributed by Masahiro Tanaka
add fa8cd1b HADOOP-15954. ABFS: Enable owner and group conversion for MSI and login user using OAuth.
add 308f316 Make upstream aware of 3.1.2 release
add 49ddd8a HDFS-14231. DataXceiver#run() should not log exceptions caused by InvalidToken exception as an error. Contributed by Kitti Nanasi.
add 911790c HDDS-1027. Add blockade Tests for datanode isolation and scm failures. Contributed by Nilotpal Nandi.
add 711d22f YARN-9253. Add UT to verify Placement Constraint in Distributed Shell. Contributed by Prabhu Joseph.
add d3de8e1 HDFS-14250. [SBN read]. msync should always direct to active NameNode to get latest stateID. Contributed by Chao Sun.
add 6aa6345 HDFS-14242. OIV WebImageViewer: NPE when param op is not specified. Contributed by Siyao Meng.
add 912d9f7 HDDS-1044. Client doesn't propogate correct error code to client on out of disk space. Contributed by Yiqun Lin.
add 1e5e08d YARN-7627. [ATSv2] When passing a non-number as metricslimit, the error message is wrong. Contributed by Charan Hebri.
add 7fa62e1 YARN-8219. Add application launch time to ATSV2. Contributed by Abhishek Modi.
add 3c96a03 YARN-8498. Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04. Contributed by Ayush Saxena.
add de804e5 HADOOP-15281. Distcp to add no-rename copy option.
add 214112b HDDS-1010. ContainerSet#getContainerMap should be renamed. Contributed by Supratim Deka.
add a65aca2 HDDS-922. Create isolated classloder to use ozonefs with any older hadoop versions. Contributed by Elek, Marton.
add d1ca943 YARN-7171: RM UI should sort memory / cores numerically. Contributed by Ahmed Hussein
add 75e8441 HDDS-1071. Make Ozone s3 acceptance test suite centos compatible. Contributed by Elek Marton.
add 546c5d7 HADOOP-16032. Distcp It should clear sub directory ACL before applying new ACL on.
add 668817a Revert "HADOOP-15954. ABFS: Enable owner and group conversion for MSI and login user using OAuth."
add 1f16550 HADOOP-15954. ABFS: Enable owner and group conversion for MSI and login user using OAuth.
add 4be8735 HDFS-14140. JournalNodeSyncer authentication is failing in secure cluster. Contributed by Surendra Singh Lilhore.
add a140a89 HDDS-1069. Temporarily disable the security acceptance tests by default in Ozone. Contributed by Marton Elek.
add 0c1bc4d HDDS-981. Block allocation should involve pipeline selection and then container selection. Contributed by Lokesh Jain.
add df7b7da HDDS-1073. Fix FindBugs issues on OzoneBucketStub#createMultipartKey. Contributed by Aravindan Vijayan.
add 394a9f7 HDDS-1033. Add FSStatistics for OzoneFileSystem. Contributed by Mukul Kumar Singh.
add 1771317 HDFS-14172. Avoid NPE when SectionName#fromString returns null. Contributed by Xiang Li.
add e0ab1bd YARN-9282. Typo in javadoc of class LinuxContainerExecutor: hadoop.security.authetication should be 'authentication'. Contributed by Charan Hebri.
add fb8c997 HDDS-1048. Remove SCMNodeStat from SCMNodeManager and use storage information from DatanodeInfo#StorageReportProto. Contributed by Nanda kumar.
add e50dc7e HDDS-1018. Update the base image of krb5 container for the secure ozone cluster. Contributed by Xiaoyu Yao.
add 965d26c HDDS-1026. Reads should fail over to alternate replica. Contributed by Shashikant Banerjee.
add ed99da8 HDDS-1078. TestRatisPipelineProvider failing because of node count mismatch. Contributed by Mukul Kumar Singh.
add a141458 HDDS-1077. TestSecureOzoneCluster does not config OM HTTP keytab. Contributed by Xiaoyu Yao.
add 2b7f828 YARN-9252. Allocation Tag Namespace support in Distributed Shell. Contributed by Prabhu Joseph.
add 0a1637c YARN-8555. Parameterize TestSchedulingRequestContainerAllocation(Async) to cover both PC handler options. Contributed by Prabhu Joseph.
add e7d1ae5 HDDS-1017. Use distributed tracing to indentify performance problems in Ozone. Contributed by Elek, Marton.
add 73b67b2 HDDS-1040. Add blockade Tests for client failures. Contributed by Nilotpal Nandi.
add 0ceb1b7 HDFS-14260. Replace synchronized method in BlockReceiver with atomic value. Contributed by BELUGA BEHR.
add 5c10630 HDFS-14261. Kerberize JournalNodeSyncer unit test. Contributed by Siyao Meng.
add ca4e46a HDDS-1075. Fix CertificateUtil#parseRSAPublicKey charsetName. Contributed by Siddharth Wagle.
add 6c999fe HADOOP-16098. Fix javadoc warnings in hadoop-aws. Contributed by Masatake Iwasaki.
add 1ce2e91 YARN-9229. Document docker registry deployment with NFS Gateway. Contributed by Eric Yang.
add 7536488 YARN-996. REST API support for node resource configuration. Contributed by Inigo Goiri.
add d48e61d HDDS-1012. Add Default CertificateClient implementation. Contributed by Ajay Kumar
add 26e6013 HDDS-1074. Remove dead variable from KeyOutputStream#addKeyLocationInfo. Contributed by Siddharth Wagle.
add a536eb5 HDDS-360. Use RocksDBStore and TableStore for SCM Metadata. Contributed by Anu Engineer.
add 4f7d32e HDDS-1081. CLOSING state containers should not be added to pipeline on SCM start. Contributed by Lokesh Jain.
add 63a9b0d HDDS-1080. Ozonefs Isolated class loader should support FsStorageStatistics. Contributed by Elek, Marton.
add 20b92cd HDDS-1050. TestSCMRestart#testPipelineWithScmRestart is failing. Contributed by Supratim Deka.
add 7806403 HDFS-14266. EC : Fsck -blockId shows null for EC Blocks if One Block Is Not Available. Contributed by Ayush Saxena.
add 3dc2523 YARN-9184. Add a system flag to allow update to latest docker images. Contributed by Zhaohui Xin
add 06d7890 HDDS-1047. Fix TestRatisPipelineProvider#testCreatePipelineWithFactor. Contributed by Nilotpal Nandi.
add 7b11b40 HADOOP-16097. Provide proper documentation for FairCallQueue. Contributed by Erik Krogen.
add 917ac9f HDDS-972. Add support for configuring multiple OMs. Contributed by Hanisha Koneru.
add cf4aecc HDDS-1034. TestOzoneRpcClient and TestOzoneRpcClientWithRatis failure. Contributed by Mukul Kumar Singh.
add 00c5ffa HADOOP-16108. Tail Follow Interval Should Allow To Specify The Sleep Interval To Save Unnecessary RPC's. Contributed by Ayush Saxena.
add 35d4f32 HDFS-14274. EC: NPE While Listing EC Policy For A Directory Following Replication Policy. Contributed by Ayush Saxena.
add 29b411d HDFS-14263. Remove unnecessary block file exists check from FsDatasetImpl#getBlockInputStream(). Contributed by Surendra Singh Lilhore
add 024c872 HDFS-13617. Allow wrapping NN QOP into token in encrypted message. Contributed by Chen Liang
add fa067aa HDDS-936. Need a tool to map containers to ozone objects. Contributed by Sarun Singla
add fd02686 HDFS-14241. Provide feedback on successful renameSnapshot and deleteSnapshot. Contributed by Siyao Meng.
add dfe0f42 YARN-7824. [UI2] Yarn Component Instance page should include link to container logs. Contributed by Akhil PB.
add 7a57974 HDDS-1096. OzoneManager#loadOMHAConfigs should use default ports in case port is not defined. Contributed by Hanisha Koneru.
add 080a421 HDFS-14262. [SBN read] Make Log.WARN message in GlobalStateIdContext more informative. Contributed by Shweta Yakkali.
add 0d7a5ac HDFS-13209. DistributedFileSystem.create should allow an option to provide StoragePolicy. Contributed by Ayush Saxena.
add 134ae8f YARN-9293. Optimize MockAMLauncher event handling. Contributed by Bibin A Chundatt.
add b66d5ae YARN-9295. [UI2] Fix label typo in Cluster Overview page. Contributed by Charan Hebri.
add 64f28f9 HDFS-14162. [SBN read] Allow Balancer to work with Observer node. Add a new ProxyCombiner allowing for multiple related protocols to be combined. Allow AlignmentContext to be passed in NameNodeProxyFactory. Contributed by Erik Krogen.
add 6c8ffdb HDDS-1100. fix asf license errors in newly added files by HDDS-936. Contributed by Dinesh Chitlangia.
add 2d83b24 HDDS-1108. Check s3bucket exists or not before MPU operations. Contributed by Bharat Viswanadham.
add dabfeab YARN-9308. fairscheduler-statedump.log gets generated regardless of service again after the merge of HDFS-7240. Contributed by Wilfred Spiegelenburg.
add 5656409 HDDS-1099. Genesis benchmark for ozone key creation in OM. Contributed by Bharat Viswanadham.
add 492e49e Revert "HDDS-1099. Genesis benchmark for ozone key creation in OM. Contributed by Bharat Viswanadham."
add 084b6a6 HDDS-1099. Genesis benchmark for ozone key creation in OM. Contributed by Bharat Viswanadham.
add 3a39d9a YARN-9284. Fix the unit of yarn.service.am-resource.memory in the document. Contributed by Masahiro Tanaka.
add 0395f22 HDDS-1068. Improve the error propagation for ozone sh. Contributed by Elek, Marton.
add 5b55f35 YARN-8295. [UI2] Improve Resource Usage tab error message when there are no data available. Contributed by Charan Hebri.
add 506bd02 HDDS-905. Create informative landing page for Ozone S3 gateway. Contributed by Elek, Marton.
add 5cb67cf HDDS-1097. Add genesis benchmark for BlockManager#allocateBlock. Contributed by Lokesh Jain.
add 75e15cc HDDS-1103.Fix rat/findbug/checkstyle errors in ozone/hdds projects. Contributed by Elek, Marton.
add 8a426dc HDDS-1028. Improve logging in SCMPipelineManager. Contributed by Lokesh Jain.
add 9385ec4 YARN-9283. Javadoc of LinuxContainerExecutor#addSchedPriorityCommand has a wrong property name as reference
add 9584b47 HDDS-1082. OutOfMemoryError because of memory leak in KeyInputStream. Contributed by Supratim Deka.
add e0fe3d1 HDDS-1110. OzoneManager need to login during init when security is enabled. Contributed by Xiaoyu Yao.
new ed8d57b HDFS-13310. The DatanodeProtocol should have a DNA_BACKUP to backup blocks. Original patch contributed by Ewan Higgs. Followup work and fixed contributed by Virajith Jalaparthi.
new f482c67 HDFS-13421. [PROVIDED Phase 2] Implement DNA_BACKUP command in Datanode. Contributed by Ewan Higgs.
new 64da704 HDFS-13794. [PROVIDED Phase 2] Teach BlockAliasMap.Writer remove method. Contributed by Ewan Higgs
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (6657cca)
\
N -- N -- N refs/heads/HDFS-12090 (64da704)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.gitignore | 3 +-
BUILDING.txt | 43 +-
LICENSE.txt | 80 +-
NOTICE.txt | 30 +-
README.txt | 2 +
dev-support/bin/create-release | 23 +-
dev-support/bin/dist-layout-stitching | 1 +
dev-support/bin/ozone-dist-layout-stitching | 161 -
dev-support/bin/ozone-dist-tar-stitching | 43 -
dev-support/bin/yetus-wrapper | 2 +-
dev-support/docker/Dockerfile | 213 +-
hadoop-assemblies/pom.xml | 4 +-
.../resources/assemblies/hadoop-registry-dist.xml | 41 +
...-src-with-hdsl.xml => hadoop-src-with-hdds.xml} | 0
.../main/resources/assemblies/hadoop-yarn-dist.xml | 15 +
hadoop-build-tools/pom.xml | 2 +-
hadoop-client-modules/hadoop-client-api/pom.xml | 4 +-
.../hadoop-client-check-invariants/pom.xml | 6 +-
.../hadoop-client-check-test-invariants/pom.xml | 6 +-
.../hadoop-client-integration-tests/pom.xml | 4 +-
.../hadoop-client-minicluster/pom.xml | 53 +-
.../hadoop-client-runtime/pom.xml | 28 +-
hadoop-client-modules/hadoop-client/pom.xml | 4 +-
hadoop-client-modules/pom.xml | 2 +-
.../hadoop-cloud-storage/pom.xml | 4 +-
hadoop-cloud-storage-project/pom.xml | 4 +-
hadoop-common-project/hadoop-annotations/pom.xml | 28 +-
hadoop-common-project/hadoop-auth-examples/pom.xml | 4 +-
hadoop-common-project/hadoop-auth/pom.xml | 6 +-
.../server/KerberosAuthenticationHandler.java | 11 +-
.../security/authentication/util/AuthToken.java | 2 +-
.../authentication/util/CertificateUtil.java | 5 +-
.../security/authentication/util/KerberosName.java | 92 +-
.../security/authentication/util/KerberosUtil.java | 2 +-
.../client/TestKerberosAuthenticator.java | 1 +
.../server/TestAuthenticationFilter.java | 19 +-
.../server/TestKerberosAuthenticationHandler.java | 150 +-
.../authentication/util/TestKerberosName.java | 23 +-
.../util/TestRandomSignerSecretProvider.java | 4 +-
.../jdiff/Apache_Hadoop_Common_3.1.2.xml | 35695 +++++++++++++++++++
hadoop-common-project/hadoop-common/pom.xml | 99 +-
.../hadoop-common/src/main/bin/hadoop | 6 +
.../hadoop-common/src/main/conf/hadoop-env.sh | 26 +-
.../hadoop-common/src/main/conf/log4j.properties | 40 +-
.../java/org/apache/hadoop/conf/Configurable.java | 10 +-
.../java/org/apache/hadoop/conf/Configuration.java | 153 +-
.../hadoop/conf/ConfigurationWithLogging.java | 14 +-
.../java/org/apache/hadoop/crypto/CipherSuite.java | 4 +-
.../java/org/apache/hadoop/crypto/CryptoCodec.java | 7 +-
.../apache/hadoop/crypto/CryptoInputStream.java | 4 +-
.../apache/hadoop/crypto/CryptoOutputStream.java | 4 +-
.../apache/hadoop/crypto/CryptoStreamUtils.java | 21 +-
.../java/org/apache/hadoop/crypto/Decryptor.java | 14 +-
.../java/org/apache/hadoop/crypto/Encryptor.java | 14 +-
.../org/apache/hadoop/crypto/OpensslCipher.java | 18 +-
.../hadoop/crypto/key/JavaKeyStoreProvider.java | 15 +-
.../org/apache/hadoop/crypto/key/KeyProvider.java | 9 +-
.../crypto/key/KeyProviderCryptoExtension.java | 9 +-
.../key/KeyProviderDelegationTokenExtension.java | 73 +-
.../hadoop/crypto/key/KeyProviderTokenIssuer.java | 4 +-
.../hadoop/crypto/key/kms/KMSClientProvider.java | 239 +-
.../key/kms/LoadBalancingKMSClientProvider.java | 75 +-
.../apache/hadoop/crypto/key/kms/ValueQueue.java | 2 +-
.../hadoop/crypto/random/OpensslSecureRandom.java | 10 +-
.../org/apache/hadoop/fs/AbstractFileSystem.java | 95 +-
.../java/org/apache/hadoop/fs/BBPartHandle.java | 5 +
.../java/org/apache/hadoop/fs/BBUploadHandle.java | 5 +
.../java/org/apache/hadoop/fs/BlockLocation.java | 13 +-
.../apache/hadoop/fs/BufferedFSInputStream.java | 4 +-
.../org/apache/hadoop/fs/ByteBufferReadable.java | 8 +-
.../org/apache/hadoop/fs/ChecksumFileSystem.java | 2 +-
.../main/java/org/apache/hadoop/fs/ChecksumFs.java | 2 +-
.../ClusterStorageCapacityExceededException.java | 51 +
.../apache/hadoop/fs/CommonConfigurationKeys.java | 2 +-
.../hadoop/fs/CommonConfigurationKeysPublic.java | 42 +-
.../main/java/org/apache/hadoop/fs/CreateFlag.java | 2 +-
.../org/apache/hadoop/fs/DelegateToFileSystem.java | 20 +
.../main/java/org/apache/hadoop/fs/FSBuilder.java | 131 +
.../org/apache/hadoop/fs/FSDataInputStream.java | 1 +
.../org/apache/hadoop/fs/FSDataOutputStream.java | 4 +-
.../hadoop/fs/FSDataOutputStreamBuilder.java | 193 +-
.../java/org/apache/hadoop/fs/FSInputChecker.java | 2 +-
.../java/org/apache/hadoop/fs/FileContext.java | 138 +-
.../org/apache/hadoop/fs/FileEncryptionInfo.java | 32 +-
.../main/java/org/apache/hadoop/fs/FileStatus.java | 34 +-
.../main/java/org/apache/hadoop/fs/FileSystem.java | 284 +-
.../hadoop/fs/FileSystemMultipartUploader.java | 65 +-
.../main/java/org/apache/hadoop/fs/FileUtil.java | 61 +-
.../org/apache/hadoop/fs/FilterFileSystem.java | 38 +
.../main/java/org/apache/hadoop/fs/FilterFs.java | 18 +
.../hadoop/fs/FutureDataInputStreamBuilder.java | 50 +
.../main/java/org/apache/hadoop/fs/Globber.java | 13 +-
.../java/org/apache/hadoop/fs/HarFileSystem.java | 2 +-
.../hadoop/fs/HasEnhancedByteBufferAccess.java | 15 +-
.../org/apache/hadoop/fs/LocalDirAllocator.java | 3 +-
.../java/org/apache/hadoop/fs/LocalFileSystem.java | 2 +-
.../org/apache/hadoop/fs/MultipartUploader.java | 91 +-
.../apache/hadoop/fs/MultipartUploaderFactory.java | 11 +
.../main/java/org/apache/hadoop/fs/Options.java | 2 +-
.../src/main/java/org/apache/hadoop/fs/Path.java | 12 +-
.../main/java/org/apache/hadoop/fs/QuotaUsage.java | 8 +-
.../org/apache/hadoop/fs/StorageStatistics.java | 10 +-
.../org/apache/hadoop/fs/TrashPolicyDefault.java | 10 +-
.../org/apache/hadoop/fs/ftp/FTPFileSystem.java | 19 +-
.../org/apache/hadoop/fs/ftp/FtpConfigKeys.java | 1 +
.../hadoop/fs/impl/AbstractFSBuilderImpl.java | 356 +
.../fs/impl/FutureDataInputStreamBuilderImpl.java | 141 +
.../org/apache/hadoop/fs/impl/FutureIOSupport.java | 191 +
.../apache/hadoop/fs/impl/WrappedIOException.java | 56 +
.../org/apache/hadoop/fs/impl/package-info.java | 49 +
.../apache/hadoop/fs/local/LocalConfigKeys.java | 1 +
.../org/apache/hadoop/fs/permission/AclEntry.java | 4 +-
.../org/apache/hadoop/fs/permission/AclStatus.java | 2 +-
.../org/apache/hadoop/fs/permission/AclUtil.java | 10 +-
.../hadoop/fs/permission/ScopedAclEntries.java | 14 +-
.../org/apache/hadoop/fs/sftp/SFTPFileSystem.java | 44 +-
.../org/apache/hadoop/fs/sftp/SFTPInputStream.java | 28 +-
.../java/org/apache/hadoop/fs/shell/Command.java | 24 +-
.../org/apache/hadoop/fs/shell/CommandFormat.java | 2 +-
.../hadoop/fs/shell/CommandWithDestination.java | 14 +-
.../org/apache/hadoop/fs/shell/CopyCommands.java | 7 +-
.../java/org/apache/hadoop/fs/shell/Count.java | 4 +-
.../main/java/org/apache/hadoop/fs/shell/Ls.java | 46 +-
.../java/org/apache/hadoop/fs/shell/PathData.java | 6 +-
.../apache/hadoop/fs/shell/SnapshotCommands.java | 3 +
.../main/java/org/apache/hadoop/fs/shell/Tail.java | 25 +-
.../hadoop/fs/shell/find/BaseExpression.java | 4 +-
.../java/org/apache/hadoop/fs/shell/find/Find.java | 4 +-
.../hadoop/fs/viewfs/ChRootedFileSystem.java | 5 +
.../org/apache/hadoop/fs/viewfs/ChRootedFs.java | 5 +
.../apache/hadoop/fs/viewfs/ViewFileSystem.java | 13 +
.../java/org/apache/hadoop/fs/viewfs/ViewFs.java | 24 +-
.../org/apache/hadoop/ha/ActiveStandbyElector.java | 32 +-
.../org/apache/hadoop/ha/FailoverController.java | 2 +-
.../main/java/org/apache/hadoop/ha/HAAdmin.java | 52 +-
.../org/apache/hadoop/ha/HAServiceProtocol.java | 18 +
.../apache/hadoop/ha/HAServiceProtocolHelper.java | 9 +
.../java/org/apache/hadoop/ha/HAServiceTarget.java | 7 +
.../main/java/org/apache/hadoop/ha/NodeFencer.java | 2 +-
.../org/apache/hadoop/ha/SshFenceByTcpPort.java | 2 +-
.../HAServiceProtocolClientSideTranslatorPB.java | 16 +
.../HAServiceProtocolServerSideTranslatorPB.java | 20 +
.../java/org/apache/hadoop/http/HttpServer2.java | 40 +-
.../org/apache/hadoop/http/IsActiveServlet.java | 71 +
.../java/org/apache/hadoop/io/EnumSetWritable.java | 2 +-
.../main/java/org/apache/hadoop/io/IOUtils.java | 2 +-
.../main/java/org/apache/hadoop/io/MD5Hash.java | 4 +-
.../java/org/apache/hadoop/io/ReadaheadPool.java | 2 +-
.../java/org/apache/hadoop/io/SecureIOUtils.java | 4 +-
.../java/org/apache/hadoop/io/SequenceFile.java | 20 +-
.../main/java/org/apache/hadoop/io/Writable.java | 4 +-
.../org/apache/hadoop/io/WritableComparable.java | 7 +-
.../java/org/apache/hadoop/io/WritableUtils.java | 6 +-
.../io/compress/BlockDecompressorStream.java | 4 +-
.../io/compress/CompressionCodecFactory.java | 26 +-
.../org/apache/hadoop/io/compress/Lz4Codec.java | 4 +-
.../hadoop/io/compress/PassthroughCodec.java | 246 +
.../org/apache/hadoop/io/compress/SnappyCodec.java | 2 +-
.../hadoop/io/compress/bzip2/Bzip2Compressor.java | 2 +-
.../io/compress/bzip2/Bzip2Decompressor.java | 6 +-
.../hadoop/io/compress/bzip2/Bzip2Factory.java | 4 +-
.../io/compress/bzip2/CBZip2InputStream.java | 26 +-
.../io/compress/bzip2/CBZip2OutputStream.java | 7 +-
.../io/compress/zlib/BuiltInGzipDecompressor.java | 8 +-
.../hadoop/io/compress/zlib/ZlibCompressor.java | 2 +-
.../hadoop/io/compress/zlib/ZlibDecompressor.java | 6 +-
.../hadoop/io/compress/zlib/ZlibFactory.java | 4 +-
.../io/compress/zstd/ZStandardCompressor.java | 2 +-
.../io/compress/zstd/ZStandardDecompressor.java | 4 +-
.../apache/hadoop/io/erasurecode/CodecUtil.java | 2 +-
.../org/apache/hadoop/io/erasurecode/ECSchema.java | 8 +-
.../io/erasurecode/rawcoder/util/GaloisField.java | 2 +-
.../org/apache/hadoop/io/file/tfile/TFile.java | 8 +-
.../org/apache/hadoop/io/file/tfile/Utils.java | 48 +-
.../org/apache/hadoop/io/nativeio/NativeIO.java | 15 +-
.../hadoop/io/retry/RetryInvocationHandler.java | 5 +
.../org/apache/hadoop/io/retry/RetryProxy.java | 4 +-
.../apache/hadoop/io/serializer/Deserializer.java | 2 +-
.../apache/hadoop/io/serializer/Serializer.java | 2 +-
.../org/apache/hadoop/ipc/AlignmentContext.java | 96 +
.../org/apache/hadoop/ipc/CallQueueManager.java | 5 +-
.../java/org/apache/hadoop/ipc/CallerContext.java | 2 +-
.../main/java/org/apache/hadoop/ipc/Client.java | 61 +-
.../java/org/apache/hadoop/ipc/ClientCache.java | 6 +-
.../org/apache/hadoop/ipc/DecayRpcScheduler.java | 5 +-
.../java/org/apache/hadoop/ipc/ExternalCall.java | 3 +-
.../java/org/apache/hadoop/ipc/FairCallQueue.java | 32 +-
.../hadoop/ipc/ObserverRetryOnActiveException.java | 37 +
.../org/apache/hadoop/ipc/ProtobufRpcEngine.java | 24 +-
.../java/org/apache/hadoop/ipc/ProxyCombiner.java | 137 +
.../src/main/java/org/apache/hadoop/ipc/RPC.java | 49 +-
.../java/org/apache/hadoop/ipc/RefreshHandler.java | 1 -
.../org/apache/hadoop/ipc/RemoteException.java | 2 +-
.../main/java/org/apache/hadoop/ipc/RpcEngine.java | 8 +-
.../main/java/org/apache/hadoop/ipc/Server.java | 248 +-
.../org/apache/hadoop/ipc/WritableRpcEngine.java | 59 +-
.../hadoop/ipc/metrics/RpcDetailedMetrics.java | 2 +-
.../java/org/apache/hadoop/jmx/JMXJsonServlet.java | 6 +-
.../org/apache/hadoop/log/LogThrottlingHelper.java | 392 +
.../apache/hadoop/metrics2/annotation/Metric.java | 5 +
.../hadoop/metrics2/lib/MutableGaugeFloat.java | 7 +
.../hadoop/metrics2/lib/MutableGaugeInt.java | 7 +
.../hadoop/metrics2/lib/MutableGaugeLong.java | 6 +
.../hadoop/metrics2/lib/MutableMetricsFactory.java | 4 +
.../metrics2/lib/MutableRollingAverages.java | 2 +-
.../org/apache/hadoop/metrics2/package-info.java | 3 +
.../apache/hadoop/metrics2/sink/GraphiteSink.java | 8 +-
.../metrics2/sink/RollingFileSystemSink.java | 2 +-
.../apache/hadoop/metrics2/sink/StatsDSink.java | 10 +-
.../org/apache/hadoop/metrics2/util/MBeans.java | 10 +-
.../hadoop/net/AbstractDNSToSwitchMapping.java | 12 +-
.../src/main/java/org/apache/hadoop/net/DNS.java | 2 +-
.../org/apache/hadoop/net/DNSToSwitchMapping.java | 2 +-
.../main/java/org/apache/hadoop/net/NetUtils.java | 18 +-
.../org/apache/hadoop/net/NetworkTopology.java | 37 +-
.../hadoop/net/NetworkTopologyWithNodeGroup.java | 4 +-
.../org/apache/hadoop/net/ScriptBasedMapping.java | 8 +-
.../net/ScriptBasedMappingWithDependency.java | 7 +-
.../org/apache/hadoop/net/SocketOutputStream.java | 6 +-
.../security/AuthenticationFilterInitializer.java | 6 +-
.../org/apache/hadoop/security/Credentials.java | 16 +
.../apache/hadoop/security/HadoopKerberosName.java | 13 +-
.../hadoop/security/IdMappingServiceProvider.java | 5 +-
.../java/org/apache/hadoop/security/KDiag.java | 5 +-
.../apache/hadoop/security/LdapGroupsMapping.java | 186 +-
.../org/apache/hadoop/security/ProviderUtils.java | 4 +-
.../hadoop/security/SaslPropertiesResolver.java | 4 +-
.../org/apache/hadoop/security/SaslRpcClient.java | 10 +-
.../org/apache/hadoop/security/SecurityUtil.java | 39 +-
.../hadoop/security/UserGroupInformation.java | 212 +-
.../hadoop/security/alias/CredentialProvider.java | 6 +-
.../hadoop/security/alias/CredentialShell.java | 12 +-
.../security/alias/JavaKeyStoreProvider.java | 6 +-
.../security/alias/LocalJavaKeyStoreProvider.java | 3 +-
.../security/authorize/AccessControlList.java | 6 +-
.../hadoop/security/authorize/ProxyUsers.java | 4 +-
.../security/ssl/FileBasedKeyStoresFactory.java | 4 +-
.../org/apache/hadoop/security/ssl/SSLFactory.java | 6 +-
.../hadoop/security/ssl/SSLHostnameVerifier.java | 26 +-
.../security/token/DelegationTokenIssuer.java | 111 +
.../hadoop/security/token/DtFileOperations.java | 53 +-
.../apache/hadoop/security/token/DtUtilShell.java | 37 +-
.../org/apache/hadoop/security/token/Token.java | 75 +-
.../AbstractDelegationTokenSecretManager.java | 12 +-
.../delegation/ZKDelegationTokenSecretManager.java | 2 +-
.../web/DelegationTokenAuthenticatedURL.java | 39 +-
.../web/DelegationTokenAuthenticationFilter.java | 6 +-
.../web/DelegationTokenAuthenticationHandler.java | 4 +-
.../web/DelegationTokenAuthenticator.java | 3 +-
.../delegation/web/DelegationTokenManager.java | 2 +-
...rberosDelegationTokenAuthenticationHandler.java | 2 +-
.../web/KerberosDelegationTokenAuthenticator.java | 2 +-
...SchemeDelegationTokenAuthenticationHandler.java | 3 +-
...PseudoDelegationTokenAuthenticationHandler.java | 2 +-
.../web/PseudoDelegationTokenAuthenticator.java | 2 +-
.../apache/hadoop/service/ServiceOperations.java | 4 +-
.../service/launcher/InterruptEscalator.java | 6 +-
.../hadoop/service/launcher/ServiceLauncher.java | 2 +-
.../hadoop/service/launcher/package-info.java | 2 +-
.../org/apache/hadoop/tools/GetGroupsBase.java | 4 +-
.../util/BlockingThreadPoolExecutorService.java | 6 +-
.../java/org/apache/hadoop/util/ClassUtil.java | 1 -
.../java/org/apache/hadoop/util/CleanerUtil.java | 199 +
.../org/apache/hadoop/util/ComparableVersion.java | 2 +-
.../org/apache/hadoop/util/CpuTimeTracker.java | 12 +-
.../java/org/apache/hadoop/util/FindClass.java | 4 +-
.../apache/hadoop/util/GenericOptionsParser.java | 21 +-
.../org/apache/hadoop/util/HttpExceptionUtils.java | 6 +-
.../org/apache/hadoop/util/JsonSerialization.java | 2 +-
.../main/java/org/apache/hadoop/util/KMSUtil.java | 13 +-
.../java/org/apache/hadoop/util/LambdaUtils.java | 59 +
.../org/apache/hadoop/util/LightWeightCache.java | 10 +-
.../java/org/apache/hadoop/util/LineReader.java | 3 -
.../java/org/apache/hadoop/util/MachineList.java | 2 +-
.../java/org/apache/hadoop/util/ProtoUtil.java | 13 +
.../hadoop/util/SemaphoredDelegatingExecutor.java | 8 +-
.../main/java/org/apache/hadoop/util/Shell.java | 14 +-
.../apache/hadoop/util/ShutdownHookManager.java | 2 +-
.../apache/hadoop/util/ShutdownThreadsHelper.java | 2 -
.../java/org/apache/hadoop/util/SignalLogger.java | 4 +-
.../java/org/apache/hadoop/util/StringUtils.java | 8 +-
.../src/main/java/org/apache/hadoop/util/Tool.java | 2 +-
.../main/java/org/apache/hadoop/util/ZKUtil.java | 4 +-
.../hadoop/util/bloom/DynamicBloomFilter.java | 9 +-
.../apache/hadoop/util/concurrent/AsyncGet.java | 2 +-
.../hadoop/util/curator/ZKCuratorManager.java | 10 +-
.../org/apache/hadoop/util/hash/JenkinsHash.java | 4 +-
.../hadoop/io/compress/zstd/ZStandardCompressor.c | 21 +-
.../io/compress/zstd/ZStandardDecompressor.c | 12 +-
.../apache/hadoop/io/erasurecode/jni_xor_decoder.c | 2 +
.../src/main/proto/HAServiceProtocol.proto | 20 +
.../hadoop-common/src/main/proto/RpcHeader.proto | 2 +
.../hadoop-common/src/main/proto/Security.proto | 1 +
.../src/main/resources/core-default.xml | 352 +-
.../src/site/markdown/CommandsManual.md | 1 +
.../src/site/markdown/CredentialProviderAPI.md | 130 +-
.../src/site/markdown/FairCallQueue.md | 150 +
.../src/site/markdown/GroupsMapping.md | 54 +-
.../hadoop-common/src/site/markdown/Metrics.md | 25 +
.../hadoop-common/src/site/markdown/SecureMode.md | 25 +-
.../src/site/markdown/SingleCluster.md.vm | 3 -
.../src/site/markdown/filesystem/filesystem.md | 98 +-
.../site/markdown/filesystem/fsdatainputstream.md | 14 +
.../filesystem/fsdatainputstreambuilder.md | 112 +
.../filesystem/fsdataoutputstreambuilder.md | 6 +-
.../src/site/markdown/filesystem/index.md | 1 +
.../site/markdown/filesystem/multipartuploader.md | 235 +
.../src/site/markdown/registry/hadoop-registry.md | 1018 +
.../src/site/markdown/registry/index.md | 31 +
.../markdown/registry/registry-configuration.md | 397 +
.../src/site/markdown/registry/registry-dns.md | 224 +
.../site/markdown/registry/registry-security.md | 0
.../registry/using-the-hadoop-service-registry.md | 273 +
.../0.1.0/{CHANGES.0.1.0.md => CHANGELOG.0.1.0.md} | 0
.../0.1.1/{CHANGES.0.1.1.md => CHANGELOG.0.1.1.md} | 0
.../{CHANGES.0.10.0.md => CHANGELOG.0.10.0.md} | 0
.../{CHANGES.0.10.1.md => CHANGELOG.0.10.1.md} | 0
.../{CHANGES.0.11.0.md => CHANGELOG.0.11.0.md} | 0
.../{CHANGES.0.11.1.md => CHANGELOG.0.11.1.md} | 0
.../{CHANGES.0.11.2.md => CHANGELOG.0.11.2.md} | 0
.../{CHANGES.0.12.0.md => CHANGELOG.0.12.0.md} | 0
.../{CHANGES.0.12.1.md => CHANGELOG.0.12.1.md} | 0
.../{CHANGES.0.12.2.md => CHANGELOG.0.12.2.md} | 0
.../{CHANGES.0.12.3.md => CHANGELOG.0.12.3.md} | 0
.../{CHANGES.0.13.0.md => CHANGELOG.0.13.0.md} | 0
.../{CHANGES.0.14.0.md => CHANGELOG.0.14.0.md} | 0
.../{CHANGES.0.14.1.md => CHANGELOG.0.14.1.md} | 0
.../{CHANGES.0.14.2.md => CHANGELOG.0.14.2.md} | 0
.../{CHANGES.0.14.3.md => CHANGELOG.0.14.3.md} | 0
.../{CHANGES.0.14.4.md => CHANGELOG.0.14.4.md} | 0
.../{CHANGES.0.15.0.md => CHANGELOG.0.15.0.md} | 0
.../{CHANGES.0.15.1.md => CHANGELOG.0.15.1.md} | 0
.../{CHANGES.0.15.2.md => CHANGELOG.0.15.2.md} | 0
.../{CHANGES.0.15.3.md => CHANGELOG.0.15.3.md} | 0
.../markdown/release/0.15.4/CHANGELOG.0.15.4.md | 31 +
.../site/markdown/release/0.15.4/CHANGES.0.15.4.md | 31 -
.../{CHANGES.0.16.0.md => CHANGELOG.0.16.0.md} | 0
.../{CHANGES.0.16.1.md => CHANGELOG.0.16.1.md} | 0
.../{CHANGES.0.16.2.md => CHANGELOG.0.16.2.md} | 0
.../{CHANGES.0.16.3.md => CHANGELOG.0.16.3.md} | 0
.../{CHANGES.0.16.4.md => CHANGELOG.0.16.4.md} | 0
.../{CHANGES.0.17.0.md => CHANGELOG.0.17.0.md} | 0
.../{CHANGES.0.17.1.md => CHANGELOG.0.17.1.md} | 0
.../{CHANGES.0.17.2.md => CHANGELOG.0.17.2.md} | 0
.../markdown/release/0.17.3/CHANGELOG.0.17.3.md | 42 +
.../site/markdown/release/0.17.3/CHANGES.0.17.3.md | 42 -
.../{CHANGES.0.18.0.md => CHANGELOG.0.18.0.md} | 0
.../{CHANGES.0.18.1.md => CHANGELOG.0.18.1.md} | 0
.../{CHANGES.0.18.2.md => CHANGELOG.0.18.2.md} | 0
.../{CHANGES.0.18.3.md => CHANGELOG.0.18.3.md} | 0
.../markdown/release/0.18.4/CHANGELOG.0.18.4.md | 45 +
.../site/markdown/release/0.18.4/CHANGES.0.18.4.md | 45 -
.../{CHANGES.0.19.0.md => CHANGELOG.0.19.0.md} | 0
.../{CHANGES.0.19.1.md => CHANGELOG.0.19.1.md} | 0
.../{CHANGES.0.19.2.md => CHANGELOG.0.19.2.md} | 0
.../0.2.0/{CHANGES.0.2.0.md => CHANGELOG.0.2.0.md} | 0
.../0.2.1/{CHANGES.0.2.1.md => CHANGELOG.0.2.1.md} | 0
.../{CHANGES.0.20.0.md => CHANGELOG.0.20.0.md} | 0
.../markdown/release/0.20.1/CHANGELOG.0.20.1.md | 127 +
.../site/markdown/release/0.20.1/CHANGES.0.20.1.md | 127 -
.../{CHANGES.0.20.2.md => CHANGELOG.0.20.2.md} | 0
...ANGES.0.20.203.0.md => CHANGELOG.0.20.203.0.md} | 0
.../release/0.20.203.1/CHANGELOG.0.20.203.1.md | 32 +
.../release/0.20.203.1/CHANGES.0.20.203.1.md | 32 -
...ANGES.0.20.204.0.md => CHANGELOG.0.20.204.0.md} | 0
...ANGES.0.20.205.0.md => CHANGELOG.0.20.205.0.md} | 0
.../markdown/release/0.20.3/CHANGELOG.0.20.3.md | 91 +
.../site/markdown/release/0.20.3/CHANGES.0.20.3.md | 91 -
.../{CHANGES.0.21.0.md => CHANGELOG.0.21.0.md} | 0
.../markdown/release/0.21.1/CHANGELOG.0.21.1.md | 123 +
.../site/markdown/release/0.21.1/CHANGES.0.21.1.md | 123 -
.../{CHANGES.0.22.0.md => CHANGELOG.0.22.0.md} | 0
.../markdown/release/0.22.1/CHANGELOG.0.22.1.md | 107 +
.../site/markdown/release/0.22.1/CHANGES.0.22.1.md | 107 -
.../{CHANGES.0.23.0.md => CHANGELOG.0.23.0.md} | 0
.../{CHANGES.0.23.1.md => CHANGELOG.0.23.1.md} | 0
.../markdown/release/0.23.10/CHANGELOG.0.23.10.md | 121 +
.../markdown/release/0.23.10/CHANGES.0.23.10.md | 120 -
.../{CHANGES.0.23.11.md => CHANGELOG.0.23.11.md} | 0
.../markdown/release/0.23.2/CHANGELOG.0.23.2.md | 177 +
.../site/markdown/release/0.23.2/CHANGES.0.23.2.md | 177 -
.../{CHANGES.0.23.3.md => CHANGELOG.0.23.3.md} | 0
.../{CHANGES.0.23.4.md => CHANGELOG.0.23.4.md} | 0
.../{CHANGES.0.23.5.md => CHANGELOG.0.23.5.md} | 0
.../markdown/release/0.23.6/CHANGELOG.0.23.6.md | 127 +
.../site/markdown/release/0.23.6/CHANGES.0.23.6.md | 127 -
.../{CHANGES.0.23.7.md => CHANGELOG.0.23.7.md} | 0
.../{CHANGES.0.23.8.md => CHANGELOG.0.23.8.md} | 0
.../{CHANGES.0.23.9.md => CHANGELOG.0.23.9.md} | 0
.../markdown/release/0.24.0/CHANGELOG.0.24.0.md | 89 +
.../site/markdown/release/0.24.0/CHANGES.0.24.0.md | 89 -
.../0.3.0/{CHANGES.0.3.0.md => CHANGELOG.0.3.0.md} | 0
.../0.3.1/{CHANGES.0.3.1.md => CHANGELOG.0.3.1.md} | 0
.../0.3.2/{CHANGES.0.3.2.md => CHANGELOG.0.3.2.md} | 0
.../0.4.0/{CHANGES.0.4.0.md => CHANGELOG.0.4.0.md} | 0
.../0.5.0/{CHANGES.0.5.0.md => CHANGELOG.0.5.0.md} | 0
.../0.6.0/{CHANGES.0.6.0.md => CHANGELOG.0.6.0.md} | 0
.../0.6.1/{CHANGES.0.6.1.md => CHANGELOG.0.6.1.md} | 0
.../0.6.2/{CHANGES.0.6.2.md => CHANGELOG.0.6.2.md} | 0
.../0.7.0/{CHANGES.0.7.0.md => CHANGELOG.0.7.0.md} | 0
.../0.7.1/{CHANGES.0.7.1.md => CHANGELOG.0.7.1.md} | 0
.../0.7.2/{CHANGES.0.7.2.md => CHANGELOG.0.7.2.md} | 0
.../0.8.0/{CHANGES.0.8.0.md => CHANGELOG.0.8.0.md} | 0
.../0.9.0/{CHANGES.0.9.0.md => CHANGELOG.0.9.0.md} | 0
.../0.9.1/{CHANGES.0.9.1.md => CHANGELOG.0.9.1.md} | 0
.../0.9.2/{CHANGES.0.9.2.md => CHANGELOG.0.9.2.md} | 0
.../1.0.0/{CHANGES.1.0.0.md => CHANGELOG.1.0.0.md} | 0
.../1.0.1/{CHANGES.1.0.1.md => CHANGELOG.1.0.1.md} | 0
.../1.0.2/{CHANGES.1.0.2.md => CHANGELOG.1.0.2.md} | 0
.../1.0.3/{CHANGES.1.0.3.md => CHANGELOG.1.0.3.md} | 0
.../1.0.4/{CHANGES.1.0.4.md => CHANGELOG.1.0.4.md} | 0
.../1.1.0/{CHANGES.1.1.0.md => CHANGELOG.1.1.0.md} | 0
.../1.1.1/{CHANGES.1.1.1.md => CHANGELOG.1.1.1.md} | 0
.../1.1.2/{CHANGES.1.1.2.md => CHANGELOG.1.1.2.md} | 0
.../site/markdown/release/1.1.3/CHANGELOG.1.1.3.md | 31 +
.../site/markdown/release/1.1.3/CHANGES.1.1.3.md | 31 -
.../1.2.0/{CHANGES.1.2.0.md => CHANGELOG.1.2.0.md} | 0
.../1.2.1/{CHANGES.1.2.1.md => CHANGELOG.1.2.1.md} | 0
.../site/markdown/release/1.2.2/CHANGELOG.1.2.2.md | 32 +
.../site/markdown/release/1.2.2/CHANGES.1.2.2.md | 32 -
.../site/markdown/release/1.3.0/CHANGELOG.1.3.0.md | 139 +
.../site/markdown/release/1.3.0/CHANGES.1.3.0.md | 139 -
...GES.2.0.0-alpha.md => CHANGELOG.2.0.0-alpha.md} | 0
...GES.2.0.1-alpha.md => CHANGELOG.2.0.1-alpha.md} | 0
...GES.2.0.2-alpha.md => CHANGELOG.2.0.2-alpha.md} | 0
.../release/2.0.3-alpha/CHANGELOG.2.0.3-alpha.md | 588 +
.../release/2.0.3-alpha/CHANGES.2.0.3-alpha.md | 588 -
...GES.2.0.4-alpha.md => CHANGELOG.2.0.4-alpha.md} | 0
...GES.2.0.5-alpha.md => CHANGELOG.2.0.5-alpha.md} | 0
...GES.2.0.6-alpha.md => CHANGELOG.2.0.6-alpha.md} | 0
...ANGES.2.1.0-beta.md => CHANGELOG.2.1.0-beta.md} | 0
...ANGES.2.1.1-beta.md => CHANGELOG.2.1.1-beta.md} | 0
.../markdown/release/2.10.0/CHANGELOG.2.10.0.md | 484 +
.../markdown/release/2.10.0/RELEASENOTES.2.10.0.md | 115 +
.../2.2.0/{CHANGES.2.2.0.md => CHANGELOG.2.2.0.md} | 0
.../site/markdown/release/2.2.1/CHANGELOG.2.2.1.md | 38 +
.../site/markdown/release/2.2.1/CHANGES.2.2.1.md | 38 -
.../2.3.0/{CHANGES.2.3.0.md => CHANGELOG.2.3.0.md} | 0
.../site/markdown/release/2.4.0/CHANGELOG.2.4.0.md | 487 +
.../site/markdown/release/2.4.0/CHANGES.2.4.0.md | 487 -
.../2.4.1/{CHANGES.2.4.1.md => CHANGELOG.2.4.1.md} | 0
.../site/markdown/release/2.5.0/CHANGELOG.2.5.0.md | 551 +
.../site/markdown/release/2.5.0/CHANGES.2.5.0.md | 551 -
.../2.5.1/{CHANGES.2.5.1.md => CHANGELOG.2.5.1.md} | 0
.../2.5.2/{CHANGES.2.5.2.md => CHANGELOG.2.5.2.md} | 0
.../site/markdown/release/2.6.0/CHANGELOG.2.6.0.md | 964 +
.../site/markdown/release/2.6.0/CHANGES.2.6.0.md | 964 -
.../site/markdown/release/2.6.1/CHANGELOG.2.6.1.md | 202 +
.../site/markdown/release/2.6.1/CHANGES.2.6.1.md | 202 -
.../2.6.2/{CHANGES.2.6.2.md => CHANGELOG.2.6.2.md} | 0
.../2.6.3/{CHANGES.2.6.3.md => CHANGELOG.2.6.3.md} | 0
.../site/markdown/release/2.6.4/CHANGELOG.2.6.4.md | 98 +
.../site/markdown/release/2.6.4/CHANGES.2.6.4.md | 98 -
.../2.6.5/{CHANGES.2.6.5.md => CHANGELOG.2.6.5.md} | 0
.../site/markdown/release/2.6.6/CHANGELOG.2.6.6.md | 57 +
.../site/markdown/release/2.6.6/CHANGES.2.6.6.md | 47 -
.../site/markdown/release/2.7.0/CHANGELOG.2.7.0.md | 988 +
.../site/markdown/release/2.7.0/CHANGES.2.7.0.md | 988 -
.../markdown/release/2.7.0/RELEASENOTES.2.7.0.md | 7 +
.../site/markdown/release/2.7.1/CHANGELOG.2.7.1.md | 175 +
.../site/markdown/release/2.7.1/CHANGES.2.7.1.md | 182 -
.../markdown/release/2.7.1/RELEASENOTES.2.7.1.md | 7 -
.../site/markdown/release/2.7.2/CHANGELOG.2.7.2.md | 208 +
.../site/markdown/release/2.7.2/CHANGES.2.7.2.md | 208 -
.../site/markdown/release/2.7.3/CHANGELOG.2.7.3.md | 284 +
.../site/markdown/release/2.7.3/CHANGES.2.7.3.md | 284 -
.../site/markdown/release/2.7.4/CHANGELOG.2.7.4.md | 326 +
.../site/markdown/release/2.7.4/CHANGES.2.7.4.md | 326 -
.../markdown/release/2.7.4/RELEASENOTES.2.7.4.md | 14 +
.../site/markdown/release/2.7.5/CHANGELOG.2.7.5.md | 93 +
.../site/markdown/release/2.7.5/CHANGES.2.7.5.md | 32 -
.../markdown/release/2.7.5/RELEASENOTES.2.7.5.md | 12 +
.../site/markdown/release/2.7.6/CHANGELOG.2.7.6.md | 92 +
.../markdown/release/2.7.6/RELEASENOTES.2.7.6.md | 42 +
.../site/markdown/release/2.7.7/CHANGELOG.2.7.7.md | 47 +
.../markdown/release/2.7.7/RELEASENOTES.2.7.7.md | 21 +
.../site/markdown/release/2.7.8/CHANGELOG.2.7.8.md | 31 +
.../markdown/release/2.7.8/RELEASENOTES.2.7.8.md | 21 +
.../site/markdown/release/2.8.0/CHANGELOG.2.8.0.md | 2999 ++
.../site/markdown/release/2.8.0/CHANGES.2.8.0.md | 2993 --
.../markdown/release/2.8.0/RELEASENOTES.2.8.0.md | 7 +
.../2.8.1/{CHANGES.2.8.1.md => CHANGELOG.2.8.1.md} | 0
.../site/markdown/release/2.8.2/CHANGELOG.2.8.2.md | 374 +
.../site/markdown/release/2.8.2/CHANGES.2.8.2.md | 385 -
.../markdown/release/2.8.2/RELEASENOTES.2.8.2.md | 9 +-
.../site/markdown/release/2.8.3/CHANGELOG.2.8.3.md | 129 +
.../site/markdown/release/2.8.3/CHANGES.2.8.3.md | 69 -
.../markdown/release/2.8.3/RELEASENOTES.2.8.3.md | 17 +
.../site/markdown/release/2.8.4/CHANGELOG.2.8.4.md | 123 +
.../markdown/release/2.8.4/RELEASENOTES.2.8.4.md | 21 +
.../site/markdown/release/2.8.5/CHANGELOG.2.8.5.md | 70 +
.../markdown/release/2.8.5/RELEASENOTES.2.8.5.md | 21 +
.../site/markdown/release/2.9.0/CHANGELOG.2.9.0.md | 2013 ++
.../site/markdown/release/2.9.0/CHANGES.2.9.0.md | 1453 -
.../markdown/release/2.9.0/RELEASENOTES.2.9.0.md | 189 +-
.../site/markdown/release/2.9.1/CHANGELOG.2.9.1.md | 279 +
.../site/markdown/release/2.9.1/CHANGES.2.9.1.md | 277 -
.../markdown/release/2.9.1/RELEASENOTES.2.9.1.md | 2 +-
.../site/markdown/release/2.9.2/CHANGELOG.2.9.2.md | 274 +
.../markdown/release/2.9.2/RELEASENOTES.2.9.2.md | 31 +
.../release/3.0.0-alpha1/CHANGELOG.3.0.0-alpha1.md | 3751 ++
.../release/3.0.0-alpha1/CHANGES.3.0.0-alpha1.md | 3750 --
.../3.0.0-alpha1/RELEASENOTES.3.0.0-alpha1.md | 14 +
.../release/3.0.0-alpha2/CHANGELOG.3.0.0-alpha2.md | 922 +
.../release/3.0.0-alpha2/CHANGES.3.0.0-alpha2.md | 919 -
.../3.0.0-alpha2/RELEASENOTES.3.0.0-alpha2.md | 17 +-
.../release/3.0.0-alpha3/CHANGELOG.3.0.0-alpha3.md | 46 +
.../release/3.0.0-alpha3/CHANGES.3.0.0-alpha3.md | 46 -
.../release/3.0.0-alpha4/CHANGELOG.3.0.0-alpha4.md | 878 +
.../release/3.0.0-alpha4/CHANGES.3.0.0-alpha4.md | 887 -
.../3.0.0-alpha4/RELEASENOTES.3.0.0-alpha4.md | 12 +-
.../release/3.0.0-beta1/CHANGELOG.3.0.0-beta1.md | 649 +
.../release/3.0.0-beta1/CHANGES.3.0.0-beta1.md | 646 -
.../3.0.0-beta1/RELEASENOTES.3.0.0-beta1.md | 13 +
.../site/markdown/release/3.0.0/CHANGELOG.3.0.0.md | 363 +
.../site/markdown/release/3.0.0/CHANGES.3.0.0.md | 360 -
.../markdown/release/3.0.0/RELEASENOTES.3.0.0.md | 7 -
.../site/markdown/release/3.0.1/CHANGELOG.3.0.1.md | 231 +
.../site/markdown/release/3.0.1/CHANGES.3.0.1.md | 241 -
.../site/markdown/release/3.0.2/CHANGELOG.3.0.2.md | 31 +
.../site/markdown/release/3.0.2/CHANGES.3.0.2.md | 31 -
.../site/markdown/release/3.0.3/CHANGELOG.3.0.3.md | 312 +
.../site/markdown/release/3.0.3/CHANGES.3.0.3.md | 309 -
.../markdown/release/3.0.3/RELEASENOTES.3.0.3.md | 7 +
.../site/markdown/release/3.0.4/CHANGELOG.3.0.4.md | 189 +
.../markdown/release/3.0.4/RELEASENOTES.3.0.4.md | 50 +
.../site/markdown/release/3.1.0/CHANGELOG.3.1.0.md | 1042 +
.../site/markdown/release/3.1.0/CHANGES.3.1.0.md | 1022 -
.../markdown/release/3.1.0/RELEASENOTES.3.1.0.md | 7 +
.../site/markdown/release/3.1.1/CHANGELOG.3.1.1.md | 502 +
.../site/markdown/release/3.1.1/CHANGES.3.1.1.md | 498 -
.../markdown/release/3.1.1/RELEASENOTES.3.1.1.md | 539 +-
.../site/markdown/release/3.1.2/CHANGES.3.1.2.md | 382 +
.../markdown/release/3.1.2/RELEASENOTES.3.1.2.md | 88 +
.../site/markdown/release/3.2.0/CHANGELOG.3.2.0.md | 1170 +
.../markdown/release/3.2.0/RELEASENOTES.3.2.0.md | 224 +
.../resources/images/faircallqueue-overview.png | Bin 0 -> 47397 bytes
.../org/apache/hadoop/cli/util/CLICommand.java | 6 +-
.../org/apache/hadoop/cli/util/CLICommandFS.java | 4 +-
.../apache/hadoop/cli/util/CLICommandTypes.java | 6 +-
.../hadoop/conf/TestCommonConfigurationFields.java | 6 +-
.../org/apache/hadoop/conf/TestConfigRedactor.java | 7 +
.../org/apache/hadoop/conf/TestConfiguration.java | 100 +
.../hadoop/conf/TestConfigurationDeprecation.java | 89 +-
.../apache/hadoop/conf/TestReconfiguration.java | 6 +-
.../apache/hadoop/crypto/key/TestKeyProvider.java | 32 +-
.../TestKeyProviderDelegationTokenExtension.java | 20 +-
.../crypto/key/kms/TestKMSClientProvider.java | 138 +
.../kms/TestLoadBalancingKMSClientProvider.java | 108 +-
.../fs/FileContextMainOperationsBaseTest.java | 95 +-
.../src/test/java/org/apache/hadoop/fs/TestDU.java | 3 +
.../org/apache/hadoop/fs/TestFileSystemTokens.java | 1 -
.../org/apache/hadoop/fs/TestFilterFileSystem.java | 4 +-
.../java/org/apache/hadoop/fs/TestFsShell.java | 2 +-
.../org/apache/hadoop/fs/TestHarFileSystem.java | 25 +
.../org/apache/hadoop/fs/TestLocalFileSystem.java | 2 +-
.../test/java/org/apache/hadoop/fs/TestTrash.java | 6 +
.../fs/contract/AbstractContractConcatTest.java | 34 +-
.../AbstractContractGetFileStatusTest.java | 30 +-
.../fs/contract/AbstractContractMkdirTest.java | 8 +
.../AbstractContractMultipartUploaderTest.java | 561 +-
.../fs/contract/AbstractContractOpenTest.java | 135 +-
.../contract/AbstractContractPathHandleTest.java | 61 +
.../hadoop/fs/contract/AbstractFSContract.java | 2 -
.../hadoop/fs/contract/ContractTestUtils.java | 50 +-
.../TestLocalFSContractMultipartUploader.java | 10 +
.../apache/hadoop/fs/ftp/TestFTPFileSystem.java | 15 +
.../apache/hadoop/fs/sftp/TestSFTPFileSystem.java | 79 +-
.../java/org/apache/hadoop/fs/shell/TestCopy.java | 4 +-
.../hadoop/fs/shell/TestCopyPreserveFlag.java | 10 +
.../java/org/apache/hadoop/fs/shell/TestLs.java | 69 +-
.../java/org/apache/hadoop/fs/shell/TestMove.java | 2 +-
.../java/org/apache/hadoop/fs/shell/TestTail.java | 57 +
.../hadoop/fs/viewfs/ViewFileSystemBaseTest.java | 5 +
.../apache/hadoop/fs/viewfs/ViewFsBaseTest.java | 13 +-
.../java/org/apache/hadoop/ha/DummyHAService.java | 18 +-
.../java/org/apache/hadoop/ha/MiniZKFCCluster.java | 4 +
.../apache/hadoop/ha/TestActiveStandbyElector.java | 23 +-
.../org/apache/hadoop/http/TestHttpServer.java | 14 +
.../apache/hadoop/http/TestIsActiveServlet.java | 95 +
.../org/apache/hadoop/http/TestSSLHttpServer.java | 204 +-
.../hadoop/http/lib/TestStaticUserWebFilter.java | 3 +-
.../io/compress/TestBlockDecompressorStream.java | 29 +
.../zstd/TestZStandardCompressorDecompressor.java | 10 +-
.../rawcoder/RawErasureCoderBenchmark.java | 4 +-
.../hadoop/io/retry/TestDefaultRetryPolicy.java | 4 +-
.../org/apache/hadoop/io/retry/TestRetryProxy.java | 6 +-
.../org/apache/hadoop/ipc/TestFairCallQueue.java | 42 +-
.../test/java/org/apache/hadoop/ipc/TestIPC.java | 126 +-
.../test/java/org/apache/hadoop/ipc/TestRPC.java | 14 +-
.../java/org/apache/hadoop/ipc/TestSaslRPC.java | 4 +-
.../java/org/apache/hadoop/ipc/TestServer.java | 5 +-
.../apache/hadoop/log/TestLogThrottlingHelper.java | 175 +
.../hadoop/metrics2/impl/TestGraphiteMetrics.java | 4 +-
.../metrics2/impl/TestMetricsSystemImpl.java | 2 +-
.../hadoop/metrics2/impl/TestMetricsVisitor.java | 2 +-
.../hadoop/metrics2/lib/TestMutableMetrics.java | 4 +-
.../metrics2/lib/TestMutableRollingAverages.java | 4 +-
.../java/org/apache/hadoop/net/StaticMapping.java | 4 +-
.../org/apache/hadoop/net/TestClusterTopology.java | 35 +
.../hadoop/security/TestAuthenticationFilter.java | 10 +-
.../apache/hadoop/security/TestCredentials.java | 57 +-
.../hadoop/security/TestLdapGroupsMapping.java | 126 +-
.../hadoop/security/TestLdapGroupsMappingBase.java | 76 +-
.../TestLdapGroupsMappingWithFailover.java | 142 +
.../TestLdapGroupsMappingWithOneQuery.java | 22 +-
.../TestLdapGroupsMappingWithPosixGroup.java | 12 +-
.../security/TestRuleBasedLdapGroupsMapping.java | 4 +-
.../hadoop/security/TestUGILoginFromKeytab.java | 56 +
.../hadoop/security/TestUserGroupInformation.java | 29 +-
.../security/http/TestXFrameOptionsFilter.java | 16 +-
.../hadoop/security/ssl/KeyStoreTestUtil.java | 126 +-
.../apache/hadoop/security/ssl/TestSSLFactory.java | 3 +-
.../hadoop/security/token/TestDtUtilShell.java | 44 +
.../apache/hadoop/security/token/TestToken.java | 22 +-
.../hadoop/service/TestServiceOperations.java | 2 +-
.../org/apache/hadoop/test/LambdaTestUtils.java | 221 +
.../org/apache/hadoop/test/MetricsAsserts.java | 21 +-
.../apache/hadoop/test/TestLambdaTestUtils.java | 114 +-
.../org/apache/hadoop/tracing/SetSpanReceiver.java | 2 +-
.../java/org/apache/hadoop/util/TestRunJar.java | 2 +-
.../org/apache/hadoop/util/TestStringUtils.java | 9 +
.../hadoop/util/curator/TestZKCuratorManager.java | 23 +
.../hadoop-common/src/test/resources/testConf.xml | 6 +-
hadoop-common-project/hadoop-kms/pom.xml | 11 +-
.../key/kms/server/KMSAuthenticationFilter.java | 23 +-
.../crypto/key/kms/server/KMSJSONReader.java | 4 +-
.../hadoop-kms/src/main/resources/kms-default.xml | 7 +
.../hadoop/crypto/key/kms/server/TestKMS.java | 413 +-
.../kms/server/TestKMSAuthenticationFilter.java | 48 +
hadoop-common-project/hadoop-minikdc/pom.xml | 4 +-
hadoop-common-project/hadoop-nfs/pom.xml | 6 +-
.../dev-support/findbugs-exclude.xml | 33 +
hadoop-common-project/hadoop-registry/pom.xml | 309 +
.../apache/hadoop/registry/cli/RegistryCli.java | 0
.../hadoop/registry/client/api/BindFlags.java | 0
.../hadoop/registry/client/api/DNSOperations.java | 0
.../registry/client/api/DNSOperationsFactory.java | 0
.../registry/client/api/RegistryConstants.java | 388 +
.../registry/client/api/RegistryOperations.java | 0
.../client/api/RegistryOperationsFactory.java | 0
.../hadoop/registry/client/api/package-info.java | 0
.../registry/client/binding/JsonSerDeser.java | 0
.../registry/client/binding/RegistryPathUtils.java | 0
.../registry/client/binding/RegistryTypeUtils.java | 0
.../registry/client/binding/RegistryUtils.java | 0
.../registry/client/binding/package-info.java | 0
.../exceptions/AuthenticationFailedException.java | 0
.../exceptions/InvalidPathnameException.java | 0
.../client/exceptions/InvalidRecordException.java | 0
.../NoChildrenForEphemeralsException.java | 0
.../exceptions/NoPathPermissionsException.java | 0
.../client/exceptions/NoRecordException.java | 0
.../client/exceptions/RegistryIOException.java | 0
.../registry/client/exceptions/package-info.java | 0
.../client/impl/FSRegistryOperationsService.java | 248 +
.../client/impl/RegistryOperationsClient.java | 0
.../hadoop/registry/client/impl/package-info.java | 0
.../client/impl/zk/BindingInformation.java | 0
.../registry/client/impl/zk/CuratorService.java | 0
.../registry/client/impl/zk/ListenerHandle.java | 0
.../registry/client/impl/zk/PathListener.java | 0
.../client/impl/zk/RegistryBindingSource.java | 0
.../client/impl/zk/RegistryInternalConstants.java | 0
.../client/impl/zk/RegistryOperationsService.java | 0
.../registry/client/impl/zk/RegistrySecurity.java | 0
.../registry/client/impl/zk/ZKPathDumper.java | 0
.../client/impl/zk/ZookeeperConfigOptions.java | 0
.../registry/client/impl/zk/package-info.java | 0
.../hadoop/registry/client/types/AddressTypes.java | 0
.../hadoop/registry/client/types/Endpoint.java | 0
.../registry/client/types/ProtocolTypes.java | 0
.../registry/client/types/RegistryPathStatus.java | 0
.../registry/client/types/ServiceRecord.java | 0
.../hadoop/registry/client/types/package-info.java | 0
.../client/types/yarn/PersistencePolicies.java | 0
.../client/types/yarn/YarnRegistryAttributes.java | 0
.../registry/conf/RegistryConfiguration.java | 41 +
.../apache/hadoop/registry/conf/package-info.java | 22 +
.../dns/ApplicationServiceRecordProcessor.java | 0
.../server/dns/BaseServiceRecordProcessor.java | 0
.../dns/ContainerServiceRecordProcessor.java | 0
.../hadoop/registry/server/dns/LookupTask.java | 0
.../server/dns/PrivilegedRegistryDNSStarter.java | 81 +
.../registry/server/dns/RecordCreatorFactory.java | 0
.../hadoop/registry/server/dns/RegistryDNS.java | 1800 +
.../registry/server/dns/RegistryDNSServer.java | 267 +
.../registry/server/dns/ReverseZoneUtils.java | 171 +
.../hadoop/registry/server/dns/SecureableZone.java | 0
.../server/dns/ServiceRecordProcessor.java | 0
.../hadoop/registry/server/dns/ZoneSelector.java | 0
.../hadoop/registry/server/dns/package-info.java | 0
.../integration/SelectByYarnPersistence.java | 0
.../registry/server/integration/package-info.java | 0
.../hadoop/registry/server/package-info.java | 0
.../server/services/AddingCompositeService.java | 0
.../server/services/DeleteCompletionCallback.java | 0
.../server/services/MicroZookeeperService.java | 0
.../server/services/MicroZookeeperServiceKeys.java | 0
.../server/services/RegistryAdminService.java | 0
.../registry/server/services/package-info.java | 0
.../src/main/tla/hadoopregistry.tla | 582 +
.../hadoop/registry/AbstractRegistryTest.java | 0
.../hadoop/registry/AbstractZKRegistryTest.java | 113 +
.../apache/hadoop/registry/RegistryTestHelper.java | 0
.../hadoop/registry/cli/TestRegistryCli.java | 0
.../registry/client/binding/TestMarshalling.java | 0
.../client/binding/TestRegistryOperationUtils.java | 0
.../client/binding/TestRegistryPathUtils.java | 0
.../registry/client/impl/CuratorEventCatcher.java | 0
.../registry/client/impl/TestCuratorService.java | 0
.../impl/TestFSRegistryOperationsService.java | 0
.../client/impl/TestMicroZookeeperService.java | 61 +
.../integration/TestYarnPolicySelector.java | 0
.../operations/TestRegistryOperations.java | 0
.../secure/AbstractSecureRegistryTest.java | 369 +
.../registry/secure/KerberosConfiguration.java | 0
.../secure/TestRegistrySecurityHelper.java | 0
.../hadoop/registry/secure/TestSecureLogins.java | 241 +
.../hadoop/registry/secure/TestSecureRegistry.java | 0
.../registry/server/dns/TestRegistryDNS.java | 0
.../registry/server/dns/TestReverseZoneUtils.java | 0
.../registry/server/dns/TestSecureRegistryDNS.java | 0
.../src/test/resources/0.17.172.in-addr.arpa.zone | 0
.../src/test/resources/log4j.properties | 0
.../src/test/resources/test.private | 0
hadoop-common-project/pom.xml | 5 +-
hadoop-dist/pom.xml | 122 +-
.../src/main/compose/ozone/docker-compose.yaml | 49 -
hadoop-dist/src/main/compose/ozone/docker-config | 33 -
hadoop-dist/src/main/compose/ozoneperf/README.md | 73 -
.../src/main/compose/ozoneperf/compose-all.sh | 18 -
.../compose/ozoneperf/docker-compose-freon.yaml | 26 -
.../src/main/compose/ozoneperf/docker-compose.yaml | 64 -
.../src/main/compose/ozoneperf/docker-config | 32 -
hadoop-dist/src/main/compose/ozoneperf/init.sh | 21 -
.../src/main/compose/ozoneperf/prometheus.yml | 24 -
hadoop-hdds/client/pom.xml | 10 +-
.../hdds/scm/ClientCredentialInterceptor.java | 65 +
.../org/apache/hadoop/hdds/scm/XceiverClient.java | 207 -
.../apache/hadoop/hdds/scm/XceiverClientGrpc.java | 319 +-
.../hadoop/hdds/scm/XceiverClientHandler.java | 202 -
.../hadoop/hdds/scm/XceiverClientInitializer.java | 74 -
.../hadoop/hdds/scm/XceiverClientManager.java | 51 +-
.../apache/hadoop/hdds/scm/XceiverClientRatis.java | 299 +-
.../hdds/scm/client/ContainerOperationClient.java | 137 +-
.../hadoop/hdds/scm/client/HddsClientUtils.java | 4 +-
.../hadoop/hdds/scm/storage/BlockInputStream.java | 407 +
.../hadoop/hdds/scm/storage/BlockOutputStream.java | 629 +
.../hadoop/hdds/scm/storage/ChunkInputStream.java | 263 -
.../hadoop/hdds/scm/storage/ChunkOutputStream.java | 235 -
hadoop-hdds/common/pom.xml | 61 +-
.../org/apache/hadoop/hdds/HddsConfigKeys.java | 168 +-
.../java/org/apache/hadoop/hdds/HddsUtils.java | 174 +-
.../org/apache/hadoop/hdds/cli/GenericCli.java | 38 +-
.../hadoop/hdds/cli/GenericParentCommand.java | 29 +
.../hadoop/hdds/cli/HddsVersionProvider.java | 2 +-
.../hdds/cli/MissingSubcommandException.java | 35 +
.../org/apache/hadoop/hdds/client/BlockID.java | 85 +-
.../hadoop/hdds/client/ContainerBlockID.java | 79 +
.../apache/hadoop/hdds/conf/HddsConfServlet.java | 3 +
.../hadoop/hdds/conf/OzoneConfiguration.java | 2 +
.../hadoop/hdds/protocol/SCMSecurityProtocol.java | 55 +
.../SCMSecurityProtocolClientSideTranslatorPB.java | 124 +
.../hdds/protocolPB/SCMSecurityProtocolPB.java | 35 +
.../SCMSecurityProtocolServerSideTranslatorPB.java | 94 +
.../hadoop/hdds/protocolPB/package-info.java | 22 +
.../org/apache/hadoop/hdds/scm/ScmConfigKeys.java | 144 +-
.../apache/hadoop/hdds/scm/XceiverClientReply.java | 73 +
.../apache/hadoop/hdds/scm/XceiverClientSpi.java | 76 +-
.../apache/hadoop/hdds/scm/client/ScmClient.java | 44 +-
.../hdds/scm/container/ContainerException.java | 46 +
.../hadoop/hdds/scm/container/ContainerID.java | 46 +-
.../hadoop/hdds/scm/container/ContainerInfo.java | 477 +
.../scm/container/ContainerNotFoundException.java | 44 +
.../ContainerReplicaNotFoundException.java | 45 +
.../container/common/helpers/AllocatedBlock.java | 35 +-
.../container/common/helpers/ContainerInfo.java | 473 -
.../common/helpers/ContainerNotOpenException.java | 36 +
.../common/helpers/ContainerWithPipeline.java | 11 +-
.../helpers/InvalidContainerStateException.java | 35 +
.../scm/container/common/helpers/Pipeline.java | 275 -
.../scm/container/common/helpers/PipelineID.java | 88 -
.../apache/hadoop/hdds/scm/pipeline/Pipeline.java | 310 +
.../hadoop/hdds/scm/pipeline/PipelineID.java | 80 +
.../scm/pipeline/PipelineNotFoundException.java | 46 +
.../pipeline/UnknownPipelineStateException.java | 46 +
.../hadoop/hdds/scm/pipeline/package-info.java | 24 +
.../hadoop/hdds/scm/protocol/LocatedContainer.java | 2 +-
.../scm/protocol/ScmBlockLocationProtocol.java | 3 +
.../hadoop/hdds/scm/protocol/ScmLocatedBlock.java | 2 +-
.../protocol/StorageContainerLocationProtocol.java | 42 +-
...lockLocationProtocolClientSideTranslatorPB.java | 75 +-
.../scm/protocolPB/ScmBlockLocationProtocolPB.java | 6 +-
...inerLocationProtocolClientSideTranslatorPB.java | 100 +-
.../StorageContainerLocationProtocolPB.java | 4 +
.../hdds/scm/storage/ContainerProtocolCalls.java | 338 +-
.../security/exception/SCMSecurityException.java | 79 +
.../hdds/security/exception/package-info.java | 23 +
.../hdds/security/token/BlockTokenException.java | 53 +
.../hdds/security/token/BlockTokenVerifier.java | 131 +
.../security/token/OzoneBlockTokenIdentifier.java | 199 +
.../security/token/OzoneBlockTokenSelector.java | 75 +
.../hadoop/hdds/security/token/TokenVerifier.java | 38 +
.../hadoop/hdds/security/token/package-info.java | 22 +
.../hadoop/hdds/security/x509/SecurityConfig.java | 462 +
.../x509/certificate/authority/BaseApprover.java | 249 +
.../certificate/authority/CertificateApprover.java | 86 +
.../certificate/authority/CertificateServer.java | 113 +
.../certificate/authority/DefaultApprover.java | 128 +
.../certificate/authority/DefaultCAServer.java | 439 +
.../authority/PKIProfiles/DefaultCAProfile.java | 46 +
.../authority/PKIProfiles/DefaultProfile.java | 333 +
.../authority/PKIProfiles/PKIProfile.java | 140 +
.../authority/PKIProfiles/package-info.java | 33 +
.../x509/certificate/authority/package-info.java | 22 +
.../x509/certificate/client/CertificateClient.java | 173 +
.../certificate/client/DNCertificateClient.java | 40 +
.../client/DefaultCertificateClient.java | 632 +
.../certificate/client/OMCertificateClient.java | 102 +
.../x509/certificate/client/package-info.java | 22 +
.../x509/certificate/utils/CertificateCodec.java | 280 +
.../x509/certificate/utils/package-info.java | 22 +
.../certificates/utils/CertificateSignRequest.java | 276 +
.../certificates/utils/SelfSignedCertificate.java | 238 +
.../x509/certificates/utils/package-info.java | 22 +
.../x509/exceptions/CertificateException.java | 87 +
.../security/x509/exceptions/package-info.java | 23 +
.../hdds/security/x509/keys/HDDSKeyGenerator.java | 118 +
.../hadoop/hdds/security/x509/keys/KeyCodec.java | 411 +
.../hdds/security/x509/keys/SecurityUtil.java | 138 +
.../hdds/security/x509/keys/package-info.java | 23 +
.../hadoop/hdds/security/x509/package-info.java | 99 +
.../hadoop/hdds/tracing/GrpcClientInterceptor.java | 57 +
.../hadoop/hdds/tracing/GrpcServerInterceptor.java | 51 +
.../apache/hadoop/hdds/tracing/StringCodec.java | 89 +
.../apache/hadoop/hdds/tracing/TraceAllMethod.java | 86 +
.../apache/hadoop/hdds/tracing/TracingUtil.java | 112 +
.../apache/hadoop/hdds/tracing/package-info.java | 23 +
.../org/apache/hadoop/ozone/OzoneConfigKeys.java | 181 +-
.../java/org/apache/hadoop/ozone/OzoneConsts.java | 95 +-
.../org/apache/hadoop/ozone/OzoneSecurityUtil.java | 60 +
.../org/apache/hadoop/ozone/audit/AuditLogger.java | 44 +-
.../apache/hadoop/ozone/audit/AuditMessage.java | 77 +-
.../org/apache/hadoop/ozone/audit/Auditable.java | 2 +-
.../org/apache/hadoop/ozone/audit/Auditor.java | 33 +
.../org/apache/hadoop/ozone/audit/DNAction.java | 48 +
.../org/apache/hadoop/ozone/audit/SCMAction.java | 45 +
.../apache/hadoop/ozone/audit/package-info.java | 16 +-
.../org/apache/hadoop/ozone/common/BlockGroup.java | 11 +-
.../org/apache/hadoop/ozone/common/Checksum.java | 249 +
.../apache/hadoop/ozone/common/ChecksumData.java | 190 +
.../ozone/common/OzoneChecksumException.java | 66 +
.../apache/hadoop/ozone/common/StorageInfo.java | 3 +-
.../ozone/container/common/helpers/BlockData.java | 273 +
.../ozone/container/common/helpers/ChunkInfo.java | 33 +-
.../helpers/ContainerCommandRequestPBHelper.java | 192 +
.../ozone/container/common/helpers/KeyData.java | 253 -
.../org/apache/hadoop/ozone/lock/ActiveLock.java | 101 +
.../org/apache/hadoop/ozone/lock/LockManager.java | 101 +
.../hadoop/ozone/lock/PooledLockFactory.java | 43 +
.../org/apache/hadoop/ozone/lock/package-info.java | 21 +
...lockLocationProtocolServerSideTranslatorPB.java | 3 +-
...inerLocationProtocolServerSideTranslatorPB.java | 106 +-
.../org/apache/hadoop/utils/HddsVersionInfo.java | 158 +-
.../apache/hadoop/utils/MetadataKeyFilters.java | 3 +-
.../apache/hadoop/utils/MetadataStoreBuilder.java | 62 +-
.../org/apache/hadoop/utils/RetriableTask.java | 78 +
.../java/org/apache/hadoop/utils/RocksDBStore.java | 7 +-
.../java/org/apache/hadoop/utils/VersionInfo.java | 97 +
.../org/apache/hadoop/utils/db/BatchOperation.java | 27 +
.../apache/hadoop/utils/db/ByteArrayKeyValue.java | 67 +
.../java/org/apache/hadoop/utils/db/Codec.java | 41 +
.../org/apache/hadoop/utils/db/CodecRegistry.java | 89 +
.../apache/hadoop/utils/db/DBConfigFromFile.java | 34 +-
.../java/org/apache/hadoop/utils/db/DBStore.java | 48 +-
.../org/apache/hadoop/utils/db/DBStoreBuilder.java | 9 +-
.../apache/hadoop/utils/db/RDBBatchOperation.java | 69 +
.../java/org/apache/hadoop/utils/db/RDBStore.java | 139 +-
.../apache/hadoop/utils/db/RDBStoreIterator.java | 41 +-
.../java/org/apache/hadoop/utils/db/RDBTable.java | 58 +-
.../org/apache/hadoop/utils/db/StringCodec.java | 45 +
.../java/org/apache/hadoop/utils/db/Table.java | 91 +-
.../org/apache/hadoop/utils/db/TableIterator.java | 18 +-
.../org/apache/hadoop/utils/db/TypedTable.java | 209 +
.../main/java/org/apache/ratis/RatisHelper.java | 147 +-
.../src/main/proto/DatanodeContainerProtocol.proto | 170 +-
.../src/main/proto/SCMSecurityProtocol.proto | 82 +
.../src/main/proto/ScmBlockLocationProtocol.proto | 8 +-
.../proto/StorageContainerLocationProtocol.proto | 76 +-
hadoop-hdds/common/src/main/proto/hdds.proto | 111 +-
.../common/src/main/resources/ozone-default.xml | 925 +-
.../token/TestOzoneBlockTokenIdentifier.java | 313 +
.../hadoop/hdds/security/token/package-info.java | 22 +
.../x509/certificate/authority/MockApprover.java | 57 +
.../certificate/authority/TestDefaultCAServer.java | 169 +
.../certificate/authority/TestDefaultProfile.java | 364 +
.../x509/certificate/authority/package-info.java | 22 +
.../client/TestCertificateClientInit.java | 206 +
.../client/TestDefaultCertificateClient.java | 336 +
.../certificate/utils/TestCertificateCodec.java | 218 +
.../x509/certificate/utils/package-info.java | 23 +
.../certificates/TestCertificateSignRequest.java | 285 +
.../x509/certificates/TestRootCertificate.java | 258 +
.../security/x509/certificates/package-info.java | 22 +
.../security/x509/keys/TestHDDSKeyGenerator.java | 87 +
.../hdds/security/x509/keys/TestKeyCodec.java | 231 +
.../hdds/security/x509/keys/package-info.java | 22 +
.../hadoop/hdds/security/x509/package-info.java | 22 +
.../org/apache/hadoop/ozone/audit/DummyAction.java | 36 +-
.../hadoop/ozone/audit/TestOzoneAuditLogger.java | 89 +-
.../apache/hadoop/ozone/common/TestChecksum.java | 101 +
.../hadoop/ozone/lease/TestLeaseManager.java | 17 +-
.../apache/hadoop/ozone/lock/TestLockManager.java | 64 +
.../org/apache/hadoop/ozone/lock/package-info.java | 21 +
.../org/apache/hadoop/utils/TestHddsIdFactory.java | 2 +-
.../org/apache/hadoop/utils/TestMetadataStore.java | 98 +-
.../org/apache/hadoop/utils/TestRetriableTask.java | 76 +
.../apache/hadoop/utils/TestRocksDBStoreMBean.java | 23 +-
.../apache/hadoop/utils/db/TestDBStoreBuilder.java | 9 +-
.../org/apache/hadoop/utils/db/TestRDBStore.java | 31 +-
.../apache/hadoop/utils/db/TestRDBTableStore.java | 70 +-
.../hadoop/utils/db/TestTypedRDBTableStore.java | 235 +
.../java/org/apache/hadoop/utils/package-info.java | 22 +
.../common/src/test/resources/log4j2.properties | 4 +-
.../dev-support/findbugsExcludeFile.xml | 12 +
hadoop-hdds/container-service/pom.xml | 28 +-
.../org/apache/hadoop/hdds/scm/HddsServerUtil.java | 78 +-
.../apache/hadoop/ozone/HddsDatanodeService.java | 153 +-
.../container/common/helpers/ContainerReport.java | 205 -
.../container/common/helpers/ContainerUtils.java | 2 +-
.../common/helpers/KeyValueContainerReport.java | 117 -
.../ozone/container/common/impl/ContainerData.java | 83 +-
.../container/common/impl/ContainerDataYaml.java | 24 +-
.../ozone/container/common/impl/ContainerSet.java | 9 +-
.../container/common/impl/HddsDispatcher.java | 385 +-
.../common/impl/OpenContainerBlockMap.java | 46 +-
.../container/common/interfaces/Container.java | 39 +-
.../common/interfaces/ContainerDispatcher.java | 15 +-
.../ozone/container/common/interfaces/Handler.java | 111 +-
.../report/CommandStatusReportPublisher.java | 4 +-
.../common/report/ContainerReportPublisher.java | 3 +-
.../common/report/PipelineReportPublisher.java | 73 +
.../common/report/ReportPublisherFactory.java | 4 +
.../common/statemachine/DatanodeStateMachine.java | 75 +-
.../common/statemachine/SCMConnectionManager.java | 7 +-
.../common/statemachine/StateContext.java | 166 +-
.../CloseContainerCommandHandler.java | 102 +-
.../commandhandler/CommandHandler.java | 7 +-
.../commandhandler/DeleteBlocksCommandHandler.java | 47 +-
.../DeleteContainerCommandHandler.java | 85 +
.../ReplicateContainerCommandHandler.java | 50 +-
.../container/common/states/DatanodeState.java | 3 +
.../common/states/datanode/InitDatanodeState.java | 22 +-
.../states/endpoint/HeartbeatEndpointTask.java | 74 +-
.../states/endpoint/RegisterEndpointTask.java | 12 +-
.../states/endpoint/VersionEndpointTask.java | 78 +-
.../transport/server/GrpcXceiverService.java | 21 +-
.../server/ServerCredentialInterceptor.java | 74 +
.../common/transport/server/XceiverServer.java | 139 +-
.../common/transport/server/XceiverServerGrpc.java | 98 +-
.../transport/server/XceiverServerHandler.java | 82 -
.../transport/server/XceiverServerInitializer.java | 64 -
.../common/transport/server/XceiverServerSpi.java | 16 +
.../common/transport/server/ratis/CSMMetrics.java | 5 +-
.../server/ratis/ContainerStateMachine.java | 751 +-
.../transport/server/ratis/DispatcherContext.java | 133 +
.../transport/server/ratis/XceiverServerRatis.java | 532 +-
.../container/common/utils/ContainerCache.java | 6 +-
.../container/common/utils/HddsVolumeUtil.java | 2 +-
.../container/common/volume/AbstractFuture.java | 1298 +
.../ozone/container/common/volume/HddsVolume.java | 51 +-
.../container/common/volume/HddsVolumeChecker.java | 421 +
.../common/volume/ThrottledAsyncChecker.java | 247 +
.../container/common/volume/TimeoutFuture.java | 161 +
.../ozone/container/common/volume/VolumeInfo.java | 21 +-
.../ozone/container/common/volume/VolumeSet.java | 331 +-
.../container/keyvalue/KeyValueBlockIterator.java | 16 +-
.../container/keyvalue/KeyValueContainer.java | 146 +-
.../container/keyvalue/KeyValueContainerData.java | 75 +-
.../ozone/container/keyvalue/KeyValueHandler.java | 635 +-
.../container/keyvalue/helpers/BlockUtils.java | 201 +
.../container/keyvalue/helpers/ChunkUtils.java | 85 +-
.../ozone/container/keyvalue/helpers/KeyUtils.java | 199 -
.../helpers/KeyValueContainerLocationUtil.java | 13 +-
.../keyvalue/helpers/KeyValueContainerUtil.java | 19 +-
.../container/keyvalue/helpers/SmallFileUtils.java | 13 +-
.../container/keyvalue/impl/BlockManagerImpl.java | 282 +
.../container/keyvalue/impl/ChunkManagerImpl.java | 98 +-
.../container/keyvalue/impl/KeyManagerImpl.java | 227 -
.../container/keyvalue/impl/package-info.java | 5 +-
.../keyvalue/interfaces/BlockManager.java | 85 +
.../keyvalue/interfaces/ChunkManager.java | 13 +-
.../container/keyvalue/interfaces/KeyManager.java | 84 -
.../keyvalue/interfaces/package-info.java | 21 +
.../background/BlockDeletingService.java | 12 +-
.../container/ozoneimpl/ContainerController.java | 138 +
.../ozone/container/ozoneimpl/ContainerReader.java | 56 +-
.../ozone/container/ozoneimpl/OzoneContainer.java | 175 +-
.../container/replication/ContainerDownloader.java | 40 +
.../replication/ContainerReplicationSource.java | 49 +
.../container/replication/ContainerReplicator.java | 27 +
.../replication/ContainerStreamingOutput.java | 45 +
.../replication/DownloadAndImportReplicator.java | 136 +
.../replication/GrpcReplicationClient.java | 169 +
.../replication/GrpcReplicationService.java | 129 +
.../OnDemandContainerReplicationSource.java | 76 +
.../replication/ReplicationSupervisor.java | 129 +
.../container/replication/ReplicationTask.java | 102 +
.../replication/SimpleContainerDownloader.java | 120 +
.../ozone/container/replication/package-info.java | 21 +
.../protocol/StorageContainerDatanodeProtocol.java | 27 +-
.../protocol/StorageContainerNodeProtocol.java | 14 +-
.../protocol/commands/CloseContainerCommand.java | 53 +-
.../protocol/commands/CommandForDatanode.java | 9 +-
.../ozone/protocol/commands/CommandStatus.java | 41 +-
.../commands/DeleteBlockCommandStatus.java | 100 +
.../protocol/commands/DeleteBlocksCommand.java | 6 +-
.../protocol/commands/DeleteContainerCommand.java | 86 +
.../commands/ReplicateContainerCommand.java | 4 -
.../ozone/protocol/commands/ReregisterCommand.java | 11 +-
.../hadoop/ozone/protocol/commands/SCMCommand.java | 2 +-
...inerDatanodeProtocolClientSideTranslatorPB.java | 23 +-
.../StorageContainerDatanodeProtocolPB.java | 6 +
...inerDatanodeProtocolServerSideTranslatorPB.java | 21 +-
.../proto/StorageContainerDatanodeProtocol.proto | 102 +-
.../hadoop/ozone/TestHddsDatanodeService.java | 96 +
.../ozone/container/common/SCMTestUtils.java | 25 +-
.../hadoop/ozone/container/common/ScmTestMock.java | 50 +-
.../container/common/TestDatanodeStateMachine.java | 57 +-
.../common/TestKeyValueContainerData.java | 13 +-
.../common/impl/TestContainerDataYaml.java | 21 +-
.../container/common/impl/TestContainerSet.java | 23 +-
.../container/common/impl/TestHddsDispatcher.java | 161 +-
.../container/common/interfaces/TestHandler.java | 29 +-
.../common/report/TestReportPublisher.java | 19 +-
.../TestCloseContainerCommandHandler.java | 296 +
.../statemachine/commandhandler/package-info.java | 22 +
.../states/endpoint/TestHeartbeatEndpointTask.java | 12 +-
.../container/common/volume/TestHddsVolume.java | 63 +-
.../common/volume/TestHddsVolumeChecker.java | 212 +
.../container/common/volume/TestVolumeSet.java | 10 +-
.../common/volume/TestVolumeSetDiskChecks.java | 182 +
.../container/common/volume/package-info.java | 22 +
.../container/keyvalue/TestBlockManagerImpl.java | 197 +
.../container/keyvalue/TestChunkManagerImpl.java | 85 +-
.../container/keyvalue/TestKeyManagerImpl.java | 191 -
.../keyvalue/TestKeyValueBlockIterator.java | 37 +-
.../container/keyvalue/TestKeyValueContainer.java | 69 +-
.../TestKeyValueContainerMarkUnhealthy.java | 172 +
.../container/keyvalue/TestKeyValueHandler.java | 114 +-
.../TestKeyValueHandlerWithUnhealthyContainer.java | 231 +
.../container/keyvalue/TestTarContainerPacker.java | 15 +-
.../ozone/container/keyvalue/package-info.java | 22 +
.../container/ozoneimpl/TestOzoneContainer.java | 18 +-
.../replication/TestReplicationSupervisor.java | 136 +
.../ozone/container/replication/package-info.java | 22 +
.../src/test/resources/additionalfields.container | 4 +-
.../test/resources/incorrect.checksum.container | 2 +
.../src/test/resources/incorrect.container | 4 +-
hadoop-hdds/docs/README.md | 55 +
hadoop-hdds/docs/archetypes/default.md | 20 +
hadoop-hdds/docs/config.yaml | 46 +
hadoop-hdds/docs/content/AuditParser.md | 72 +
hadoop-hdds/docs/content/BucketCommands.md | 131 +
hadoop-hdds/docs/content/BuildingSources.md | 54 +
hadoop-hdds/docs/content/CommandShell.md | 108 +
hadoop-hdds/docs/content/Concepts.md | 108 +
hadoop-hdds/docs/content/Dozone.md | 109 +
hadoop-hdds/docs/content/Freon.md | 64 +
hadoop-hdds/docs/content/Hdds.md | 65 +
hadoop-hdds/docs/content/JavaApi.md | 172 +
hadoop-hdds/docs/content/KeyCommands.md | 144 +
hadoop-hdds/docs/content/OzoneFS.md | 95 +
hadoop-hdds/docs/content/OzoneManager.md | 77 +
hadoop-hdds/docs/content/Prometheus.md | 94 +
hadoop-hdds/docs/content/RealCluster.md | 74 +
hadoop-hdds/docs/content/Rest.md | 544 +
hadoop-hdds/docs/content/RunningViaDocker.md | 73 +
hadoop-hdds/docs/content/RunningWithHDFS.md | 77 +
hadoop-hdds/docs/content/S3.md | 130 +
hadoop-hdds/docs/content/S3Commands.md | 41 +
hadoop-hdds/docs/content/SCMCLI.md | 29 +
hadoop-hdds/docs/content/Settings.md | 142 +
hadoop-hdds/docs/content/VolumeCommands.md | 116 +
hadoop-hdds/docs/content/_index.md | 38 +
hadoop-hdds/docs/content/genconf.md | 28 +
hadoop-hdds/docs/dev-support/bin/generate-site.sh | 29 +
hadoop-hdds/docs/pom.xml | 81 +
hadoop-hdds/docs/static/NOTES.md | 20 +
.../docs/static/OzoneOverview.png | Bin
.../docs/static/OzoneOverview.svg | 0
.../docs/static/SCMBlockDiagram.png | Bin
hadoop-hdds/docs/static/ozone-logo.png | Bin 0 -> 40237 bytes
.../themes/ozonedoc/layouts/_default/single.html | 38 +
.../docs/themes/ozonedoc/layouts/index.html | 37 +
.../themes/ozonedoc/layouts/partials/footer.html | 22 +
.../themes/ozonedoc/layouts/partials/header.html | 34 +
.../themes/ozonedoc/layouts/partials/navbar.html | 37 +
.../themes/ozonedoc/layouts/partials/sidebar.html | 58 +
.../ozonedoc/static/css/bootstrap-theme.min.css | 0
.../static/css/bootstrap-theme.min.css.map | 0
.../themes/ozonedoc/static/css/bootstrap.min.css | 0
.../ozonedoc/static/css/bootstrap.min.css.map | 0
.../docs/themes/ozonedoc/static/css/ozonedoc.css | 138 +
.../static/fonts/glyphicons-halflings-regular.eot | Bin
.../static}/fonts/glyphicons-halflings-regular.svg | 0
.../static/fonts/glyphicons-halflings-regular.ttf | Bin
.../static/fonts/glyphicons-halflings-regular.woff | Bin
.../fonts/glyphicons-halflings-regular.woff2 | Bin
.../themes/ozonedoc/static/js/bootstrap.min.js | 0
.../docs/themes/ozonedoc/static/js/jquery.min.js | 0
.../docs/themes/ozonedoc/static/js/ozonedoc.js | 0
.../docs/themes/ozonedoc/theme.toml | 0
hadoop-hdds/framework/pom.xml | 10 +-
.../apache/hadoop/hdds/server/BaseHttpServer.java | 28 +-
.../hadoop/hdds/server/PrometheusMetricsSink.java | 133 +
.../hadoop/hdds/server/PrometheusServlet.java | 46 +
.../org/apache/hadoop/hdds/server/ServerUtils.java | 49 +-
.../hadoop/hdds/server/ServiceRuntimeInfoImpl.java | 15 +-
.../hadoop/hdds/server/events/EventQueue.java | 25 +-
.../hadoop/hdds/server/events/EventWatcher.java | 32 +-
.../hdds/server/TestPrometheusMetricsSink.java | 96 +
.../hdds/server/events/TestEventWatcher.java | 49 +-
.../hadoop/hdds/server/events/package-info.java | 22 +
hadoop-hdds/pom.xml | 224 +-
hadoop-hdds/server-scm/pom.xml | 73 +-
.../java/org/apache/hadoop/hdds/scm/ScmUtils.java | 82 +
.../hadoop/hdds/scm/block/BlockManagerImpl.java | 426 +-
.../block/DatanodeDeletedBlockTransactions.java | 58 +-
.../hadoop/hdds/scm/block/DeletedBlockLogImpl.java | 289 +-
.../hdds/scm/block/PendingDeleteHandler.java | 3 +
.../hdds/scm/block/PendingDeleteStatusList.java | 10 +-
.../hdds/scm/block/SCMBlockDeletingService.java | 17 +-
.../hdds/scm/chillmode/ChillModeExitRule.java | 32 +
.../hdds/scm/chillmode/ChillModePrecheck.java | 69 +
.../hdds/scm/chillmode/ChillModeRestrictedOps.java | 41 +
.../hdds/scm/chillmode/ContainerChillModeRule.java | 111 +
.../hdds/scm/chillmode/DataNodeChillModeRule.java | 83 +
.../hdds/scm/chillmode/PipelineChillModeRule.java | 108 +
.../apache/hadoop/hdds/scm/chillmode/Precheck.java | 29 +
.../hdds/scm/chillmode/SCMChillModeManager.java | 175 +
.../hadoop/hdds/scm/chillmode/package-info.java | 18 +
.../scm/command/CommandStatusReportHandler.java | 30 +-
.../scm/container/CloseContainerEventHandler.java | 111 +-
.../hdds/scm/container/ContainerManager.java | 155 +
.../hdds/scm/container/ContainerMapping.java | 772 -
.../hdds/scm/container/ContainerReplica.java | 231 +
.../hdds/scm/container/ContainerReportHandler.java | 218 +-
.../hdds/scm/container/ContainerStateManager.java | 498 +-
.../container/DeleteContainerCommandWatcher.java | 56 +
.../IncrementalContainerReportHandler.java | 80 +
.../apache/hadoop/hdds/scm/container/Mapping.java | 138 -
.../hdds/scm/container/ReportHandlerHelper.java | 365 +
.../hdds/scm/container/SCMContainerManager.java | 432 +
.../hdds/scm/container/closer/ContainerCloser.java | 194 -
.../placement/algorithms/SCMCommonPolicy.java | 10 +-
.../algorithms/SCMContainerPlacementRandom.java | 1 +
.../placement/metrics/DatanodeMetric.java | 3 +-
.../container/placement/metrics/LongMetric.java | 1 -
.../container/placement/metrics/SCMNodeMetric.java | 5 +-
.../container/placement/metrics/SCMNodeStat.java | 10 +-
.../replication/ReplicationActivityStatus.java | 56 +-
.../container/replication/ReplicationManager.java | 210 +-
.../scm/container/states/ContainerQueryKey.java | 110 +
.../hdds/scm/container/states/ContainerState.java | 32 +-
.../scm/container/states/ContainerStateMap.java | 441 +-
.../apache/hadoop/hdds/scm/events/SCMEvents.java | 95 +-
.../hadoop/hdds/scm/exceptions/SCMException.java | 3 +-
.../hadoop/hdds/scm/exceptions/package-info.java | 29 +-
.../metadata/DeletedBlocksTransactionCodec.java | 46 +
.../apache/hadoop/hdds/scm/metadata/LongCodec.java | 39 +
.../hadoop/hdds/scm/metadata/SCMMetadataStore.java | 73 +
.../hdds/scm/metadata/SCMMetadataStoreRDBImpl.java | 148 +
.../hadoop/hdds/scm/metadata/package-info.java | 21 +
.../apache/hadoop/hdds/scm/node/CommandQueue.java | 8 +-
.../apache/hadoop/hdds/scm/node/DatanodeInfo.java | 7 +-
.../hadoop/hdds/scm/node/DeadNodeHandler.java | 114 +-
.../hadoop/hdds/scm/node/NewNodeHandler.java | 23 +-
.../apache/hadoop/hdds/scm/node/NodeManager.java | 96 +-
.../hadoop/hdds/scm/node/NodeManagerMXBean.java | 26 +-
.../hadoop/hdds/scm/node/NodeReportHandler.java | 2 +-
.../hadoop/hdds/scm/node/NodeStateManager.java | 92 +-
.../scm/node/NonHealthyToHealthyNodeHandler.java | 48 +
.../hadoop/hdds/scm/node/SCMNodeManager.java | 400 +-
.../hdds/scm/node/SCMNodeStorageStatMap.java | 2 +-
.../hadoop/hdds/scm/node/StaleNodeHandler.java | 42 +-
.../hdds/scm/node/states/Node2ContainerMap.java | 142 +-
.../hdds/scm/node/states/Node2ObjectsMap.java | 164 +
.../hdds/scm/node/states/Node2PipelineMap.java | 77 +
.../hadoop/hdds/scm/node/states/NodeStateMap.java | 77 +-
.../hadoop/hdds/scm/node/states/ReportResult.java | 105 +-
.../hdds/scm/pipeline/PipelineActionHandler.java | 74 +
.../hadoop/hdds/scm/pipeline/PipelineFactory.java | 57 +
.../hadoop/hdds/scm/pipeline/PipelineManager.java | 70 +
.../hadoop/hdds/scm/pipeline/PipelineProvider.java | 35 +
.../hdds/scm/pipeline/PipelineReportHandler.java | 107 +
.../hdds/scm/pipeline/PipelineStateManager.java | 131 +
.../hadoop/hdds/scm/pipeline/PipelineStateMap.java | 299 +
.../hdds/scm/pipeline/RatisPipelineProvider.java | 138 +
.../hdds/scm/pipeline/RatisPipelineUtils.java | 275 +
.../hdds/scm/pipeline/SCMPipelineManager.java | 274 +
.../hdds/scm/pipeline/SimplePipelineProvider.java | 75 +
.../hadoop/hdds/scm/pipeline/package-info.java | 24 +
.../hdds/scm/pipelines/Node2PipelineMap.java | 124 -
.../hadoop/hdds/scm/pipelines/PipelineManager.java | 207 -
.../hdds/scm/pipelines/PipelineSelector.java | 477 -
.../hadoop/hdds/scm/pipelines/package-info.java | 38 -
.../hdds/scm/pipelines/ratis/RatisManagerImpl.java | 144 -
.../hdds/scm/pipelines/ratis/package-info.java | 18 -
.../standalone/StandaloneManagerImpl.java | 137 -
.../scm/pipelines/standalone/package-info.java | 18 -
.../hdds/scm/server/SCMBlockProtocolServer.java | 108 +-
.../hdds/scm/server/SCMClientProtocolServer.java | 330 +-
.../hadoop/hdds/scm/server/SCMConfigurator.java | 202 +
.../scm/server/SCMDatanodeHeartbeatDispatcher.java | 143 +-
.../hdds/scm/server/SCMDatanodeProtocolServer.java | 186 +-
.../apache/hadoop/hdds/scm/server/SCMMXBean.java | 17 +
.../hdds/scm/server/SCMSecurityProtocolServer.java | 168 +
.../apache/hadoop/hdds/scm/server/SCMStorage.java | 73 -
.../hadoop/hdds/scm/server/SCMStorageConfig.java | 73 +
.../hdds/scm/server/StorageContainerManager.java | 604 +-
.../server/StorageContainerManagerHttpServer.java | 5 +-
.../commands/RetriableDatanodeEventWatcher.java | 58 +
.../ozone/protocol/commands/package-info.java | 18 +
.../main/{ => resources}/webapps/scm/index.html | 0
.../src/main/{ => resources}/webapps/scm/main.html | 0
.../main/resources/webapps/scm/scm-overview.html | 60 +
.../src/main/resources/webapps/scm/scm.js | 58 +
.../src/main/webapps/scm/scm-overview.html | 60 -
hadoop-hdds/server-scm/src/main/webapps/scm/scm.js | 54 -
.../org/apache/hadoop/hdds/scm/HddsTestUtils.java | 110 +
.../hadoop/hdds/scm/HddsWhiteboxTestUtils.java | 103 +
.../hadoop/hdds/scm/TestHddsServerUtils.java | 206 +
.../scm/TestStorageContainerManagerHttpServer.java | 1 +
.../java/org/apache/hadoop/hdds/scm/TestUtils.java | 125 +-
.../hadoop/hdds/scm/block/TestBlockManager.java | 266 +-
.../hadoop/hdds/scm/block/TestDeletedBlockLog.java | 118 +-
.../scm/chillmode/TestSCMChillModeManager.java | 278 +
.../command/TestCommandStatusReportHandler.java | 30 +-
.../hadoop/hdds/scm/container/MockNodeManager.java | 191 +-
.../container/TestCloseContainerEventHandler.java | 72 +-
.../hdds/scm/container/TestContainerMapping.java | 385 -
.../scm/container/TestContainerReportHandler.java | 799 +-
.../scm/container/TestContainerReportHelper.java | 113 +
.../scm/container/TestContainerStateManager.java | 128 +
.../TestIncrementalContainerReportHandler.java | 218 +
.../scm/container/TestSCMContainerManager.java | 231 +
.../scm/container/closer/TestContainerCloser.java | 228 -
.../TestSCMContainerPlacementCapacity.java | 3 +
.../TestSCMContainerPlacementRandom.java | 3 +
.../replication/TestReplicationActivityStatus.java | 63 +
.../replication/TestReplicationManager.java | 204 +-
.../hdds/scm/node/TestContainerPlacement.java | 30 +-
.../hadoop/hdds/scm/node/TestDeadNodeHandler.java | 332 +
.../hadoop/hdds/scm/node/TestNodeManager.java | 1142 -
.../hdds/scm/node/TestNodeReportHandler.java | 9 +-
.../hadoop/hdds/scm/node/TestSCMNodeManager.java | 980 +
.../hdds/scm/node/TestSCMNodeStorageStatMap.java | 16 +-
.../scm/node/states/TestNode2ContainerMap.java | 39 +-
.../scm/server/TestSCMClientProtocolServer.java | 60 +
.../server/TestSCMDatanodeHeartbeatDispatcher.java | 43 +-
.../scm/server/TestSCMSecurityProtocolServer.java | 60 +
.../ozone/container/common/TestEndPoint.java | 48 +-
.../container/placement/TestDatanodeMetrics.java | 15 +-
.../testutils/ReplicationNodeManagerMock.java | 149 +-
hadoop-hdds/tools/pom.xml | 12 +-
.../hdds/scm/cli/ChillModeCheckSubcommand.java | 61 +
.../hadoop/hdds/scm/cli/ChillModeCommands.java | 60 +
.../hdds/scm/cli/ChillModeExitSubcommand.java | 57 +
.../org/apache/hadoop/hdds/scm/cli/SCMCLI.java | 9 +-
.../hdds/scm/cli/container/InfoSubcommand.java | 13 +-
.../hdds/scm/cli/container/ListSubcommand.java | 4 +-
.../scm/cli/pipeline/ClosePipelineSubcommand.java | 53 +
.../scm/cli/pipeline/ListPipelinesSubcommand.java | 48 +
.../hadoop/hdds/scm/cli/pipeline/package-info.java | 22 +
hadoop-hdfs-project/hadoop-hdfs-client/pom.xml | 6 +-
.../src/main/java/org/apache/hadoop/fs/Hdfs.java | 5 +
.../org/apache/hadoop/fs/HdfsBlockLocation.java | 4 +
.../src/main/java/org/apache/hadoop/fs/XAttr.java | 2 +-
.../java/org/apache/hadoop/hdfs/AddBlockFlag.java | 8 +-
.../org/apache/hadoop/hdfs/ClientGSIContext.java | 90 +
.../java/org/apache/hadoop/hdfs/DFSClient.java | 83 +-
.../hadoop/hdfs/DFSInotifyEventInputStream.java | 2 +-
.../org/apache/hadoop/hdfs/DFSInputStream.java | 20 +-
.../apache/hadoop/hdfs/DFSOpsCountStatistics.java | 23 +
.../org/apache/hadoop/hdfs/DFSOutputStream.java | 27 +-
.../java/org/apache/hadoop/hdfs/DFSPacket.java | 16 +-
.../apache/hadoop/hdfs/DFSStripedInputStream.java | 22 +
.../apache/hadoop/hdfs/DFSStripedOutputStream.java | 17 +-
.../java/org/apache/hadoop/hdfs/DFSUtilClient.java | 91 +-
.../java/org/apache/hadoop/hdfs/DataStreamer.java | 1 +
.../apache/hadoop/hdfs/DistributedFileSystem.java | 132 +-
.../java/org/apache/hadoop/hdfs/HdfsKMSUtil.java | 60 +-
.../apache/hadoop/hdfs/NameNodeProxiesClient.java | 26 +-
.../java/org/apache/hadoop/hdfs/PeerCache.java | 8 +-
.../org/apache/hadoop/hdfs/ReaderStrategy.java | 15 -
.../java/org/apache/hadoop/hdfs/StripeReader.java | 23 +-
.../org/apache/hadoop/hdfs/client/HdfsAdmin.java | 12 +-
.../hadoop/hdfs/client/HdfsClientConfigKeys.java | 8 +
.../hdfs/client/impl/BlockReaderFactory.java | 12 +-
.../hdfs/client/impl/BlockReaderLocalLegacy.java | 2 +-
.../hadoop/hdfs/client/impl/DfsClientConf.java | 8 +
.../hadoop/hdfs/client/impl/LeaseRenewer.java | 2 +-
.../java/org/apache/hadoop/hdfs/inotify/Event.java | 14 +
.../hdfs/protocol/ClientDatanodeProtocol.java | 2 +-
.../hadoop/hdfs/protocol/ClientProtocol.java | 110 +-
.../apache/hadoop/hdfs/protocol/DatanodeInfo.java | 83 +-
.../hadoop/hdfs/protocol/HdfsPathHandle.java | 4 +-
.../hdfs/protocol/NoECPolicySetException.java | 37 +
.../hdfs/protocol/QuotaExceededException.java | 5 +-
.../hdfs/protocol/ReconfigurationProtocol.java | 2 +-
.../hadoop/hdfs/protocol/ReencryptionStatus.java | 12 +-
.../hadoop/hdfs/protocol/SnapshotDiffReport.java | 65 +
.../ClientNamenodeProtocolTranslatorPB.java | 45 +-
.../hadoop/hdfs/protocolPB/PBHelperClient.java | 76 +-
.../security/token/block/BlockTokenIdentifier.java | 4 +-
.../server/datanode/ReplicaNotFoundException.java | 8 +-
.../ha/AbstractNNFailoverProxyProvider.java | 19 +-
.../server/namenode/ha/ClientHAProxyFactory.java | 12 +
.../hdfs/server/namenode/ha/HAProxyFactory.java | 9 +
.../namenode/ha/IPFailoverProxyProvider.java | 2 +-
.../namenode/ha/ObserverReadProxyProvider.java | 357 +
.../ObserverReadProxyProviderWithIPFailover.java | 142 +
.../hadoop/hdfs/server/namenode/ha/ReadOnly.java | 57 +
.../namenode/ha/RequestHedgingProxyProvider.java | 209 +-
.../hdfs/shortcircuit/DfsClientShmManager.java | 5 +-
.../hdfs/shortcircuit/ShortCircuitCache.java | 4 +-
.../hdfs/shortcircuit/ShortCircuitReplica.java | 2 +-
.../apache/hadoop/hdfs/util/ByteArrayManager.java | 4 +-
.../hadoop/hdfs/util/CombinedHostsFileReader.java | 41 +-
.../org/apache/hadoop/hdfs/util/IOUtilsClient.java | 13 +-
.../org/apache/hadoop/hdfs/util/LongBitFormat.java | 8 +
.../apache/hadoop/hdfs/util/StripedBlockUtil.java | 124 +-
.../org/apache/hadoop/hdfs/web/JsonUtilClient.java | 20 +-
.../apache/hadoop/hdfs/web/WebHdfsFileSystem.java | 79 +-
.../hadoop/hdfs/web/resources/ECPolicyParam.java | 42 +
.../hadoop/hdfs/web/resources/GetOpParam.java | 14 +-
.../hadoop/hdfs/web/resources/PostOpParam.java | 2 +
.../hadoop/hdfs/web/resources/PutOpParam.java | 5 +
.../src/main/proto/ClientNamenodeProtocol.proto | 19 +
.../src/main/proto/inotify.proto | 1 +
.../apache/hadoop/hdfs/protocol/TestReadOnly.java | 103 +
.../ha/TestRequestHedgingProxyProvider.java | 103 +-
.../hadoop/hdfs/web/TestByteRangeInputStream.java | 4 +-
.../apache/hadoop/hdfs/web/TestTokenAspect.java | 9 +-
hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml | 8 +-
.../hadoop/fs/http/client/HttpFSFileSystem.java | 50 +-
.../apache/hadoop/fs/http/server/FSOperations.java | 161 +
.../fs/http/server/HttpFSParametersProvider.java | 43 +-
.../apache/hadoop/fs/http/server/HttpFSServer.java | 162 +-
.../src/main/resources/httpfs-default.xml | 7 +
.../hadoop/fs/http/client/BaseTestHttpFSWith.java | 384 +-
.../hadoop/fs/http/server/TestHttpFSServer.java | 438 +-
.../fs/http/server/TestHttpFSServerWebServer.java | 11 +
.../org/apache/hadoop/test/TestHdfsHelper.java | 24 +-
.../hadoop-hdfs-native-client/pom.xml | 6 +-
.../hadoop-hdfs-native-client/src/CMakeLists.txt | 34 +-
.../src/main/native/libhdfs/exception.c | 2 +-
.../native/libhdfs/os/posix/thread_local_storage.c | 65 +-
.../libhdfs/os/windows/thread_local_storage.c | 63 +-
.../src/main/native/libhdfspp/CMakeLists.txt | 4 +-
.../main/native/libhdfspp/lib/proto/CMakeLists.txt | 1 +
hadoop-hdfs-project/hadoop-hdfs-nfs/pom.xml | 8 +-
.../apache/hadoop/hdfs/nfs/nfs3/OpenFileCtx.java | 4 +-
.../hadoop/hdfs/nfs/nfs3/OpenFileCtxCache.java | 6 +-
hadoop-hdfs-project/hadoop-hdfs-rbf/pom.xml | 6 +-
.../RouterAdminProtocolServerSideTranslatorPB.java | 2 +-
.../federation/metrics/NamenodeBeanMetrics.java | 5 +
.../resolver/ActiveNamenodeResolver.java | 11 +-
.../resolver/FederationNamenodeServiceState.java | 3 +
.../resolver/FileSubclusterResolver.java | 1 +
.../federation/resolver/MountTableResolver.java | 41 +-
.../MultipleDestinationMountTableResolver.java | 4 +-
.../federation/resolver/NamenodeStatusReport.java | 5 +-
.../server/federation/resolver/PathLocation.java | 6 +-
.../resolver/order/AvailableSpaceResolver.java | 2 +-
.../federation/resolver/order/HashResolver.java | 10 +-
.../federation/resolver/order/LocalResolver.java | 2 +-
.../federation/resolver/order/RandomResolver.java | 22 +-
.../server/federation/resolver/package-info.java | 5 +-
.../federation/router/ConnectionContext.java | 10 +-
.../server/federation/router/ConnectionPool.java | 3 +-
.../federation/router/IsRouterActiveServlet.java | 37 +
.../federation/router/NameserviceManager.java | 9 +
.../hdfs/server/federation/router/Quota.java | 58 +-
.../server/federation/router/RBFConfigKeys.java | 4 +
.../hdfs/server/federation/router/Router.java | 5 +-
.../federation/router/RouterAdminServer.java | 27 +-
.../federation/router/RouterClientProtocol.java | 26 +-
.../server/federation/router/RouterHttpServer.java | 13 +-
.../federation/router/RouterQuotaManager.java | 6 +-
.../server/federation/router/RouterQuotaUsage.java | 8 +-
.../server/federation/router/RouterRpcClient.java | 23 +-
.../server/federation/router/RouterRpcServer.java | 33 +-
.../federation/router/RouterSafemodeService.java | 26 +-
.../federation/router/RouterStateManager.java | 9 +
.../federation/router/RouterWebHdfsMethods.java | 9 +-
.../server/federation/store/CachedRecordStore.java | 4 +-
.../server/federation/store/MembershipStore.java | 16 +-
.../hdfs/server/federation/store/RouterStore.java | 10 +-
.../server/federation/store/StateStoreService.java | 19 +-
.../server/federation/store/StateStoreUtils.java | 4 +
.../store/driver/StateStoreRecordOperations.java | 15 +-
.../store/driver/impl/StateStoreBaseImpl.java | 1 -
.../driver/impl/StateStoreFileSystemImpl.java | 16 +-
.../hdfs/server/federation/store/package-info.java | 12 +-
.../federation/store/records/BaseRecord.java | 4 +-
.../federation/store/records/MembershipState.java | 3 -
.../federation/store/records/MountTable.java | 26 +-
.../server/federation/store/records/Query.java | 2 +-
.../federation/utils/ConsistentHashRing.java | 2 +-
.../hadoop/hdfs/tools/federation/RouterAdmin.java | 15 +
.../src/main/resources/hdfs-rbf-default.xml | 8 +
.../src/main/webapps/router/federationhealth.html | 3 +-
.../src/main/webapps/router/federationhealth.js | 8 +-
.../src/site/markdown/HDFSRouterFederation.md | 32 +-
.../contract/router/web/RouterWebHDFSContract.java | 1 +
.../server/federation/FederationTestUtils.java | 4 +-
.../resolver/TestMountTableResolver.java | 26 +
.../resolver/order/TestAvailableSpaceResolver.java | 2 +-
.../resolver/order/TestLocalResolver.java | 2 +-
.../server/federation/router/TestRouterAdmin.java | 2 +-
.../federation/router/TestRouterAdminCLI.java | 59 +-
.../server/federation/router/TestRouterRpc.java | 2 +-
.../federation/store/driver/TestStateStoreZK.java | 53 +
.../dev-support/jdiff/Apache_Hadoop_HDFS_2.9.2.xml | 312 +
.../dev-support/jdiff/Apache_Hadoop_HDFS_3.1.2.xml | 676 +
.../dev-support/jdiff/Apache_Hadoop_HDFS_3.2.0.xml | 674 +
hadoop-hdfs-project/hadoop-hdfs/pom.xml | 15 +-
.../java/org/apache/hadoop/hdfs/DFSConfigKeys.java | 30 +-
.../main/java/org/apache/hadoop/hdfs/DFSUtil.java | 19 +-
.../java/org/apache/hadoop/hdfs/HdfsDtFetcher.java | 7 +-
.../org/apache/hadoop/hdfs/NameNodeProxies.java | 124 +-
.../org/apache/hadoop/hdfs/SWebHdfsDtFetcher.java | 7 +-
.../org/apache/hadoop/hdfs/WebHdfsDtFetcher.java | 7 +-
.../apache/hadoop/hdfs/net/DomainPeerServer.java | 6 +-
.../org/apache/hadoop/hdfs/net/TcpPeerServer.java | 6 +-
.../hadoop/hdfs/protocol/BlockListAsLongs.java | 2 +-
.../hadoop/hdfs/protocol/CacheDirective.java | 3 +-
...ientNamenodeProtocolServerSideTranslatorPB.java | 51 +-
.../apache/hadoop/hdfs/protocolPB/PBHelper.java | 4 +
.../hadoop/hdfs/qjournal/client/AsyncLogger.java | 7 +
.../hdfs/qjournal/client/AsyncLoggerSet.java | 20 +-
.../hdfs/qjournal/client/IPCLoggerChannel.java | 14 +
.../hdfs/qjournal/client/QuorumJournalManager.java | 145 +-
.../hdfs/qjournal/client/QuorumOutputStream.java | 13 +-
.../hdfs/qjournal/protocol/QJournalProtocol.java | 24 +-
.../QJournalProtocolServerSideTranslatorPB.java | 16 +-
.../protocolPB/QJournalProtocolTranslatorPB.java | 20 +
.../qjournal/server/GetJournalEditServlet.java | 7 +-
.../hadoop/hdfs/qjournal/server/Journal.java | 77 +-
.../hdfs/qjournal/server/JournalMetrics.java | 20 +-
.../hadoop/hdfs/qjournal/server/JournalNode.java | 10 +-
.../hdfs/qjournal/server/JournalNodeRpcServer.java | 12 +-
.../hdfs/qjournal/server/JournalNodeSyncer.java | 25 +-
.../hdfs/qjournal/server/JournaledEditsCache.java | 417 +
.../token/block/BlockPoolTokenSecretManager.java | 34 +-
.../token/block/BlockTokenSecretManager.java | 62 +-
.../delegation/DelegationTokenSecretManager.java | 8 +-
.../hdfs/server/aliasmap/InMemoryAliasMap.java | 221 +-
.../aliasmap/InMemoryLevelDBAliasMapServer.java | 8 +
.../hadoop/hdfs/server/balancer/Balancer.java | 23 +-
.../hadoop/hdfs/server/balancer/Dispatcher.java | 6 +-
.../hdfs/server/balancer/NameNodeConnector.java | 18 +-
.../AvailableSpaceBlockPlacementPolicy.java | 8 +-
.../hdfs/server/blockmanagement/BlockManager.java | 129 +-
.../blockmanagement/BlockManagerSafeMode.java | 27 +-
.../blockmanagement/BlockPlacementPolicy.java | 1 -
.../BlockPlacementPolicyDefault.java | 51 +-
.../blockmanagement/CombinedHostFileManager.java | 6 +-
.../server/blockmanagement/CorruptReplicasMap.java | 2 +-
.../blockmanagement/DatanodeAdminManager.java | 11 +-
.../server/blockmanagement/DatanodeDescriptor.java | 4 +-
.../server/blockmanagement/DatanodeManager.java | 13 +-
.../server/blockmanagement/HeartbeatManager.java | 2 +-
.../server/blockmanagement/HostFileManager.java | 14 +-
.../hdfs/server/blockmanagement/HostSet.java | 8 +-
.../server/blockmanagement/SlowPeerTracker.java | 5 +-
.../hdfs/server/common/ECTopologyVerifier.java | 135 +
.../hdfs/server/common/HdfsServerConstants.java | 4 +-
.../hadoop/hdfs/server/common/JspHelper.java | 14 +-
.../hdfs/server/common/MetricsLoggerTask.java | 6 +-
.../hadoop/hdfs/server/common/TokenVerifier.java | 35 +
.../org/apache/hadoop/hdfs/server/common/Util.java | 10 +-
.../hdfs/server/datanode/BPOfferService.java | 12 +-
.../hdfs/server/datanode/BPServiceActor.java | 40 +-
.../hdfs/server/datanode/BlockPoolManager.java | 15 +-
.../server/datanode/BlockPoolSliceStorage.java | 60 +-
.../hadoop/hdfs/server/datanode/BlockReceiver.java | 83 +-
.../hdfs/server/datanode/BlockRecoveryWorker.java | 15 +-
.../hadoop/hdfs/server/datanode/BlockScanner.java | 6 +-
.../hadoop/hdfs/server/datanode/BlockSender.java | 34 +-
.../apache/hadoop/hdfs/server/datanode/DNConf.java | 2 +-
.../hadoop/hdfs/server/datanode/DataNode.java | 22 +-
.../hadoop/hdfs/server/datanode/DataStorage.java | 4 +-
.../hadoop/hdfs/server/datanode/DataXceiver.java | 3 +-
.../hdfs/server/datanode/DirectoryScanner.java | 621 +-
.../hadoop/hdfs/server/datanode/DiskBalancer.java | 21 +-
.../hdfs/server/datanode/FileIoProvider.java | 3 -
.../datanode/IncrementalBlockReportManager.java | 5 +-
.../server/datanode/LocalReplicaInPipeline.java | 57 +-
.../server/datanode/ProfilingFileIoEvents.java | 7 +-
.../hdfs/server/datanode/ReplicaInPipeline.java | 14 +
.../hdfs/server/datanode/ShortCircuitRegistry.java | 7 +-
.../hadoop/hdfs/server/datanode/VolumeScanner.java | 22 +-
.../server/datanode/checker/AbstractFuture.java | 14 +-
.../hdfs/server/datanode/checker/AsyncChecker.java | 2 +-
.../datanode/checker/DatasetVolumeChecker.java | 2 +-
.../datanode/checker/StorageLocationChecker.java | 2 +-
.../datanode/checker/ThrottledAsyncChecker.java | 8 +-
.../datanode/erasurecode/ErasureCodingWorker.java | 3 +-
.../datanode/erasurecode/StripedBlockReader.java | 14 +-
.../server/datanode/erasurecode/StripedReader.java | 17 +-
.../datanode/erasurecode/StripedReconstructor.java | 3 +-
.../AvailableSpaceVolumeChoosingPolicy.java | 9 +-
.../server/datanode/fsdataset/FsDatasetSpi.java | 12 +-
.../server/datanode/fsdataset/FsVolumeSpi.java | 49 +-
.../fsdataset/RoundRobinVolumeChoosingPolicy.java | 7 +-
.../datanode/fsdataset/impl/BlockPoolSlice.java | 187 +-
.../fsdataset/impl/FsDatasetAsyncDiskService.java | 7 +-
.../datanode/fsdataset/impl/FsDatasetImpl.java | 16 +-
.../datanode/fsdataset/impl/FsDatasetUtil.java | 30 +-
.../datanode/fsdataset/impl/FsVolumeImpl.java | 203 +-
.../datanode/fsdataset/impl/FsVolumeList.java | 5 +-
.../fsdataset/impl/ProvidedVolumeImpl.java | 50 +-
.../impl/RamDiskAsyncLazyPersistService.java | 7 +-
.../fsdataset/impl/RamDiskReplicaTracker.java | 7 +-
.../server/datanode/fsdataset/impl/ReplicaMap.java | 25 +
.../fsdataset/impl/ReservedSpaceCalculator.java | 2 +-
.../server/datanode/metrics/DataNodeMetrics.java | 33 +-
.../server/datanode/metrics/OutlierDetector.java | 3 +-
.../server/datanode/web/DatanodeHttpServer.java | 6 +-
.../web/RestCsrfPreventionFilterHandler.java | 4 +-
.../datanode/web/SimpleHttpProxyHandler.java | 4 +-
.../datanode/web/webhdfs/DataNodeUGIProvider.java | 6 +-
.../datanode/web/webhdfs/ExceptionHandler.java | 4 +-
.../server/datanode/web/webhdfs/HdfsWriter.java | 8 +-
.../datanode/web/webhdfs/WebHdfsHandler.java | 10 +-
.../server/diskbalancer/DiskBalancerException.java | 1 -
.../server/diskbalancer/command/PlanCommand.java | 3 +-
.../datamodel/DiskBalancerCluster.java | 11 +-
.../datamodel/DiskBalancerDataNode.java | 10 +-
.../server/diskbalancer/planner/GreedyPlanner.java | 26 +-
.../server/diskbalancer/planner/package-info.java | 2 +-
.../org/apache/hadoop/hdfs/server/mover/Mover.java | 12 +-
.../hdfs/server/namenode/AclEntryStatusFormat.java | 109 +-
.../hadoop/hdfs/server/namenode/AclStorage.java | 18 +-
.../hadoop/hdfs/server/namenode/CacheManager.java | 80 +-
.../hadoop/hdfs/server/namenode/CachePool.java | 2 -
.../hdfs/server/namenode/CheckpointConf.java | 7 +-
.../hadoop/hdfs/server/namenode/Checkpointer.java | 30 +-
.../namenode/ContentSummaryComputationContext.java | 10 +-
.../hadoop/hdfs/server/namenode/DfsServlet.java | 7 +-
.../server/namenode/ECTopologyVerifierResult.java | 45 +
.../server/namenode/EditLogBackupOutputStream.java | 7 +-
.../server/namenode/EditLogFileInputStream.java | 52 +-
.../server/namenode/EditLogFileOutputStream.java | 11 +-
.../hdfs/server/namenode/EditsDoubleBuffer.java | 7 +-
.../server/namenode/EncryptionZoneManager.java | 49 +-
.../namenode/ErasureCodingPolicyManager.java | 163 +-
.../hdfs/server/namenode/FSDirErasureCodingOp.java | 4 +
.../server/namenode/FSDirStatAndListingOp.java | 9 +
.../hdfs/server/namenode/FSDirWriteFileOp.java | 26 +-
.../hadoop/hdfs/server/namenode/FSDirectory.java | 19 +-
.../hadoop/hdfs/server/namenode/FSEditLog.java | 31 +-
.../hdfs/server/namenode/FSEditLogAsync.java | 69 +-
.../hdfs/server/namenode/FSEditLogLoader.java | 58 +-
.../hadoop/hdfs/server/namenode/FSEditLogOp.java | 734 +-
.../hadoop/hdfs/server/namenode/FSImage.java | 28 +-
.../hadoop/hdfs/server/namenode/FSImageFormat.java | 6 +-
.../hdfs/server/namenode/FSImageFormatPBINode.java | 109 +-
.../server/namenode/FSImageFormatProtobuf.java | 37 +-
.../FSImagePreTransactionalStorageInspector.java | 10 +-
.../FSImageTransactionalStorageInspector.java | 6 +-
.../hadoop/hdfs/server/namenode/FSNamesystem.java | 161 +-
.../hdfs/server/namenode/FSNamesystemLock.java | 127 +-
.../hdfs/server/namenode/FSPermissionChecker.java | 6 +-
.../hdfs/server/namenode/FileJournalManager.java | 7 +-
.../hdfs/server/namenode/GlobalStateIdContext.java | 160 +
.../apache/hadoop/hdfs/server/namenode/INode.java | 10 +-
.../hdfs/server/namenode/INodeReference.java | 6 +-
.../server/namenode/INodeWithAdditionalFields.java | 36 +-
.../hadoop/hdfs/server/namenode/INodesInPath.java | 10 +-
.../hadoop/hdfs/server/namenode/ImageServlet.java | 46 +-
.../namenode/InotifyFSEditLogOpTranslator.java | 3 +
.../server/namenode/IsNameNodeActiveServlet.java | 33 +
.../hdfs/server/namenode/JournalManager.java | 2 +-
.../hadoop/hdfs/server/namenode/JournalSet.java | 22 +-
.../hadoop/hdfs/server/namenode/LeaseManager.java | 2 +-
.../hdfs/server/namenode/MetaRecoveryContext.java | 9 +-
.../server/namenode/NNStorageRetentionManager.java | 6 +-
.../hadoop/hdfs/server/namenode/NNUpgradeUtil.java | 7 +-
.../hadoop/hdfs/server/namenode/NameCache.java | 6 +-
.../hadoop/hdfs/server/namenode/NameNode.java | 97 +-
.../hdfs/server/namenode/NameNodeHttpServer.java | 37 +-
.../server/namenode/NameNodeLayoutVersion.java | 3 +-
.../hdfs/server/namenode/NameNodeMXBean.java | 7 +
.../server/namenode/NameNodeResourceChecker.java | 9 +-
.../hdfs/server/namenode/NameNodeRpcServer.java | 112 +-
.../hadoop/hdfs/server/namenode/NamenodeFsck.java | 166 +-
.../apache/hadoop/hdfs/server/namenode/Quota.java | 5 +-
.../server/namenode/QuotaByStorageTypeEntry.java | 6 +-
.../namenode/RedundantEditLogInputStream.java | 27 +-
.../hdfs/server/namenode/ReencryptionHandler.java | 2 +-
.../hdfs/server/namenode/SecondaryNameNode.java | 16 +-
.../hdfs/server/namenode/SerialNumberManager.java | 152 +-
.../hdfs/server/namenode/SerialNumberMap.java | 43 +-
.../server/namenode/StartupProgressServlet.java | 2 +-
.../hdfs/server/namenode/StoragePolicySummary.java | 13 +-
.../hdfs/server/namenode/TransferFsImage.java | 30 +-
.../hadoop/hdfs/server/namenode/XAttrFormat.java | 95 +-
.../server/namenode/XAttrPermissionFilter.java | 4 +-
.../hadoop/hdfs/server/namenode/XAttrStorage.java | 19 +-
.../hdfs/server/namenode/ha/BootstrapStandby.java | 104 +-
.../hdfs/server/namenode/ha/EditLogTailer.java | 50 +-
.../server/namenode/ha/NameNodeHAProxyFactory.java | 9 +-
.../hdfs/server/namenode/ha/StandbyState.java | 19 +-
.../namenode/metrics/ECBlockGroupsMBean.java | 5 +
.../server/namenode/metrics/NameNodeMetrics.java | 6 +
.../namenode/snapshot/AbstractINodeDiffList.java | 8 +-
.../namenode/snapshot/DiffListBySkipList.java | 9 +-
.../snapshot/DirectorySnapshottableFeature.java | 7 +-
.../server/namenode/snapshot/SnapshotDiffInfo.java | 50 +-
.../namenode/sps/BlockStorageMovementNeeded.java | 5 +-
.../server/namenode/sps/DatanodeCacheManager.java | 2 +-
.../namenode/sps/StoragePolicySatisfyManager.java | 14 +-
.../startupprogress/StartupProgressView.java | 4 +-
.../server/namenode/top/metrics/TopMetrics.java | 17 +-
.../server/namenode/top/window/RollingWindow.java | 18 +-
.../namenode/top/window/RollingWindowManager.java | 2 +-
.../web/resources/NamenodeWebHdfsMethods.java | 87 +-
.../hdfs/server/protocol/BalancerProtocols.java | 30 +
.../protocol/BlockStorageMovementCommand.java | 11 +-
.../hdfs/server/protocol/DatanodeProtocol.java | 2 +-
.../hdfs/server/protocol/NamenodeProtocol.java | 5 +-
.../hadoop/hdfs/server/protocol/ServerCommand.java | 6 +-
.../sps/ExternalSPSBlockMoveTaskHandler.java | 2 +
.../org/apache/hadoop/hdfs/tools/DFSAdmin.java | 6 +-
.../org/apache/hadoop/hdfs/tools/DFSHAAdmin.java | 6 +-
.../hadoop/hdfs/tools/DFSZKFailoverController.java | 20 +-
.../java/org/apache/hadoop/hdfs/tools/DFSck.java | 23 +-
.../org/apache/hadoop/hdfs/tools/DebugAdmin.java | 8 +-
.../hadoop/hdfs/tools/DelegationTokenFetcher.java | 66 +-
.../java/org/apache/hadoop/hdfs/tools/ECAdmin.java | 113 +-
.../org/apache/hadoop/hdfs/tools/GetGroups.java | 6 +-
.../hadoop/hdfs/tools/NNHAServiceTarget.java | 5 +
.../hadoop/hdfs/tools/StoragePolicyAdmin.java | 20 +-
.../OfflineEditsBinaryLoader.java | 10 +-
.../offlineEditsViewer/OfflineEditsViewer.java | 4 +-
.../offlineEditsViewer/OfflineEditsVisitor.java | 2 +-
.../offlineEditsViewer/StatisticsEditsVisitor.java | 4 +-
.../tools/offlineImageViewer/FSImageHandler.java | 14 +-
.../tools/offlineImageViewer/FSImageLoader.java | 33 +-
.../NameDistributionVisitor.java | 4 +-
.../OfflineImageReconstructor.java | 10 +-
.../offlineImageViewer/OfflineImageViewer.java | 9 +-
.../offlineImageViewer/OfflineImageViewerPB.java | 65 +-
.../offlineImageViewer/PBImageCorruption.java | 107 +
.../PBImageCorruptionDetector.java | 344 +
.../PBImageDelimitedTextWriter.java | 30 +-
.../offlineImageViewer/PBImageTextWriter.java | 233 +-
.../tools/offlineImageViewer/PBImageXmlWriter.java | 28 +-
.../tools/offlineImageViewer/WebImageViewer.java | 7 +-
.../hadoop/hdfs/util/AtomicFileOutputStream.java | 6 +-
.../java/org/apache/hadoop/hdfs/util/Diff.java | 16 +-
.../hadoop/hdfs/util/LightWeightHashSet.java | 7 +-
.../org/apache/hadoop/hdfs/util/MD5FileUtils.java | 8 +-
.../hadoop/hdfs/util/PersistentLongFile.java | 10 +-
.../java/org/apache/hadoop/hdfs/util/XMLUtils.java | 4 +-
.../java/org/apache/hadoop/hdfs/web/JsonUtil.java | 32 +
.../hdfs/web/resources/ExceptionHandler.java | 7 +-
.../hadoop-hdfs/src/main/proto/HdfsServer.proto | 1 +
.../src/main/proto/QJournalProtocol.proto | 18 +
.../hadoop-hdfs/src/main/proto/fsimage.proto | 1 +
.../src/main/resources/hdfs-default.xml | 140 +-
.../src/main/webapps/datanode/datanode.html | 5 +-
.../src/main/webapps/hdfs/dfshealth.html | 11 +-
.../src/main/webapps/hdfs/explorer.html | 6 +-
.../hadoop-hdfs/src/main/webapps/hdfs/explorer.js | 28 +
.../src/main/webapps/journal/index.html | 2 +-
.../src/main/webapps/secondary/status.html | 2 +-
.../src/main/webapps/static/dfs-dust.js | 4 +-
.../hadoop-hdfs/src/main/webapps/static/hadoop.css | 12 +-
.../site/markdown/CentralizedCacheManagement.md | 7 +-
.../hadoop-hdfs/src/site/markdown/HDFSCommands.md | 5 +-
.../src/site/markdown/HDFSDiskbalancer.md | 5 +-
.../site/markdown/HDFSHighAvailabilityWithQJM.md | 43 +-
.../hadoop-hdfs/src/site/markdown/HdfsDesign.md | 7 +-
.../src/site/markdown/HdfsImageViewer.md | 36 +-
.../src/site/markdown/HdfsPermissionsGuide.md | 2 +
.../src/site/markdown/HdfsProvidedStorage.md | 2 +-
.../src/site/markdown/ObserverNameNode.md | 187 +
.../hadoop-hdfs/src/site/markdown/WebHDFS.md | 341 +
.../org/apache/hadoop/cli/TestCacheAdminCLI.java | 7 +-
.../hadoop/fs/TestEnhancedByteBufferAccess.java | 10 +-
.../apache/hadoop/fs/TestHdfsNativeCodeLoader.java | 7 +-
.../java/org/apache/hadoop/fs/TestUnbuffer.java | 8 +-
.../hdfs/TestHDFSContractMultipartUploader.java | 15 +
.../fs/contract/hdfs/TestHDFSContractOpen.java | 2 +-
.../apache/hadoop/hdfs/AdminStatesBaseTest.java | 7 +-
.../org/apache/hadoop/hdfs/AppendTestUtil.java | 6 +-
.../apache/hadoop/hdfs/BenchmarkThroughput.java | 13 +-
.../java/org/apache/hadoop/hdfs/DFSTestUtil.java | 76 +-
.../org/apache/hadoop/hdfs/FileAppendTest4.java | 7 +-
.../org/apache/hadoop/hdfs/MiniDFSCluster.java | 104 +-
.../hadoop/hdfs/MiniDFSClusterWithNodeGroup.java | 7 +-
.../org/apache/hadoop/hdfs/MiniDFSNNTopology.java | 17 +
.../hdfs/ReadStripedFileWithDecodingHelper.java | 5 +-
.../org/apache/hadoop/hdfs/TestAbandonBlock.java | 7 +-
.../org/apache/hadoop/hdfs/TestAclsEndToEnd.java | 8 +-
.../hadoop/hdfs/TestAppendSnapshotTruncate.java | 7 +-
.../apache/hadoop/hdfs/TestBalancerBandwidth.java | 7 +-
.../hadoop/hdfs/TestBlockMissingException.java | 7 +-
.../hadoop/hdfs/TestBlockTokenWrappingQOP.java | 190 +
.../TestClientProtocolForPipelineRecovery.java | 2 +
.../hadoop/hdfs/TestClientReportBadBlock.java | 8 +-
.../java/org/apache/hadoop/hdfs/TestConnCache.java | 6 +-
.../apache/hadoop/hdfs/TestDFSClientFailover.java | 13 +-
.../apache/hadoop/hdfs/TestDFSClientRetries.java | 86 +-
.../org/apache/hadoop/hdfs/TestDFSFinalize.java | 6 +-
.../hdfs/TestDFSInotifyEventInputStream.java | 106 +-
.../apache/hadoop/hdfs/TestDFSOutputStream.java | 4 +-
.../org/apache/hadoop/hdfs/TestDFSPermission.java | 7 +-
.../org/apache/hadoop/hdfs/TestDFSRollback.java | 6 +-
.../java/org/apache/hadoop/hdfs/TestDFSShell.java | 6 +-
.../apache/hadoop/hdfs/TestDFSStartupVersions.java | 6 +-
.../hadoop/hdfs/TestDFSStorageStateRecovery.java | 6 +-
.../hadoop/hdfs/TestDFSStripedInputStream.java | 10 +-
...estDFSStripedInputStreamWithRandomECPolicy.java | 8 +-
.../hadoop/hdfs/TestDFSStripedOutputStream.java | 42 +-
.../TestDFSStripedOutputStreamWithFailure.java | 41 +-
...dOutputStreamWithFailureWithRandomECPolicy.java | 8 +-
...stDFSStripedOutputStreamWithRandomECPolicy.java | 8 +-
.../org/apache/hadoop/hdfs/TestDFSUpgrade.java | 7 +-
.../hadoop/hdfs/TestDFSUpgradeFromImage.java | 7 +-
.../org/apache/hadoop/hdfs/TestDataStream.java | 6 +-
.../hadoop/hdfs/TestDataTransferProtocol.java | 6 +-
.../hadoop/hdfs/TestDatanodeRegistration.java | 7 +-
.../org/apache/hadoop/hdfs/TestDatanodeReport.java | 6 +-
.../apache/hadoop/hdfs/TestDisableConnCache.java | 6 +-
.../hadoop/hdfs/TestDistributedFileSystem.java | 227 +-
.../hdfs/TestDistributedFileSystemWithECFile.java | 44 +
.../apache/hadoop/hdfs/TestEncryptedTransfer.java | 17 +-
.../apache/hadoop/hdfs/TestEncryptionZones.java | 48 +-
.../hadoop/hdfs/TestErasureCodingAddConfig.java | 79 +
.../hdfs/TestErasureCodingMultipleRacks.java | 18 +-
.../hadoop/hdfs/TestExternalBlockReader.java | 8 +-
.../org/apache/hadoop/hdfs/TestFileAppend4.java | 11 +-
.../org/apache/hadoop/hdfs/TestFileCreation.java | 2 +-
.../apache/hadoop/hdfs/TestHAAuxiliaryPort.java | 112 +
.../apache/hadoop/hdfs/TestHDFSServerPorts.java | 7 +-
.../java/org/apache/hadoop/hdfs/TestHDFSTrash.java | 6 +-
.../hdfs/TestInjectionForSimulatedStorage.java | 6 +-
.../org/apache/hadoop/hdfs/TestLargeBlock.java | 7 +-
.../java/org/apache/hadoop/hdfs/TestLease.java | 25 +-
.../org/apache/hadoop/hdfs/TestLeaseRecovery2.java | 30 +-
.../apache/hadoop/hdfs/TestMaintenanceState.java | 2 +
.../apache/hadoop/hdfs/TestMissingBlocksAlert.java | 8 +-
.../apache/hadoop/hdfs/TestParallelReadUtil.java | 6 +-
.../java/org/apache/hadoop/hdfs/TestPipelines.java | 6 +-
.../java/org/apache/hadoop/hdfs/TestPread.java | 3 +-
.../java/org/apache/hadoop/hdfs/TestQuota.java | 8 +
.../hdfs/TestReadStripedFileWithMissingBlocks.java | 8 +-
.../hadoop/hdfs/TestReconstructStripedFile.java | 7 +-
.../TestReplaceDatanodeFailureReplication.java | 8 +-
.../hadoop/hdfs/TestReplaceDatanodeOnFailure.java | 7 +-
.../org/apache/hadoop/hdfs/TestReplication.java | 9 +-
.../org/apache/hadoop/hdfs/TestRollingUpgrade.java | 7 +-
.../java/org/apache/hadoop/hdfs/TestSafeMode.java | 6 +-
.../hdfs/TestStateAlignmentContextWithHA.java | 347 +
.../hdfs/TestUnsetAndChangeDirectoryEcPolicy.java | 31 +-
.../java/org/apache/hadoop/hdfs/TestWriteRead.java | 8 +-
.../hadoop/hdfs/TestWriteReadStripedFile.java | 19 +-
.../hdfs/TestWriteStripedFileWithFailure.java | 8 +-
.../client/impl/TestBlockReaderIoProvider.java | 4 +-
.../hdfs/client/impl/TestBlockReaderLocal.java | 10 +-
.../client/impl/TestBlockReaderLocalMetrics.java | 23 +-
.../hadoop/hdfs/net/TestDFSNetworkTopology.java | 8 +-
.../hadoop/hdfs/protocol/TestBlockListAsLongs.java | 8 +-
.../hadoop/hdfs/protocol/TestLayoutVersion.java | 3 +-
.../hadoop/hdfs/protocol/TestLocatedBlock.java | 9 +-
.../datatransfer/sasl/TestSaslDataTransfer.java | 4 +-
.../hadoop/hdfs/qjournal/MiniJournalCluster.java | 7 +-
.../hadoop/hdfs/qjournal/MiniQJMHACluster.java | 11 +-
.../apache/hadoop/hdfs/qjournal/TestNNWithQJM.java | 15 +-
.../hadoop/hdfs/qjournal/TestSecureNNWithQJM.java | 2 -
.../hdfs/qjournal/client/TestEpochsAreUnique.java | 7 +-
.../hdfs/qjournal/client/TestQJMWithFaults.java | 6 +-
.../qjournal/client/TestQuorumJournalManager.java | 150 +-
.../client/TestQuorumJournalManagerUnit.java | 113 +-
.../hdfs/qjournal/server/JournalTestUtil.java | 48 +
.../hadoop/hdfs/qjournal/server/TestJournal.java | 47 +
.../TestJournalNodeRespectsBindHostKeys.java | 6 +-
.../hdfs/qjournal/server/TestJournalNodeSync.java | 90 +-
.../qjournal/server/TestJournaledEditsCache.java | 257 +
.../hadoop/hdfs/security/TestDelegationToken.java | 11 +-
.../security/TestDelegationTokenForProxyUser.java | 7 +-
.../hdfs/security/token/block/TestBlockToken.java | 26 +-
.../server/aliasmap/ITestInMemoryAliasMap.java | 30 +
.../hdfs/server/aliasmap/TestInMemoryAliasMap.java | 45 -
.../hadoop/hdfs/server/balancer/TestBalancer.java | 12 +-
.../balancer/TestBalancerWithHANameNodes.java | 101 +-
.../TestBalancerWithMultipleNameNodes.java | 12 +-
.../server/balancer/TestBalancerWithNodeGroup.java | 6 +-
.../blockmanagement/BaseReplicationPolicyTest.java | 3 +-
.../hdfs/server/blockmanagement/TestBlockInfo.java | 8 +-
.../blockmanagement/TestBlockInfoStriped.java | 45 +
.../server/blockmanagement/TestBlockManager.java | 14 +-
.../blockmanagement/TestBlockManagerSafeMode.java | 83 +-
.../TestBlockReportRateLimiting.java | 7 +-
.../TestBlocksWithNotEnoughRacks.java | 79 +-
.../blockmanagement/TestCachedBlocksList.java | 7 +-
.../blockmanagement/TestCorruptReplicaInfo.java | 8 +-
.../blockmanagement/TestDatanodeManager.java | 7 +-
.../TestNameNodePrunesMissingStorages.java | 7 +-
.../blockmanagement/TestPendingReconstruction.java | 2 +
.../blockmanagement/TestRBWBlockInvalidation.java | 47 +-
...tReconstructStripedBlocksWithRackAwareness.java | 11 +-
.../blockmanagement/TestReplicationPolicy.java | 30 +-
.../TestSequentialBlockGroupId.java | 8 +-
.../blockmanagement/TestSequentialBlockId.java | 7 +-
.../hdfs/server/common/TestGetUriFromString.java | 7 +-
.../impl/TestInMemoryLevelDBAliasMapClient.java | 9 -
.../hdfs/server/datanode/BlockReportTestBase.java | 11 +-
.../hdfs/server/datanode/DataNodeTestUtils.java | 8 +-
.../server/datanode/InternalDataNodeTestUtils.java | 18 +-
.../hdfs/server/datanode/SimulatedFSDataset.java | 240 +-
.../hdfs/server/datanode/TestBPOfferService.java | 91 +-
.../hadoop/hdfs/server/datanode/TestBatchIbr.java | 11 +-
.../datanode/TestBlockCountersInPendingIBR.java | 4 +-
.../TestBlockHasMultipleReplicasOnSameDN.java | 7 +-
.../hdfs/server/datanode/TestBlockPoolManager.java | 11 +-
.../hdfs/server/datanode/TestBlockRecovery.java | 47 +-
.../hdfs/server/datanode/TestBlockReplacement.java | 6 +-
.../datanode/TestBpServiceActorScheduler.java | 7 +-
.../datanode/TestDataNodeErasureCodingMetrics.java | 8 +-
.../server/datanode/TestDataNodeFaultInjector.java | 8 +-
.../datanode/TestDataNodeHotSwapVolumes.java | 12 +-
.../server/datanode/TestDataNodeInitStorage.java | 7 +-
.../hdfs/server/datanode/TestDataNodeLifeline.java | 16 +-
.../hdfs/server/datanode/TestDataNodeMXBean.java | 7 +-
.../hdfs/server/datanode/TestDataNodeMetrics.java | 81 +-
.../server/datanode/TestDataNodeMetricsLogger.java | 6 +-
.../TestDataNodeMultipleRegistrations.java | 38 +-
.../datanode/TestDataNodeReconfiguration.java | 7 +-
.../datanode/TestDataNodeRollingUpgrade.java | 7 +-
.../server/datanode/TestDataNodeTcpNoDelay.java | 8 +-
.../server/datanode/TestDataNodeVolumeFailure.java | 7 +-
.../TestDataNodeVolumeFailureReporting.java | 14 +-
.../server/datanode/TestDataNodeVolumeMetrics.java | 8 +-
.../datanode/TestDataXceiverBackwardsCompat.java | 15 +-
.../datanode/TestDataXceiverLazyPersistHint.java | 20 +-
.../datanode/TestDatanodeProtocolRetryPolicy.java | 33 +-
.../hdfs/server/datanode/TestDatanodeRegister.java | 9 +-
.../hdfs/server/datanode/TestDirectoryScanner.java | 254 +-
.../TestDnRespectsBlockReportSplitThreshold.java | 17 +-
.../hdfs/server/datanode/TestFsDatasetCache.java | 22 +-
.../datanode/TestIncrementalBlockReports.java | 11 +-
.../datanode/TestIncrementalBrVariations.java | 21 +-
.../server/datanode/TestReadOnlySharedStorage.java | 7 +-
.../hdfs/server/datanode/TestStorageReport.java | 16 +-
.../hdfs/server/datanode/TestTransferRbw.java | 7 +-
.../server/datanode/TestTriggerBlockReport.java | 10 +-
.../datanode/checker/TestDatasetVolumeChecker.java | 10 +-
.../checker/TestDatasetVolumeCheckerFailures.java | 20 +-
.../checker/TestDatasetVolumeCheckerTimeout.java | 23 +-
.../checker/TestThrottledAsyncChecker.java | 2 +-
.../checker/TestThrottledAsyncCheckerTimeout.java | 29 +-
.../extdataset/ExternalReplicaInPipeline.java | 6 +
.../datanode/extdataset/ExternalVolumeImpl.java | 10 +-
.../fsdataset/impl/FsDatasetImplTestUtils.java | 8 +-
.../fsdataset/impl/LazyPersistTestCase.java | 7 +-
.../datanode/fsdataset/impl/TestFsDatasetImpl.java | 17 +-
.../datanode/fsdataset/impl/TestFsVolumeList.java | 64 +-
.../fsdataset/impl/TestLazyPersistFiles.java | 15 +-
.../datanode/fsdataset/impl/TestProvidedImpl.java | 13 +-
.../fsdataset/impl/TestSpaceReservation.java | 6 +-
.../hdfs/server/diskbalancer/TestDiskBalancer.java | 65 +-
.../TestDiskBalancerWithMockMover.java | 7 +-
.../hadoop/hdfs/server/mover/TestStorageMover.java | 22 +-
.../hdfs/server/namenode/FSImageTestUtil.java | 32 +-
.../server/namenode/NNThroughputBenchmark.java | 20 +-
.../hdfs/server/namenode/NameNodeAdapter.java | 13 +
.../server/namenode/OfflineEditsViewerHelper.java | 8 +-
.../hdfs/server/namenode/TestAddBlockRetry.java | 11 +-
.../hdfs/server/namenode/TestAddStripedBlocks.java | 120 +-
.../hdfs/server/namenode/TestAllowFormat.java | 8 +-
.../hdfs/server/namenode/TestAuditLogAtDebug.java | 14 +-
.../hdfs/server/namenode/TestAuditLogger.java | 2 +-
.../namenode/TestAuditLoggerWithCommands.java | 2 +-
.../hdfs/server/namenode/TestBackupNode.java | 13 +-
.../TestBlockPlacementPolicyRackFaultTolerant.java | 4 +-
.../hdfs/server/namenode/TestCacheDirectives.java | 55 +-
.../hdfs/server/namenode/TestCheckpoint.java | 20 +-
.../hadoop/hdfs/server/namenode/TestClusterId.java | 9 +-
.../namenode/TestCommitBlockSynchronization.java | 10 +-
.../hdfs/server/namenode/TestDeadDatanode.java | 7 +-
.../namenode/TestDefaultBlockPlacementPolicy.java | 4 +-
.../hdfs/server/namenode/TestDeleteRace.java | 7 +-
.../server/namenode/TestDiskspaceQuotaUpdate.java | 10 +-
.../hadoop/hdfs/server/namenode/TestEditLog.java | 18 +-
.../namenode/TestEditLogFileInputStream.java | 28 +-
.../namenode/TestEditLogJournalFailures.java | 46 +-
.../hdfs/server/namenode/TestEditLogRace.java | 169 +-
.../server/namenode/TestEnabledECPolicies.java | 103 +-
.../hdfs/server/namenode/TestFSDirAttrOp.java | 7 +-
.../hdfs/server/namenode/TestFSDirWriteFileOp.java | 14 +-
.../hdfs/server/namenode/TestFSDirectory.java | 7 +-
.../hdfs/server/namenode/TestFSEditLogLoader.java | 54 +-
.../hadoop/hdfs/server/namenode/TestFSImage.java | 42 +-
.../server/namenode/TestFSImageWithSnapshot.java | 4 +-
.../hdfs/server/namenode/TestFSNamesystemLock.java | 41 +-
.../server/namenode/TestFSPermissionChecker.java | 2 +-
.../server/namenode/TestFavoredNodesEndToEnd.java | 8 +-
.../server/namenode/TestFileJournalManager.java | 23 +-
.../hdfs/server/namenode/TestFileTruncate.java | 12 +-
.../hadoop/hdfs/server/namenode/TestFsck.java | 22 +-
.../namenode/TestFsckWithMultipleNameNodes.java | 7 +-
.../hdfs/server/namenode/TestGetImageServlet.java | 9 +-
.../hdfs/server/namenode/TestHDFSConcat.java | 7 +-
.../hdfs/server/namenode/TestHostsFiles.java | 8 +-
.../hadoop/hdfs/server/namenode/TestINodeFile.java | 8 +-
.../server/namenode/TestLargeDirectoryDelete.java | 9 +-
.../server/namenode/TestListCorruptFileBlocks.java | 74 +-
.../hdfs/server/namenode/TestListOpenFiles.java | 9 +-
.../namenode/TestNNStorageRetentionFunctional.java | 6 +-
.../namenode/TestNNStorageRetentionManager.java | 4 +-
.../hdfs/server/namenode/TestNameEditsConfigs.java | 6 +-
.../hdfs/server/namenode/TestNameNodeMXBean.java | 68 +
.../namenode/TestNameNodeMetadataConsistency.java | 2 +
.../server/namenode/TestNameNodeMetricsLogger.java | 6 +-
.../server/namenode/TestNameNodeReconfigure.java | 8 +-
.../hdfs/server/namenode/TestNameNodeRecovery.java | 9 +-
.../namenode/TestNameNodeRespectsBindHostKeys.java | 7 +-
.../server/namenode/TestNameNodeStatusMXBean.java | 6 +-
.../namenode/TestNamenodeCapacityReport.java | 7 +-
.../server/namenode/TestNamenodeRetryCache.java | 10 +-
.../server/namenode/TestQuotaByStorageType.java | 33 +-
.../hdfs/server/namenode/TestSaveNamespace.java | 80 +-
.../hadoop/hdfs/server/namenode/TestStartup.java | 7 +-
.../hdfs/server/namenode/TestStorageRestore.java | 12 +-
.../hdfs/server/namenode/TestStripedINodeFile.java | 8 +-
.../TestUpgradeDomainBlockPlacementPolicy.java | 22 +-
.../hadoop/hdfs/server/namenode/ha/HATestUtil.java | 142 +-
.../server/namenode/ha/TestBootstrapAliasmap.java | 99 +
.../server/namenode/ha/TestBootstrapStandby.java | 13 +-
.../namenode/ha/TestConsistentReadsObserver.java | 280 +
.../server/namenode/ha/TestDFSUpgradeWithHA.java | 7 +-
.../hdfs/server/namenode/ha/TestDNFencing.java | 13 +-
.../namenode/ha/TestDelegationTokensWithHA.java | 8 +-
.../hdfs/server/namenode/ha/TestEditLogTailer.java | 102 +-
.../namenode/ha/TestEditLogsDuringFailover.java | 10 +-
.../server/namenode/ha/TestFailureToReadEdits.java | 18 +-
.../hadoop/hdfs/server/namenode/ha/TestHAFsck.java | 4 +-
.../hdfs/server/namenode/ha/TestHAMetrics.java | 11 +-
.../hdfs/server/namenode/ha/TestHASafeMode.java | 13 +-
.../server/namenode/ha/TestHAStateTransitions.java | 12 +-
.../namenode/ha/TestInitializeSharedEdits.java | 7 +-
.../server/namenode/ha/TestMultiObserverNode.java | 159 +
.../hdfs/server/namenode/ha/TestObserverNode.java | 364 +
.../namenode/ha/TestObserverReadProxyProvider.java | 403 +
.../server/namenode/ha/TestPipelinesFailover.java | 7 +-
.../server/namenode/ha/TestRetryCacheWithHA.java | 11 +-
.../server/namenode/ha/TestSeveralNameNodes.java | 7 +-
.../namenode/ha/TestStandbyBlockManagement.java | 6 +-
.../server/namenode/ha/TestStandbyCheckpoints.java | 16 +-
.../namenode/ha/TestStandbyInProgressTail.java | 172 +-
.../hdfs/server/namenode/ha/TestStandbyIsHot.java | 6 +-
.../namenode/metrics/TestNameNodeMetrics.java | 15 +-
...TestINodeFileUnderConstructionWithSnapshot.java | 6 +-
.../snapshot/TestOpenFilesWithSnapshot.java | 8 +-
.../namenode/snapshot/TestRenameWithSnapshots.java | 45 +-
.../server/namenode/snapshot/TestSnapshot.java | 4 +-
.../namenode/snapshot/TestSnapshotDiffReport.java | 6 +
.../namenode/snapshot/TestSnapshotManager.java | 4 +-
.../TestBlockStorageMovementAttemptedItems.java | 1 -
.../resources/TestWebHdfsCreatePermissions.java | 7 +-
.../web/resources/TestWebHdfsDataLocality.java | 7 +-
.../sps/TestExternalStoragePolicySatisfier.java | 3 +-
.../hdfs/shortcircuit/TestShortCircuitCache.java | 9 +-
.../org/apache/hadoop/hdfs/tools/TestDFSAdmin.java | 8 +-
.../apache/hadoop/hdfs/tools/TestDFSHAAdmin.java | 26 +-
.../hdfs/tools/TestDFSHAAdminMiniCluster.java | 57 +-
.../hdfs/tools/TestDelegationTokenFetcher.java | 7 +-
.../org/apache/hadoop/hdfs/tools/TestECAdmin.java | 245 +
.../TestStoragePolicySatisfyAdminCommands.java | 22 +
.../offlineEditsViewer/TestOfflineEditsViewer.java | 8 +-
.../offlineImageViewer/TestOfflineImageViewer.java | 365 +-
.../TestOfflineImageViewerForAcl.java | 8 +-
.../TestOfflineImageViewerForContentSummary.java | 8 +-
.../TestOfflineImageViewerForXAttr.java | 8 +-
.../offlineImageViewer/TestPBImageCorruption.java | 55 +
.../hadoop/hdfs/util/TestLightWeightHashSet.java | 8 +-
.../hadoop/hdfs/util/TestLightWeightLinkedSet.java | 10 +-
.../hdfs/web/TestFSMainOperationsWebHdfs.java | 4 +-
.../org/apache/hadoop/hdfs/web/TestWebHDFS.java | 279 +-
.../hadoop/hdfs/web/TestWebHdfsTimeouts.java | 18 +-
.../apache/hadoop/hdfs/web/TestWebHdfsTokens.java | 3 +-
.../hdfs/web/TestWebHdfsWithMultipleNameNodes.java | 12 +-
.../apache/hadoop/hdfs/web/WebHdfsTestUtil.java | 7 +-
.../hadoop/hdfs/web/resources/TestParam.java | 14 +-
.../org/apache/hadoop/security/TestPermission.java | 7 +-
.../hadoop/security/TestPermissionSymlinks.java | 7 +-
.../apache/hadoop/test/MiniDFSClusterManager.java | 2 +-
.../apache/hadoop/tools/TestHdfsConfigFields.java | 1 +
.../src/test/resources/testErasureCodingConf.xml | 70 +-
.../src/test/resources/testMultipleCorruption.csv | 22 +
.../test/resources/testMultipleFileCorruption.csv | 21 +
.../test/resources/testSingleFileCorruption.csv | 18 +
.../test/resources/testSingleFolderCorruption.csv | 18 +
hadoop-hdfs-project/pom.xml | 4 +-
.../jdiff/Apache_Hadoop_MapReduce_Core_3.1.2.xml | 28085 +++++++++++++++
.../Apache_Hadoop_MapReduce_JobClient_3.1.2.xml | 16 +
.../hadoop-mapreduce-client-app/pom.xml | 29 +-
.../java/org/apache/hadoop/mapred/YarnChild.java | 28 +-
.../mapreduce/v2/app/TaskHeartbeatHandler.java | 34 +-
.../mapreduce/v2/app/client/MRClientService.java | 15 +-
.../mapreduce/v2/app/job/impl/TaskAttemptImpl.java | 15 +
.../mapreduce/v2/app/rm/RMContainerAllocator.java | 14 +-
.../v2/app/speculate/DefaultSpeculator.java | 4 +-
.../mapreduce/v2/app/webapp/AMWebServices.java | 9 +-
.../v2/app/webapp/JAXBContextResolver.java | 7 +-
.../hadoop/mapreduce/v2/app/webapp/TaskPage.java | 3 +-
.../v2/app/webapp/dao/MapTaskAttemptInfo.java | 39 +
.../v2/app/webapp/dao/ReduceTaskAttemptInfo.java | 11 +-
.../v2/app/webapp/dao/TaskAttemptInfo.java | 14 +-
.../v2/app/webapp/dao/TaskAttemptsInfo.java | 18 +-
.../hadoop/mapred/TestLocalContainerLauncher.java | 2 +-
.../hadoop/mapred/TestTaskAttemptListenerImpl.java | 14 +-
.../org/apache/hadoop/mapred/TestYarnChild.java | 118 +
.../jobhistory/TestJobHistoryEventHandler.java | 2 +-
.../hadoop/mapreduce/v2/app/MRAppBenchmark.java | 20 +-
.../v2/app/TestKillAMPreemptionPolicy.java | 14 +-
.../hadoop/mapreduce/v2/app/TestMRAppMaster.java | 2 +-
.../mapreduce/v2/app/TestStagingCleanup.java | 4 +-
.../mapreduce/v2/app/TestTaskHeartbeatHandler.java | 55 +-
.../v2/app/commit/TestCommitterEventHandler.java | 10 +-
.../mapreduce/v2/app/job/impl/TestJobImpl.java | 2 +-
.../mapreduce/v2/app/job/impl/TestTaskAttempt.java | 122 +-
.../job/impl/TestTaskAttemptContainerRequest.java | 11 +-
.../v2/app/launcher/TestContainerLauncher.java | 9 +
.../v2/app/launcher/TestContainerLauncherImpl.java | 11 +-
.../v2/app/local/TestLocalContainerAllocator.java | 2 +-
.../mapreduce/v2/app/metrics/TestMRAppMetrics.java | 7 +
.../mapreduce/v2/app/rm/TestRMCommunicator.java | 15 +-
.../v2/app/rm/TestRMContainerAllocator.java | 24 +-
.../mapreduce/v2/app/webapp/TestAMWebApp.java | 158 +-
.../v2/app/webapp/TestAMWebServicesAttempts.java | 3 +
.../hadoop-mapreduce-client-common/pom.xml | 11 +-
.../hadoop/mapreduce/v2/util/MRWebAppUtil.java | 6 +-
.../mapred/TestLocalDistributedCacheManager.java | 4 +-
.../hadoop-mapreduce-client-core/pom.xml | 12 +-
.../org/apache/hadoop/mapred/FileInputFormat.java | 8 +-
.../org/apache/hadoop/mapred/LineRecordReader.java | 12 +-
.../org/apache/hadoop/mapred/OutputFormat.java | 3 +
.../main/java/org/apache/hadoop/mapreduce/Job.java | 10 +-
.../hadoop/mapreduce/JobResourceUploader.java | 2 +-
.../java/org/apache/hadoop/mapreduce/MRConfig.java | 2 +-
.../org/apache/hadoop/mapreduce/MRJobConfig.java | 47 +
.../org/apache/hadoop/mapreduce/OutputFormat.java | 3 +
.../mapreduce/counters/FileSystemCounterGroup.java | 56 +-
.../hadoop/mapreduce/jobhistory/EventWriter.java | 2 +-
.../mapreduce/jobhistory/JobHistoryParser.java | 2 +-
.../mapreduce/lib/input/FileInputFormat.java | 10 +-
.../lib/input/FixedLengthRecordReader.java | 14 +-
.../mapreduce/lib/input/LineRecordReader.java | 12 +-
.../mapreduce/lib/input/NLineInputFormat.java | 12 +-
.../mapreduce/lib/output/FileOutputCommitter.java | 28 +-
.../src/main/resources/mapred-default.xml | 41 +-
.../src/site/markdown/MapredAppMasterRest.md | 2 +-
.../markdown/PluggableShuffleAndPluggableSort.md | 78 +-
.../java/org/apache/hadoop/mapred/TestMapTask.java | 2 +-
.../java/org/apache/hadoop/mapred/TestTask.java | 4 +-
.../hadoop/mapreduce/TestJobMonitorAndPrint.java | 6 +-
.../hadoop/mapreduce/TestJobResourceUploader.java | 4 +-
.../TestJobResourceUploaderWithSharedCache.java | 6 +-
.../hadoop/mapreduce/lib/db/DriverForTest.java | 2 +-
.../lib/output/TestFileOutputCommitter.java | 33 +
.../hadoop/mapreduce/security/TestTokenCache.java | 2 +-
.../mapreduce/split/TestJobSplitWriterWithEC.java | 128 +
.../hadoop/mapreduce/task/reduce/TestFetcher.java | 1 -
.../hadoop/mapreduce/task/reduce/TestMerger.java | 2 +-
.../org/apache/hadoop/mapreduce/tools/TestCLI.java | 2 +-
.../hadoop-mapreduce-client-hs-plugins/pom.xml | 4 +-
.../hadoop-mapreduce-client-hs/pom.xml | 4 +-
.../hadoop/mapreduce/v2/hs/CompletedTask.java | 5 +-
.../mapreduce/v2/hs/CompletedTaskAttempt.java | 2 +-
.../hadoop/mapreduce/v2/hs/HistoryFileManager.java | 12 +-
.../hadoop/mapreduce/v2/hs/webapp/HsTaskPage.java | 3 +-
.../mapreduce/v2/hs/webapp/HsTasksBlock.java | 5 +-
.../mapreduce/v2/hs/webapp/HsWebServices.java | 9 +-
.../v2/hs/webapp/JAXBContextResolver.java | 4 +-
.../mapreduce/v2/hs/TestHistoryFileManager.java | 52 +
...stHistoryServerFileSystemStateStoreService.java | 11 +-
.../v2/hs/TestJHSDelegationTokenSecretManager.java | 78 +-
.../hadoop/mapreduce/v2/hs/TestJobHistory.java | 4 +-
.../hadoop/mapreduce/v2/hs/webapp/TestBlocks.java | 2 +-
.../mapreduce/v2/hs/webapp/TestHsJobBlock.java | 2 +-
.../hadoop-mapreduce-client-jobclient/pom.xml | 11 +-
.../apache/hadoop/mapred/ResourceMgrDelegate.java | 29 +
.../test/java/org/apache/hadoop/fs/TestDFSIO.java | 1 +
.../apache/hadoop/mapred/JobClientUnitTest.java | 2 +-
.../apache/hadoop/mapred/TestClientRedirect.java | 25 +
.../hadoop/mapred/TestClientServiceDelegate.java | 2 +-
.../hadoop/mapred/TestMRCJCFileInputFormat.java | 2 +-
.../org/apache/hadoop/mapred/TestYARNRunner.java | 46 +-
.../hadoop/mapreduce/MiniHadoopClusterManager.java | 2 +-
.../mapreduce/TestYarnClientProtocolProvider.java | 2 +-
.../TestUmbilicalProtocolWithJobToken.java | 7 +-
.../hadoop-mapreduce-client-nativetask/pom.xml | 26 +-
.../nativetask/handlers/TestCombineHandler.java | 11 +-
.../handlers/TestNativeCollectorOnlyHandler.java | 8 +-
.../mapred/nativetask/serde/TestKVSerializer.java | 28 +-
.../hadoop-mapreduce-client-shuffle/pom.xml | 11 +-
.../org/apache/hadoop/mapred/ShuffleHandler.java | 7 +-
.../apache/hadoop/mapred/TestShuffleHandler.java | 2 +-
.../hadoop-mapreduce-client-uploader/pom.xml | 10 +-
.../hadoop/mapred/uploader/FrameworkUploader.java | 52 +
.../mapred/uploader/TestFrameworkUploader.java | 58 +
.../hadoop-mapreduce-client/pom.xml | 6 +-
.../hadoop-mapreduce-examples/pom.xml | 4 +-
.../org/apache/hadoop/examples/WordMedian.java | 13 +-
hadoop-mapreduce-project/pom.xml | 6 +-
hadoop-maven-plugins/pom.xml | 8 +-
hadoop-minicluster/pom.xml | 10 +-
hadoop-ozone/Jenkinsfile | 116 +
hadoop-ozone/acceptance-test/README.md | 48 -
.../acceptance-test/dev-support/bin/robot-all.sh | 18 -
.../dev-support/bin/robot-dnd-all.sh | 57 -
.../acceptance-test/dev-support/bin/robot.sh | 38 -
.../acceptance-test/dev-support/docker/Dockerfile | 21 -
.../dev-support/docker/docker-compose.yaml | 23 -
hadoop-ozone/acceptance-test/pom.xml | 59 -
.../acceptance-test/src/test/acceptance/basic/.env | 17 -
.../src/test/acceptance/basic/basic.robot | 50 -
.../src/test/acceptance/basic/docker-compose.yaml | 50 -
.../src/test/acceptance/basic/docker-config | 33 -
.../src/test/acceptance/basic/ozone-shell.robot | 86 -
.../src/test/acceptance/commonlib.robot | 78 -
.../src/test/acceptance/ozonefs/.env | 17 -
.../test/acceptance/ozonefs/docker-compose.yaml | 59 -
.../src/test/acceptance/ozonefs/docker-config | 34 -
.../src/test/acceptance/ozonefs/ozonefs.robot | 39 -
.../test/acceptance/ozonefs/ozonesinglenode.robot | 49 -
hadoop-ozone/client/pom.xml | 10 +-
.../org/apache/hadoop/ozone/client/BucketArgs.java | 34 +-
.../apache/hadoop/ozone/client/ObjectStore.java | 233 +-
.../apache/hadoop/ozone/client/OzoneBucket.java | 141 +-
.../apache/hadoop/ozone/client/OzoneClient.java | 7 +
.../hadoop/ozone/client/OzoneClientUtils.java | 27 -
.../org/apache/hadoop/ozone/client/OzoneKey.java | 16 +-
.../hadoop/ozone/client/OzoneKeyDetails.java | 18 +-
.../client/OzoneMultipartUploadPartListParts.java | 107 +
.../apache/hadoop/ozone/client/OzoneVolume.java | 57 +-
.../org/apache/hadoop/ozone/client/VolumeArgs.java | 28 +-
.../ozone/client/io/BlockOutputStreamEntry.java | 334 +
.../ozone/client/io/ChunkGroupInputStream.java | 321 -
.../ozone/client/io/ChunkGroupOutputStream.java | 709 -
.../hadoop/ozone/client/io/KeyInputStream.java | 338 +
.../hadoop/ozone/client/io/KeyOutputStream.java | 674 +
.../hadoop/ozone/client/io/OzoneInputStream.java | 11 +-
.../hadoop/ozone/client/io/OzoneOutputStream.java | 14 +-
.../ozone/client/protocol/ClientProtocol.java | 184 +-
.../hadoop/ozone/client/rest/RestClient.java | 160 +-
.../apache/hadoop/ozone/client/rpc/RpcClient.java | 407 +-
.../hadoop/ozone/client/TestHddsClientUtils.java | 143 +-
hadoop-ozone/common/pom.xml | 15 +-
hadoop-ozone/common/src/main/bin/ozone | 105 +-
hadoop-ozone/common/src/main/bin/ozone-config.sh | 51 +
hadoop-ozone/common/src/main/bin/start-ozone.sh | 116 +-
hadoop-ozone/common/src/main/bin/stop-ozone.sh | 41 +-
.../apache/hadoop/hdds/protocol/StorageType.java | 64 +
.../apache/hadoop/hdds/protocol/package-info.java | 20 +
.../main/java/org/apache/hadoop/ozone/OmUtils.java | 212 +-
.../ozone/OzoneIllegalArgumentException.java | 40 +
.../org/apache/hadoop/ozone/audit/OMAction.java | 53 +-
.../ozone/client/rest/response/BucketInfo.java | 2 +-
.../hadoop/ozone/client/rest/response/KeyInfo.java | 20 +
.../apache/hadoop/ozone/freon/OzoneGetConf.java | 2 +
.../org/apache/hadoop/ozone/om/OMConfigKeys.java | 126 +
.../apache/hadoop/ozone/om/OMMetadataManager.java | 292 +
.../apache/hadoop/ozone/om/OzoneManagerLock.java | 244 +
.../hadoop/ozone/om/codec/OmBucketInfoCodec.java | 52 +
.../hadoop/ozone/om/codec/OmKeyInfoCodec.java | 52 +
.../ozone/om/codec/OmMultipartKeyInfoCodec.java | 56 +
.../hadoop/ozone/om/codec/OmVolumeArgsCodec.java | 52 +
.../hadoop/ozone/om/codec/VolumeListCodec.java | 51 +
.../apache/hadoop/ozone/om/codec/package-info.java | 24 +
.../hadoop/ozone/om/exceptions/OMException.java | 182 +
.../hadoop/ozone/om/exceptions/package-info.java | 0
.../hadoop/ozone/om/helpers/KeyValueUtil.java | 54 +
.../hadoop/ozone/om/helpers/OmBucketArgs.java | 58 +-
.../hadoop/ozone/om/helpers/OmBucketInfo.java | 101 +-
.../apache/hadoop/ozone/om/helpers/OmKeyArgs.java | 99 +-
.../apache/hadoop/ozone/om/helpers/OmKeyInfo.java | 91 +-
.../hadoop/ozone/om/helpers/OmKeyLocationInfo.java | 75 +-
.../helpers/OmMultipartCommitUploadPartInfo.java | 34 +
.../hadoop/ozone/om/helpers/OmMultipartInfo.java | 77 +
.../ozone/om/helpers/OmMultipartKeyInfo.java | 106 +
.../om/helpers/OmMultipartUploadCompleteInfo.java | 70 +
.../ozone/om/helpers/OmMultipartUploadList.java | 63 +
.../om/helpers/OmMultipartUploadListParts.java | 84 +
.../hadoop/ozone/om/helpers/OmOzoneAclMap.java | 1 +
.../apache/hadoop/ozone/om/helpers/OmPartInfo.java | 60 +
.../hadoop/ozone/om/helpers/OmVolumeArgs.java | 83 +-
.../hadoop/ozone/om/helpers/S3SecretValue.java | 81 +
.../hadoop/ozone/om/helpers/ServiceInfo.java | 14 +-
.../hadoop/ozone/om/helpers/WithMetadata.java | 45 +
.../ozone/om/protocol/OzoneManagerProtocol.java | 129 +-
.../om/protocol/OzoneManagerSecurityProtocol.java | 67 +
...OzoneManagerProtocolClientSideTranslatorPB.java | 998 +-
.../om/protocolPB/OzoneManagerProtocolPB.java | 7 +
.../apache/hadoop/ozone/protocolPB/OMPBHelper.java | 38 +
.../security/OzoneBlockTokenSecretManager.java | 194 +
.../OzoneDelegationTokenSecretManager.java | 470 +
.../security/OzoneDelegationTokenSelector.java | 51 +
.../hadoop/ozone/security/OzoneSecretKey.java | 176 +
.../hadoop/ozone/security/OzoneSecretManager.java | 280 +
.../hadoop/ozone/security/OzoneSecretStore.java | 249 +
.../ozone/security/OzoneSecurityException.java | 104 +
.../ozone/security/OzoneTokenIdentifier.java | 217 +
.../ozone/security/acl/IAccessAuthorizer.java | 153 +
.../hadoop/ozone/security/acl/IOzoneObj.java | 24 +
.../ozone/security/acl/OzoneAccessAuthorizer.java | 29 +
.../ozone/security/acl/OzoneAclException.java | 71 +
.../apache/hadoop/ozone/security/acl/OzoneObj.java | 109 +
.../hadoop/ozone/security/acl/OzoneObjInfo.java | 118 +
.../hadoop/ozone/security/acl/RequestContext.java | 122 +
.../hadoop/ozone/security/acl/package-info.java | 22 +
.../apache/hadoop/ozone/security/package-info.java | 21 +
.../apache/hadoop/ozone/util/OzoneVersionInfo.java | 174 +-
.../org/apache/hadoop/ozone/util/package-info.java | 22 +
.../hadoop/ozone/web/handlers/BucketArgs.java | 2 +-
.../hadoop/ozone/web/handlers/VolumeArgs.java | 1 +
.../hadoop/ozone/web/response/BucketInfo.java | 8 +-
.../apache/hadoop/ozone/web/response/KeyInfo.java | 21 +
.../src/main/proto/OzoneManagerProtocol.proto | 562 +-
.../common/src/main/shellprofile.d/hadoop-ozone.sh | 24 -
.../java/org/apache/hadoop/ozone/TestOmUtils.java | 91 +
.../om/codec/TestOmMultipartKeyInfoCodec.java | 54 +
.../apache/hadoop/ozone/om/codec/package-info.java | 24 +
.../ozone/om/exceptions/TestResultCodes.java | 49 +
.../hadoop/ozone/om/helpers/TestOmBucketInfo.java | 46 +
.../hadoop/ozone/om/helpers/TestOmKeyInfo.java | 52 +
.../hadoop/ozone/om/helpers/package-info.java | 21 +
.../security/TestOzoneBlockTokenSecretManager.java | 147 +
.../TestOzoneDelegationTokenSecretManager.java | 217 +
.../ozone/security/acl/TestOzoneObjInfo.java | 88 +
.../apache/hadoop/ozone/web/TestBucketInfo.java | 25 +-
.../org/apache/hadoop/ozone/web/TestQuota.java | 55 +-
hadoop-ozone/datanode/pom.xml | 49 +
hadoop-ozone/dev-support/checks/acceptance.sh | 18 +
hadoop-ozone/dev-support/checks/author.sh | 22 +
hadoop-ozone/dev-support/checks/build.sh | 18 +
hadoop-ozone/dev-support/checks/checkstyle.sh | 23 +
hadoop-ozone/dev-support/checks/findbugs.sh | 34 +
hadoop-ozone/dev-support/checks/isolation.sh | 24 +
hadoop-ozone/dev-support/checks/rat.sh | 24 +
hadoop-ozone/dev-support/checks/unit.sh | 24 +
hadoop-ozone/dev-support/docker/Dockerfile | 66 +
hadoop-ozone/dist/Dockerfile | 21 +
hadoop-ozone/dist/README.md | 85 +
.../dist/dev-support/bin/dist-layout-stitching | 120 +
.../dist/dev-support/bin/dist-tar-stitching | 45 +
hadoop-ozone/dist/pom.xml | 214 +
hadoop-ozone/dist/skaffold.yaml | 25 +
hadoop-ozone/dist/src/main/blockade/README.md | 44 +
.../src/main/blockade/blockadeUtils/__init__.py | 14 +
.../src/main/blockade/blockadeUtils/blockade.py | 99 +
.../src/main/blockade/clusterUtils/__init__.py | 0
.../main/blockade/clusterUtils/cluster_utils.py | 300 +
hadoop-ozone/dist/src/main/blockade/conftest.py | 96 +
.../main/blockade/test_blockade_client_failure.py | 124 +
.../blockade/test_blockade_datanode_isolation.py | 111 +
.../dist/src/main/blockade/test_blockade_flaky.py | 61 +
.../main/blockade/test_blockade_mixed_failure.py | 117 +
...t_blockade_mixed_failure_three_nodes_isolate.py | 144 +
.../test_blockade_mixed_failure_two_nodes.py | 121 +
.../main/blockade/test_blockade_scm_isolation.py | 111 +
.../dist}/src/main/compose/README.md | 0
.../main/compose/common/grafana/conf/grafana.ini | 20 +
.../grafana/dashboards/Ozone - Object Metrics.json | 1344 +
.../grafana/dashboards/Ozone - RPC Metrics.json | 875 +
.../grafana/provisioning/dashboards/dashboards.yml | 22 +
.../provisioning/datasources/datasources.yml | 25 +
.../main/compose/common/prometheus/prometheus.yml | 25 +
hadoop-ozone/dist/src/main/compose/ozone-hdfs/.env | 17 +
.../main/compose/ozone-hdfs/docker-compose.yaml | 70 +
.../dist/src/main/compose/ozone-hdfs/docker-config | 81 +
.../dist}/src/main/compose/ozone/.env | 0
.../src/main/compose/ozone/docker-compose.yaml | 50 +
.../dist/src/main/compose/ozone/docker-config | 76 +
.../main/compose/ozoneblockade/docker-compose.yaml | 58 +
.../src/main/compose/ozoneblockade/docker-config | 77 +
.../src/main/compose/ozonefs/docker-compose.yaml | 69 +
.../dist/src/main/compose/ozonefs/docker-config | 37 +
.../dist}/src/main/compose/ozoneperf/.env | 0
.../dist/src/main/compose/ozoneperf/README.md | 52 +
.../src/main/compose/ozoneperf/docker-compose.yaml | 75 +
.../dist/src/main/compose/ozoneperf/docker-config | 38 +
.../src/main/compose/ozones3/docker-compose.yaml | 59 +
.../dist/src/main/compose/ozones3/docker-config | 79 +
.../main/compose/ozonescripts/.ssh/authorized_keys | 16 +
.../dist/src/main/compose/ozonescripts/.ssh/config | 18 +
.../src/main/compose/ozonescripts/.ssh/environment | 16 +
.../dist/src/main/compose/ozonescripts/.ssh/id_rsa | 42 +
.../src/main/compose/ozonescripts/.ssh/id_rsa.pub | 16 +
.../dist/src/main/compose/ozonescripts/Dockerfile | 33 +
.../dist/src/main/compose/ozonescripts/README.md | 38 +
.../main/compose/ozonescripts/docker-compose.yaml | 42 +
.../src/main/compose/ozonescripts/docker-config | 40 +
.../dist/src/main/compose/ozonescripts/ps.sh | 17 +
.../dist/src/main/compose/ozonescripts/start.sh | 24 +
.../dist/src/main/compose/ozonescripts/stop.sh | 17 +
.../dist/src/main/compose/ozonesecure}/.env | 0
.../dist/src/main/compose/ozonesecure/README.md | 22 +
.../main/compose/ozonesecure/docker-compose.yaml | 75 +
.../src/main/compose/ozonesecure/docker-config | 104 +
.../docker-image/docker-krb5/Dockerfile-krb5 | 34 +
.../ozonesecure/docker-image/docker-krb5/README.md | 34 +
.../ozonesecure/docker-image/docker-krb5/kadm5.acl | 20 +
.../ozonesecure/docker-image/docker-krb5/krb5.conf | 40 +
.../docker-image/docker-krb5/launcher.sh | 25 +
.../ozonesecure/docker-image/runner/Dockerfile | 39 +
.../ozonesecure/docker-image/runner/build.sh | 26 +
.../docker-image/runner/scripts/envtoconf.py | 115 +
.../docker-image/runner/scripts/krb5.conf | 38 +
.../docker-image/runner/scripts/starter.sh | 100 +
.../docker-image/runner/scripts/transformation.py | 150 +
.../main/compose/ozonetrace/docker-compose.yaml | 65 +
.../dist/src/main/compose/ozonetrace/docker-config | 84 +
.../dist/src/main/conf/dn-audit-log4j2.properties | 90 +
hadoop-ozone/dist/src/main/conf/log4j.properties | 157 +
.../dist/src/main/conf/om-audit-log4j2.properties | 90 +
.../{common => dist}/src/main/conf/ozone-site.xml | 0
.../dist/src/main/conf/scm-audit-log4j2.properties | 90 +
.../dist/src/main/k8s/ozone/config-configmap.yaml | 34 +
.../main/k8s/ozone/datanode-public-service.yaml | 29 +
.../dist/src/main/k8s/ozone/datanode-service.yaml | 29 +
.../src/main/k8s/ozone/datanode-statefulset.yaml | 52 +
.../dist/src/main/k8s/ozone/om-public-service.yaml | 29 +
.../dist/src/main/k8s/ozone/om-service.yaml | 29 +
.../dist/src/main/k8s/ozone/om-statefulset.yaml | 72 +
.../src/main/k8s/ozone/s3g-public-service.yaml | 29 +
.../dist/src/main/k8s/ozone/s3g-service.yaml | 29 +
.../dist/src/main/k8s/ozone/s3g-statefulset.yaml | 49 +
.../src/main/k8s/ozone/scm-public-service.yaml | 29 +
.../dist/src/main/k8s/ozone/scm-service.yaml | 29 +
.../dist/src/main/k8s/ozone/scm-statefulset.yaml | 67 +
.../dist/src/main/k8s/prometheus/configmap.yaml | 51 +
.../dist/src/main/k8s/prometheus/deployment.yaml | 46 +
.../k8s/prometheus/prometheus-public-serivce.yaml | 28 +
.../dist/src/main/k8s/prometheus/role.yaml | 31 +
.../dist/src/main/k8s/prometheus/rolebindng.yaml | 27 +
.../main/k8s/prometheus/scm-public-service.yaml | 29 +
.../src/main/k8s/prometheus/service-account.yaml | 19 +
.../dist/src/main/k8s/prometheus/service.yaml | 25 +
hadoop-ozone/dist/src/main/ozone/README.txt | 51 +
hadoop-ozone/dist/src/main/smoketest/.env | 17 +
hadoop-ozone/dist/src/main/smoketest/README.md | 30 +
.../main/smoketest/auditparser/auditparser.robot | 40 +
.../dist/src/main/smoketest/basic/basic.robot | 47 +
.../src/main/smoketest/basic/ozone-shell.robot | 85 +
.../dist/src/main/smoketest/commonlib.robot | 31 +
.../dist/src/main/smoketest/ozonefs/ozonefs.robot | 112 +
.../src/main/smoketest/s3/MultipartUpload.robot | 207 +
hadoop-ozone/dist/src/main/smoketest/s3/README.md | 27 +
.../dist/src/main/smoketest/s3/__init__.robot | 21 +
.../dist/src/main/smoketest/s3/awss3.robot | 47 +
.../dist/src/main/smoketest/s3/buckethead.robot | 34 +
.../dist/src/main/smoketest/s3/bucketlist.robot | 32 +
.../dist/src/main/smoketest/s3/commonawslib.robot | 72 +
.../dist/src/main/smoketest/s3/objectcopy.robot | 66 +
.../dist/src/main/smoketest/s3/objectdelete.robot | 72 +
.../src/main/smoketest/s3/objectmultidelete.robot | 48 +
.../dist/src/main/smoketest/s3/objectputget.robot | 156 +
.../dist/src/main/smoketest/s3/webui.robot | 34 +
.../src/main/smoketest/security/ozone-secure.robot | 111 +
hadoop-ozone/dist/src/main/smoketest/test.sh | 159 +
hadoop-ozone/docs/README.md | 52 -
hadoop-ozone/docs/archetypes/default.md | 17 -
hadoop-ozone/docs/config.toml | 23 -
hadoop-ozone/docs/content/CommandShell.md | 158 -
hadoop-ozone/docs/content/GettingStarted.md | 369 -
hadoop-ozone/docs/content/Metrics.md | 170 -
hadoop-ozone/docs/content/Rest.md | 553 -
hadoop-ozone/docs/content/_index.md | 102 -
hadoop-ozone/docs/dev-support/bin/generate-site.sh | 29 -
hadoop-ozone/docs/pom.xml | 75 -
.../themes/ozonedoc/layouts/_default/single.html | 32 -
.../docs/themes/ozonedoc/layouts/index.html | 21 -
.../themes/ozonedoc/layouts/partials/footer.html | 19 -
.../themes/ozonedoc/layouts/partials/header.html | 31 -
.../themes/ozonedoc/layouts/partials/navbar.html | 33 -
.../themes/ozonedoc/layouts/partials/sidebar.html | 43 -
.../docs/themes/ozonedoc/static/css/ozonedoc.css | 128 -
.../static/fonts/glyphicons-halflings-regular.svg | 288 -
hadoop-ozone/integration-test/pom.xml | 19 +-
.../scm/container/TestContainerStateManager.java | 415 -
.../TestContainerStateManagerIntegration.java | 389 +
.../hdds/scm/pipeline/TestNode2PipelineMap.java | 85 +-
.../hadoop/hdds/scm/pipeline/TestNodeFailure.java | 134 +
.../hdds/scm/pipeline/TestPipelineClose.java | 205 +-
.../scm/pipeline/TestPipelineStateManager.java | 426 +
.../scm/pipeline/TestRatisPipelineProvider.java | 138 +
.../hdds/scm/pipeline/TestRatisPipelineUtils.java | 95 +
.../hdds/scm/pipeline/TestSCMPipelineManager.java | 188 +
.../hadoop/hdds/scm/pipeline/TestSCMRestart.java | 112 +
.../scm/pipeline/TestSimplePipelineProvider.java | 103 +
.../hadoop/hdds/scm/pipeline/package-info.java | 22 +
.../org/apache/hadoop/ozone/MiniOzoneCluster.java | 140 +-
.../apache/hadoop/ozone/MiniOzoneClusterImpl.java | 258 +-
.../hadoop/ozone/MiniOzoneHAClusterImpl.java | 261 +
.../org/apache/hadoop/ozone/OzoneTestUtils.java | 90 +-
.../org/apache/hadoop/ozone/RatisTestHelper.java | 29 +-
.../TestContainerStateMachineIdempotency.java | 124 +
.../apache/hadoop/ozone/TestMiniOzoneCluster.java | 104 +-
.../hadoop/ozone/TestOzoneConfigurationFields.java | 15 +-
.../hadoop/ozone/TestSecureOzoneCluster.java | 595 +
.../hadoop/ozone/TestStorageContainerManager.java | 133 +-
.../ozone/TestStorageContainerManagerHelper.java | 25 +-
.../ozone/client/CertificateClientTestImpl.java | 151 +
.../apache/hadoop/ozone/client/package-info.java | 20 +
.../ozone/client/rest/TestOzoneRestClient.java | 495 -
.../hadoop/ozone/client/rest/package-info.java | 23 -
.../apache/hadoop/ozone/client/rpc/TestBCSID.java | 148 +
.../rpc/TestCloseContainerHandlingByClient.java | 454 +-
.../client/rpc/TestContainerStateMachine.java | 152 +
.../rpc/TestContainerStateMachineFailures.java | 153 +
.../client/rpc/TestFailureHandlingByClient.java | 248 +
.../client/rpc/TestHybridPipelineOnDatanode.java | 166 +
.../ozone/client/rpc/TestOzoneRpcClient.java | 943 +-
.../client/rpc/TestOzoneRpcClientAbstract.java | 2040 ++
.../client/rpc/TestOzoneRpcClientWithRatis.java | 58 +
.../hadoop/ozone/client/rpc/TestReadRetries.java | 223 +
.../ozone/client/rpc/TestSecureOzoneRpcClient.java | 239 +
.../ozone/container/ContainerTestHelper.java | 306 +-
.../ozone/container/TestContainerReplication.java | 179 +
.../container/common/TestBlockDeletingService.java | 20 +-
.../container/common/helpers/TestBlockData.java | 132 +
.../container/common/helpers/TestKeyData.java | 119 -
.../common/impl/TestCloseContainerHandler.java | 260 -
.../impl/TestContainerDeletionChoosingPolicy.java | 13 +-
.../common/impl/TestContainerPersistence.java | 334 +-
.../commandhandler/TestBlockDeletion.java | 174 +-
.../TestCloseContainerByPipeline.java | 121 +-
.../commandhandler/TestCloseContainerHandler.java | 25 +-
.../commandhandler/TestDeleteContainerHandler.java | 277 +
.../TestReplicateContainerHandler.java | 70 -
.../statemachine/commandhandler/package-info.java | 21 +
.../transport/server/ratis/TestCSMMetrics.java | 42 +-
.../container/metrics/TestContainerMetrics.java | 50 +-
.../container/ozoneimpl/TestOzoneContainer.java | 168 +-
.../ozoneimpl/TestOzoneContainerRatis.java | 8 +-
.../ozoneimpl/TestOzoneContainerWithTLS.java | 190 +
.../ozoneimpl/TestSecureOzoneContainer.java | 227 +
.../container/server/TestContainerServer.java | 162 +-
.../server/TestContainerStateMachine.java | 201 -
.../server/TestSecureContainerServer.java | 239 +
.../hadoop/ozone/freon/TestDataValidate.java | 140 -
.../org/apache/hadoop/ozone/freon/TestFreon.java | 129 -
.../TestGenerateOzoneRequiredConfigurations.java | 131 -
.../ozone/om/TestContainerReportWithKeys.java | 5 +-
.../ozone/om/TestMultipleContainerReadWrite.java | 9 +-
.../org/apache/hadoop/ozone/om/TestOmAcls.java | 175 +
.../hadoop/ozone/om/TestOmBlockVersioning.java | 2 +-
.../org/apache/hadoop/ozone/om/TestOmInit.java | 103 +
.../org/apache/hadoop/ozone/om/TestOmMetrics.java | 175 +-
.../org/apache/hadoop/ozone/om/TestOmSQLCli.java | 285 -
.../apache/hadoop/ozone/om/TestOzoneManager.java | 332 +-
.../ozone/om/TestOzoneManagerConfiguration.java | 343 +
.../apache/hadoop/ozone/om/TestOzoneManagerHA.java | 156 +
.../apache/hadoop/ozone/om/TestScmChillMode.java | 371 +
.../ozone/ozShell/TestOzoneDatanodeShell.java | 220 +
.../hadoop/ozone/ozShell/TestOzoneShell.java | 857 +-
.../java/org/apache/hadoop/ozone/package-info.java | 22 +
.../hadoop/ozone/scm/TestAllocateContainer.java | 11 +-
.../hadoop/ozone/scm/TestContainerSQLCli.java | 246 -
.../hadoop/ozone/scm/TestContainerSmallFile.java | 76 +-
.../scm/TestGetCommittedBlockLengthAndPutKey.java | 123 +-
.../org/apache/hadoop/ozone/scm/TestSCMMXBean.java | 107 +-
.../hadoop/ozone/scm/TestSCMNodeManagerMXBean.java | 112 +
.../hadoop/ozone/scm/TestXceiverClientManager.java | 106 +-
.../hadoop/ozone/scm/TestXceiverClientMetrics.java | 12 +-
.../ozone/web/TestOzoneRestWithMiniCluster.java | 9 +-
.../apache/hadoop/ozone/web/TestOzoneVolumes.java | 4 +
.../hadoop/ozone/web/client/TestBuckets.java | 47 +-
.../hadoop/ozone/web/client/TestBucketsRatis.java | 1 +
.../apache/hadoop/ozone/web/client/TestKeys.java | 126 +-
.../hadoop/ozone/web/client/TestKeysRatis.java | 2 -
.../apache/hadoop/ozone/web/client/TestVolume.java | 131 +-
.../hadoop/ozone/web/client/TestVolumeRatis.java | 3 +-
.../hadoop/ozone/web/client/package-info.java | 22 +
.../org/apache/hadoop/ozone/web/package-info.java | 22 +
.../src/test/resources/log4j.properties | 5 +-
.../integration-test/src/test/resources/ssl/ca.crt | 27 +
.../integration-test/src/test/resources/ssl/ca.key | 54 +
.../src/test/resources/ssl/client.crt | 27 +
.../src/test/resources/ssl/client.csr | 26 +
.../src/test/resources/ssl/client.key | 51 +
.../src/test/resources/ssl/client.pem | 52 +
.../src/test/resources/ssl/generate.sh | 34 +
.../src/test/resources/ssl/server.crt | 27 +
.../src/test/resources/ssl/server.csr | 26 +
.../src/test/resources/ssl/server.key | 51 +
.../src/test/resources/ssl/server.pem | 52 +
hadoop-ozone/objectstore-service/pom.xml | 67 +-
.../hdfs/server/datanode/ObjectStoreHandler.java | 4 +-
.../hadoop/ozone/web/OzoneHddsDatanodeService.java | 4 +-
.../ozone/web/handlers/BucketProcessTemplate.java | 6 +-
.../hadoop/ozone/web/handlers/VolumeHandler.java | 6 +-
.../hadoop/ozone/web/interfaces/Accounting.java | 2 +-
.../apache/hadoop/ozone/web/interfaces/Bucket.java | 1 +
.../ozone/web/interfaces/StorageHandler.java | 1 +
.../apache/hadoop/ozone/web/interfaces/Volume.java | 13 +-
.../web/storage/DistributedStorageHandler.java | 96 +-
.../org/apache/hadoop/ozone/TestErrorCode.java | 53 -
.../org/apache/hadoop/ozone/web/TestErrorCode.java | 53 +
.../org/apache/hadoop/ozone/web/package-info.java | 22 +
hadoop-ozone/ozone-manager/pom.xml | 70 +-
.../org/apache/hadoop/ozone/om/BucketManager.java | 2 +-
.../apache/hadoop/ozone/om/BucketManagerImpl.java | 99 +-
.../apache/hadoop/ozone/om/KeyDeletingService.java | 163 +-
.../org/apache/hadoop/ozone/om/KeyManager.java | 89 +-
.../org/apache/hadoop/ozone/om/KeyManagerImpl.java | 975 +-
.../apache/hadoop/ozone/om/OMMetadataManager.java | 247 -
.../java/org/apache/hadoop/ozone/om/OMMetrics.java | 154 +
.../org/apache/hadoop/ozone/om/OMNodeDetails.java | 111 +
.../java/org/apache/hadoop/ozone/om/OMStorage.java | 6 +-
.../hadoop/ozone/om/OmMetadataManagerImpl.java | 421 +-
.../org/apache/hadoop/ozone/om/OmMetricsInfo.java | 43 +
.../org/apache/hadoop/ozone/om/OzoneManager.java | 1629 +-
.../hadoop/ozone/om/OzoneManagerHttpServer.java | 5 +-
.../apache/hadoop/ozone/om/S3BucketManager.java | 81 +
.../hadoop/ozone/om/S3BucketManagerImpl.java | 238 +
.../apache/hadoop/ozone/om/S3SecretManager.java | 30 +
.../hadoop/ozone/om/S3SecretManagerImpl.java | 82 +
.../hadoop/ozone/om/ServiceListJSONServlet.java | 5 +-
.../org/apache/hadoop/ozone/om/VolumeManager.java | 2 +-
.../apache/hadoop/ozone/om/VolumeManagerImpl.java | 244 +-
.../hadoop/ozone/om/exceptions/OMException.java | 118 -
.../hadoop/ozone/om/ratis/OMRatisHelper.java | 115 +
.../ozone/om/ratis/OzoneManagerRatisClient.java | 181 +
.../ozone/om/ratis/OzoneManagerRatisServer.java | 358 +
.../ozone/om/ratis/OzoneManagerStateMachine.java | 157 +
.../apache/hadoop/ozone/om/ratis/package-info.java | 22 +
...OzoneManagerProtocolServerSideTranslatorPB.java | 562 +-
.../protocolPB/OzoneManagerRequestHandler.java | 879 +
.../apache/hadoop/ozone/web/ozShell/Handler.java | 136 +-
.../hadoop/ozone/web/ozShell/OzoneAddress.java | 257 +
.../org/apache/hadoop/ozone/web/ozShell/Shell.java | 430 +-
.../ozone/web/ozShell/bucket/BucketCommands.java | 67 +
.../web/ozShell/bucket/CreateBucketHandler.java | 50 +-
.../web/ozShell/bucket/DeleteBucketHandler.java | 49 +-
.../web/ozShell/bucket/InfoBucketHandler.java | 52 +-
.../web/ozShell/bucket/ListBucketHandler.java | 98 +-
.../ozone/web/ozShell/bucket/S3BucketMapping.java | 66 +
.../web/ozShell/bucket/UpdateBucketHandler.java | 72 +-
.../ozone/web/ozShell/keys/DeleteKeyHandler.java | 51 +-
.../ozone/web/ozShell/keys/GetKeyHandler.java | 92 +-
.../ozone/web/ozShell/keys/InfoKeyHandler.java | 56 +-
.../hadoop/ozone/web/ozShell/keys/KeyCommands.java | 67 +
.../ozone/web/ozShell/keys/ListKeyHandler.java | 100 +-
.../ozone/web/ozShell/keys/PutKeyHandler.java | 94 +-
.../ozone/web/ozShell/keys/RenameKeyHandler.java | 73 +
.../ozone/web/ozShell/s3/GetS3SecretHandler.java | 49 +
.../hadoop/ozone/web/ozShell/s3/S3Commands.java | 60 +
.../hadoop/ozone/web/ozShell/s3/package-info.java | 21 +
.../web/ozShell/token/CancelTokenHandler.java | 72 +
.../ozone/web/ozShell/token/GetTokenHandler.java | 77 +
.../ozone/web/ozShell/token/PrintTokenHandler.java | 71 +
.../ozone/web/ozShell/token/RenewTokenHandler.java | 75 +
.../ozone/web/ozShell/token/TokenCommands.java | 64 +
.../ozone/web/ozShell/token/package-info.java | 26 +
.../web/ozShell/volume/CreateVolumeHandler.java | 79 +-
.../web/ozShell/volume/DeleteVolumeHandler.java | 43 +-
.../web/ozShell/volume/InfoVolumeHandler.java | 43 +-
.../web/ozShell/volume/ListVolumeHandler.java | 102 +-
.../web/ozShell/volume/UpdateVolumeHandler.java | 61 +-
.../ozone/web/ozShell/volume/VolumeCommands.java | 67 +
.../webapps/ozoneManager/index.html | 0
.../{ => resources}/webapps/ozoneManager/main.css | 0
.../{ => resources}/webapps/ozoneManager/main.html | 0
.../webapps/ozoneManager/om-metrics.html | 0
.../webapps/ozoneManager/ozoneManager.js | 0
.../ozone/om/ScmBlockLocationTestIngClient.java | 181 +
.../hadoop/ozone/om/TestBucketManagerImpl.java | 60 +-
.../apache/hadoop/ozone/om/TestChunkStreams.java | 141 +-
.../hadoop/ozone/om/TestKeyDeletingService.java | 205 +
.../apache/hadoop/ozone/om/TestKeyManagerImpl.java | 200 +
.../hadoop/ozone/om/TestOzoneManagerLock.java | 193 +
.../hadoop/ozone/om/TestS3BucketManager.java | 163 +
.../om/ratis/TestOzoneManagerRatisServer.java | 155 +
.../ozone/security/TestOzoneManagerBlockToken.java | 251 +
.../ozone/security/TestOzoneTokenIdentifier.java | 300 +
.../apache/hadoop/ozone/security/package-info.java | 21 +
.../hadoop/ozone/web/ozShell/TestOzoneAddress.java | 101 +
.../hadoop/ozone/web/ozShell/package-info.java | 21 +
hadoop-ozone/ozonefs-lib-legacy/pom.xml | 111 +
.../src/main/resources/ozonefs.txt | 21 +
hadoop-ozone/ozonefs-lib/pom.xml | 96 +
hadoop-ozone/ozonefs/pom.xml | 81 +-
.../org/apache/hadoop/fs/ozone/BasicKeyInfo.java | 53 +
.../hadoop/fs/ozone/FilteredClassLoader.java | 86 +
.../apache/hadoop/fs/ozone/OzoneClientAdapter.java | 55 +
.../hadoop/fs/ozone/OzoneClientAdapterFactory.java | 122 +
.../hadoop/fs/ozone/OzoneClientAdapterImpl.java | 246 +
.../apache/hadoop/fs/ozone/OzoneFSInputStream.java | 16 +-
.../hadoop/fs/ozone/OzoneFSOutputStream.java | 5 +-
.../hadoop/fs/ozone/OzoneFSStorageStatistics.java | 126 +
.../apache/hadoop/fs/ozone/OzoneFileSystem.java | 503 +-
.../java/org/apache/hadoop/fs/ozone/Statistic.java | 119 +
.../services/org.apache.hadoop.fs.FileSystem | 16 +
.../hadoop/fs/ozone/TestOzoneFSInputStream.java | 4 +-
.../hadoop/fs/ozone/TestOzoneFileInterfaces.java | 60 +-
.../hadoop/fs/ozone/TestOzoneFileSystem.java | 275 +
.../hadoop/fs/ozone/TestOzoneFsRenameDir.java | 121 +
.../contract/ITestOzoneContractGetFileStatus.java | 6 +-
.../org/apache/hadoop/fs/ozone/package-info.java | 22 +
.../ozonefs/src/test/resources/log4j.properties | 1 +
hadoop-ozone/pom.xml | 258 +-
hadoop-ozone/s3gateway/pom.xml | 220 +
.../s3/CommonHeadersContainerResponseFilter.java | 48 +
.../java/org/apache/hadoop/ozone/s3/Gateway.java | 66 +
.../apache/hadoop/ozone/s3/GatewayApplication.java | 29 +
.../apache/hadoop/ozone/s3/HeaderPreprocessor.java | 55 +
.../hadoop/ozone/s3/OzoneClientProducer.java | 48 +
.../hadoop/ozone/s3/OzoneConfigurationHolder.java | 43 +
.../apache/hadoop/ozone/s3/RequestIdentifier.java | 48 +
.../hadoop/ozone/s3/S3GatewayConfigKeys.java | 55 +
.../hadoop/ozone/s3/S3GatewayHttpServer.java | 85 +
.../hadoop/ozone/s3/SignedChunksInputStream.java | 139 +
.../hadoop/ozone/s3/VirtualHostStyleFilter.java | 165 +
.../ozone/s3/commontypes/BucketMetadata.java | 53 +
.../hadoop/ozone/s3/commontypes/CommonPrefix.java | 47 +
.../ozone/s3/commontypes/IsoDateAdapter.java | 47 +
.../hadoop/ozone/s3/commontypes/KeyMetadata.java | 87 +
.../hadoop/ozone/s3/commontypes/package-info.java | 29 +
.../hadoop/ozone/s3/endpoint/BucketEndpoint.java | 316 +
.../endpoint/CompleteMultipartUploadRequest.java | 77 +
.../endpoint/CompleteMultipartUploadResponse.java | 78 +
.../ozone/s3/endpoint/CopyObjectResponse.java | 63 +
.../hadoop/ozone/s3/endpoint/EndpointBase.java | 216 +
.../ozone/s3/endpoint/ListBucketResponse.java | 55 +
.../ozone/s3/endpoint/ListObjectResponse.java | 180 +
.../ozone/s3/endpoint/ListPartsResponse.java | 196 +
.../ozone/s3/endpoint/MultiDeleteRequest.java | 96 +
.../endpoint/MultiDeleteRequestUnmarshaller.java | 84 +
.../ozone/s3/endpoint/MultiDeleteResponse.java | 154 +
.../endpoint/MultipartUploadInitiateResponse.java | 69 +
.../hadoop/ozone/s3/endpoint/ObjectEndpoint.java | 739 +
.../hadoop/ozone/s3/endpoint/RootEndpoint.java | 81 +
.../ozone/s3/endpoint/XmlNamespaceFilter.java | 54 +
.../hadoop/ozone/s3/endpoint/package-info.java | 30 +
.../hadoop/ozone/s3/exception/OS3Exception.java | 161 +
.../ozone/s3/exception/OS3ExceptionMapper.java | 50 +
.../hadoop/ozone/s3/exception/S3ErrorTable.java | 99 +
.../hadoop/ozone/s3/exception/package-info.java | 21 +
.../s3/header/AuthenticationHeaderParser.java | 65 +
.../ozone/s3/header/AuthorizationHeaderV2.java | 97 +
.../ozone/s3/header/AuthorizationHeaderV4.java | 146 +
.../apache/hadoop/ozone/s3/header/Credential.java | 94 +
.../hadoop/ozone/s3/header/package-info.java | 22 +
.../hadoop/ozone/s3/io/S3WrapperInputStream.java | 79 +
.../apache/hadoop/ozone/s3/io/package-info.java | 23 +
.../org/apache/hadoop/ozone/s3/package-info.java | 22 +
.../apache/hadoop/ozone/s3/util/ContinueToken.java | 173 +
.../apache/hadoop/ozone/s3/util/RFC1123Util.java | 98 +
.../apache/hadoop/ozone/s3/util/RangeHeader.java | 89 +
.../ozone/s3/util/RangeHeaderParserUtil.java | 95 +
.../org/apache/hadoop/ozone/s3/util/S3Consts.java | 53 +
.../apache/hadoop/ozone/s3/util/S3StorageType.java | 55 +
.../apache/hadoop/ozone/s3/util/package-info.java | 22 +
.../src/main/resources/META-INF/beans.xml | 20 +
.../s3gateway/src/main/resources/browser.html | 617 +
.../resources/webapps/s3gateway/WEB-INF/beans.xml | 20 +
.../resources/webapps/s3gateway/WEB-INF/web.xml | 36 +
.../main/resources/webapps/static/images/ozone.ico | Bin 0 -> 1150 bytes
.../src/main/resources/webapps/static/index.html | 79 +
.../hadoop/ozone/client/ObjectStoreStub.java | 244 +
.../hadoop/ozone/client/OzoneBucketStub.java | 312 +
.../hadoop/ozone/client/OzoneClientStub.java | 37 +
.../hadoop/ozone/client/OzoneOutputStreamStub.java | 73 +
.../hadoop/ozone/client/OzoneVolumeStub.java | 110 +
.../apache/hadoop/ozone/client/package-info.java | 21 +
.../ozone/s3/TestSignedChunksInputStream.java | 114 +
.../ozone/s3/TestVirtualHostStyleFilter.java | 220 +
.../s3/endpoint/TestAbortMultipartUpload.java | 83 +
.../hadoop/ozone/s3/endpoint/TestBucketDelete.java | 100 +
.../hadoop/ozone/s3/endpoint/TestBucketGet.java | 381 +
.../hadoop/ozone/s3/endpoint/TestBucketHead.java | 71 +
.../ozone/s3/endpoint/TestBucketResponse.java | 38 +
.../s3/endpoint/TestInitiateMultipartUpload.java | 79 +
.../hadoop/ozone/s3/endpoint/TestListParts.java | 129 +
.../TestMultiDeleteRequestUnmarshaller.java | 78 +
.../s3/endpoint/TestMultipartUploadComplete.java | 222 +
.../hadoop/ozone/s3/endpoint/TestObjectDelete.java | 60 +
.../hadoop/ozone/s3/endpoint/TestObjectGet.java | 90 +
.../hadoop/ozone/s3/endpoint/TestObjectHead.java | 104 +
.../ozone/s3/endpoint/TestObjectMultiDelete.java | 116 +
.../hadoop/ozone/s3/endpoint/TestObjectPut.java | 270 +
.../hadoop/ozone/s3/endpoint/TestPartUpload.java | 126 +
.../hadoop/ozone/s3/endpoint/TestRootList.java | 72 +
.../hadoop/ozone/s3/endpoint/package-info.java | 21 +
.../ozone/s3/exception/TestOS3Exception.java | 50 +
.../hadoop/ozone/s3/exception/package-info.java | 22 +
.../ozone/s3/header/TestAuthorizationHeaderV2.java | 88 +
.../ozone/s3/header/TestAuthorizationHeaderV4.java | 97 +
.../org/apache/hadoop/ozone/s3/package-info.java | 21 +
.../hadoop/ozone/s3/util/TestContinueToken.java | 50 +
.../hadoop/ozone/s3/util/TestRFC1123Util.java | 44 +
.../ozone/s3/util/TestRangeHeaderParserUtil.java | 93 +
hadoop-ozone/tools/pom.xml | 48 +-
.../hadoop/ozone/audit/parser/AuditParser.java | 55 +
.../ozone/audit/parser/common/DatabaseHelper.java | 245 +
.../ozone/audit/parser/common/ParserConsts.java | 35 +
.../ozone/audit/parser/common/package-info.java | 20 +
.../audit/parser/handler/LoadCommandHandler.java | 52 +
.../audit/parser/handler/QueryCommandHandler.java | 57 +
.../parser/handler/TemplateCommandHandler.java | 61 +
.../ozone/audit/parser/handler/package-info.java | 20 +
.../ozone/audit/parser/model/AuditEntry.java | 188 +
.../ozone/audit/parser/model/package-info.java | 20 +
.../hadoop/ozone/audit/parser/package-info.java | 20 +
.../java/org/apache/hadoop/ozone/freon/Freon.java | 1137 +-
.../org/apache/hadoop/ozone/freon/ProgressBar.java | 147 +
.../hadoop/ozone/freon/RandomKeyGenerator.java | 981 +
.../apache/hadoop/ozone/fsck/BlockIdDetails.java | 83 +
.../apache/hadoop/ozone/fsck/ContainerMapper.java | 134 +
.../org/apache/hadoop/ozone/fsck/package-info.java | 44 +
.../GenerateOzoneRequiredConfigurations.java | 156 +-
.../ozone/genesis/BenchMarkBlockManager.java | 150 +
.../ozone/genesis/BenchMarkContainerStateMap.java | 65 +-
.../ozone/genesis/BenchMarkDatanodeDispatcher.java | 121 +-
.../ozone/genesis/BenchMarkMetadataStoreReads.java | 3 +
.../genesis/BenchMarkMetadataStoreWrites.java | 4 +-
.../ozone/genesis/BenchMarkOMKeyAllocation.java | 135 +
.../ozone/genesis/BenchMarkRocksDbStore.java | 6 +-
.../org/apache/hadoop/ozone/genesis/Genesis.java | 11 +-
.../apache/hadoop/ozone/genesis/GenesisUtil.java | 8 +-
.../org/apache/hadoop/ozone/scm/cli/SQLCLI.java | 14 +-
.../tools/src/main/resources/commands.properties | 22 +
.../hadoop/ozone/audit/parser/TestAuditParser.java | 191 +
.../hadoop/ozone/audit/parser/package-info.java | 21 +
.../hadoop/ozone/freon/TestDataValidate.java | 121 +
.../freon/TestFreonWithDatanodeFastRestart.java | 137 +
.../ozone/freon/TestFreonWithDatanodeRestart.java | 108 +
.../ozone/freon/TestFreonWithPipelineDestroy.java | 109 +
.../apache/hadoop/ozone/freon/TestProgressBar.java | 73 +
.../hadoop/ozone/freon/TestRandomKeyGenerator.java | 112 +
.../apache/hadoop/ozone/freon/package-info.java | 22 +
.../hadoop/ozone/fsck/TestContainerMapper.java | 117 +
.../org/apache/hadoop/ozone/fsck/package-info.java | 44 +
.../TestGenerateOzoneRequiredConfigurations.java | 232 +
.../apache/hadoop/ozone/genconf/package-info.java | 22 +
.../org/apache/hadoop/ozone/om/TestOmSQLCli.java | 286 +
.../org/apache/hadoop/ozone/om/package-info.java | 22 +
.../hadoop/ozone/scm/TestContainerSQLCli.java | 260 +
.../org/apache/hadoop/ozone/scm/package-info.java | 22 +
.../tools/src/test/resources/commands.properties | 22 +
.../tools/src/test/resources/testaudit.log | 15 +
hadoop-project-dist/pom.xml | 40 +-
hadoop-project/pom.xml | 380 +-
hadoop-project/src/site/site.xml | 10 +-
hadoop-tools/hadoop-aliyun/pom.xml | 8 +-
.../fs/aliyun/oss/AliyunOSSBlockOutputStream.java | 5 +-
.../fs/aliyun/oss/AliyunOSSCopyFileTask.java | 7 +-
.../hadoop/fs/aliyun/oss/AliyunOSSFileSystem.java | 46 +-
.../fs/aliyun/oss/AliyunOSSFileSystemStore.java | 63 +-
.../hadoop/fs/aliyun/oss/AliyunOSSUtils.java | 8 +-
.../org/apache/hadoop/fs/aliyun/oss/Constants.java | 2 +
.../java/org/apache/hadoop/fs/aliyun/oss/OSS.java | 43 +
.../apache/hadoop/fs/aliyun/oss/OSSFileStatus.java | 38 +
.../src/site/markdown/tools/hadoop-aliyun/index.md | 55 +-
.../hadoop/fs/aliyun/oss/AliyunOSSTestUtils.java | 19 +-
.../fs/aliyun/oss/TestAliyunCredentials.java | 25 +-
.../aliyun/oss/TestAliyunOSSBlockOutputStream.java | 74 +-
.../oss/TestAliyunOSSFileSystemContract.java | 27 +-
.../aliyun/oss/TestAliyunOSSFileSystemStore.java | 10 +-
.../fs/aliyun/oss/contract/AliyunOSSContract.java | 9 +
.../oss/contract/TestAliyunOSSContractDistCp.java | 1 -
.../aliyun/oss/fileContext/TestOSSFileContext.java | 28 +
.../fileContext/TestOSSFileContextCreateMkdir.java | 40 +
.../TestOSSFileContextMainOperations.java | 71 +
.../fileContext/TestOSSFileContextStatistics.java | 69 +
.../oss/fileContext/TestOSSFileContextURI.java | 51 +
.../oss/fileContext/TestOSSFileContextUtil.java | 40 +
.../apache/hadoop/fs/aliyun/oss/yarn/TestOSS.java | 83 +
.../fs/aliyun/oss/yarn/TestOSSMiniYarnCluster.java | 160 +
hadoop-tools/hadoop-archive-logs/pom.xml | 6 +-
hadoop-tools/hadoop-archives/pom.xml | 6 +-
.../hadoop-aws/dev-support/findbugs-exclude.xml | 5 +
hadoop-tools/hadoop-aws/pom.xml | 31 +-
.../hadoop/fs/s3a/AWSCredentialProviderList.java | 80 +-
.../hadoop/fs/s3a/BasicAWSCredentialsProvider.java | 62 -
.../java/org/apache/hadoop/fs/s3a/Constants.java | 108 +-
.../hadoop/fs/s3a/DefaultS3ClientFactory.java | 11 +-
.../hadoop/fs/s3a/InconsistentAmazonS3Client.java | 3 +-
.../apache/hadoop/fs/s3a/InternalConstants.java | 53 +
.../java/org/apache/hadoop/fs/s3a/Invoker.java | 2 +-
.../main/java/org/apache/hadoop/fs/s3a/S3A.java | 12 +-
.../apache/hadoop/fs/s3a/S3AEncryptionMethods.java | 43 +-
.../org/apache/hadoop/fs/s3a/S3AFileSystem.java | 689 +-
.../org/apache/hadoop/fs/s3a/S3AInputStream.java | 43 +-
.../apache/hadoop/fs/s3a/S3AInstrumentation.java | 36 +-
.../apache/hadoop/fs/s3a/S3AMultipartUploader.java | 35 +-
.../org/apache/hadoop/fs/s3a/S3AOpContext.java | 27 +-
.../org/apache/hadoop/fs/s3a/S3AReadOpContext.java | 97 +-
.../org/apache/hadoop/fs/s3a/S3ARetryPolicy.java | 24 +-
.../java/org/apache/hadoop/fs/s3a/S3AUtils.java | 211 +-
.../org/apache/hadoop/fs/s3a/S3ClientFactory.java | 6 +-
.../apache/hadoop/fs/s3a/S3ObjectAttributes.java | 27 +-
.../fs/s3a/SimpleAWSCredentialsProvider.java | 56 +-
.../java/org/apache/hadoop/fs/s3a/Statistic.java | 9 +-
.../fs/s3a/TemporaryAWSCredentialsProvider.java | 103 +-
.../apache/hadoop/fs/s3a/WriteOperationHelper.java | 109 +-
.../fs/s3a/auth/AbstractAWSCredentialProvider.java | 70 +
.../auth/AbstractSessionCredentialsProvider.java | 171 +
.../fs/s3a/auth/AssumedRoleCredentialProvider.java | 41 +-
.../s3a/auth/IAMInstanceCredentialsProvider.java | 75 +
.../fs/s3a/auth/MarshalledCredentialBinding.java | 208 +
.../fs/s3a/auth/MarshalledCredentialProvider.java | 92 +
.../hadoop/fs/s3a/auth/MarshalledCredentials.java | 409 +
.../hadoop/fs/s3a/auth/NoAuthWithAWSException.java | 8 +-
.../fs/s3a/auth/NoAwsCredentialsException.java | 69 +
.../org/apache/hadoop/fs/s3a/auth/RoleModel.java | 97 +-
.../apache/hadoop/fs/s3a/auth/RolePolicies.java | 190 +-
.../hadoop/fs/s3a/auth/STSClientFactory.java | 173 +-
.../fs/s3a/auth/delegation/AWSPolicyProvider.java | 59 +
.../fs/s3a/auth/delegation/AbstractDTService.java | 154 +
.../delegation/AbstractDelegationTokenBinding.java | 307 +
.../delegation/AbstractS3ATokenIdentifier.java | 307 +
.../s3a/auth/delegation/DelegationConstants.java | 165 +
.../delegation/DelegationTokenIOException.java | 50 +
.../delegation/EncryptionSecretOperations.java | 73 +
.../fs/s3a/auth/delegation/EncryptionSecrets.java | 221 +
.../delegation/FullCredentialsTokenBinding.java | 172 +
.../delegation/FullCredentialsTokenIdentifier.java | 50 +
.../fs/s3a/auth/delegation/RoleTokenBinding.java | 176 +
.../s3a/auth/delegation/RoleTokenIdentifier.java | 49 +
.../s3a/auth/delegation/S3ADelegationTokens.java | 686 +
.../fs/s3a/auth/delegation/S3ADtFetcher.java | 80 +
.../s3a/auth/delegation/SessionTokenBinding.java | 424 +
.../auth/delegation/SessionTokenIdentifier.java | 148 +
.../fs/s3a/auth/delegation/package-info.java | 34 +
.../apache/hadoop/fs/s3a/auth/package-info.java | 6 +-
.../apache/hadoop/fs/s3a/commit/DurationInfo.java | 39 +-
.../hadoop/fs/s3a/s3guard/DDBPathMetadata.java | 16 +-
.../hadoop/fs/s3a/s3guard/DirListingMetadata.java | 21 +-
.../fs/s3a/s3guard/DynamoDBMetadataStore.java | 579 +-
.../hadoop/fs/s3a/s3guard/ExpirableMetadata.java | 39 +
.../hadoop/fs/s3a/s3guard/LocalMetadataEntry.java | 3 +
.../hadoop/fs/s3a/s3guard/LocalMetadataStore.java | 55 +-
.../apache/hadoop/fs/s3a/s3guard/PathMetadata.java | 2 +-
.../s3guard/PathMetadataDynamoDBTranslation.java | 61 +-
.../org/apache/hadoop/fs/s3a/s3guard/S3Guard.java | 63 +-
.../s3a/s3guard/S3GuardDataAccessRetryPolicy.java | 47 +
.../apache/hadoop/fs/s3a/s3guard/S3GuardTool.java | 241 +-
.../fs/s3a/select/InternalSelectConstants.java | 77 +
.../apache/hadoop/fs/s3a/select/SelectBinding.java | 431 +
.../hadoop/fs/s3a/select/SelectConstants.java | 296 +
.../hadoop/fs/s3a/select/SelectInputStream.java | 457 +
.../apache/hadoop/fs/s3a/select/SelectTool.java | 355 +
.../apache/hadoop/fs/s3a/select/package-info.java | 27 +
.../apache/hadoop/fs/s3native/S3xLoginHelper.java | 123 +-
.../org.apache.hadoop.security.token.DtFetcher | 18 +
...rg.apache.hadoop.security.token.TokenIdentifier | 20 +
.../markdown/tools/hadoop-aws/assumed_roles.md | 289 +-
.../hadoop-aws/delegation_token_architecture.md | 466 +
.../markdown/tools/hadoop-aws/delegation_tokens.md | 870 +
.../src/site/markdown/tools/hadoop-aws/index.md | 119 +-
.../site/markdown/tools/hadoop-aws/s3_select.md | 1100 +
.../src/site/markdown/tools/hadoop-aws/s3guard.md | 192 +-
.../src/site/markdown/tools/hadoop-aws/testing.md | 189 +-
.../tools/hadoop-aws/troubleshooting_s3a.md | 62 +-
.../fs/contract/s3a/ITestS3AContractDistCp.java | 33 +
.../s3a/ITestS3AContractMultipartUploader.java | 64 +-
.../apache/hadoop/fs/s3a/AbstractS3ATestBase.java | 6 +-
.../fs/s3a/ITestS3AAWSCredentialsProvider.java | 24 +-
.../hadoop/fs/s3a/ITestS3AConfiguration.java | 51 +-
.../hadoop/fs/s3a/ITestS3ACopyFromLocalFile.java | 15 +-
.../hadoop/fs/s3a/ITestS3ACredentialsInURL.java | 164 -
.../ITestS3AEncryptionSSECBlockOutputStream.java | 45 -
...ptionSSEKMSUserDefinedKeyBlockOutputStream.java | 50 -
.../ITestS3AEncryptionSSES3BlockOutputStream.java | 44 -
.../hadoop/fs/s3a/ITestS3AFailureHandling.java | 11 +-
.../hadoop/fs/s3a/ITestS3AFileSystemContract.java | 5 +
.../fs/s3a/ITestS3ATemporaryCredentials.java | 364 +-
.../hadoop/fs/s3a/ITestS3GuardListConsistency.java | 18 +-
.../org/apache/hadoop/fs/s3a/ITestS3GuardTtl.java | 105 +
.../apache/hadoop/fs/s3a/MockS3AFileSystem.java | 19 +-
.../apache/hadoop/fs/s3a/MockS3ClientFactory.java | 5 +-
.../org/apache/hadoop/fs/s3a/S3ATestConstants.java | 20 +
.../org/apache/hadoop/fs/s3a/S3ATestUtils.java | 324 +-
.../fs/s3a/TestS3AAWSCredentialsProvider.java | 261 +-
.../apache/hadoop/fs/s3a/TestS3AGetFileStatus.java | 33 +-
.../fs/s3a/TestS3AMultipartUploaderSupport.java | 2 +-
.../apache/hadoop/fs/s3a/TestSSEConfiguration.java | 56 +-
.../apache/hadoop/fs/s3a/auth/ITestAssumeRole.java | 102 +-
.../s3a/auth/ITestAssumedRoleCommitOperations.java | 2 +-
.../apache/hadoop/fs/s3a/auth/RoleTestUtils.java | 47 +-
.../fs/s3a/auth/TestMarshalledCredentials.java | 138 +
.../s3a/auth/delegation/AbstractDelegationIT.java | 207 +
.../auth/delegation/CountInvocationsProvider.java | 52 +
.../hadoop/fs/s3a/auth/delegation/Csvout.java | 103 +
.../auth/delegation/ILoadTestRoleCredentials.java | 38 +
.../delegation/ILoadTestSessionCredentials.java | 295 +
.../s3a/auth/delegation/ITestDelegatedMRJob.java | 272 +
.../delegation/ITestRoleDelegationInFileystem.java | 68 +
.../auth/delegation/ITestRoleDelegationTokens.java | 122 +
.../ITestSessionDelegationInFileystem.java | 727 +
.../delegation/ITestSessionDelegationTokens.java | 282 +
.../delegation/MiniKerberizedHadoopCluster.java | 378 +
.../delegation/TestS3ADelegationTokenSupport.java | 171 +
.../hadoop/fs/s3a/commit/AbstractCommitITest.java | 2 +-
.../fs/s3a/commit/AbstractITCommitMRJob.java | 17 +-
.../fs/s3a/commit/AbstractITCommitProtocol.java | 14 -
.../fs/s3a/commit/staging/StagingTestBase.java | 30 +-
.../s3a/commit/staging/TestStagingCommitter.java | 6 +-
.../TestStagingDirectoryOutputCommitter.java | 22 +-
.../staging/TestStagingPartitionedFileListing.java | 4 +-
.../staging/TestStagingPartitionedJobCommit.java | 4 +-
.../staging/TestStagingPartitionedTaskCommit.java | 24 +-
.../fileContext/ITestS3AFileContextStatistics.java | 20 +-
.../s3a/s3guard/AbstractS3GuardToolTestBase.java | 193 +-
.../fs/s3a/s3guard/ITestDynamoDBMetadataStore.java | 147 +-
.../s3guard/ITestDynamoDBMetadataStoreScale.java | 595 +-
.../fs/s3a/s3guard/ITestS3GuardToolDynamoDB.java | 104 +-
.../fs/s3a/s3guard/ITestS3GuardToolLocal.java | 16 +-
.../fs/s3a/s3guard/MetadataStoreTestBase.java | 31 +-
.../fs/s3a/s3guard/S3GuardToolTestHelper.java | 89 +
.../fs/s3a/s3guard/TestDynamoDBMiscOperations.java | 94 +
.../TestPathMetadataDynamoDBTranslation.java | 48 +-
.../apache/hadoop/fs/s3a/s3guard/TestS3Guard.java | 6 +-
.../scale/AbstractITestS3AMetadataStoreScale.java | 24 +-
.../apache/hadoop/fs/s3a/scale/NanoTimerStats.java | 192 +
.../hadoop/fs/s3a/select/AbstractS3SelectTest.java | 746 +
.../org/apache/hadoop/fs/s3a/select/CsvFile.java | 138 +
.../apache/hadoop/fs/s3a/select/ITestS3Select.java | 967 +
.../hadoop/fs/s3a/select/ITestS3SelectCLI.java | 347 +
.../hadoop/fs/s3a/select/ITestS3SelectLandsat.java | 432 +
.../hadoop/fs/s3a/select/ITestS3SelectMRJob.java | 206 +
.../fs/s3a/yarn/ITestS3AMiniYarnCluster.java | 38 +-
.../hadoop/fs/s3native/TestS3xLoginHelper.java | 70 +-
.../java/org/apache/hadoop/mapreduce/MockJob.java | 116 +
.../hadoop-aws/src/test/resources/core-site.xml | 10 +
hadoop-tools/hadoop-azure-datalake/pom.xml | 4 +-
hadoop-tools/hadoop-azure/pom.xml | 419 +-
.../src/config/checkstyle-suppressions.xml | 49 +
.../fs/azure/AzureNativeFileSystemStore.java | 67 +-
.../hadoop/fs/azure/ClientThrottlingAnalyzer.java | 2 +-
.../hadoop/fs/azure/PageBlobOutputStream.java | 21 +-
.../java/org/apache/hadoop/fs/azurebfs/Abfs.java | 46 +
.../hadoop/fs/azurebfs/AbfsConfiguration.java | 638 +
.../java/org/apache/hadoop/fs/azurebfs/Abfss.java | 46 +
.../hadoop/fs/azurebfs/AzureBlobFileSystem.java | 1086 +
.../fs/azurebfs/AzureBlobFileSystemStore.java | 1038 +
.../fs/azurebfs/SecureAzureBlobFileSystem.java | 39 +
.../fs/azurebfs/constants/AbfsHttpConstants.java | 95 +
.../fs/azurebfs/constants/ConfigurationKeys.java | 114 +
.../constants/FileSystemConfigurations.java | 71 +
.../azurebfs/constants/FileSystemUriSchemes.java | 42 +
.../constants/HttpHeaderConfigurations.java | 63 +
.../fs/azurebfs/constants/HttpQueryParams.java | 41 +
.../hadoop/fs/azurebfs/constants/package-info.java | 22 +
.../ConfigurationValidationAnnotations.java | 104 +
.../contracts/annotations/package-info.java | 22 +
.../diagnostics/ConfigurationValidator.java | 37 +
.../contracts/diagnostics/package-info.java | 22 +
.../exceptions/AbfsRestOperationException.java | 103 +
.../exceptions/AzureBlobFileSystemException.java | 56 +
.../ConfigurationPropertyNotFoundException.java | 32 +
.../FileSystemOperationUnhandledException.java | 33 +
.../InvalidAbfsRestOperationException.java | 40 +
.../exceptions/InvalidAclOperationException.java | 33 +
.../InvalidConfigurationValueException.java | 37 +
.../InvalidFileSystemPropertyException.java | 33 +
.../exceptions/InvalidUriAuthorityException.java | 33 +
.../contracts/exceptions/InvalidUriException.java | 33 +
.../contracts/exceptions/KeyProviderException.java | 42 +
.../contracts/exceptions/TimeoutException.java | 33 +
.../exceptions/TokenAccessProviderException.java | 36 +
.../contracts/exceptions/package-info.java | 22 +
.../hadoop/fs/azurebfs/contracts/package-info.java | 22 +
.../contracts/services/AzureServiceErrorCode.java | 115 +
.../contracts/services/ListResultEntrySchema.java | 239 +
.../contracts/services/ListResultSchema.java | 58 +
.../contracts/services/ReadBufferStatus.java | 29 +
.../azurebfs/contracts/services/package-info.java | 22 +
.../Base64StringConfigurationBasicValidator.java | 50 +
.../BooleanConfigurationBasicValidator.java | 50 +
.../diagnostics/ConfigurationBasicValidator.java | 67 +
.../IntegerConfigurationBasicValidator.java | 68 +
.../LongConfigurationBasicValidator.java | 63 +
.../StringConfigurationBasicValidator.java | 43 +
.../fs/azurebfs/diagnostics/package-info.java | 22 +
.../extensions/AbfsAuthorizationException.java | 41 +
.../fs/azurebfs/extensions/AbfsAuthorizer.java | 57 +
.../extensions/CustomDelegationTokenManager.java | 70 +
.../extensions/CustomTokenProviderAdaptee.java | 75 +
.../fs/azurebfs/extensions/package-info.java | 32 +
.../fs/azurebfs/oauth2/AccessTokenProvider.java | 98 +
.../fs/azurebfs/oauth2/AzureADAuthenticator.java | 346 +
.../hadoop/fs/azurebfs/oauth2/AzureADToken.java | 47 +
.../azurebfs/oauth2/ClientCredsTokenProvider.java | 62 +
.../oauth2/CustomTokenProviderAdapter.java | 58 +
.../fs/azurebfs/oauth2/IdentityTransformer.java | 278 +
.../fs/azurebfs/oauth2/MsiTokenProvider.java | 48 +
.../hadoop/fs/azurebfs/oauth2/QueryParams.java | 69 +
.../oauth2/RefreshTokenBasedTokenProvider.java | 57 +
.../azurebfs/oauth2/UserPasswordTokenProvider.java | 56 +
.../hadoop/fs/azurebfs/oauth2/package-info.java | 18 +
.../org/apache/hadoop/fs/azurebfs/package.html | 31 +
.../security/AbfsDelegationTokenIdentifier.java | 49 +
.../security/AbfsDelegationTokenManager.java | 88 +
.../fs/azurebfs/security/AbfsTokenRenewer.java | 96 +
.../hadoop/fs/azurebfs/security/package-info.java | 23 +
.../hadoop/fs/azurebfs/services/AbfsAclHelper.java | 278 +
.../hadoop/fs/azurebfs/services/AbfsClient.java | 588 +
.../services/AbfsClientThrottlingAnalyzer.java | 272 +
.../services/AbfsClientThrottlingIntercept.java | 135 +
.../fs/azurebfs/services/AbfsHttpHeader.java | 40 +
.../fs/azurebfs/services/AbfsHttpOperation.java | 446 +
.../fs/azurebfs/services/AbfsInputStream.java | 391 +
.../fs/azurebfs/services/AbfsOutputStream.java | 392 +
.../fs/azurebfs/services/AbfsPermission.java | 114 +
.../fs/azurebfs/services/AbfsRestOperation.java | 208 +
.../azurebfs/services/AbfsRestOperationType.java | 43 +
.../fs/azurebfs/services/AbfsUriQueryBuilder.java | 64 +
.../hadoop/fs/azurebfs/services/AuthType.java | 27 +
.../azurebfs/services/ExponentialRetryPolicy.java | 144 +
.../hadoop/fs/azurebfs/services/KeyProvider.java | 43 +
.../hadoop/fs/azurebfs/services/ReadBuffer.java | 139 +
.../fs/azurebfs/services/ReadBufferManager.java | 395 +
.../fs/azurebfs/services/ReadBufferWorker.java | 72 +
.../fs/azurebfs/services/SharedKeyCredentials.java | 510 +
.../services/ShellDecryptionKeyProvider.java | 71 +
.../fs/azurebfs/services/SimpleKeyProvider.java | 54 +
.../hadoop/fs/azurebfs/services/package-info.java | 22 +
.../apache/hadoop/fs/azurebfs/utils/Base64.java | 329 +
.../fs/azurebfs/utils/SSLSocketFactoryEx.java | 243 +
.../apache/hadoop/fs/azurebfs/utils/UriUtils.java | 78 +
.../hadoop/fs/azurebfs/utils/package-info.java | 22 +
...rg.apache.hadoop.security.token.TokenIdentifier | 1 +
.../org.apache.hadoop.security.token.TokenRenewer | 1 +
.../hadoop-azure/src/site/markdown/abfs.md | 82 +
.../src/site/markdown/testing_azure.md | 230 +-
.../fs/azure/AzureBlobStorageTestAccount.java | 22 +-
.../hadoop/fs/azure/ITestContainerChecks.java | 6 -
.../ITestFileSystemOperationExceptionMessage.java | 3 +-
.../fs/azure/ITestNativeAzureFileSystemLive.java | 4 +-
.../fs/azure/ITestNativeFileSystemStatistics.java | 99 +
.../fs/azure/ITestOutputStreamSemantics.java | 43 +
.../hadoop/fs/azure/ITestWasbRemoteCallHelper.java | 69 +-
.../fs/azure/ITestWasbUriAndConfiguration.java | 26 -
.../fs/azure/NativeAzureFileSystemBaseTest.java | 80 +-
.../fs/azure/TestKeyPageBlobDirectories.java | 170 +
.../fs/azure/integration/AzureTestConstants.java | 6 +-
.../fs/azure/integration/AzureTestUtils.java | 69 +
.../ITestAzureFileSystemInstrumentation.java | 57 +-
.../fs/azure/metrics/TestRollingWindowAverage.java | 4 +-
.../fs/azurebfs/AbstractAbfsIntegrationTest.java | 345 +
.../hadoop/fs/azurebfs/AbstractAbfsScaleTest.java | 59 +
.../fs/azurebfs/AbstractAbfsTestWithTimeout.java | 70 +
.../apache/hadoop/fs/azurebfs/ITestAbfsClient.java | 76 +
.../fs/azurebfs/ITestAbfsIdentityTransformer.java | 301 +
.../fs/azurebfs/ITestAbfsReadWriteAndSeek.java | 89 +
.../azurebfs/ITestAbfsRestOperationException.java | 75 +
.../azurebfs/ITestAzureBlobFileSystemAppend.java | 79 +
.../ITestAzureBlobFileSystemAuthorization.java | 366 +
.../ITestAzureBlobFileSystemBackCompat.java | 86 +
.../fs/azurebfs/ITestAzureBlobFileSystemCopy.java | 96 +
.../azurebfs/ITestAzureBlobFileSystemCreate.java | 107 +
.../azurebfs/ITestAzureBlobFileSystemDelete.java | 133 +
.../fs/azurebfs/ITestAzureBlobFileSystemE2E.java | 226 +
.../azurebfs/ITestAzureBlobFileSystemE2EScale.java | 120 +
.../ITestAzureBlobFileSystemFileStatus.java | 125 +
.../azurebfs/ITestAzureBlobFileSystemFinalize.java | 62 +
.../fs/azurebfs/ITestAzureBlobFileSystemFlush.java | 382 +
.../ITestAzureBlobFileSystemInitAndCreate.java | 52 +
.../ITestAzureBlobFileSystemListStatus.java | 239 +
.../fs/azurebfs/ITestAzureBlobFileSystemMkDir.java | 48 +
.../fs/azurebfs/ITestAzureBlobFileSystemOauth.java | 178 +
.../ITestAzureBlobFileSystemPermission.java | 108 +
.../ITestAzureBlobFileSystemRandomRead.java | 586 +
.../azurebfs/ITestAzureBlobFileSystemRename.java | 152 +
.../ITestAzureBlobFileSystemRenameUnicode.java | 98 +
.../fs/azurebfs/ITestAzureBlobFilesystemAcl.java | 1410 +
.../hadoop/fs/azurebfs/ITestClientUrlScheme.java | 101 +
.../fs/azurebfs/ITestFileSystemInitialization.java | 77 +
.../fs/azurebfs/ITestFileSystemProperties.java | 119 +
.../fs/azurebfs/ITestFileSystemRegistration.java | 113 +
.../fs/azurebfs/ITestGetNameSpaceEnabled.java | 96 +
.../fs/azurebfs/ITestOauthOverAbfsScheme.java | 63 +
.../fs/azurebfs/ITestWasbAbfsCompatibility.java | 194 +
.../TestAbfsConfigurationFieldsValidation.java | 179 +
.../fs/azurebfs/TestAccountConfiguration.java | 285 +
.../azurebfs/constants/TestConfigurationKeys.java | 42 +
.../hadoop/fs/azurebfs/constants/package-info.java | 22 +
.../azurebfs/contract/ABFSContractTestBinding.java | 67 +
.../azurebfs/contract/AbfsFileSystemContract.java | 64 +
.../ITestAbfsFileSystemContractAppend.java | 61 +
.../ITestAbfsFileSystemContractConcat.java | 51 +
.../ITestAbfsFileSystemContractCreate.java | 52 +
.../ITestAbfsFileSystemContractDelete.java | 52 +
.../ITestAbfsFileSystemContractDistCp.java | 52 +
.../ITestAbfsFileSystemContractGetFileStatus.java | 51 +
.../contract/ITestAbfsFileSystemContractMkdir.java | 52 +
.../contract/ITestAbfsFileSystemContractOpen.java | 52 +
.../ITestAbfsFileSystemContractRename.java | 52 +
.../ITestAbfsFileSystemContractRootDirectory.java | 57 +
.../ITestAbfsFileSystemContractSecureDistCp.java | 49 +
.../contract/ITestAbfsFileSystemContractSeek.java | 52 +
.../ITestAbfsFileSystemContractSetTimes.java | 51 +
.../contract/ITestAzureBlobFileSystemBasics.java | 105 +
.../hadoop/fs/azurebfs/contract/package-info.java | 22 +
.../diagnostics/TestConfigurationValidators.java | 121 +
.../fs/azurebfs/diagnostics/package-info.java | 22 +
.../fs/azurebfs/extensions/MockAbfsAuthorizer.java | 87 +
.../fs/azurebfs/extensions/package-info.java | 22 +
.../apache/hadoop/fs/azurebfs/package-info.java | 22 +
.../fs/azurebfs/services/TestAbfsClient.java | 94 +
.../services/TestAbfsClientThrottlingAnalyzer.java | 177 +
.../fs/azurebfs/services/TestQueryParams.java | 72 +
.../services/TestShellDecryptionKeyProvider.java | 92 +
.../hadoop/fs/azurebfs/services/package-info.java | 22 +
.../hadoop/fs/azurebfs/utils/AbfsTestUtils.java | 85 +
.../hadoop/fs/azurebfs/utils/AclTestHelpers.java | 119 +
.../hadoop/fs/azurebfs/utils/Parallelized.java | 60 +
.../hadoop/fs/azurebfs/utils/TestUriUtils.java | 48 +
.../hadoop/fs/azurebfs/utils/package-info.java | 22 +
.../hadoop-azure/src/test/resources/abfs.xml | 64 +
.../hadoop-azure/src/test/resources/azure-test.xml | 56 +-
.../src/test/resources/log4j.properties | 34 +
hadoop-tools/hadoop-datajoin/pom.xml | 6 +-
hadoop-tools/hadoop-distcp/pom.xml | 6 +-
.../apache/hadoop/tools/CopyListingFileStatus.java | 13 +-
.../org/apache/hadoop/tools/DistCpConstants.java | 7 +-
.../org/apache/hadoop/tools/DistCpContext.java | 4 +
.../apache/hadoop/tools/DistCpOptionSwitch.java | 16 +-
.../org/apache/hadoop/tools/DistCpOptions.java | 19 +
.../org/apache/hadoop/tools/OptionsParser.java | 4 +-
.../apache/hadoop/tools/mapred/CopyCommitter.java | 8 +-
.../org/apache/hadoop/tools/mapred/CopyMapper.java | 6 +-
.../tools/mapred/RetriableFileCopyCommand.java | 52 +-
.../org/apache/hadoop/tools/util/DistCpUtils.java | 1 +
.../hadoop-distcp/src/site/markdown/DistCp.md.vm | 7 +-
.../hadoop/tools/TestCopyListingFileStatus.java | 10 +
.../org/apache/hadoop/tools/TestDistCpOptions.java | 34 +-
.../tools/contract/AbstractContractDistCpTest.java | 68 +-
.../hadoop/tools/mapred/TestCopyCommitter.java | 4 -
.../apache/hadoop/tools/util/TestDistCpUtils.java | 88 +-
hadoop-tools/hadoop-extras/pom.xml | 16 +-
hadoop-tools/hadoop-fs2img/pom.xml | 6 +-
.../hadoop/hdfs/server/namenode/ImageWriter.java | 5 +-
.../namenode/ITestProvidedImplementation.java | 139 +-
hadoop-tools/hadoop-gridmix/pom.xml | 16 +-
.../hadoop/mapred/gridmix/TestGridMixClasses.java | 8 +-
hadoop-tools/hadoop-kafka/pom.xml | 6 +-
.../hadoop/metrics2/impl/TestKafkaMetrics.java | 4 +-
hadoop-tools/hadoop-openstack/pom.xml | 4 +-
hadoop-tools/hadoop-pipes/pom.xml | 4 +-
hadoop-tools/hadoop-resourceestimator/pom.xml | 2 +-
hadoop-tools/hadoop-rumen/pom.xml | 4 +-
.../tools/rumen/Pre21JobHistoryConstants.java | 11 +-
hadoop-tools/hadoop-sls/pom.xml | 4 +-
.../java/org/apache/hadoop/yarn/sls/SLSRunner.java | 40 +-
.../hadoop/yarn/sls/appmaster/AMSimulator.java | 10 +-
.../hadoop/yarn/sls/appmaster/MRAMSimulator.java | 9 +-
.../yarn/sls/appmaster/StreamAMSimulator.java | 5 +-
.../hadoop/yarn/sls/nodemanager/NodeInfo.java | 13 +-
.../yarn/sls/resourcemanager/MockAMLauncher.java | 54 +-
.../hadoop/yarn/sls/scheduler/RMNodeWrapper.java | 6 +
.../hadoop/yarn/sls/appmaster/TestAMSimulator.java | 10 +-
hadoop-tools/hadoop-streaming/pom.xml | 16 +-
.../streaming/mapreduce/StreamInputFormat.java | 14 +-
hadoop-tools/hadoop-tools-dist/pom.xml | 4 +-
hadoop-tools/pom.xml | 4 +-
hadoop-yarn-project/hadoop-yarn/bin/yarn | 6 +
.../hadoop-yarn/conf/container-executor.cfg | 1 +
hadoop-yarn-project/hadoop-yarn/conf/yarn-env.sh | 1 +
.../hadoop-yarn/dev-support/findbugs-exclude.xml | 16 -
.../jdiff/Apache_Hadoop_YARN_Client_3.1.2.xml | 2947 ++
.../jdiff/Apache_Hadoop_YARN_Common_3.1.2.xml | 3566 ++
.../hadoop-yarn/hadoop-yarn-api/pom.xml | 6 +-
.../hadoop/yarn/api/ApplicationClientProtocol.java | 55 +
.../hadoop/yarn/api/ApplicationConstants.java | 53 +-
.../yarn/api/ContainerManagementProtocol.java | 19 +
.../org/apache/hadoop/yarn/api/CsiAdaptorPB.java | 31 +
.../apache/hadoop/yarn/api/CsiAdaptorPlugin.java | 50 +
.../apache/hadoop/yarn/api/CsiAdaptorProtocol.java | 83 +
.../GetAttributesToNodesRequest.java | 74 +
.../GetAttributesToNodesResponse.java | 65 +
.../GetClusterNodeAttributesRequest.java | 47 +
.../GetClusterNodeAttributesResponse.java | 73 +
.../GetLocalizationStatusesRequest.java | 69 +
.../GetLocalizationStatusesResponse.java | 87 +
.../GetNodesToAttributesRequest.java | 65 +
.../GetNodesToAttributesResponse.java | 63 +
.../api/protocolrecords/GetPluginInfoRequest.java | 30 +
.../api/protocolrecords/GetPluginInfoResponse.java | 43 +
.../protocolrecords/NodePublishVolumeRequest.java | 94 +
.../protocolrecords/NodePublishVolumeResponse.java | 31 +
.../NodeUnpublishVolumeRequest.java | 44 +
.../NodeUnpublishVolumeResponse.java | 31 +
.../ValidateVolumeCapabilitiesRequest.java | 117 +
.../ValidateVolumeCapabilitiesResponse.java | 46 +
.../hadoop/yarn/api/records/ApplicationReport.java | 49 +-
.../apache/hadoop/yarn/api/records/Container.java | 17 +-
.../hadoop/yarn/api/records/ContainerReport.java | 20 +-
.../hadoop/yarn/api/records/ContainerStatus.java | 18 +
.../hadoop/yarn/api/records/LocalizationState.java | 36 +
.../yarn/api/records/LocalizationStatus.java | 95 +
.../hadoop/yarn/api/records/NodeAttribute.java | 92 +
.../hadoop/yarn/api/records/NodeAttributeInfo.java | 62 +
.../hadoop/yarn/api/records/NodeAttributeKey.java | 66 +
.../yarn/api/records/NodeAttributeOpCode.java | 43 +
.../hadoop/yarn/api/records/NodeAttributeType.java | 35 +
.../apache/hadoop/yarn/api/records/NodeReport.java | 13 +
.../apache/hadoop/yarn/api/records/NodeState.java | 12 +
.../yarn/api/records/NodeToAttributeValue.java | 57 +
.../apache/hadoop/yarn/api/records/Resource.java | 7 +-
.../yarn/api/records/ResourceInformation.java | 75 +-
.../yarn/api/records/ResourceUtilization.java | 8 +-
.../yarn/api/records/ShellContainerCommand.java | 32 +
.../yarn/api/resource/PlacementConstraint.java | 40 +-
.../yarn/api/resource/PlacementConstraints.java | 19 +
.../apache/hadoop/yarn/conf/YarnConfiguration.java | 240 +-
.../api/ResourceManagerAdministrationProtocol.java | 13 +-
.../AttributeMappingOperationType.java | 42 +
.../api/protocolrecords/NodeToAttributes.java | 59 +
.../NodesToAttributesMappingRequest.java | 69 +
.../NodesToAttributesMappingResponse.java | 31 +
.../util/constraint/PlacementConstraintParser.java | 193 +-
.../hadoop/yarn/util/csi/CsiConfigUtils.java | 79 +
.../apache/hadoop/yarn/util/csi/package-info.java | 21 +
.../hadoop/yarn/util/resource/ResourceUtils.java | 145 +-
.../src/main/proto/YarnCsiAdaptor.proto | 40 +
.../main/proto/applicationclient_protocol.proto | 3 +
.../main/proto/containermanagement_protocol.proto | 4 +
.../resourcemanager_administration_protocol.proto | 1 +
...arn_server_resourcemanager_service_protos.proto | 16 +
.../src/main/proto/yarn_csi_adaptor.proto | 93 +
.../src/main/proto/yarn_protos.proto | 49 +
.../src/main/proto/yarn_service_protos.proto | 51 +
.../resource/TestPlacementConstraintParser.java | 160 +-
.../hadoop/yarn/conf/TestResourceInformation.java | 44 +
.../yarn/conf/TestYarnConfigurationFields.java | 2 +
.../pom.xml | 7 +-
.../distributedshell/ApplicationMaster.java | 118 +-
.../yarn/applications/distributedshell/Client.java | 102 +-
.../distributedshell/PlacementSpec.java | 19 +-
.../distributedshell/TestDSAppMaster.java | 8 +-
.../TestDSWithMultipleNodeManager.java | 361 +
.../distributedshell/TestDistributedShell.java | 195 +-
.../TestDistributedShellWithNodeLabels.java | 165 -
.../src/test/resources/a.txt | 15 +
.../src/test/resources/b.txt | 15 +
.../pom.xml | 4 +-
.../hadoop-yarn-services-api/pom.xml | 6 +-
.../yarn/service/client/ApiServiceClient.java | 129 +-
.../hadoop/yarn/service/webapp/ApiServer.java | 268 +-
.../YARN-Simplified-V1-API-Layer-For-Services.yaml | 164 +-
.../hadoop/yarn/service/ServiceClientTest.java | 2 +-
.../service/client/TestSecureApiServiceClient.java | 115 +-
.../hadoop-yarn-services-core/pom.xml | 6 +-
.../hadoop/yarn/service/ClientAMProtocol.java | 11 +
.../hadoop/yarn/service/ClientAMService.java | 41 +-
.../apache/hadoop/yarn/service/ServiceEvent.java | 14 +-
.../hadoop/yarn/service/ServiceEventType.java | 3 +-
.../apache/hadoop/yarn/service/ServiceManager.java | 332 +-
.../apache/hadoop/yarn/service/ServiceMaster.java | 7 +
.../hadoop/yarn/service/ServiceScheduler.java | 162 +-
.../yarn/service/UpgradeComponentsFinder.java | 8 +-
.../hadoop/yarn/service/api/records/Artifact.java | 1 -
.../hadoop/yarn/service/api/records/Component.java | 27 +-
.../service/api/records/ComponentContainers.java | 97 +
.../yarn/service/api/records/ConfigFile.java | 4 +-
.../yarn/service/api/records/ConfigFormat.java | 5 +-
.../yarn/service/api/records/Configuration.java | 1 -
.../hadoop/yarn/service/api/records/Container.java | 45 +-
.../yarn/service/api/records/ContainerState.java | 2 +-
.../hadoop/yarn/service/api/records/Error.java | 1 -
.../service/api/records/KerberosPrincipal.java | 2 -
.../service/api/records/LocalizationStatus.java | 132 +
.../service/api/records/PlacementConstraint.java | 3 -
.../yarn/service/api/records/PlacementPolicy.java | 3 -
.../yarn/service/api/records/PlacementScope.java | 3 -
.../yarn/service/api/records/PlacementType.java | 3 -
.../yarn/service/api/records/ReadinessCheck.java | 1 -
.../hadoop/yarn/service/api/records/Resource.java | 1 -
.../service/api/records/ResourceInformation.java | 37 +-
.../hadoop/yarn/service/api/records/Service.java | 16 +-
.../yarn/service/api/records/ServiceState.java | 8 +-
.../yarn/service/api/records/ServiceStatus.java | 1 -
.../hadoop/yarn/service/client/ServiceClient.java | 285 +-
.../hadoop/yarn/service/component/Component.java | 350 +-
.../yarn/service/component/ComponentEvent.java | 20 +-
.../yarn/service/component/ComponentEventType.java | 4 +-
.../yarn/service/component/ComponentState.java | 3 +-
.../yarn/service/component/NeverRestartPolicy.java | 11 +-
.../service/component/OnFailureRestartPolicy.java | 12 +-
.../component/instance/ComponentInstance.java | 556 +-
.../instance/ComponentInstanceEventType.java | 3 +-
.../component/instance/ComponentInstanceState.java | 4 +-
.../hadoop/yarn/service/conf/YarnServiceConf.java | 5 +-
.../yarn/service/conf/YarnServiceConstants.java | 2 +
.../containerlaunch/ContainerLaunchService.java | 32 +-
.../service/exceptions/RestApiErrorMessages.java | 2 -
.../pb/client/ClientAMProtocolPBClientImpl.java | 27 +
.../pb/service/ClientAMProtocolPBServiceImpl.java | 26 +
.../yarn/service/monitor/ServiceMonitor.java | 5 +-
.../service/provider/AbstractProviderService.java | 12 +-
.../yarn/service/provider/ProviderService.java | 40 +-
.../yarn/service/provider/ProviderUtils.java | 28 +-
.../ServiceTimelineMetricsConstants.java | 2 +
.../timelineservice/ServiceTimelinePublisher.java | 2 +
.../hadoop/yarn/service/utils/ConfigUtils.java | 56 -
.../hadoop/yarn/service/utils/FilterUtils.java | 25 +-
.../apache/hadoop/yarn/service/utils/HttpUtil.java | 123 +
.../yarn/service/utils/PublishedConfiguration.java | 9 -
.../utils/PublishedConfigurationOutputter.java | 2 -
.../hadoop/yarn/service/utils/ServiceApiUtil.java | 143 +-
.../yarn/service/utils/SliderFileSystem.java | 49 +
.../src/main/proto/ClientAMProtocol.proto | 19 +-
.../yarn/service/MockRunningServiceContext.java | 65 +-
.../apache/hadoop/yarn/service/MockServiceAM.java | 8 +-
.../hadoop/yarn/service/ServiceTestUtils.java | 32 +-
.../TestDefaultUpgradeComponentsFinder.java | 40 +-
.../apache/hadoop/yarn/service/TestServiceAM.java | 86 +-
.../hadoop/yarn/service/TestServiceManager.java | 158 +-
.../yarn/service/TestYarnNativeServices.java | 57 +-
.../hadoop/yarn/service/client/TestServiceCLI.java | 17 +
.../yarn/service/client/TestServiceClient.java | 71 +-
.../yarn/service/component/TestComponent.java | 300 +-
.../TestComponentDecommissionInstances.java | 147 +
.../component/TestComponentRestartPolicy.java | 4 +-
.../component/instance/TestComponentInstance.java | 302 +-
.../hadoop/yarn/service/conf/ExampleAppJson.java | 1 +
.../yarn/service/conf/TestAppJsonResolve.java | 19 +
.../yarn/service/monitor/TestServiceMonitor.java | 15 +-
.../yarn/service/provider/TestProviderUtils.java | 11 +-
.../providers/TestAbstractClientProvider.java | 4 +-
.../providers/TestDefaultClientProvider.java | 4 +-
.../hadoop/yarn/service/utils/TestFilterUtils.java | 39 +-
.../yarn/service/utils/TestServiceApiUtil.java | 72 +-
.../src/test/resources/log4j.properties | 0
.../yarn/service/conf/examples/external3.json | 27 +
.../hadoop-yarn-services/pom.xml | 2 +-
.../hadoop-yarn-submarine/README.md | 6 +-
.../hadoop-yarn-submarine/pom.xml | 12 +-
.../base/ubuntu-16.04/Dockerfile.cpu.tf_1.8.0 | 69 +
.../base/ubuntu-16.04/Dockerfile.gpu.tf_1.8.0 | 67 +
.../src/main/docker/build-all.sh | 32 +
.../ubuntu-16.04/Dockerfile.cpu.tf_1.8.0 | 22 +
.../ubuntu-16.04/Dockerfile.gpu.tf_1.8.0 | 22 +
.../cifar10_estimator_tf_1.8.0/README.md | 542 +
.../cifar10_estimator_tf_1.8.0/cifar10.py | 113 +
.../cifar10_estimator_tf_1.8.0/cifar10_main.py | 521 +
.../cifar10_estimator_tf_1.8.0/cifar10_model.py | 80 +
.../cifar10_estimator_tf_1.8.0/cifar10_utils.py | 154 +
.../generate_cifar10_tfrecords.py | 114 +
.../cifar10_estimator_tf_1.8.0/model_base.py | 219 +
.../zeppelin-notebook-example/Dockerfile.gpu | 75 +
.../zeppelin-notebook-example/run_container.sh | 22 +
.../docker/zeppelin-notebook-example/shiro.ini | 120 +
.../zeppelin-notebook-example/zeppelin-site.xml | 569 +
.../hadoop/yarn/submarine/client/cli/Cli.java | 3 -
.../yarn/submarine/client/cli/CliConstants.java | 11 +
.../hadoop/yarn/submarine/client/cli/CliUtils.java | 158 +-
.../yarn/submarine/client/cli/RunJobCli.java | 81 +-
.../submarine/client/cli/param/Localization.java | 133 +
.../yarn/submarine/client/cli/param/Quicklink.java | 71 +
.../client/cli/param/RunJobParameters.java | 140 +-
.../submarine/client/cli/param/package-info.java | 19 +
.../yarn/submarine/common/ClientContext.java | 15 +-
.../yarn/submarine/common/api/JobStatus.java | 4 +-
.../common/conf/SubmarineConfiguration.java | 15 +
.../common/fs/DefaultRemoteDirectoryManager.java | 92 +-
.../common/fs/RemoteDirectoryManager.java | 22 +-
.../common/FSBasedSubmarineStorageImpl.java | 18 +-
.../yarn/submarine/runtimes/common/JobMonitor.java | 6 +
.../yarnservice/YarnServiceJobMonitor.java | 26 +-
.../yarnservice/YarnServiceJobSubmitter.java | 596 +-
.../runtimes/yarnservice/YarnServiceUtils.java | 93 +-
.../src/site/DeveloperGuide.md | 26 -
.../hadoop-yarn-submarine/src/site/QuickStart.md | 134 -
.../src/site/markdown/DeveloperGuide.md | 24 +
.../src/site/markdown/Examples.md | 21 +
.../src/site/markdown/HowToInstall.md | 36 +
.../src/site/markdown/Index.md | 44 +
.../src/site/markdown/InstallationGuide.md | 615 +
.../markdown/InstallationGuideChineseVersion.md | 757 +
.../src/site/markdown/QuickStart.md | 218 +
.../markdown/RunningDistributedCifar10TFJobs.md | 162 +
.../src/site/markdown/RunningZeppelinOnYARN.md | 37 +
.../src/site/markdown/TestAndTroubleshooting.md | 165 +
.../src/site/markdown/WriteDockerfile.md | 117 +
.../src/site/resources/css/site.css | 0
.../src/site/resources/images/job-logs-ui.png | Bin 0 -> 229944 bytes
.../resources/images/multiple-tensorboard-jobs.png | Bin 0 -> 184717 bytes
.../site/resources/images/submarine-installer.gif | Bin 0 -> 546547 bytes
.../site/resources/images/tensorboard-service.png | Bin 0 -> 107567 bytes
.../hadoop-yarn-submarine}/src/site/site.xml | 0
.../submarine/client/cli/TestRunJobCliParsing.java | 124 +-
.../cli/yarnservice/TestYarnServiceRunJobCli.java | 1129 +-
.../yarn/submarine/common/MockClientContext.java | 9 +-
.../common/fs/MockRemoteDirectoryManager.java | 97 +-
.../hadoop-yarn/hadoop-yarn-applications/pom.xml | 4 +-
.../hadoop-yarn/hadoop-yarn-client/pom.xml | 14 +-
.../apache/hadoop/yarn/client/api/AHSClient.java | 10 +-
.../yarn/client/api/ContainerShellWebSocket.java | 163 +
.../apache/hadoop/yarn/client/api/NMClient.java | 34 +
.../apache/hadoop/yarn/client/api/YarnClient.java | 75 +-
.../yarn/client/api/async/AMRMClientAsync.java | 11 +
.../client/api/async/impl/AMRMClientAsyncImpl.java | 9 +
.../yarn/client/api/impl/AHSv2ClientImpl.java | 157 +
.../yarn/client/api/impl/AMRMClientImpl.java | 5 +
.../hadoop/yarn/client/api/impl/NMClientImpl.java | 56 +
.../yarn/client/api/impl/YarnClientImpl.java | 287 +-
.../hadoop/yarn/client/cli/ApplicationCLI.java | 93 +-
.../apache/hadoop/yarn/client/cli/ClusterCLI.java | 19 +
.../org/apache/hadoop/yarn/client/cli/LogsCLI.java | 118 +-
.../hadoop/yarn/client/cli/NodeAttributesCLI.java | 718 +
.../org/apache/hadoop/yarn/client/cli/NodeCLI.java | 15 +-
.../apache/hadoop/yarn/client/cli/QueueCLI.java | 2 +-
.../apache/hadoop/yarn/client/cli/RMAdminCLI.java | 15 +-
.../org/apache/hadoop/yarn/client/cli/TopCLI.java | 1 +
.../org/apache/hadoop/yarn/client/cli/YarnCLI.java | 15 +-
.../hadoop/yarn/client/util/YarnClientUtils.java | 67 +
.../client/api/async/impl/TestAMRMClientAsync.java | 8 +-
.../client/api/async/impl/TestNMClientAsync.java | 4 +-
.../yarn/client/api/impl/BaseAMRMClientTest.java | 4 +-
.../hadoop/yarn/client/api/impl/TestAHSClient.java | 6 +-
.../yarn/client/api/impl/TestAHSv2ClientImpl.java | 249 +
.../yarn/client/api/impl/TestAMRMClient.java | 2 +-
.../impl/TestAMRMClientPlacementConstraints.java | 232 +-
.../client/api/impl/TestSharedCacheClientImpl.java | 2 +-
.../yarn/client/api/impl/TestYarnClient.java | 52 +-
.../yarn/client/api/impl/TestYarnClientImpl.java | 7 +-
.../hadoop/yarn/client/cli/TestClusterCLI.java | 63 +-
.../apache/hadoop/yarn/client/cli/TestLogsCLI.java | 50 +-
.../yarn/client/cli/TestNodeAttributesCLI.java | 537 +
.../hadoop/yarn/client/cli/TestRMAdminCLI.java | 24 +-
.../apache/hadoop/yarn/client/cli/TestYarnCLI.java | 162 +-
.../hadoop-yarn/hadoop-yarn-common/pom.xml | 28 +-
.../ApplicationClientProtocolPBClientImpl.java | 55 +
.../ContainerManagementProtocolPBClientImpl.java | 22 +
.../pb/client/CsiAdaptorProtocolPBClientImpl.java | 127 +
.../ApplicationClientProtocolPBServiceImpl.java | 65 +
.../ContainerManagementProtocolPBServiceImpl.java | 20 +
.../service/CsiAdaptorProtocolPBServiceImpl.java | 111 +
.../pb/PlacementConstraintFromProtoConverter.java | 10 +-
.../pb/PlacementConstraintToProtoConverter.java | 11 +
.../impl/pb/GetAttributesToNodesRequestPBImpl.java | 176 +
.../pb/GetAttributesToNodesResponsePBImpl.java | 207 +
.../pb/GetClusterNodeAttributesRequestPBImpl.java | 75 +
.../pb/GetClusterNodeAttributesResponsePBImpl.java | 160 +
.../pb/GetLocalizationStatusesRequestPBImpl.java | 156 +
.../pb/GetLocalizationStatusesResponsePBImpl.java | 260 +
.../impl/pb/GetNodesToAttributesRequestPBImpl.java | 132 +
.../pb/GetNodesToAttributesResponsePBImpl.java | 181 +
.../impl/pb/GetPluginInfoRequestPBImpl.java | 60 +
.../impl/pb/GetPluginInfoResponsePBImpl.java | 84 +
.../impl/pb/NodePublishVolumeRequestPBImpl.java | 201 +
.../impl/pb/NodePublishVolumeResponsePBImpl.java | 62 +
.../impl/pb/NodeUnpublishVolumeRequestPBImpl.java | 89 +
.../impl/pb/NodeUnpublishVolumeResponsePBImpl.java | 61 +
.../ValidateVolumeCapabilitiesRequestPBImpl.java | 121 +
.../ValidateVolumeCapabilitiesResponsePBImpl.java | 87 +
.../records/impl/pb/ApplicationReportPBImpl.java | 12 +
.../yarn/api/records/impl/pb/ContainerPBImpl.java | 44 +-
.../api/records/impl/pb/ContainerReportPBImpl.java | 23 +-
.../api/records/impl/pb/ContainerStatusPBImpl.java | 21 +
.../records/impl/pb/LocalizationStatusPBImpl.java | 192 +
.../records/impl/pb/NodeAttributeInfoPBImpl.java | 147 +
.../records/impl/pb/NodeAttributeKeyPBImpl.java | 140 +
.../api/records/impl/pb/NodeAttributePBImpl.java | 170 +
.../yarn/api/records/impl/pb/NodeReportPBImpl.java | 44 +-
.../impl/pb/NodeToAttributeValuePBImpl.java | 137 +
.../yarn/api/records/impl/pb/ProtoUtils.java | 66 +
.../yarn/api/records/impl/pb/ResourcePBImpl.java | 23 +
.../apache/hadoop/yarn/client/AMRMClientUtils.java | 10 +-
.../hadoop/yarn/client/api/AppAdminClient.java | 25 +
.../yarn/client/api/TimelineReaderClient.java | 120 +
.../client/api/impl/TimelineReaderClientImpl.java | 241 +
.../LogAggregationFileController.java | 74 +-
.../hadoop/yarn/nodelabels/AbstractLabel.java | 71 +
.../nodelabels/AttributeExpressionOperation.java | 26 +
.../hadoop/yarn/nodelabels/AttributeValue.java | 53 +
.../yarn/nodelabels/CommonNodeLabelsManager.java | 32 +-
.../yarn/nodelabels/FileSystemNodeLabelsStore.java | 270 +-
.../hadoop/yarn/nodelabels/NodeAttributeStore.java | 77 +
.../yarn/nodelabels/NodeAttributesManager.java | 137 +
.../hadoop/yarn/nodelabels/NodeLabelUtil.java | 204 +
.../hadoop/yarn/nodelabels/NodeLabelsStore.java | 32 +-
.../nodelabels/NonAppendableFSNodeLabelStore.java | 46 +-
.../hadoop/yarn/nodelabels/RMNodeAttribute.java | 98 +
.../apache/hadoop/yarn/nodelabels/RMNodeLabel.java | 110 +-
.../yarn/nodelabels/StringAttributeValue.java | 61 +
.../yarn/nodelabels/store/AbstractFSNodeStore.java | 213 +
.../yarn/nodelabels/store/FSStoreOpHandler.java | 131 +
.../hadoop/yarn/nodelabels/store/StoreOp.java | 49 +
.../nodelabels/store/op/AddClusterLabelOp.java | 73 +
.../store/op/AddNodeToAttributeLogOp.java | 71 +
.../yarn/nodelabels/store/op/FSNodeStoreLogOp.java | 52 +
.../nodelabels/store/op/NodeAttributeMirrorOp.java | 64 +
.../nodelabels/store/op/NodeLabelMirrorOp.java | 85 +
.../yarn/nodelabels/store/op/NodeToLabelOp.java | 75 +
.../nodelabels/store/op/RemoveClusterLabelOp.java | 75 +
.../store/op/RemoveNodeToAttributeLogOp.java | 71 +
.../store/op/ReplaceNodeToAttributeLogOp.java | 73 +
.../yarn/nodelabels/store/op/package-info.java | 21 +
.../hadoop/yarn/nodelabels/store/package-info.java | 21 +
...eManagerAdministrationProtocolPBClientImpl.java | 26 +-
...ManagerAdministrationProtocolPBServiceImpl.java | 31 +-
.../impl/pb/NodeToAttributesPBImpl.java | 164 +
.../pb/NodesToAttributesMappingRequestPBImpl.java | 197 +
.../pb/NodesToAttributesMappingResponsePBImpl.java | 50 +
.../RemoveFromClusterNodeLabelsRequestPBImpl.java | 24 +-
.../server/metrics/AppAttemptMetricsConstants.java | 69 +
.../metrics/ApplicationMetricsConstants.java | 125 +
.../server/metrics/ContainerMetricsConstants.java | 86 +
.../hadoop/yarn/server/metrics/package-info.java | 22 +
.../util/resource/DefaultResourceCalculator.java | 2 +-
.../util/resource/DominantResourceCalculator.java | 36 +-
.../yarn/util/resource/ResourceCalculator.java | 18 +
.../hadoop/yarn/util/resource/Resources.java | 36 +-
.../util/timeline/TimelineEntityV2Converter.java | 473 +
.../org/apache/hadoop/yarn/webapp/Dispatcher.java | 12 +-
.../org/apache/hadoop/yarn/webapp/WebApps.java | 25 +-
.../apache/hadoop/yarn/webapp/view/JQueryUI.java | 6 +-
.../static/dt-1.10.18/css/custom_datatable.css | 68 +
.../webapps/static/dt-1.10.18/css/demo_page.css | 108 +
.../webapps/static/dt-1.10.18/css/demo_table.css | 544 +
.../static/dt-1.10.18/css/jquery.dataTables.css | 466 +
.../webapps/static/dt-1.10.18/css/jui-dt.css | 352 +
.../images/Sorting icons.psd | Bin
.../images/back_disabled.jpg | Bin
.../images/back_enabled.jpg | Bin
.../{dt-1.10.7 => dt-1.10.18}/images/favicon.ico | Bin
.../images/forward_disabled.jpg | Bin
.../images/forward_enabled.jpg | Bin
.../{dt-1.10.7 => dt-1.10.18}/images/sort_asc.png | Bin
.../images/sort_asc_disabled.png | Bin
.../{dt-1.10.7 => dt-1.10.18}/images/sort_both.png | Bin
.../{dt-1.10.7 => dt-1.10.18}/images/sort_desc.png | Bin
.../images/sort_desc_disabled.png | Bin
.../static/dt-1.10.18/js/jquery.dataTables.min.js | 184 +
.../webapps/static/dt-1.10.7/css/demo_page.css | 110 -
.../webapps/static/dt-1.10.7/css/demo_table.css | 538 -
.../webapps/static/dt-1.10.7/css/jui-dt.css | 322 -
.../static/dt-1.10.7/js/jquery.dataTables.min.js | 160 -
.../resources/webapps/static/yarn.dt.plugins.js | 23 +
.../src/main/resources/yarn-default.xml | 242 +-
.../apache/hadoop/yarn/TestContainerLaunchRPC.java | 9 +
.../yarn/TestContainerResourceIncreaseRPC.java | 9 +
.../apache/hadoop/yarn/api/TestPBImplRecords.java | 114 +-
.../apache/hadoop/yarn/api/TestResourcePBImpl.java | 128 +-
.../yarn/client/api/impl/TestTimelineClient.java | 33 +-
.../api/impl/TestTimelineClientForATS1_5.java | 2 +-
.../api/impl/TestTimelineReaderClientImpl.java | 157 +
.../impl/pb/TestRpcClientFactoryPBImpl.java | 2 +-
.../impl/pb/TestRpcServerFactoryPBImpl.java | 2 +-
.../nodelabels/DummyCommonNodeLabelsManager.java | 8 +-
.../nodelabels/TestFileSystemNodeLabelsStore.java | 16 +-
.../hadoop/yarn/nodelabels/TestNodeLabelUtil.java | 122 +
.../resourcetypes/ResourceTypesTestHelper.java | 22 +
.../CustomResourceTypesConfigurationProvider.java | 186 +
.../yarn/util/resource/TestResourceCalculator.java | 26 +-
.../yarn/util/resource/TestResourceUtils.java | 192 +-
.../hadoop/yarn/util/resource/TestResources.java | 8 +-
.../resources/resource-types/resource-types-5.xml | 53 +
.../resources/resource-types/resource-types-6.xml | 58 +
.../dev-support/findbugs-exclude.xml | 21 +
.../hadoop-yarn/hadoop-yarn-csi/pom.xml | 203 +
.../hadoop/yarn/csi/adaptor/CsiAdaptorFactory.java | 73 +
.../csi/adaptor/CsiAdaptorProtocolService.java | 114 +
.../yarn/csi/adaptor/CsiAdaptorServices.java | 108 +
.../yarn/csi/adaptor/DefaultCsiAdaptorImpl.java | 132 +
.../hadoop/yarn/csi/adaptor/package-info.java | 21 +
.../apache/hadoop/yarn/csi/client/CsiClient.java | 49 +
.../hadoop/yarn/csi/client/CsiClientImpl.java | 82 +
.../hadoop/yarn/csi/client/CsiGrpcClient.java | 126 +
.../hadoop/yarn/csi/client/package-info.java | 21 +
.../org/apache/hadoop/yarn/csi/package-info.java | 21 +
.../GetPluginInfoResponseProtoTranslator.java | 44 +
.../NodePublishVolumeRequestProtoTranslator.java | 77 +
.../NodeUnpublishVolumeRequestProtoTranslator.java | 49 +
.../yarn/csi/translator/ProtoTranslator.java | 49 +
.../csi/translator/ProtoTranslatorFactory.java | 78 +
...teVolumeCapabilitiesRequestProtoTranslator.java | 93 +
...nVolumeCapabilitiesResponseProtoTranslator.java | 48 +
.../hadoop/yarn/csi/translator/package-info.java | 21 +
.../apache/hadoop/yarn/csi/utils/GrpcHelper.java | 52 +
.../apache/hadoop/yarn/csi/utils/package-info.java | 21 +
.../hadoop-yarn-csi/src/main/proto/csi.proto | 1114 +
.../hadoop/yarn/csi/adaptor/MockCsiAdaptor.java | 85 +
.../yarn/csi/adaptor/TestCsiAdaptorService.java | 448 +
.../adaptor/TestGetPluginInfoRequestResponse.java | 66 +
.../csi/adaptor/TestNodePublishVolumeRequest.java | 55 +
.../TestValidateVolumeCapabilityRequest.java | 113 +
.../TestValidateVolumeCapabilityResponse.java | 61 +
.../hadoop/yarn/csi/adaptor/package-info.java | 21 +
.../hadoop/yarn/csi/client/FakeCsiDriver.java | 65 +
.../yarn/csi/client/FakeCsiIdentityService.java | 42 +
.../hadoop/yarn/csi/client/ICsiClientTest.java | 53 +
.../hadoop/yarn/csi/client/TestCsiClient.java | 77 +
.../hadoop/yarn/csi/client/package-info.java | 21 +
.../hadoop-yarn/hadoop-yarn-registry/pom.xml | 239 +-
.../registry/client/api/RegistryConstants.java | 401 -
.../client/impl/FSRegistryOperationsService.java | 249 -
.../server/dns/PrivilegedRegistryDNSStarter.java | 80 -
.../hadoop/registry/server/dns/RegistryDNS.java | 1800 -
.../registry/server/dns/RegistryDNSServer.java | 267 -
.../registry/server/dns/ReverseZoneUtils.java | 171 -
.../hadoop-yarn-registry/src/main/resources/.keep | 0
.../src/main/tla/yarnregistry.tla | 582 -
.../hadoop/registry/AbstractZKRegistryTest.java | 113 -
.../client/impl/TestMicroZookeeperService.java | 60 -
.../secure/AbstractSecureRegistryTest.java | 368 -
.../hadoop/registry/secure/TestSecureLogins.java | 231 -
.../pom.xml | 8 +-
.../ApplicationHistoryManagerImpl.java | 23 +-
.../ApplicationHistoryManagerOnTimelineStore.java | 33 +-
.../records/ContainerHistoryData.java | 13 +
.../webapp/AHSWebServices.java | 250 +-
.../timeline/KeyValueBasedTimelineStore.java | 98 +-
.../yarn/server/timeline/MemoryTimelineStore.java | 37 +-
.../server/timeline/TimelineStoreMapAdapter.java | 9 +-
...stApplicationHistoryManagerOnTimelineStore.java | 1 +
.../webapp/TestAHSWebServices.java | 8 +-
.../timeline/webapp/TestTimelineWebServices.java | 2 +-
.../hadoop-yarn-server-common/pom.xml | 8 +-
.../yarn/server/AMHeartbeatRequestHandler.java | 33 +-
.../hadoop/yarn/server/AMRMClientRelayer.java | 476 +-
.../api/protocolrecords/NodeHeartbeatRequest.java | 21 +
.../api/protocolrecords/NodeHeartbeatResponse.java | 26 +-
.../RegisterNodeManagerRequest.java | 16 +
.../RegisterNodeManagerResponse.java | 5 +
.../impl/pb/NodeHeartbeatRequestPBImpl.java | 65 +
.../impl/pb/NodeHeartbeatResponsePBImpl.java | 106 +-
.../impl/pb/RegisterNodeManagerRequestPBImpl.java | 53 +
.../impl/pb/RegisterNodeManagerResponsePBImpl.java | 15 +
.../amrmproxy/BroadcastAMRMProxyPolicy.java | 4 +-
.../amrmproxy/FederationAMRMProxyPolicy.java | 8 +-
.../policies/amrmproxy/HomeAMRMProxyPolicy.java | 5 +-
.../LocalityMulticastAMRMProxyPolicy.java | 52 +-
.../policies/amrmproxy/RejectAMRMProxyPolicy.java | 4 +-
.../policies/router/LoadBasedRouterPolicy.java | 6 +-
.../federation/utils/FederationRegistryClient.java | 10 +-
.../utils/FederationStateStoreFacade.java | 5 +-
.../server/metrics/AMRMClientRelayerMetrics.java | 368 +
.../server/metrics/AppAttemptMetricsConstants.java | 69 -
.../metrics/ApplicationMetricsConstants.java | 122 -
.../server/metrics/ContainerMetricsConstants.java | 83 -
.../hadoop/yarn/server/metrics/package-info.java | 18 +
.../hadoop/yarn/server/security/AMSecretKeys.java | 45 +
.../yarn/server/uam/UnmanagedAMPoolManager.java | 73 +-
.../server/uam/UnmanagedApplicationManager.java | 68 +-
.../hadoop/yarn/server/utils/BuilderUtils.java | 6 +-
.../yarn/server/utils/YarnServerBuilderUtils.java | 68 +
.../yarn/server/volume/csi/CsiConstants.java | 37 +
.../server/volume/csi/VolumeCapabilityRange.java | 107 +
.../hadoop/yarn/server/volume/csi/VolumeId.java | 59 +
.../yarn/server/volume/csi/VolumeMetaData.java | 227 +
.../csi/exception/InvalidVolumeException.java | 28 +
.../volume/csi/exception/VolumeException.java | 34 +
.../csi/exception/VolumeProvisioningException.java | 32 +
.../server/volume/csi/exception/package-info.java | 27 +
.../yarn/server/volume/csi/package-info.java | 27 +
.../hadoop/yarn/server/webapp/LogWebService.java | 506 +
.../yarn/server/webapp/LogWebServiceUtils.java | 258 +
.../hadoop/yarn/server/webapp/WebPageUtils.java | 3 +-
.../yarn/server/webapp/YarnWebServiceParams.java | 1 +
.../hadoop/yarn/server/webapp/dao/AppInfo.java | 4 +-
.../yarn/server/webapp/dao/ContainerInfo.java | 7 +-
.../proto/yarn_server_common_service_protos.proto | 10 +
.../test/java/org/apache/hadoop/yarn/TestRPC.java | 9 +
.../hadoop/yarn/TestYarnServerApiClasses.java | 55 +-
.../yarn/server/MockResourceManagerFacade.java | 142 +-
.../hadoop/yarn/server/TestAMRMClientRelayer.java | 37 +-
.../api/protocolrecords/TestProtocolRecords.java | 27 +-
.../policies/BaseFederationPoliciesTest.java | 5 +-
.../TestBroadcastAMRMProxyFederationPolicy.java | 10 +-
.../amrmproxy/TestHomeAMRMProxyPolicy.java | 10 +-
.../TestLocalityMulticastAMRMProxyPolicy.java | 41 +-
.../amrmproxy/TestRejectAMRMProxyPolicy.java | 5 +-
.../policies/router/TestLoadBasedRouterPolicy.java | 31 +
.../utils/FederationPoliciesTestUtil.java | 2 +-
.../utils/FederationStateStoreTestUtil.java | 2 +-
.../utils/TestFederationStateStoreFacade.java | 17 +
.../metrics/TestAMRMClientRelayerMetrics.java | 509 +
.../uam/TestUnmanagedApplicationManager.java | 78 +-
.../yarn/server/webapp/TestLogWebService.java | 126 +
.../hadoop-yarn-server-nodemanager/pom.xml | 17 +-
.../yarn/server/nodemanager/ContainerExecutor.java | 33 +-
.../hadoop/yarn/server/nodemanager/Context.java | 5 +
.../nodemanager/DefaultContainerExecutor.java | 47 +-
.../server/nodemanager/LinuxContainerExecutor.java | 117 +-
.../nodemanager/LocalDirsHandlerService.java | 5 +-
.../yarn/server/nodemanager/NodeManager.java | 83 +-
.../yarn/server/nodemanager/NodeStatusUpdater.java | 14 +
.../server/nodemanager/NodeStatusUpdaterImpl.java | 426 +-
.../WindowsSecureContainerExecutor.java | 6 +-
.../amrmproxy/FederationInterceptor.java | 776 +-
.../nodemanager/api/deviceplugin/Device.java | 234 +
.../nodemanager/api/deviceplugin/DevicePlugin.java | 69 +
.../api/deviceplugin/DevicePluginScheduler.java | 38 +
.../api/deviceplugin/DeviceRegisterRequest.java | 73 +
.../api/deviceplugin/DeviceRuntimeSpec.java | 137 +
.../api/deviceplugin/MountDeviceSpec.java | 131 +
.../api/deviceplugin/MountVolumeSpec.java | 118 +
.../nodemanager/api/deviceplugin/VolumeSpec.java | 114 +
.../api/deviceplugin/YarnRuntimeType.java | 42 +
.../nodemanager/api/deviceplugin/package-info.java | 19 +
.../nodemanager/containermanager/AuxServices.java | 919 +-
.../AuxiliaryServiceWithCustomClassLoader.java | 8 +-
.../containermanager/ContainerManagerImpl.java | 145 +-
.../containermanager/container/Container.java | 13 +
.../containermanager/container/ContainerImpl.java | 98 +-
.../container/SlidingWindowRetryPolicy.java | 4 +
.../launcher/ContainerCleanup.java | 240 +
.../containermanager/launcher/ContainerLaunch.java | 331 +-
.../launcher/ContainerRelaunch.java | 26 +-
.../launcher/ContainersLauncher.java | 54 +-
.../linux/privileged/PrivilegedOperation.java | 7 +-
.../privileged/PrivilegedOperationExecutor.java | 57 +-
.../resources/CGroupsBlkioResourceHandlerImpl.java | 5 +
.../resources/CGroupsCpuResourceHandlerImpl.java | 5 +
.../linux/resources/CGroupsHandler.java | 2 +-
.../linux/resources/CGroupsHandlerImpl.java | 21 +-
.../CGroupsMemoryResourceHandlerImpl.java | 24 +-
.../linux/resources/DefaultOOMHandler.java | 45 +-
.../linux/resources/MemoryResourceHandler.java | 10 -
.../resources/NetworkPacketTaggingHandlerImpl.java | 5 +
.../linux/resources/ResourceHandlerChain.java | 6 +
.../linux/resources/ResourceHandlerModule.java | 52 +-
.../TrafficControlBandwidthHandlerImpl.java | 7 +-
.../resources/fpga/FpgaResourceHandlerImpl.java | 14 +-
.../linux/resources/gpu/GpuResourceAllocator.java | 10 +-
.../resources/gpu/GpuResourceHandlerImpl.java | 16 +-
.../resources/numa/NumaResourceHandlerImpl.java | 12 +-
.../runtime/DefaultLinuxContainerRuntime.java | 138 +-
.../runtime/DelegatingLinuxContainerRuntime.java | 122 +-
.../linux/runtime/DockerLinuxContainerRuntime.java | 408 +-
.../runtime/JavaSandboxLinuxContainerRuntime.java | 32 +-
.../linux/runtime/LinuxContainerRuntime.java | 11 +
.../runtime/LinuxContainerRuntimeConstants.java | 4 +
.../linux/runtime/docker/DockerClient.java | 8 +-
.../runtime/docker/DockerCommandExecutor.java | 73 +-
.../linux/runtime/docker/DockerExecCommand.java | 62 +
.../linux/runtime/docker/DockerInspectCommand.java | 23 +-
.../linux/runtime/docker/DockerRmCommand.java | 11 +-
.../linux/runtime/docker/DockerRunCommand.java | 17 +
.../localizer/ContainerLocalizer.java | 29 +-
.../localizer/ResourceLocalizationService.java | 101 +-
.../containermanager/localizer/ResourceSet.java | 45 +-
.../containermanager/monitor/ContainerMetrics.java | 8 +-
.../monitor/ContainerStopMonitoringEvent.java | 12 +
.../monitor/ContainersMonitor.java | 6 +-
.../monitor/ContainersMonitorImpl.java | 127 +-
.../records/AuxServiceConfiguration.java | 137 +
.../containermanager/records/AuxServiceFile.java | 137 +
.../containermanager/records/AuxServiceRecord.java | 175 +
.../records/AuxServiceRecords.java | 46 +
.../containermanager/records/package-info.java | 29 +
.../resourceplugin/ResourcePluginManager.java | 197 +-
.../deviceframework/AssignedDevice.java | 78 +
.../deviceframework/DeviceMappingManager.java | 381 +
.../deviceframework/DevicePluginAdapter.java | 123 +
.../deviceframework/DeviceResourceHandlerImpl.java | 154 +
.../deviceframework/DeviceResourceUpdaterImpl.java | 66 +
.../deviceframework/package-info.java | 19 +
.../resourceplugin/fpga/FpgaResourcePlugin.java | 5 +
.../resourceplugin/fpga/IntelFpgaOpenclPlugin.java | 4 +-
.../gpu/GpuDockerCommandPluginFactory.java | 4 +
.../resourceplugin/gpu/GpuResourcePlugin.java | 5 +
.../gpu/NvidiaDockerV2CommandPlugin.java | 111 +
.../resourceplugin/package-info.java | 19 +
.../containermanager/runtime/ContainerRuntime.java | 26 +-
.../runtime/ContainerRuntimeConstants.java | 4 +
.../AllocationBasedResourceUtilizationTracker.java | 27 +-
.../scheduler/ContainerScheduler.java | 19 +-
.../volume/csi/ContainerVolumePublisher.java | 207 +
.../containermanager/volume/csi/package-info.java | 22 +
.../nodemanager/executor/ContainerExecContext.java | 112 +
.../executor/ContainerStartContext.java | 36 +
.../nodemanager/metrics/NodeManagerMetrics.java | 21 +
.../AbstractNodeDescriptorsProvider.java | 197 +
.../nodelabels/AbstractNodeLabelsProvider.java | 149 -
.../ConfigurationNodeAttributesProvider.java | 156 +
.../ConfigurationNodeLabelsProvider.java | 13 +-
.../nodelabels/NodeAttributesProvider.java | 32 +
.../nodelabels/NodeDescriptorsProvider.java | 45 +
.../nodelabels/NodeDescriptorsScriptRunner.java | 84 +
.../nodemanager/nodelabels/NodeLabelsProvider.java | 22 +-
.../ScriptBasedNodeAttributesProvider.java | 158 +
.../nodelabels/ScriptBasedNodeLabelsProvider.java | 126 +-
.../nodemanager/nodelabels/package-info.java | 28 +
.../recovery/NMLeveldbStateStoreService.java | 173 +-
.../nodemanager/recovery/NMStateStoreService.java | 29 +-
.../timelineservice/NMTimelinePublisher.java | 276 +-
.../webapp/ContainerShellWebSocket.java | 198 +
.../webapp/ContainerShellWebSocketServlet.java | 36 +
.../nodemanager/webapp/JAXBContextResolver.java | 3 +
.../server/nodemanager/webapp/NMWebServices.java | 92 +
.../server/nodemanager/webapp/TerminalServlet.java | 49 +
.../yarn/server/nodemanager/webapp/WebServer.java | 13 +
.../webapp/dao/AuxiliaryServiceInfo.java | 60 +
.../webapp/dao/AuxiliaryServicesInfo.java | 55 +
.../webapp/dao/NMDeviceResourceInfo.java | 56 +
.../container-executor/impl/container-executor.c | 754 +-
.../container-executor/impl/container-executor.h | 65 +-
.../src/main/native/container-executor/impl/main.c | 117 +-
.../impl/modules/fpga/fpga-module.c | 5 +-
.../impl/modules/gpu/gpu-module.c | 5 +-
.../src/main/native/container-executor/impl/util.h | 7 +-
.../container-executor/impl/utils/docker-util.c | 158 +-
.../container-executor/impl/utils/docker-util.h | 44 +-
.../container-executor/impl/utils/string-utils.c | 9 +
.../container-executor/impl/utils/string-utils.h | 6 +
.../test/test-container-executor.c | 532 +-
.../test/utils/test_docker_util.cc | 203 +-
.../native/oom-listener/impl/oom_listener_main.c | 4 +-
.../oom-listener/test/oom_listener_test_main.cc | 14 +-
.../src/main/resources/TERMINAL/css/style.css | 33 +
.../src/main/resources/TERMINAL/terminal.template | 127 +
.../TERMINAL/xterm/dist/addons/attach/attach.js | 104 +
.../xterm/dist/addons/attach/attach.js.map | 1 +
.../TERMINAL/xterm/dist/addons/fit/fit.js | 51 +
.../TERMINAL/xterm/dist/addons/fit/fit.js.map | 1 +
.../xterm/dist/addons/fullscreen/fullscreen.css | 10 +
.../xterm/dist/addons/fullscreen/fullscreen.js | 27 +
.../xterm/dist/addons/fullscreen/fullscreen.js.map | 1 +
.../TERMINAL/xterm/dist/addons/search/search.js | 166 +
.../xterm/dist/addons/search/search.js.map | 1 +
.../xterm/dist/addons/terminado/terminado.js | 69 +
.../xterm/dist/addons/terminado/terminado.js.map | 1 +
.../xterm/dist/addons/webLinks/webLinks.js | 41 +
.../xterm/dist/addons/webLinks/webLinks.js.map | 1 +
.../xterm/dist/addons/winptyCompat/winptyCompat.js | 31 +
.../dist/addons/winptyCompat/winptyCompat.js.map | 1 +
.../TERMINAL/xterm/dist/addons/zmodem/zmodem.js | 45 +
.../xterm/dist/addons/zmodem/zmodem.js.map | 1 +
.../main/resources/TERMINAL/xterm/dist/xterm.css | 164 +
.../main/resources/TERMINAL/xterm/dist/xterm.js | 8907 +++++
.../resources/TERMINAL/xterm/dist/xterm.js.map | 1 +
.../server/nodemanager/DummyContainerManager.java | 3 +-
.../server/nodemanager/TestContainerExecutor.java | 16 +
.../nodemanager/TestDefaultContainerExecutor.java | 175 +-
.../nodemanager/TestLinuxContainerExecutor.java | 35 +-
.../TestLinuxContainerExecutorWithMocks.java | 135 +-
.../yarn/server/nodemanager/TestNodeManager.java | 2 +-
.../server/nodemanager/TestNodeManagerReboot.java | 2 +-
.../server/nodemanager/TestNodeManagerResync.java | 3 +-
.../server/nodemanager/TestNodeStatusUpdater.java | 32 +-
.../TestNodeStatusUpdaterForAttributes.java | 439 +
.../TestNodeStatusUpdaterForLabels.java | 52 +-
.../nodemanager/amrmproxy/BaseAMRMProxyTest.java | 10 +
.../amrmproxy/TestAMRMProxyService.java | 1 -
.../amrmproxy/TestFederationInterceptor.java | 432 +-
.../amrmproxy/TestableFederationInterceptor.java | 126 +-
.../containermanager/BaseContainerManagerTest.java | 18 +-
.../containermanager/TestAuxServices.java | 447 +-
.../containermanager/TestContainerManager.java | 133 +-
.../TestContainerManagerRecovery.java | 7 +-
.../nodemanager/containermanager/TestNMProxy.java | 2 +-
.../application/TestApplication.java | 11 +-
.../containermanager/container/TestContainer.java | 68 +-
.../container/TestSlidingWindowRetryPolicy.java | 26 +-
.../task/DockerContainerDeletionMatcher.java | 5 +-
.../deletion/task/FileDeletionMatcher.java | 5 +-
.../launcher/TestContainerCleanup.java | 108 +
.../launcher/TestContainerLaunch.java | 234 +-
.../launcher/TestContainerRelaunch.java | 36 +-
.../launcher/TestContainersLauncher.java | 28 +-
.../privileged/MockPrivilegedOperationCaptor.java | 10 +-
.../TestCGroupElasticMemoryController.java | 2 +-
.../linux/resources/TestCGroupsHandlerImpl.java | 42 +-
.../TestCGroupsMemoryResourceHandlerImpl.java | 44 -
.../linux/resources/TestDefaultOOMHandler.java | 436 +-
.../TestNetworkPacketTaggingHandlerImpl.java | 4 +-
.../TestTrafficControlBandwidthHandlerImpl.java | 2 +-
.../linux/resources/TestTrafficController.java | 4 +-
.../resources/fpga/TestFpgaResourceHandler.java | 10 +-
.../resources/gpu/TestGpuResourceHandler.java | 27 +-
.../resources/numa/TestNumaResourceAllocator.java | 6 +-
.../numa/TestNumaResourceHandlerImpl.java | 6 +-
.../linux/runtime/MockLinuxContainerRuntime.java | 75 +
.../TestDelegatingLinuxContainerRuntime.java | 76 +-
.../linux/runtime/TestDockerContainerRuntime.java | 541 +-
.../linux/runtime/docker/TestDockerClient.java | 46 +
.../runtime/docker/TestDockerCommandExecutor.java | 29 +-
.../linux/runtime/docker/TestDockerRmCommand.java | 35 +-
.../linux/runtime/docker/TestDockerRunCommand.java | 12 +-
.../localizer/TestContainerLocalizer.java | 38 +-
.../localizer/TestLocalResourcesTrackerImpl.java | 6 +-
.../localizer/TestLocalizedResource.java | 22 +-
.../localizer/TestResourceLocalizationService.java | 75 +-
.../localizer/TestResourceSet.java | 106 +
.../sharedcache/TestSharedCacheUploader.java | 4 +-
.../logaggregation/TestAppLogAggregatorImpl.java | 2 +-
.../logaggregation/TestLogAggregationService.java | 9 +-
.../loghandler/TestNonAggregatingLogHandler.java | 16 +-
.../monitor/TestContainerMetrics.java | 6 +-
.../TestContainersMonitorResourceChange.java | 14 +
.../resourceplugin/TestResourcePluginManager.java | 302 +-
.../deviceframework/FakeTestDevicePlugin1.java | 77 +
.../deviceframework/FakeTestDevicePlugin2.java | 27 +
.../deviceframework/FakeTestDevicePlugin3.java | 65 +
.../deviceframework/FakeTestDevicePlugin4.java | 54 +
.../deviceframework/FakeTestDevicePlugin5.java | 56 +
.../deviceframework/TestDeviceMappingManager.java | 366 +
.../deviceframework/TestDevicePluginAdapter.java | 658 +
.../resourceplugin/fpga/TestFpgaDiscoverer.java | 68 +-
.../gpu/TestNvidiaDockerV2CommandPlugin.java | 130 +
...tAllocationBasedResourceUtilizationTracker.java | 18 -
.../scheduler/TestContainerSchedulerQueuing.java | 32 +-
.../scheduler/TestContainerSchedulerRecovery.java | 81 +-
.../TestConfigurationNodeAttributesProvider.java | 261 +
.../TestConfigurationNodeLabelsProvider.java | 26 +-
.../TestScriptBasedNodeAttributesProvider.java | 251 +
.../TestScriptBasedNodeLabelsProvider.java | 18 +-
.../recovery/NMMemoryStateStoreService.java | 18 +-
.../recovery/TestNMLeveldbStateStoreService.java | 271 +-
.../timelineservice/TestNMTimelinePublisher.java | 2 +
.../webapp/ContainerShellClientSocketTest.java | 80 +
.../server/nodemanager/webapp/MockContainer.java | 20 +
.../webapp/TestNMContainerWebSocket.java | 185 +
.../webapp/TestNMWebServicesAuxServices.java | 334 +
.../nodemanager/webapp/TestNMWebTerminal.java | 124 +
.../resources/resource-types-pluggable-devices.xml | 23 +
.../hadoop-yarn-server-resourcemanager/pom.xml | 10 +-
.../yarn/server/resourcemanager/AdminService.java | 135 +
.../resourcemanager/ApplicationMasterService.java | 15 +-
.../server/resourcemanager/ClientRMService.java | 130 +-
.../resourcemanager/DefaultAMSProcessor.java | 3 +-
.../IsResourceManagerActiveServlet.java | 38 +
.../OpportunisticContainerAllocatorAMService.java | 8 +-
.../resourcemanager/RMActiveServiceContext.java | 72 +-
.../yarn/server/resourcemanager/RMAppManager.java | 13 +-
.../yarn/server/resourcemanager/RMAuditLogger.java | 81 +-
.../yarn/server/resourcemanager/RMContext.java | 26 +-
.../yarn/server/resourcemanager/RMContextImpl.java | 49 +-
.../yarn/server/resourcemanager/RMServerUtils.java | 31 +-
.../server/resourcemanager/ResourceManager.java | 45 +-
.../resourcemanager/ResourceTrackerService.java | 146 +-
.../resourcemanager/amlauncher/AMLauncher.java | 29 +
.../metrics/AbstractSystemMetricsPublisher.java | 5 +
.../metrics/CombinedSystemMetricsPublisher.java | 6 +
.../metrics/NoOpSystemMetricPublisher.java | 4 +
.../metrics/SystemMetricsPublisher.java | 2 +
.../metrics/TimelineServiceV2Publisher.java | 17 +
.../AbstractPreemptableResourceCalculator.java | 2 +-
.../ProportionalCapacityPreemptionPolicy.java | 3 +
.../monitor/capacity/TempQueuePerPartition.java | 2 +-
.../nodelabels/FileSystemNodeAttributeStore.java | 102 +
.../nodelabels/NodeAttributesManagerImpl.java | 755 +
.../nodelabels/NodeAttributesStoreEvent.java | 52 +
.../nodelabels/NodeAttributesStoreEventType.java | 26 +
.../nodelabels/NodeLabelsUtils.java | 19 +
.../placement/AppNameMappingPlacementRule.java | 12 +-
.../resourcemanager/placement/PlacementRule.java | 4 +-
.../placement/UserGroupMappingPlacementRule.java | 11 +-
.../recovery/FileSystemRMStateStore.java | 53 +
.../recovery/LeveldbRMStateStore.java | 62 +
.../recovery/MemoryRMStateStore.java | 18 +
.../resourcemanager/recovery/NullRMStateStore.java | 9 +
.../resourcemanager/recovery/RMStateStore.java | 102 +-
.../recovery/RMStateStoreEventType.java | 1 +
.../recovery/RMStateStoreProxyCAEvent.java | 49 +
.../resourcemanager/recovery/ZKRMStateStore.java | 60 +-
.../resourcemanager/reservation/InMemoryPlan.java | 4 +-
.../server/resourcemanager/rmapp/RMAppImpl.java | 67 +-
.../rmapp/attempt/RMAppAttemptImpl.java | 14 +-
.../resourcemanager/rmcontainer/RMContainer.java | 5 +
.../rmcontainer/RMContainerImpl.java | 18 +-
.../yarn/server/resourcemanager/rmnode/RMNode.java | 6 +
.../server/resourcemanager/rmnode/RMNodeImpl.java | 47 +-
.../rmnode/UpdatedContainerInfo.java | 14 +-
.../scheduler/AbstractYarnScheduler.java | 72 +-
.../scheduler/ClusterNodeTracker.java | 13 +-
.../scheduler/ContainerUpdateContext.java | 2 +
.../resourcemanager/scheduler/QueueMetrics.java | 130 +-
.../scheduler/QueueMetricsForCustomResources.java | 158 +
.../resourcemanager/scheduler/ResourceLimits.java | 24 +
.../resourcemanager/scheduler/SchedulerNode.java | 11 +
.../resourcemanager/scheduler/SchedulerUtils.java | 76 +-
.../resourcemanager/scheduler/YarnScheduler.java | 9 +-
.../scheduler/capacity/AbstractCSQueue.java | 55 +-
.../scheduler/capacity/CSQueue.java | 4 +
.../scheduler/capacity/CSQueueMetrics.java | 39 +
.../scheduler/capacity/CSQueueUtils.java | 17 +
.../scheduler/capacity/CapacityScheduler.java | 162 +-
.../capacity/CapacitySchedulerConfiguration.java | 94 +-
.../scheduler/capacity/LeafQueue.java | 4 +
.../scheduler/capacity/ManagedParentQueue.java | 20 +-
.../scheduler/capacity/ParentQueue.java | 24 +-
.../allocator/RegularContainerAllocator.java | 21 +-
.../scheduler/common/fica/FiCaSchedulerApp.java | 19 +-
.../constraint/PlacementConstraintsUtil.java | 148 +-
.../processor/PlacementConstraintProcessor.java | 12 +-
.../event/NodeAttributesUpdateSchedulerEvent.java | 41 +
.../scheduler/event/SchedulerEventType.java | 1 +
.../scheduler/fair/AllocationConfiguration.java | 13 +
.../scheduler/fair/ConfigurableResource.java | 5 +-
.../scheduler/fair/FSAppAttempt.java | 5 +-
.../scheduler/fair/FSLeafQueue.java | 38 +
.../scheduler/fair/FSParentQueue.java | 37 +-
.../scheduler/fair/FSPreemptionThread.java | 57 +-
.../resourcemanager/scheduler/fair/FSQueue.java | 9 +
.../scheduler/fair/FairScheduler.java | 123 +-
.../scheduler/fair/FairSchedulerConfiguration.java | 97 +-
.../scheduler/fair/QueueManager.java | 137 +-
.../fair/allocation/AllocationFileQueueParser.java | 7 +
.../scheduler/fair/allocation/QueueProperties.java | 16 +-
.../scheduler/fair/policies/ComputeFairShares.java | 73 +-
.../policies/DominantResourceFairnessPolicy.java | 2 +-
.../scheduler/fifo/FifoAppAttempt.java | 10 +-
.../placement/LocalityAppPlacementAllocator.java | 4 +
.../security/DelegationTokenRenewer.java | 17 +-
.../resourcemanager/security/ProxyCAManager.java | 85 +
.../resourcemanager/volume/csi/VolumeBuilder.java | 106 +
.../resourcemanager/volume/csi/VolumeManager.java | 74 +
.../volume/csi/VolumeManagerImpl.java | 177 +
.../resourcemanager/volume/csi/VolumeStates.java | 60 +
.../csi/event/ControllerPublishVolumeEvent.java | 30 +
.../volume/csi/event/ValidateVolumeEvent.java | 30 +
.../volume/csi/event/VolumeEvent.java | 43 +
.../volume/csi/event/VolumeEventType.java | 29 +
.../volume/csi/event/package-info.java | 27 +
.../volume/csi/lifecycle/Volume.java | 45 +
.../volume/csi/lifecycle/VolumeImpl.java | 219 +
.../volume/csi/lifecycle/VolumeState.java | 35 +
.../volume/csi/lifecycle/package-info.java | 27 +
.../resourcemanager/volume/csi/package-info.java | 27 +
.../volume/csi/processor/VolumeAMSProcessor.java | 173 +
.../volume/csi/processor/package-info.java | 27 +
.../volume/csi/provisioner/VolumeProvisioner.java | 32 +
.../csi/provisioner/VolumeProvisioningResults.java | 87 +
.../csi/provisioner/VolumeProvisioningTask.java | 66 +
.../volume/csi/provisioner/package-info.java | 27 +
.../resourcemanager/volume/package-info.java | 27 +
.../resourcemanager/webapp/FairSchedulerPage.java | 4 +
.../webapp/JAXBContextResolver.java | 3 +-
.../server/resourcemanager/webapp/NodesPage.java | 3 +-
.../server/resourcemanager/webapp/RMWSConsts.java | 3 +
.../resourcemanager/webapp/RMWebAppFilter.java | 3 +-
.../webapp/RMWebServiceProtocol.java | 15 +
.../resourcemanager/webapp/RMWebServices.java | 75 +-
.../resourcemanager/webapp/dao/AppAttemptInfo.java | 6 +
.../webapp/dao/FairSchedulerQueueInfo.java | 9 +-
.../webapp/dao/NodeAttributeInfo.java | 65 +
.../webapp/dao/NodeAttributesInfo.java | 49 +
.../resourcemanager/webapp/dao/NodeInfo.java | 19 +
.../webapp/dao/ReservationListInfo.java | 4 +
.../webapp/dao/ResourceOptionInfo.java | 65 +
.../resourcemanager/webapp/dao/SchedulerInfo.java | 2 +
.../resourcemanager/webapp/dao/package-info.java | 27 +
.../server/resourcemanager/AppManagerTestBase.java | 107 +
.../ApplicationMasterServiceTestBase.java | 582 +
.../hadoop/yarn/server/resourcemanager/MockAM.java | 27 +
.../resourcemanager/MockMemoryRMStateStore.java | 1 +
.../hadoop/yarn/server/resourcemanager/MockNM.java | 3 +
.../yarn/server/resourcemanager/MockNodes.java | 58 +-
.../hadoop/yarn/server/resourcemanager/MockRM.java | 48 +
.../resourcemanager/NodeAttributeTestUtils.java | 54 +
.../yarn/server/resourcemanager/NodeManager.java | 9 +
.../resourcemanager/TestAMAuthorization.java | 9 +
.../server/resourcemanager/TestAppManager.java | 100 +-
.../TestAppManagerWithFairScheduler.java | 177 +
.../resourcemanager/TestApplicationACLs.java | 2 +-
.../TestApplicationMasterLauncher.java | 97 +-
.../TestApplicationMasterService.java | 993 -
.../TestApplicationMasterServiceCapacity.java | 207 +
.../TestApplicationMasterServiceFair.java | 118 +
.../TestApplicationMasterServiceInterceptor.java | 217 +
.../resourcemanager/TestClientRMService.java | 401 +-
.../server/resourcemanager/TestClientRMTokens.java | 6 +-
...stOpportunisticContainerAllocatorAMService.java | 78 +
.../hadoop/yarn/server/resourcemanager/TestRM.java | 14 +-
.../server/resourcemanager/TestRMAdminService.java | 199 +-
.../server/resourcemanager/TestRMAuditLogger.java | 23 +-
.../resourcemanager/TestRMEmbeddedElector.java | 2 +-
.../resourcemanager/TestRMNodeTransitions.java | 17 +-
.../yarn/server/resourcemanager/TestRMRestart.java | 74 +-
.../server/resourcemanager/TestRMServerUtils.java | 136 +-
.../TestResourceTrackerService.java | 736 +-
.../TestWorkPreservingRMRestart.java | 2 +-
.../applicationsmanager/MockAsm.java | 12 +-
.../TestRMAppLogAggregationStatus.java | 2 +-
.../metrics/TestSystemMetricsPublisherForV2.java | 4 +-
...ionalCapacityPreemptionPolicyMockFramework.java | 19 +-
.../TestPreemptionForQueueWithPriorities.java | 2 +-
.../TestProportionalCapacityPreemptionPolicy.java | 19 +-
...lCapacityPreemptionPolicyForNodePartitions.java | 2 +-
...acityPreemptionPolicyForReservedContainers.java | 2 +-
...lCapacityPreemptionPolicyInterQueueWithDRF.java | 2 +-
...ortionalCapacityPreemptionPolicyIntraQueue.java | 92 +-
...cityPreemptionPolicyIntraQueueFairOrdering.java | 2 +-
...apacityPreemptionPolicyIntraQueueUserLimit.java | 2 +-
...lCapacityPreemptionPolicyIntraQueueWithDRF.java | 2 +-
...alCapacityPreemptionPolicyPreemptToBalance.java | 2 +-
.../nodelabels/NullRMNodeLabelsManager.java | 7 +
.../TestFileSystemNodeAttributeStore.java | 279 +
.../nodelabels/TestNodeAttributesManager.java | 364 +
.../recovery/RMStateStoreTestBase.java | 33 +
.../recovery/TestFSRMStateStore.java | 1 +
.../recovery/TestLeveldbRMStateStore.java | 6 +
.../recovery/TestZKRMStateStore.java | 1 +
.../reservation/ReservationSystemTestUtil.java | 8 +-
.../TestCapacitySchedulerPlanFollower.java | 12 +-
.../reservation/TestFairSchedulerPlanFollower.java | 10 +-
.../reservation/TestReservationInputValidator.java | 2 +-
.../planning/TestSimpleCapacityReplanner.java | 2 +-
.../rmapp/TestNodesListManager.java | 11 +-
.../rmapp/TestRMAppTransitions.java | 172 +-
.../rmapp/attempt/TestRMAppAttemptTransitions.java | 106 +-
.../rmcontainer/TestRMContainerImpl.java | 16 +-
.../scheduler/AppMetricsChecker.java | 122 +
.../resourcemanager/scheduler/QueueInfo.java | 90 +
.../scheduler/QueueMetricsTestData.java | 105 +
.../scheduler/ResourceMetricsChecker.java | 190 +
.../TestConfigurationMutationACLPolicies.java | 4 +-
.../scheduler/TestQueueMetrics.java | 772 +-
.../TestQueueMetricsForCustomResources.java | 645 +
.../scheduler/TestSchedulerUtils.java | 199 +-
.../scheduler/capacity/TestApplicationLimits.java | 14 +-
.../capacity/TestApplicationLimitsByPartition.java | 13 +-
.../capacity/TestCSAllocateCustomResource.java | 193 +
.../scheduler/capacity/TestCapacityScheduler.java | 365 +-
.../TestCapacitySchedulerAsyncScheduling.java | 83 +
.../scheduler/capacity/TestChildQueueOrder.java | 9 +-
.../capacity/TestContainerAllocation.java | 203 +
.../scheduler/capacity/TestLeafQueue.java | 11 +-
.../capacity/TestNodeLabelContainerAllocation.java | 62 +
.../scheduler/capacity/TestParentQueue.java | 4 +-
.../scheduler/capacity/TestQueueState.java | 3 +-
.../scheduler/capacity/TestQueueStateManager.java | 2 +-
.../scheduler/capacity/TestReservations.java | 2 +-
.../TestSchedulingRequestContainerAllocation.java | 228 +-
...tSchedulingRequestContainerAllocationAsync.java | 53 +-
.../scheduler/capacity/TestUtils.java | 24 +-
.../scheduler/fair/FairSchedulerTestBase.java | 4 +-
.../scheduler/fair/FakeSchedulable.java | 17 +-
.../fair/TestAllocationFileLoaderService.java | 36 +-
.../fair/TestApplicationMasterServiceWithFS.java | 174 +
.../scheduler/fair/TestComputeFairShares.java | 89 +-
.../scheduler/fair/TestContinuousScheduling.java | 5 +-
.../scheduler/fair/TestFSParentQueue.java | 12 +-
.../scheduler/fair/TestFSSchedulerNode.java | 2 +-
.../scheduler/fair/TestFairScheduler.java | 243 +-
.../fair/TestFairSchedulerConfiguration.java | 320 +-
.../fair/TestFairSchedulerPreemption.java | 155 +-
.../scheduler/fair/TestQueueManager.java | 246 +-
.../fair/allocationfile/AllocationFileQueue.java | 6 +-
.../allocationfile/AllocationFileQueueBuilder.java | 6 +
.../AllocationFileQueueProperties.java | 12 +
.../TestSingleConstraintAppPlacementAllocator.java | 196 +-
.../security/TestClientToAMTokens.java | 2 +-
.../security/TestDelegationTokenRenewer.java | 112 +-
.../security/TestProxyCAManager.java | 105 +
.../security/TestRMAuthenticationFilter.java | 3 +-
.../security/TestRMDelegationTokens.java | 75 +-
.../volume/csi/TestVolumeCapabilityRange.java | 67 +
.../volume/csi/TestVolumeLifecycle.java | 191 +
.../volume/csi/TestVolumeMetaData.java | 178 +
.../volume/csi/TestVolumeProcessor.java | 328 +
.../webapp/TestApplicationsRequestBuilder.java | 4 +-
.../resourcemanager/webapp/TestRMWebApp.java | 2 +-
.../resourcemanager/webapp/TestRMWebServices.java | 2 +-
.../webapp/TestRMWebServicesAppAttempts.java | 397 +
.../TestRMWebServicesAppCustomResourceTypes.java | 170 +
.../webapp/TestRMWebServicesApps.java | 498 +-
.../TestRMWebServicesAppsCustomResourceTypes.java | 83 +-
...estRMWebServicesCustomResourceTypesCommons.java | 98 +
.../webapp/TestRMWebServicesNodeLabels.java | 2 +-
.../webapp/TestRMWebServicesNodes.java | 191 +-
.../CustomResourceTypesConfigurationProvider.java | 138 -
.../FairSchedulerJsonVerifications.java | 4 +-
.../FairSchedulerXmlVerifications.java | 4 +-
.../TestRMWebServicesFairScheduler.java | 0
...ebServicesFairSchedulerCustomResourceTypes.java | 9 +-
.../webapp/helper/AppInfoJsonVerifications.java | 4 +-
.../webapp/helper/AppInfoXmlVerifications.java | 4 +-
.../webapp/helper/BufferedClientResponse.java | 11 +-
.../helper/JsonCustomResourceTypeTestcase.java | 9 +
.../helper/ResourceRequestsJsonVerifications.java | 4 +-
.../helper/ResourceRequestsXmlVerifications.java | 4 +-
.../helper/XmlCustomResourceTypeTestCase.java | 9 +
.../src/test/resources/resource-types-test.xml | 22 +
.../hadoop-yarn-server-router/pom.xml | 13 +-
.../apache/hadoop/yarn/server/router/Router.java | 8 +
.../yarn/server/router/clientrm/ClientMethod.java | 71 +
.../clientrm/DefaultClientRequestInterceptor.java | 25 +
.../clientrm/FederationClientInterceptor.java | 181 +-
.../router/clientrm/RouterClientRMService.java | 28 +
.../router/clientrm/RouterYarnClientUtils.java | 55 +
.../rmadmin/DefaultRMAdminRequestInterceptor.java | 9 +
.../router/rmadmin/RouterRMAdminService.java | 10 +
.../webapp/DefaultRequestInterceptorREST.java | 12 +
.../router/webapp/FederationInterceptorREST.java | 223 +-
.../server/router/webapp/RouterWebServices.java | 19 +
.../hadoop/yarn/server/router/TestRouter.java | 38 +
.../PassThroughClientRequestInterceptor.java | 25 +
.../clientrm/TestFederationClientInterceptor.java | 46 +-
.../router/clientrm/TestRouterYarnClientUtils.java | 57 +
.../PassThroughRMAdminRequestInterceptor.java | 9 +
.../webapp/MockDefaultRequestInterceptorREST.java | 12 +
.../router/webapp/MockRESTRequestInterceptor.java | 9 +-
.../webapp/PassThroughRESTRequestInterceptor.java | 9 +
.../webapp/TestFederationInterceptorREST.java | 22 +
.../router/webapp/TestRouterWebServicesREST.java | 54 +-
.../hadoop-yarn-server-sharedcachemanager/pom.xml | 6 +-
.../server/sharedcachemanager/TestCleanerTask.java | 7 +-
.../TestSCMAdminProtocolService.java | 4 +-
.../store/TestInMemorySCMStore.java | 3 +-
.../hadoop-yarn-server-tests/pom.xml | 8 +-
.../hadoop/yarn/server/TestDiskFailures.java | 38 +-
.../TestTimelineServiceClientIntegration.java | 2 +-
.../security/TestTimelineAuthFilterForV2.java | 4 +-
.../pom.xml | 6 +-
.../server/timeline/LevelDBCacheTimelineStore.java | 14 +-
.../timeline/TestEntityGroupFSTimelineStore.java | 6 +-
.../pom.xml | 13 +-
.../reader/DummyTimelineReaderMetrics.java | 39 +
.../TestTimelineReaderWebServicesHBaseStorage.java | 11 +
.../storage/DataGeneratorForTest.java | 2 +-
.../pom.xml | 10 +-
.../storage/HBaseTimelineSchemaCreator.java | 378 +
.../storage/TimelineSchemaCreator.java | 378 -
.../pom.xml | 4 +-
.../pom.xml | 4 +-
.../pom.xml | 12 +-
.../pom.xml | 4 +-
.../pom.xml | 4 +-
.../hadoop-yarn-server-timelineservice/pom.xml | 6 +-
.../collector/NodeTimelineCollectorManager.java | 15 +-
.../collector/TimelineCollectorManager.java | 2 +-
.../collector/TimelineCollectorWebService.java | 16 +-
.../PerNodeAggTimelineCollectorMetrics.java | 117 +
.../metrics/TimelineReaderMetrics.java | 114 +
.../timelineservice/metrics/package-info.java | 28 +
.../reader/TimelineReaderServer.java | 4 +-
.../reader/TimelineReaderWebServices.java | 197 +-
.../reader/TimelineReaderWebServicesUtils.java | 21 +-
...TimelineReaderWhitelistAuthorizationFilter.java | 2 +-
.../storage/FileSystemTimelineReaderImpl.java | 87 +-
.../storage/NoOpTimelineReaderImpl.java | 80 +
.../storage/NoOpTimelineWriterImpl.java | 88 +
.../timelineservice/storage/SchemaCreator.java | 28 +
.../storage/TimelineSchemaCreator.java | 80 +
.../collector/TestNMTimelineCollectorManager.java | 8 +-
.../TestPerNodeAggTimelineCollectorMetrics.java | 56 +
.../TestPerNodeTimelineCollectorsAuxService.java | 2 +-
.../collector/TestTimelineCollector.java | 2 +-
.../reader/TestTimelineReaderMetrics.java | 56 +
.../TestTimelineReaderWebServicesBasicAcl.java | 11 +-
...TimelineReaderWhitelistAuthorizationFilter.java | 2 +-
.../storage/DummyTimelineSchemaCreator.java | 29 +
.../storage/TestTimelineSchemaCreator.java | 41 +
.../hadoop-yarn-server-web-proxy/pom.xml | 14 +-
.../hadoop/yarn/server/webproxy/ProxyCA.java | 441 +
.../hadoop/yarn/server/webproxy/WebAppProxy.java | 15 +-
.../yarn/server/webproxy/WebAppProxyServer.java | 10 +-
.../yarn/server/webproxy/WebAppProxyServlet.java | 90 +-
.../hadoop/yarn/server/webproxy/TestProxyCA.java | 627 +
.../server/webproxy/TestWebAppProxyServer.java | 31 +-
.../server/webproxy/TestWebAppProxyServlet.java | 58 +-
.../server/webproxy/amfilter/TestAmFilter.java | 3 +-
.../hadoop-yarn/hadoop-yarn-server/pom.xml | 4 +-
.../hadoop-yarn/hadoop-yarn-site/pom.xml | 4 +-
.../src/site/markdown/CapacityScheduler.md | 16 +-
.../src/site/markdown/DockerContainers.md | 218 +-
.../src/site/markdown/FairScheduler.md | 2 +
.../src/site/markdown/NodeAttributes.md | 156 +
.../src/site/markdown/NodeManager.md | 59 +-
.../src/site/markdown/NodeManagerCGroupsMemory.md | 12 +-
.../src/site/markdown/NodeManagerCgroups.md | 4 +-
.../src/site/markdown/NodeManagerRest.md | 96 +
.../site/markdown/OpportunisticContainers.md.vm | 1 +
.../src/site/markdown/PlacementConstraints.md.vm | 4 +-
.../src/site/markdown/ResourceManagerHA.md | 5 +
.../src/site/markdown/ResourceManagerRest.md | 114 +
.../src/site/markdown/SecureContainer.md | 3 +
.../src/site/markdown/TimelineServiceV2.md | 35 +-
.../src/site/markdown/UsingFPGA.md | 2 +-
.../src/site/markdown/UsingGpus.md | 9 +-
.../src/site/markdown/YarnApplicationSecurity.md | 84 +-
.../src/site/markdown/YarnCommands.md | 3 +
.../hadoop-yarn-site/src/site/markdown/YarnUI2.md | 2 +-
.../src/site/markdown/registry/index.md | 30 -
.../markdown/registry/registry-configuration.md | 453 -
.../registry/using-the-yarn-service-registry.md | 273 -
.../src/site/markdown/registry/yarn-registry.md | 1033 -
.../site/markdown/yarn-service/Configurations.md | 3 +-
.../src/site/markdown/yarn-service/Examples.md | 2 +-
.../src/site/markdown/yarn-service/Overview.md | 2 +-
.../src/site/markdown/yarn-service/QuickStart.md | 1 +
.../src/site/markdown/yarn-service/RegistryDNS.md | 200 -
.../site/markdown/yarn-service/ServiceDiscovery.md | 83 +-
.../site/markdown/yarn-service/ServiceUpgrade.md | 38 +-
.../site/markdown/yarn-service/YarnServiceAPI.md | 64 +-
.../hadoop-yarn/hadoop-yarn-ui/pom.xml | 53 +-
.../src/main/webapp/app/adapters/yarn-app-log.js | 11 +-
.../main/webapp/app/adapters/yarn-app-timeline.js | 6 +-
.../webapp/app/adapters/yarn-component-instance.js | 8 +-
.../main/webapp/app/adapters/yarn-flow-activity.js | 4 +
.../src/main/webapp/app/adapters/yarn-log.js | 8 +-
.../webapp/app/adapters/yarn-service-component.js | 8 +-
.../main/webapp/app/adapters/yarn-service-info.js | 8 +-
.../src/main/webapp/app/adapters/yarn-service.js | 32 +
.../app/adapters/yarn-timeline-appattempt.js | 8 +
.../webapp/app/adapters/yarn-timeline-container.js | 4 +
.../main/webapp/app/components/timeline-view.js | 55 +-
.../webapp/app/controllers/app-table-columns.js | 14 +-
.../src/main/webapp/app/controllers/application.js | 10 +-
.../src/main/webapp/app/controllers/yarn-app.js | 20 +-
.../webapp/app/controllers/yarn-app/components.js | 2 +-
.../main/webapp/app/controllers/yarn-app/logs.js | 16 +-
.../controllers/yarn-component-instance/info.js | 5 +-
.../controllers/yarn-component-instances/info.js | 16 +-
.../src/main/webapp/app/helpers/log-files-comma.js | 2 +-
.../src/main/webapp/app/initializers/env.js | 1 +
.../src/main/webapp/app/initializers/loader.js | 65 +-
.../src/main/webapp/app/models/cluster-metric.js | 8 +-
.../src/main/webapp/app/models/yarn-app-attempt.js | 7 +
.../src/main/webapp/app/models/yarn-app.js | 15 +-
.../webapp/app/models/yarn-component-instance.js | 18 +
.../src/main/webapp/app/models/yarn-container.js | 8 +
.../src/main/webapp/app/models/yarn-service.js | 30 +
.../webapp/app/models/yarn-timeline-appattempt.js | 1 +
.../webapp/app/models/yarn-timeline-container.js | 8 +
.../hadoop-yarn-ui/src/main/webapp/app/router.js | 13 +-
.../src/main/webapp/app/routes/application.js | 12 +-
.../src/main/webapp/app/routes/yarn-app-attempt.js | 5 +
.../src/main/webapp/app/routes/yarn-app.js | 12 +
.../src/main/webapp/app/routes/yarn-app/logs.js | 6 +
.../app/routes/yarn-component-instance/info.js | 9 +-
.../src/main/webapp/app/routes/yarn-log-service.js | 66 +
.../main/webapp/app/routes/yarn-node-container.js | 8 +-
.../webapp/app/serializers/yarn-app-attempt.js | 1 +
.../main/webapp/app/serializers/yarn-app-log.js | 4 +-
.../src/main/webapp/app/serializers/yarn-app.js | 4 +-
.../app/serializers/yarn-component-instance.js | 2 +-
.../main/webapp/app/serializers/yarn-container.js | 5 +-
.../webapp/app/serializers/yarn-flowrun-brief.js | 11 +-
.../main/webapp/app/serializers/yarn-flowrun.js | 11 +-
.../app/serializers/yarn-service-component.js | 2 +-
.../main/webapp/app/serializers/yarn-service.js | 44 +
.../app/serializers/yarn-timeline-appattempt.js | 1 +
.../app/serializers/yarn-timeline-container.js | 3 +-
.../src/main/webapp/app/styles/app.scss | 8 +
.../src/main/webapp/app/styles/compose-box.scss | 6 +-
.../src/main/webapp/app/styles/layout.scss | 8 +
.../src/main/webapp/app/styles/yarn-app.scss | 5 +
.../app/templates/components/app-attempt-table.hbs | 14 +-
.../app/templates/components/container-table.hbs | 20 +-
.../app/templates/components/timeline-view.hbs | 2 +-
.../src/main/webapp/app/templates/error.hbs | 2 +-
.../src/main/webapp/app/templates/notfound.hbs | 2 +-
.../main/webapp/app/templates/yarn-app-attempt.hbs | 81 +-
.../src/main/webapp/app/templates/yarn-app.hbs | 38 +-
.../webapp/app/templates/yarn-app/attempts.hbs | 2 +-
.../main/webapp/app/templates/yarn-app/charts.hbs | 2 +-
.../main/webapp/app/templates/yarn-app/configs.hbs | 88 +-
.../main/webapp/app/templates/yarn-app/info.hbs | 2 +-
.../main/webapp/app/templates/yarn-app/logs.hbs | 49 +-
.../main/webapp/app/templates/yarn-apps/apps.hbs | 2 +
.../app/templates/yarn-component-instance.hbs | 22 +-
.../app/templates/yarn-component-instance/info.hbs | 16 +-
.../app/templates/yarn-component-instances.hbs | 33 +-
.../main/webapp/app/templates/yarn-log-service.hbs | 37 +
.../main/webapp/app/templates/yarn-node/info.hbs | 4 +-
.../src/main/webapp/app/templates/yarn-queue.hbs | 11 -
.../main/webapp/app/templates/yarn-queue/apps.hbs | 5 +-
.../main/webapp/app/templates/yarn-services.hbs | 4 +-
.../src/main/webapp/app/utils/converter.js | 2 +-
.../hadoop-yarn-ui/src/main/webapp/bower.json | 2 +-
.../src/main/webapp/config/default-config.js | 1 +
.../src/main/webapp/public/crossdomain.xml | 7 +-
hadoop-yarn-project/hadoop-yarn/pom.xml | 5 +-
hadoop-yarn-project/pom.xml | 4 +-
pom.xml | 11 +-
start-build-env.sh | 15 +-
4387 files changed, 366183 insertions(+), 79848 deletions(-)
delete mode 100755 dev-support/bin/ozone-dist-layout-stitching
delete mode 100755 dev-support/bin/ozone-dist-tar-stitching
create mode 100644 hadoop-assemblies/src/main/resources/assemblies/hadoop-registry-dist.xml
rename hadoop-assemblies/src/main/resources/assemblies/{hadoop-src-with-hdsl.xml => hadoop-src-with-hdds.xml} (100%)
create mode 100644 hadoop-common-project/hadoop-common/dev-support/jdiff/Apache_Hadoop_Common_3.1.2.xml
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ClusterStorageCapacityExceededException.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSBuilder.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FutureDataInputStreamBuilder.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/AbstractFSBuilderImpl.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/FutureDataInputStreamBuilderImpl.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/FutureIOSupport.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/WrappedIOException.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/package-info.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/http/IsActiveServlet.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/PassthroughCodec.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/AlignmentContext.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/ObserverRetryOnActiveException.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/ProxyCombiner.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/log/LogThrottlingHelper.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/DelegationTokenIssuer.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/CleanerUtil.java
create mode 100644 hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/LambdaUtils.java
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/FairCallQueue.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/filesystem/fsdatainputstreambuilder.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/filesystem/multipartuploader.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/registry/hadoop-registry.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/registry/index.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/registry/registry-configuration.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/registry/registry-dns.md
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site => hadoop-common-project/hadoop-common}/src/site/markdown/registry/registry-security.md (100%)
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/registry/using-the-hadoop-service-registry.md
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.1.0/{CHANGES.0.1.0.md => CHANGELOG.0.1.0.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.1.1/{CHANGES.0.1.1.md => CHANGELOG.0.1.1.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.10.0/{CHANGES.0.10.0.md => CHANGELOG.0.10.0.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.10.1/{CHANGES.0.10.1.md => CHANGELOG.0.10.1.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.11.0/{CHANGES.0.11.0.md => CHANGELOG.0.11.0.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.11.1/{CHANGES.0.11.1.md => CHANGELOG.0.11.1.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.11.2/{CHANGES.0.11.2.md => CHANGELOG.0.11.2.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.12.0/{CHANGES.0.12.0.md => CHANGELOG.0.12.0.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.12.1/{CHANGES.0.12.1.md => CHANGELOG.0.12.1.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.12.2/{CHANGES.0.12.2.md => CHANGELOG.0.12.2.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.12.3/{CHANGES.0.12.3.md => CHANGELOG.0.12.3.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.13.0/{CHANGES.0.13.0.md => CHANGELOG.0.13.0.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.14.0/{CHANGES.0.14.0.md => CHANGELOG.0.14.0.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.14.1/{CHANGES.0.14.1.md => CHANGELOG.0.14.1.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.14.2/{CHANGES.0.14.2.md => CHANGELOG.0.14.2.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.14.3/{CHANGES.0.14.3.md => CHANGELOG.0.14.3.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.14.4/{CHANGES.0.14.4.md => CHANGELOG.0.14.4.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.15.0/{CHANGES.0.15.0.md => CHANGELOG.0.15.0.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.15.1/{CHANGES.0.15.1.md => CHANGELOG.0.15.1.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.15.2/{CHANGES.0.15.2.md => CHANGELOG.0.15.2.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.15.3/{CHANGES.0.15.3.md => CHANGELOG.0.15.3.md} (100%)
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/0.15.4/CHANGELOG.0.15.4.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/0.15.4/CHANGES.0.15.4.md
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.16.0/{CHANGES.0.16.0.md => CHANGELOG.0.16.0.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.16.1/{CHANGES.0.16.1.md => CHANGELOG.0.16.1.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.16.2/{CHANGES.0.16.2.md => CHANGELOG.0.16.2.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.16.3/{CHANGES.0.16.3.md => CHANGELOG.0.16.3.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.16.4/{CHANGES.0.16.4.md => CHANGELOG.0.16.4.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.17.0/{CHANGES.0.17.0.md => CHANGELOG.0.17.0.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.17.1/{CHANGES.0.17.1.md => CHANGELOG.0.17.1.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.17.2/{CHANGES.0.17.2.md => CHANGELOG.0.17.2.md} (100%)
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/0.17.3/CHANGELOG.0.17.3.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/0.17.3/CHANGES.0.17.3.md
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.0/{CHANGES.0.18.0.md => CHANGELOG.0.18.0.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.1/{CHANGES.0.18.1.md => CHANGELOG.0.18.1.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.2/{CHANGES.0.18.2.md => CHANGELOG.0.18.2.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.3/{CHANGES.0.18.3.md => CHANGELOG.0.18.3.md} (100%)
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.4/CHANGELOG.0.18.4.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.4/CHANGES.0.18.4.md
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.19.0/{CHANGES.0.19.0.md => CHANGELOG.0.19.0.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.19.1/{CHANGES.0.19.1.md => CHANGELOG.0.19.1.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.19.2/{CHANGES.0.19.2.md => CHANGELOG.0.19.2.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.2.0/{CHANGES.0.2.0.md => CHANGELOG.0.2.0.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.2.1/{CHANGES.0.2.1.md => CHANGELOG.0.2.1.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.0/{CHANGES.0.20.0.md => CHANGELOG.0.20.0.md} (100%)
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.1/CHANGELOG.0.20.1.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.1/CHANGES.0.20.1.md
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.2/{CHANGES.0.20.2.md => CHANGELOG.0.20.2.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.203.0/{CHANGES.0.20.203.0.md => CHANGELOG.0.20.203.0.md} (100%)
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.203.1/CHANGELOG.0.20.203.1.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.203.1/CHANGES.0.20.203.1.md
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.204.0/{CHANGES.0.20.204.0.md => CHANGELOG.0.20.204.0.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.205.0/{CHANGES.0.20.205.0.md => CHANGELOG.0.20.205.0.md} (100%)
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.3/CHANGELOG.0.20.3.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.3/CHANGES.0.20.3.md
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.21.0/{CHANGES.0.21.0.md => CHANGELOG.0.21.0.md} (100%)
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/0.21.1/CHANGELOG.0.21.1.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/0.21.1/CHANGES.0.21.1.md
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.22.0/{CHANGES.0.22.0.md => CHANGELOG.0.22.0.md} (100%)
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/0.22.1/CHANGELOG.0.22.1.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/0.22.1/CHANGES.0.22.1.md
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.0/{CHANGES.0.23.0.md => CHANGELOG.0.23.0.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.1/{CHANGES.0.23.1.md => CHANGELOG.0.23.1.md} (100%)
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.10/CHANGELOG.0.23.10.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.10/CHANGES.0.23.10.md
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.11/{CHANGES.0.23.11.md => CHANGELOG.0.23.11.md} (100%)
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.2/CHANGELOG.0.23.2.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.2/CHANGES.0.23.2.md
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.3/{CHANGES.0.23.3.md => CHANGELOG.0.23.3.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.4/{CHANGES.0.23.4.md => CHANGELOG.0.23.4.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.5/{CHANGES.0.23.5.md => CHANGELOG.0.23.5.md} (100%)
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.6/CHANGELOG.0.23.6.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.6/CHANGES.0.23.6.md
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.7/{CHANGES.0.23.7.md => CHANGELOG.0.23.7.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.8/{CHANGES.0.23.8.md => CHANGELOG.0.23.8.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.9/{CHANGES.0.23.9.md => CHANGELOG.0.23.9.md} (100%)
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/0.24.0/CHANGELOG.0.24.0.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/0.24.0/CHANGES.0.24.0.md
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.3.0/{CHANGES.0.3.0.md => CHANGELOG.0.3.0.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.3.1/{CHANGES.0.3.1.md => CHANGELOG.0.3.1.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.3.2/{CHANGES.0.3.2.md => CHANGELOG.0.3.2.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.4.0/{CHANGES.0.4.0.md => CHANGELOG.0.4.0.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.5.0/{CHANGES.0.5.0.md => CHANGELOG.0.5.0.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.6.0/{CHANGES.0.6.0.md => CHANGELOG.0.6.0.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.6.1/{CHANGES.0.6.1.md => CHANGELOG.0.6.1.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.6.2/{CHANGES.0.6.2.md => CHANGELOG.0.6.2.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.7.0/{CHANGES.0.7.0.md => CHANGELOG.0.7.0.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.7.1/{CHANGES.0.7.1.md => CHANGELOG.0.7.1.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.7.2/{CHANGES.0.7.2.md => CHANGELOG.0.7.2.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.8.0/{CHANGES.0.8.0.md => CHANGELOG.0.8.0.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.9.0/{CHANGES.0.9.0.md => CHANGELOG.0.9.0.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.9.1/{CHANGES.0.9.1.md => CHANGELOG.0.9.1.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/0.9.2/{CHANGES.0.9.2.md => CHANGELOG.0.9.2.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/1.0.0/{CHANGES.1.0.0.md => CHANGELOG.1.0.0.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/1.0.1/{CHANGES.1.0.1.md => CHANGELOG.1.0.1.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/1.0.2/{CHANGES.1.0.2.md => CHANGELOG.1.0.2.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/1.0.3/{CHANGES.1.0.3.md => CHANGELOG.1.0.3.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/1.0.4/{CHANGES.1.0.4.md => CHANGELOG.1.0.4.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/1.1.0/{CHANGES.1.1.0.md => CHANGELOG.1.1.0.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/1.1.1/{CHANGES.1.1.1.md => CHANGELOG.1.1.1.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/1.1.2/{CHANGES.1.1.2.md => CHANGELOG.1.1.2.md} (100%)
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/1.1.3/CHANGELOG.1.1.3.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/1.1.3/CHANGES.1.1.3.md
rename hadoop-common-project/hadoop-common/src/site/markdown/release/1.2.0/{CHANGES.1.2.0.md => CHANGELOG.1.2.0.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/1.2.1/{CHANGES.1.2.1.md => CHANGELOG.1.2.1.md} (100%)
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/1.2.2/CHANGELOG.1.2.2.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/1.2.2/CHANGES.1.2.2.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/1.3.0/CHANGELOG.1.3.0.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/1.3.0/CHANGES.1.3.0.md
rename hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.0-alpha/{CHANGES.2.0.0-alpha.md => CHANGELOG.2.0.0-alpha.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.1-alpha/{CHANGES.2.0.1-alpha.md => CHANGELOG.2.0.1-alpha.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.2-alpha/{CHANGES.2.0.2-alpha.md => CHANGELOG.2.0.2-alpha.md} (100%)
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.3-alpha/CHANGELOG.2.0.3-alpha.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.3-alpha/CHANGES.2.0.3-alpha.md
rename hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.4-alpha/{CHANGES.2.0.4-alpha.md => CHANGELOG.2.0.4-alpha.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.5-alpha/{CHANGES.2.0.5-alpha.md => CHANGELOG.2.0.5-alpha.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.6-alpha/{CHANGES.2.0.6-alpha.md => CHANGELOG.2.0.6-alpha.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/2.1.0-beta/{CHANGES.2.1.0-beta.md => CHANGELOG.2.1.0-beta.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/2.1.1-beta/{CHANGES.2.1.1-beta.md => CHANGELOG.2.1.1-beta.md} (100%)
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.10.0/CHANGELOG.2.10.0.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.10.0/RELEASENOTES.2.10.0.md
rename hadoop-common-project/hadoop-common/src/site/markdown/release/2.2.0/{CHANGES.2.2.0.md => CHANGELOG.2.2.0.md} (100%)
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.2.1/CHANGELOG.2.2.1.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.2.1/CHANGES.2.2.1.md
rename hadoop-common-project/hadoop-common/src/site/markdown/release/2.3.0/{CHANGES.2.3.0.md => CHANGELOG.2.3.0.md} (100%)
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.4.0/CHANGELOG.2.4.0.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.4.0/CHANGES.2.4.0.md
rename hadoop-common-project/hadoop-common/src/site/markdown/release/2.4.1/{CHANGES.2.4.1.md => CHANGELOG.2.4.1.md} (100%)
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.5.0/CHANGELOG.2.5.0.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.5.0/CHANGES.2.5.0.md
rename hadoop-common-project/hadoop-common/src/site/markdown/release/2.5.1/{CHANGES.2.5.1.md => CHANGELOG.2.5.1.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/2.5.2/{CHANGES.2.5.2.md => CHANGELOG.2.5.2.md} (100%)
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.6.0/CHANGELOG.2.6.0.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.6.0/CHANGES.2.6.0.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.6.1/CHANGELOG.2.6.1.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.6.1/CHANGES.2.6.1.md
rename hadoop-common-project/hadoop-common/src/site/markdown/release/2.6.2/{CHANGES.2.6.2.md => CHANGELOG.2.6.2.md} (100%)
rename hadoop-common-project/hadoop-common/src/site/markdown/release/2.6.3/{CHANGES.2.6.3.md => CHANGELOG.2.6.3.md} (100%)
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.6.4/CHANGELOG.2.6.4.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.6.4/CHANGES.2.6.4.md
rename hadoop-common-project/hadoop-common/src/site/markdown/release/2.6.5/{CHANGES.2.6.5.md => CHANGELOG.2.6.5.md} (100%)
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.6.6/CHANGELOG.2.6.6.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.6.6/CHANGES.2.6.6.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.7.0/CHANGELOG.2.7.0.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.7.0/CHANGES.2.7.0.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.7.1/CHANGELOG.2.7.1.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.7.1/CHANGES.2.7.1.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.7.2/CHANGELOG.2.7.2.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.7.2/CHANGES.2.7.2.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.7.3/CHANGELOG.2.7.3.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.7.3/CHANGES.2.7.3.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.7.4/CHANGELOG.2.7.4.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.7.4/CHANGES.2.7.4.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.7.5/CHANGELOG.2.7.5.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.7.5/CHANGES.2.7.5.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.7.6/CHANGELOG.2.7.6.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.7.6/RELEASENOTES.2.7.6.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.7.7/CHANGELOG.2.7.7.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.7.7/RELEASENOTES.2.7.7.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.7.8/CHANGELOG.2.7.8.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.7.8/RELEASENOTES.2.7.8.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.8.0/CHANGELOG.2.8.0.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.8.0/CHANGES.2.8.0.md
rename hadoop-common-project/hadoop-common/src/site/markdown/release/2.8.1/{CHANGES.2.8.1.md => CHANGELOG.2.8.1.md} (100%)
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.8.2/CHANGELOG.2.8.2.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.8.2/CHANGES.2.8.2.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.8.3/CHANGELOG.2.8.3.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.8.3/CHANGES.2.8.3.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.8.4/CHANGELOG.2.8.4.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.8.4/RELEASENOTES.2.8.4.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.8.5/CHANGELOG.2.8.5.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.8.5/RELEASENOTES.2.8.5.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.9.0/CHANGELOG.2.9.0.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.9.0/CHANGES.2.9.0.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.9.1/CHANGELOG.2.9.1.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.9.1/CHANGES.2.9.1.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.9.2/CHANGELOG.2.9.2.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/2.9.2/RELEASENOTES.2.9.2.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/3.0.0-alpha1/CHANGELOG.3.0.0-alpha1.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/3.0.0-alpha1/CHANGES.3.0.0-alpha1.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/3.0.0-alpha2/CHANGELOG.3.0.0-alpha2.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/3.0.0-alpha2/CHANGES.3.0.0-alpha2.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/3.0.0-alpha3/CHANGELOG.3.0.0-alpha3.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/3.0.0-alpha3/CHANGES.3.0.0-alpha3.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/3.0.0-alpha4/CHANGELOG.3.0.0-alpha4.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/3.0.0-alpha4/CHANGES.3.0.0-alpha4.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/3.0.0-beta1/CHANGELOG.3.0.0-beta1.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/3.0.0-beta1/CHANGES.3.0.0-beta1.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/3.0.0/CHANGELOG.3.0.0.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/3.0.0/CHANGES.3.0.0.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/3.0.1/CHANGELOG.3.0.1.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/3.0.1/CHANGES.3.0.1.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/3.0.2/CHANGELOG.3.0.2.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/3.0.2/CHANGES.3.0.2.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/3.0.3/CHANGELOG.3.0.3.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/3.0.3/CHANGES.3.0.3.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/3.0.4/CHANGELOG.3.0.4.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/3.0.4/RELEASENOTES.3.0.4.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/3.1.0/CHANGELOG.3.1.0.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/3.1.0/CHANGES.3.1.0.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/3.1.1/CHANGELOG.3.1.1.md
delete mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/3.1.1/CHANGES.3.1.1.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/3.1.2/CHANGES.3.1.2.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/3.1.2/RELEASENOTES.3.1.2.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/3.2.0/CHANGELOG.3.2.0.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/markdown/release/3.2.0/RELEASENOTES.3.2.0.md
create mode 100644 hadoop-common-project/hadoop-common/src/site/resources/images/faircallqueue-overview.png
create mode 100644 hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/crypto/key/kms/TestKMSClientProvider.java
create mode 100644 hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestTail.java
create mode 100644 hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestIsActiveServlet.java
create mode 100644 hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/log/TestLogThrottlingHelper.java
create mode 100644 hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestLdapGroupsMappingWithFailover.java
create mode 100644 hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMSAuthenticationFilter.java
create mode 100644 hadoop-common-project/hadoop-registry/dev-support/findbugs-exclude.xml
create mode 100644 hadoop-common-project/hadoop-registry/pom.xml
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/cli/RegistryCli.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/api/BindFlags.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/api/DNSOperations.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/api/DNSOperationsFactory.java (100%)
create mode 100644 hadoop-common-project/hadoop-registry/src/main/java/org/apache/hadoop/registry/client/api/RegistryConstants.java
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/api/RegistryOperations.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/api/RegistryOperationsFactory.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/api/package-info.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/binding/JsonSerDeser.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/binding/RegistryPathUtils.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/binding/RegistryTypeUtils.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/binding/RegistryUtils.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/binding/package-info.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/exceptions/AuthenticationFailedException.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/exceptions/InvalidPathnameException.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/exceptions/InvalidRecordException.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/exceptions/NoChildrenForEphemeralsException.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/exceptions/NoPathPermissionsException.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/exceptions/NoRecordException.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/exceptions/RegistryIOException.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/exceptions/package-info.java (100%)
create mode 100644 hadoop-common-project/hadoop-registry/src/main/java/org/apache/hadoop/registry/client/impl/FSRegistryOperationsService.java
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/impl/RegistryOperationsClient.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/impl/package-info.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/impl/zk/BindingInformation.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/impl/zk/CuratorService.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/impl/zk/ListenerHandle.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/impl/zk/PathListener.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/impl/zk/RegistryBindingSource.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/impl/zk/RegistryInternalConstants.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/impl/zk/RegistryOperationsService.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/impl/zk/RegistrySecurity.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/impl/zk/ZKPathDumper.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/impl/zk/ZookeeperConfigOptions.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/impl/zk/package-info.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/types/AddressTypes.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/types/Endpoint.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/types/ProtocolTypes.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/types/RegistryPathStatus.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/types/ServiceRecord.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/types/package-info.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/types/yarn/PersistencePolicies.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/client/types/yarn/YarnRegistryAttributes.java (100%)
create mode 100644 hadoop-common-project/hadoop-registry/src/main/java/org/apache/hadoop/registry/conf/RegistryConfiguration.java
create mode 100644 hadoop-common-project/hadoop-registry/src/main/java/org/apache/hadoop/registry/conf/package-info.java
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/server/dns/ApplicationServiceRecordProcessor.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/server/dns/BaseServiceRecordProcessor.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/server/dns/ContainerServiceRecordProcessor.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/server/dns/LookupTask.java (100%)
create mode 100644 hadoop-common-project/hadoop-registry/src/main/java/org/apache/hadoop/registry/server/dns/PrivilegedRegistryDNSStarter.java
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/server/dns/RecordCreatorFactory.java (100%)
create mode 100644 hadoop-common-project/hadoop-registry/src/main/java/org/apache/hadoop/registry/server/dns/RegistryDNS.java
create mode 100644 hadoop-common-project/hadoop-registry/src/main/java/org/apache/hadoop/registry/server/dns/RegistryDNSServer.java
create mode 100644 hadoop-common-project/hadoop-registry/src/main/java/org/apache/hadoop/registry/server/dns/ReverseZoneUtils.java
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/server/dns/SecureableZone.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/server/dns/ServiceRecordProcessor.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/server/dns/ZoneSelector.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/server/dns/package-info.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/server/integration/SelectByYarnPersistence.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/server/integration/package-info.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/server/package-info.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/server/services/AddingCompositeService.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/server/services/DeleteCompletionCallback.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/server/services/MicroZookeeperService.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/server/services/MicroZookeeperServiceKeys.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/server/services/RegistryAdminService.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/main/java/org/apache/hadoop/registry/server/services/package-info.java (100%)
create mode 100644 hadoop-common-project/hadoop-registry/src/main/tla/hadoopregistry.tla
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/test/java/org/apache/hadoop/registry/AbstractRegistryTest.java (100%)
create mode 100644 hadoop-common-project/hadoop-registry/src/test/java/org/apache/hadoop/registry/AbstractZKRegistryTest.java
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/test/java/org/apache/hadoop/registry/RegistryTestHelper.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/test/java/org/apache/hadoop/registry/cli/TestRegistryCli.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/test/java/org/apache/hadoop/registry/client/binding/TestMarshalling.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/test/java/org/apache/hadoop/registry/client/binding/TestRegistryOperationUtils.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/test/java/org/apache/hadoop/registry/client/binding/TestRegistryPathUtils.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/test/java/org/apache/hadoop/registry/client/impl/CuratorEventCatcher.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/test/java/org/apache/hadoop/registry/client/impl/TestCuratorService.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/test/java/org/apache/hadoop/registry/client/impl/TestFSRegistryOperationsService.java (100%)
create mode 100644 hadoop-common-project/hadoop-registry/src/test/java/org/apache/hadoop/registry/client/impl/TestMicroZookeeperService.java
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/test/java/org/apache/hadoop/registry/integration/TestYarnPolicySelector.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/test/java/org/apache/hadoop/registry/operations/TestRegistryOperations.java (100%)
create mode 100644 hadoop-common-project/hadoop-registry/src/test/java/org/apache/hadoop/registry/secure/AbstractSecureRegistryTest.java
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/test/java/org/apache/hadoop/registry/secure/KerberosConfiguration.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/test/java/org/apache/hadoop/registry/secure/TestRegistrySecurityHelper.java (100%)
create mode 100644 hadoop-common-project/hadoop-registry/src/test/java/org/apache/hadoop/registry/secure/TestSecureLogins.java
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/test/java/org/apache/hadoop/registry/secure/TestSecureRegistry.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/test/java/org/apache/hadoop/registry/server/dns/TestRegistryDNS.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/test/java/org/apache/hadoop/registry/server/dns/TestReverseZoneUtils.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/test/java/org/apache/hadoop/registry/server/dns/TestSecureRegistryDNS.java (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/test/resources/0.17.172.in-addr.arpa.zone (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/test/resources/log4j.properties (100%)
rename {hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry => hadoop-common-project/hadoop-registry}/src/test/resources/test.private (100%)
delete mode 100644 hadoop-dist/src/main/compose/ozone/docker-compose.yaml
delete mode 100644 hadoop-dist/src/main/compose/ozone/docker-config
delete mode 100644 hadoop-dist/src/main/compose/ozoneperf/README.md
delete mode 100755 hadoop-dist/src/main/compose/ozoneperf/compose-all.sh
delete mode 100644 hadoop-dist/src/main/compose/ozoneperf/docker-compose-freon.yaml
delete mode 100644 hadoop-dist/src/main/compose/ozoneperf/docker-compose.yaml
delete mode 100644 hadoop-dist/src/main/compose/ozoneperf/docker-config
delete mode 100755 hadoop-dist/src/main/compose/ozoneperf/init.sh
delete mode 100644 hadoop-dist/src/main/compose/ozoneperf/prometheus.yml
create mode 100644 hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/ClientCredentialInterceptor.java
delete mode 100644 hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/XceiverClient.java
delete mode 100644 hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/XceiverClientHandler.java
delete mode 100644 hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/XceiverClientInitializer.java
create mode 100644 hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockInputStream.java
create mode 100644 hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java
delete mode 100644 hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/ChunkInputStream.java
delete mode 100644 hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/ChunkOutputStream.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/cli/GenericParentCommand.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/cli/MissingSubcommandException.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/client/ContainerBlockID.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/protocol/SCMSecurityProtocol.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/protocolPB/SCMSecurityProtocolClientSideTranslatorPB.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/protocolPB/SCMSecurityProtocolPB.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/protocolPB/SCMSecurityProtocolServerSideTranslatorPB.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/protocolPB/package-info.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/XceiverClientReply.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerException.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerInfo.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerNotFoundException.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerReplicaNotFoundException.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/common/helpers/ContainerInfo.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/common/helpers/ContainerNotOpenException.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/common/helpers/InvalidContainerStateException.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/common/helpers/Pipeline.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/common/helpers/PipelineID.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/pipeline/Pipeline.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineID.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineNotFoundException.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/pipeline/UnknownPipelineStateException.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/pipeline/package-info.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/exception/SCMSecurityException.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/exception/package-info.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/token/BlockTokenException.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/token/BlockTokenVerifier.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/token/OzoneBlockTokenIdentifier.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/token/OzoneBlockTokenSelector.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/token/TokenVerifier.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/token/package-info.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/SecurityConfig.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/authority/BaseApprover.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/authority/CertificateApprover.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/authority/CertificateServer.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/authority/DefaultApprover.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/authority/DefaultCAServer.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/authority/PKIProfiles/DefaultCAProfile.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/authority/PKIProfiles/DefaultProfile.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/authority/PKIProfiles/PKIProfile.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/authority/PKIProfiles/package-info.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/authority/package-info.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/client/CertificateClient.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/client/DNCertificateClient.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/client/DefaultCertificateClient.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/client/OMCertificateClient.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/client/package-info.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/utils/CertificateCodec.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/utils/package-info.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificates/utils/CertificateSignRequest.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificates/utils/SelfSignedCertificate.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificates/utils/package-info.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/exceptions/CertificateException.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/exceptions/package-info.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/keys/HDDSKeyGenerator.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/keys/KeyCodec.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/keys/SecurityUtil.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/keys/package-info.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/package-info.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/tracing/GrpcClientInterceptor.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/tracing/GrpcServerInterceptor.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/tracing/StringCodec.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/tracing/TraceAllMethod.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/tracing/TracingUtil.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/tracing/package-info.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneSecurityUtil.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/audit/Auditor.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/audit/DNAction.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/audit/SCMAction.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/common/Checksum.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/common/ChecksumData.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/common/OzoneChecksumException.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/container/common/helpers/BlockData.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/container/common/helpers/ContainerCommandRequestPBHelper.java
delete mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/container/common/helpers/KeyData.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/lock/ActiveLock.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/lock/LockManager.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/lock/PooledLockFactory.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/lock/package-info.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/RetriableTask.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/VersionInfo.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/db/BatchOperation.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/db/ByteArrayKeyValue.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/db/Codec.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/db/CodecRegistry.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/db/RDBBatchOperation.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/db/StringCodec.java
create mode 100644 hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/db/TypedTable.java
create mode 100644 hadoop-hdds/common/src/main/proto/SCMSecurityProtocol.proto
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/token/TestOzoneBlockTokenIdentifier.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/token/package-info.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/certificate/authority/MockApprover.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/certificate/authority/TestDefaultCAServer.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/certificate/authority/TestDefaultProfile.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/certificate/authority/package-info.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/certificate/client/TestCertificateClientInit.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/certificate/client/TestDefaultCertificateClient.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/certificate/utils/TestCertificateCodec.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/certificate/utils/package-info.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/certificates/TestCertificateSignRequest.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/certificates/TestRootCertificate.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/certificates/package-info.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/keys/TestHDDSKeyGenerator.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/keys/TestKeyCodec.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/keys/package-info.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/package-info.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/common/TestChecksum.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/lock/TestLockManager.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/lock/package-info.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/utils/TestRetriableTask.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/utils/db/TestTypedRDBTableStore.java
create mode 100644 hadoop-hdds/common/src/test/java/org/apache/hadoop/utils/package-info.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/helpers/ContainerReport.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/helpers/KeyValueContainerReport.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/report/PipelineReportPublisher.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/commandhandler/DeleteContainerCommandHandler.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ServerCredentialInterceptor.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/XceiverServerHandler.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/XceiverServerInitializer.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/DispatcherContext.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/AbstractFuture.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/HddsVolumeChecker.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/ThrottledAsyncChecker.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/TimeoutFuture.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/helpers/BlockUtils.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/helpers/KeyUtils.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/BlockManagerImpl.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/KeyManagerImpl.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/interfaces/BlockManager.java
delete mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/interfaces/KeyManager.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/interfaces/package-info.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/ozoneimpl/ContainerController.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/ContainerDownloader.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/ContainerReplicationSource.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/ContainerReplicator.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/ContainerStreamingOutput.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/DownloadAndImportReplicator.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/GrpcReplicationClient.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/GrpcReplicationService.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/OnDemandContainerReplicationSource.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/ReplicationSupervisor.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/ReplicationTask.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/SimpleContainerDownloader.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/package-info.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/protocol/commands/DeleteBlockCommandStatus.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/protocol/commands/DeleteContainerCommand.java
create mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/TestHddsDatanodeService.java
create mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/statemachine/commandhandler/TestCloseContainerCommandHandler.java
create mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/statemachine/commandhandler/package-info.java
create mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/volume/TestHddsVolumeChecker.java
create mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/volume/TestVolumeSetDiskChecks.java
create mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/volume/package-info.java
create mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/keyvalue/TestBlockManagerImpl.java
delete mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/keyvalue/TestKeyManagerImpl.java
create mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/keyvalue/TestKeyValueContainerMarkUnhealthy.java
create mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/keyvalue/TestKeyValueHandlerWithUnhealthyContainer.java
create mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/keyvalue/package-info.java
create mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/replication/TestReplicationSupervisor.java
create mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/replication/package-info.java
create mode 100644 hadoop-hdds/docs/README.md
create mode 100644 hadoop-hdds/docs/archetypes/default.md
create mode 100644 hadoop-hdds/docs/config.yaml
create mode 100644 hadoop-hdds/docs/content/AuditParser.md
create mode 100644 hadoop-hdds/docs/content/BucketCommands.md
create mode 100644 hadoop-hdds/docs/content/BuildingSources.md
create mode 100644 hadoop-hdds/docs/content/CommandShell.md
create mode 100644 hadoop-hdds/docs/content/Concepts.md
create mode 100644 hadoop-hdds/docs/content/Dozone.md
create mode 100644 hadoop-hdds/docs/content/Freon.md
create mode 100644 hadoop-hdds/docs/content/Hdds.md
create mode 100644 hadoop-hdds/docs/content/JavaApi.md
create mode 100644 hadoop-hdds/docs/content/KeyCommands.md
create mode 100644 hadoop-hdds/docs/content/OzoneFS.md
create mode 100644 hadoop-hdds/docs/content/OzoneManager.md
create mode 100644 hadoop-hdds/docs/content/Prometheus.md
create mode 100644 hadoop-hdds/docs/content/RealCluster.md
create mode 100644 hadoop-hdds/docs/content/Rest.md
create mode 100644 hadoop-hdds/docs/content/RunningViaDocker.md
create mode 100644 hadoop-hdds/docs/content/RunningWithHDFS.md
create mode 100644 hadoop-hdds/docs/content/S3.md
create mode 100644 hadoop-hdds/docs/content/S3Commands.md
create mode 100644 hadoop-hdds/docs/content/SCMCLI.md
create mode 100644 hadoop-hdds/docs/content/Settings.md
create mode 100644 hadoop-hdds/docs/content/VolumeCommands.md
create mode 100644 hadoop-hdds/docs/content/_index.md
create mode 100644 hadoop-hdds/docs/content/genconf.md
create mode 100755 hadoop-hdds/docs/dev-support/bin/generate-site.sh
create mode 100644 hadoop-hdds/docs/pom.xml
create mode 100644 hadoop-hdds/docs/static/NOTES.md
rename {hadoop-ozone => hadoop-hdds}/docs/static/OzoneOverview.png (100%)
rename {hadoop-ozone => hadoop-hdds}/docs/static/OzoneOverview.svg (100%)
rename {hadoop-ozone => hadoop-hdds}/docs/static/SCMBlockDiagram.png (100%)
create mode 100644 hadoop-hdds/docs/static/ozone-logo.png
create mode 100644 hadoop-hdds/docs/themes/ozonedoc/layouts/_default/single.html
create mode 100644 hadoop-hdds/docs/themes/ozonedoc/layouts/index.html
create mode 100644 hadoop-hdds/docs/themes/ozonedoc/layouts/partials/footer.html
create mode 100644 hadoop-hdds/docs/themes/ozonedoc/layouts/partials/header.html
create mode 100644 hadoop-hdds/docs/themes/ozonedoc/layouts/partials/navbar.html
create mode 100644 hadoop-hdds/docs/themes/ozonedoc/layouts/partials/sidebar.html
rename {hadoop-ozone => hadoop-hdds}/docs/themes/ozonedoc/static/css/bootstrap-theme.min.css (100%)
rename {hadoop-ozone => hadoop-hdds}/docs/themes/ozonedoc/static/css/bootstrap-theme.min.css.map (100%)
rename {hadoop-ozone => hadoop-hdds}/docs/themes/ozonedoc/static/css/bootstrap.min.css (100%)
rename {hadoop-ozone => hadoop-hdds}/docs/themes/ozonedoc/static/css/bootstrap.min.css.map (100%)
create mode 100644 hadoop-hdds/docs/themes/ozonedoc/static/css/ozonedoc.css
rename {hadoop-ozone => hadoop-hdds}/docs/themes/ozonedoc/static/fonts/glyphicons-halflings-regular.eot (100%)
copy {hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/bootstrap-3.3.7 => hadoop-hdds/docs/themes/ozonedoc/static}/fonts/glyphicons-halflings-regular.svg (100%)
rename {hadoop-ozone => hadoop-hdds}/docs/themes/ozonedoc/static/fonts/glyphicons-halflings-regular.ttf (100%)
rename {hadoop-ozone => hadoop-hdds}/docs/themes/ozonedoc/static/fonts/glyphicons-halflings-regular.woff (100%)
rename {hadoop-ozone => hadoop-hdds}/docs/themes/ozonedoc/static/fonts/glyphicons-halflings-regular.woff2 (100%)
rename {hadoop-ozone => hadoop-hdds}/docs/themes/ozonedoc/static/js/bootstrap.min.js (100%)
rename {hadoop-ozone => hadoop-hdds}/docs/themes/ozonedoc/static/js/jquery.min.js (100%)
rename {hadoop-ozone => hadoop-hdds}/docs/themes/ozonedoc/static/js/ozonedoc.js (100%)
rename {hadoop-ozone => hadoop-hdds}/docs/themes/ozonedoc/theme.toml (100%)
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/PrometheusMetricsSink.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/PrometheusServlet.java
create mode 100644 hadoop-hdds/framework/src/test/java/org/apache/hadoop/hdds/server/TestPrometheusMetricsSink.java
create mode 100644 hadoop-hdds/framework/src/test/java/org/apache/hadoop/hdds/server/events/package-info.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/ScmUtils.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/chillmode/ChillModeExitRule.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/chillmode/ChillModePrecheck.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/chillmode/ChillModeRestrictedOps.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/chillmode/ContainerChillModeRule.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/chillmode/DataNodeChillModeRule.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/chillmode/PipelineChillModeRule.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/chillmode/Precheck.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/chillmode/SCMChillModeManager.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/chillmode/package-info.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerManager.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerMapping.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerReplica.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/DeleteContainerCommandWatcher.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/IncrementalContainerReportHandler.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/Mapping.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ReportHandlerHelper.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/SCMContainerManager.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/closer/ContainerCloser.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/states/ContainerQueryKey.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/metadata/DeletedBlocksTransactionCodec.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/metadata/LongCodec.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/metadata/SCMMetadataStore.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/metadata/SCMMetadataStoreRDBImpl.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/metadata/package-info.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/NonHealthyToHealthyNodeHandler.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/states/Node2ObjectsMap.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/states/Node2PipelineMap.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineActionHandler.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineFactory.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineManager.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineProvider.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineReportHandler.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineStateManager.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineStateMap.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/RatisPipelineProvider.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/RatisPipelineUtils.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/SCMPipelineManager.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/SimplePipelineProvider.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/package-info.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipelines/Node2PipelineMap.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipelines/PipelineManager.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipelines/PipelineSelector.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipelines/package-info.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipelines/ratis/RatisManagerImpl.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipelines/ratis/package-info.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipelines/standalone/StandaloneManagerImpl.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipelines/standalone/package-info.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMConfigurator.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMSecurityProtocolServer.java
delete mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMStorage.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMStorageConfig.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/ozone/protocol/commands/RetriableDatanodeEventWatcher.java
create mode 100644 hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/ozone/protocol/commands/package-info.java
rename hadoop-hdds/server-scm/src/main/{ => resources}/webapps/scm/index.html (100%)
rename hadoop-hdds/server-scm/src/main/{ => resources}/webapps/scm/main.html (100%)
create mode 100644 hadoop-hdds/server-scm/src/main/resources/webapps/scm/scm-overview.html
create mode 100644 hadoop-hdds/server-scm/src/main/resources/webapps/scm/scm.js
delete mode 100644 hadoop-hdds/server-scm/src/main/webapps/scm/scm-overview.html
delete mode 100644 hadoop-hdds/server-scm/src/main/webapps/scm/scm.js
create mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/HddsTestUtils.java
create mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/HddsWhiteboxTestUtils.java
create mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/TestHddsServerUtils.java
create mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/chillmode/TestSCMChillModeManager.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/TestContainerMapping.java
create mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/TestContainerReportHelper.java
create mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/TestContainerStateManager.java
create mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/TestIncrementalContainerReportHandler.java
create mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/TestSCMContainerManager.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/closer/TestContainerCloser.java
create mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/replication/TestReplicationActivityStatus.java
create mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/node/TestDeadNodeHandler.java
delete mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/node/TestNodeManager.java
create mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/node/TestSCMNodeManager.java
create mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/server/TestSCMClientProtocolServer.java
create mode 100644 hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/server/TestSCMSecurityProtocolServer.java
create mode 100644 hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/ChillModeCheckSubcommand.java
create mode 100644 hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/ChillModeCommands.java
create mode 100644 hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/ChillModeExitSubcommand.java
create mode 100644 hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/pipeline/ClosePipelineSubcommand.java
create mode 100644 hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/pipeline/ListPipelinesSubcommand.java
create mode 100644 hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/pipeline/package-info.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/ClientGSIContext.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/NoECPolicySetException.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/ObserverReadProxyProvider.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/ObserverReadProxyProviderWithIPFailover.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/ReadOnly.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/ECPolicyParam.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-client/src/test/java/org/apache/hadoop/hdfs/protocol/TestReadOnly.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/IsRouterActiveServlet.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/dev-support/jdiff/Apache_Hadoop_HDFS_2.9.2.xml
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/dev-support/jdiff/Apache_Hadoop_HDFS_3.1.2.xml
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/dev-support/jdiff/Apache_Hadoop_HDFS_3.2.0.xml
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/server/JournaledEditsCache.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/ECTopologyVerifier.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/TokenVerifier.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ECTopologyVerifierResult.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/GlobalStateIdContext.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/IsNameNodeActiveServlet.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/BalancerProtocols.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/PBImageCorruption.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/PBImageCorruptionDetector.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ObserverNameNode.md
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockTokenWrappingQOP.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestErasureCodingAddConfig.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestHAAuxiliaryPort.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestStateAlignmentContextWithHA.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/qjournal/server/JournalTestUtil.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/qjournal/server/TestJournaledEditsCache.java
delete mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/aliasmap/TestInMemoryAliasMap.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestBootstrapAliasmap.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestConsistentReadsObserver.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestMultiObserverNode.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestObserverNode.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestObserverReadProxyProvider.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestECAdmin.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestPBImageCorruption.java
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testMultipleCorruption.csv
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testMultipleFileCorruption.csv
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testSingleFileCorruption.csv
create mode 100644 hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testSingleFolderCorruption.csv
create mode 100644 hadoop-mapreduce-project/dev-support/jdiff/Apache_Hadoop_MapReduce_Core_3.1.2.xml
create mode 100644 hadoop-mapreduce-project/dev-support/jdiff/Apache_Hadoop_MapReduce_JobClient_3.1.2.xml
create mode 100644 hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/MapTaskAttemptInfo.java
create mode 100644 hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapred/TestYarnChild.java
create mode 100644 hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/split/TestJobSplitWriterWithEC.java
create mode 100644 hadoop-ozone/Jenkinsfile
delete mode 100644 hadoop-ozone/acceptance-test/README.md
delete mode 100755 hadoop-ozone/acceptance-test/dev-support/bin/robot-all.sh
delete mode 100755 hadoop-ozone/acceptance-test/dev-support/bin/robot-dnd-all.sh
delete mode 100755 hadoop-ozone/acceptance-test/dev-support/bin/robot.sh
delete mode 100644 hadoop-ozone/acceptance-test/dev-support/docker/Dockerfile
delete mode 100644 hadoop-ozone/acceptance-test/dev-support/docker/docker-compose.yaml
delete mode 100644 hadoop-ozone/acceptance-test/pom.xml
delete mode 100644 hadoop-ozone/acceptance-test/src/test/acceptance/basic/.env
delete mode 100644 hadoop-ozone/acceptance-test/src/test/acceptance/basic/basic.robot
delete mode 100644 hadoop-ozone/acceptance-test/src/test/acceptance/basic/docker-compose.yaml
delete mode 100644 hadoop-ozone/acceptance-test/src/test/acceptance/basic/docker-config
delete mode 100644 hadoop-ozone/acceptance-test/src/test/acceptance/basic/ozone-shell.robot
delete mode 100644 hadoop-ozone/acceptance-test/src/test/acceptance/commonlib.robot
delete mode 100644 hadoop-ozone/acceptance-test/src/test/acceptance/ozonefs/.env
delete mode 100644 hadoop-ozone/acceptance-test/src/test/acceptance/ozonefs/docker-compose.yaml
delete mode 100644 hadoop-ozone/acceptance-test/src/test/acceptance/ozonefs/docker-config
delete mode 100644 hadoop-ozone/acceptance-test/src/test/acceptance/ozonefs/ozonefs.robot
delete mode 100644 hadoop-ozone/acceptance-test/src/test/acceptance/ozonefs/ozonesinglenode.robot
create mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/OzoneMultipartUploadPartListParts.java
create mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/io/BlockOutputStreamEntry.java
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/io/ChunkGroupInputStream.java
delete mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/io/ChunkGroupOutputStream.java
create mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/io/KeyInputStream.java
create mode 100644 hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/io/KeyOutputStream.java
create mode 100755 hadoop-ozone/common/src/main/bin/ozone-config.sh
mode change 100644 => 100755 hadoop-ozone/common/src/main/bin/start-ozone.sh
mode change 100644 => 100755 hadoop-ozone/common/src/main/bin/stop-ozone.sh
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/hdds/protocol/StorageType.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/hdds/protocol/package-info.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/OzoneIllegalArgumentException.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/OMMetadataManager.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/OzoneManagerLock.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/codec/OmBucketInfoCodec.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/codec/OmKeyInfoCodec.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/codec/OmMultipartKeyInfoCodec.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/codec/OmVolumeArgsCodec.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/codec/VolumeListCodec.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/codec/package-info.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/exceptions/OMException.java
rename hadoop-ozone/{ozone-manager => common}/src/main/java/org/apache/hadoop/ozone/om/exceptions/package-info.java (100%)
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/KeyValueUtil.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmMultipartCommitUploadPartInfo.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmMultipartInfo.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmMultipartKeyInfo.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmMultipartUploadCompleteInfo.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmMultipartUploadList.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmMultipartUploadListParts.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmPartInfo.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/S3SecretValue.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/WithMetadata.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/protocol/OzoneManagerSecurityProtocol.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/OzoneBlockTokenSecretManager.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/OzoneDelegationTokenSecretManager.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/OzoneDelegationTokenSelector.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/OzoneSecretKey.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/OzoneSecretManager.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/OzoneSecretStore.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/OzoneSecurityException.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/OzoneTokenIdentifier.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/acl/IAccessAuthorizer.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/acl/IOzoneObj.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/acl/OzoneAccessAuthorizer.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/acl/OzoneAclException.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/acl/OzoneObj.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/acl/OzoneObjInfo.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/acl/RequestContext.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/acl/package-info.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/security/package-info.java
create mode 100644 hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/util/package-info.java
create mode 100644 hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/TestOmUtils.java
create mode 100644 hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/om/codec/TestOmMultipartKeyInfoCodec.java
create mode 100644 hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/om/codec/package-info.java
create mode 100644 hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/om/exceptions/TestResultCodes.java
create mode 100644 hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/om/helpers/TestOmBucketInfo.java
create mode 100644 hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/om/helpers/TestOmKeyInfo.java
create mode 100644 hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/om/helpers/package-info.java
create mode 100644 hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/security/TestOzoneBlockTokenSecretManager.java
create mode 100644 hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/security/TestOzoneDelegationTokenSecretManager.java
create mode 100644 hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/security/acl/TestOzoneObjInfo.java
create mode 100644 hadoop-ozone/datanode/pom.xml
create mode 100755 hadoop-ozone/dev-support/checks/acceptance.sh
create mode 100755 hadoop-ozone/dev-support/checks/author.sh
create mode 100755 hadoop-ozone/dev-support/checks/build.sh
create mode 100755 hadoop-ozone/dev-support/checks/checkstyle.sh
create mode 100755 hadoop-ozone/dev-support/checks/findbugs.sh
create mode 100755 hadoop-ozone/dev-support/checks/isolation.sh
create mode 100755 hadoop-ozone/dev-support/checks/rat.sh
create mode 100755 hadoop-ozone/dev-support/checks/unit.sh
create mode 100644 hadoop-ozone/dev-support/docker/Dockerfile
create mode 100644 hadoop-ozone/dist/Dockerfile
create mode 100644 hadoop-ozone/dist/README.md
create mode 100755 hadoop-ozone/dist/dev-support/bin/dist-layout-stitching
create mode 100755 hadoop-ozone/dist/dev-support/bin/dist-tar-stitching
create mode 100644 hadoop-ozone/dist/pom.xml
create mode 100644 hadoop-ozone/dist/skaffold.yaml
create mode 100644 hadoop-ozone/dist/src/main/blockade/README.md
create mode 100644 hadoop-ozone/dist/src/main/blockade/blockadeUtils/__init__.py
create mode 100644 hadoop-ozone/dist/src/main/blockade/blockadeUtils/blockade.py
copy hadoop-tools/hadoop-resourceestimator/src/main/resources/webapps/ResourceEstimatorServer/.gitignore => hadoop-ozone/dist/src/main/blockade/clusterUtils/__init__.py (100%)
create mode 100644 hadoop-ozone/dist/src/main/blockade/clusterUtils/cluster_utils.py
create mode 100644 hadoop-ozone/dist/src/main/blockade/conftest.py
create mode 100644 hadoop-ozone/dist/src/main/blockade/test_blockade_client_failure.py
create mode 100644 hadoop-ozone/dist/src/main/blockade/test_blockade_datanode_isolation.py
create mode 100644 hadoop-ozone/dist/src/main/blockade/test_blockade_flaky.py
create mode 100644 hadoop-ozone/dist/src/main/blockade/test_blockade_mixed_failure.py
create mode 100644 hadoop-ozone/dist/src/main/blockade/test_blockade_mixed_failure_three_nodes_isolate.py
create mode 100644 hadoop-ozone/dist/src/main/blockade/test_blockade_mixed_failure_two_nodes.py
create mode 100644 hadoop-ozone/dist/src/main/blockade/test_blockade_scm_isolation.py
rename {hadoop-dist => hadoop-ozone/dist}/src/main/compose/README.md (100%)
create mode 100644 hadoop-ozone/dist/src/main/compose/common/grafana/conf/grafana.ini
create mode 100644 hadoop-ozone/dist/src/main/compose/common/grafana/dashboards/Ozone - Object Metrics.json
create mode 100644 hadoop-ozone/dist/src/main/compose/common/grafana/dashboards/Ozone - RPC Metrics.json
create mode 100755 hadoop-ozone/dist/src/main/compose/common/grafana/provisioning/dashboards/dashboards.yml
create mode 100755 hadoop-ozone/dist/src/main/compose/common/grafana/provisioning/datasources/datasources.yml
create mode 100644 hadoop-ozone/dist/src/main/compose/common/prometheus/prometheus.yml
create mode 100644 hadoop-ozone/dist/src/main/compose/ozone-hdfs/.env
create mode 100644 hadoop-ozone/dist/src/main/compose/ozone-hdfs/docker-compose.yaml
create mode 100644 hadoop-ozone/dist/src/main/compose/ozone-hdfs/docker-config
rename {hadoop-dist => hadoop-ozone/dist}/src/main/compose/ozone/.env (100%)
create mode 100644 hadoop-ozone/dist/src/main/compose/ozone/docker-compose.yaml
create mode 100644 hadoop-ozone/dist/src/main/compose/ozone/docker-config
create mode 100644 hadoop-ozone/dist/src/main/compose/ozoneblockade/docker-compose.yaml
create mode 100644 hadoop-ozone/dist/src/main/compose/ozoneblockade/docker-config
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonefs/docker-compose.yaml
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonefs/docker-config
copy {hadoop-dist => hadoop-ozone/dist}/src/main/compose/ozoneperf/.env (100%)
create mode 100644 hadoop-ozone/dist/src/main/compose/ozoneperf/README.md
create mode 100644 hadoop-ozone/dist/src/main/compose/ozoneperf/docker-compose.yaml
create mode 100644 hadoop-ozone/dist/src/main/compose/ozoneperf/docker-config
create mode 100644 hadoop-ozone/dist/src/main/compose/ozones3/docker-compose.yaml
create mode 100644 hadoop-ozone/dist/src/main/compose/ozones3/docker-config
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonescripts/.ssh/authorized_keys
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonescripts/.ssh/config
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonescripts/.ssh/environment
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonescripts/.ssh/id_rsa
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonescripts/.ssh/id_rsa.pub
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonescripts/Dockerfile
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonescripts/README.md
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonescripts/docker-compose.yaml
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonescripts/docker-config
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonescripts/ps.sh
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonescripts/start.sh
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonescripts/stop.sh
rename {hadoop-dist/src/main/compose/ozoneperf => hadoop-ozone/dist/src/main/compose/ozonesecure}/.env (100%)
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonesecure/README.md
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonesecure/docker-compose.yaml
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonesecure/docker-config
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonesecure/docker-image/docker-krb5/Dockerfile-krb5
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonesecure/docker-image/docker-krb5/README.md
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonesecure/docker-image/docker-krb5/kadm5.acl
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonesecure/docker-image/docker-krb5/krb5.conf
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonesecure/docker-image/docker-krb5/launcher.sh
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonesecure/docker-image/runner/Dockerfile
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure/docker-image/runner/build.sh
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure/docker-image/runner/scripts/envtoconf.py
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonesecure/docker-image/runner/scripts/krb5.conf
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure/docker-image/runner/scripts/starter.sh
create mode 100755 hadoop-ozone/dist/src/main/compose/ozonesecure/docker-image/runner/scripts/transformation.py
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonetrace/docker-compose.yaml
create mode 100644 hadoop-ozone/dist/src/main/compose/ozonetrace/docker-config
create mode 100644 hadoop-ozone/dist/src/main/conf/dn-audit-log4j2.properties
create mode 100644 hadoop-ozone/dist/src/main/conf/log4j.properties
create mode 100644 hadoop-ozone/dist/src/main/conf/om-audit-log4j2.properties
rename hadoop-ozone/{common => dist}/src/main/conf/ozone-site.xml (100%)
create mode 100644 hadoop-ozone/dist/src/main/conf/scm-audit-log4j2.properties
create mode 100644 hadoop-ozone/dist/src/main/k8s/ozone/config-configmap.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/ozone/datanode-public-service.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/ozone/datanode-service.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/ozone/datanode-statefulset.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/ozone/om-public-service.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/ozone/om-service.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/ozone/om-statefulset.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/ozone/s3g-public-service.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/ozone/s3g-service.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/ozone/s3g-statefulset.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/ozone/scm-public-service.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/ozone/scm-service.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/ozone/scm-statefulset.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/prometheus/configmap.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/prometheus/deployment.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/prometheus/prometheus-public-serivce.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/prometheus/role.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/prometheus/rolebindng.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/prometheus/scm-public-service.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/prometheus/service-account.yaml
create mode 100644 hadoop-ozone/dist/src/main/k8s/prometheus/service.yaml
create mode 100644 hadoop-ozone/dist/src/main/ozone/README.txt
create mode 100644 hadoop-ozone/dist/src/main/smoketest/.env
create mode 100644 hadoop-ozone/dist/src/main/smoketest/README.md
create mode 100644 hadoop-ozone/dist/src/main/smoketest/auditparser/auditparser.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/basic/basic.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/basic/ozone-shell.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/commonlib.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/ozonefs/ozonefs.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/s3/MultipartUpload.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/s3/README.md
create mode 100644 hadoop-ozone/dist/src/main/smoketest/s3/__init__.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/s3/awss3.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/s3/buckethead.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/s3/bucketlist.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/s3/commonawslib.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/s3/objectcopy.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/s3/objectdelete.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/s3/objectmultidelete.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/s3/objectputget.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/s3/webui.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/security/ozone-secure.robot
create mode 100755 hadoop-ozone/dist/src/main/smoketest/test.sh
delete mode 100644 hadoop-ozone/docs/README.md
delete mode 100644 hadoop-ozone/docs/archetypes/default.md
delete mode 100644 hadoop-ozone/docs/config.toml
delete mode 100644 hadoop-ozone/docs/content/CommandShell.md
delete mode 100644 hadoop-ozone/docs/content/GettingStarted.md
delete mode 100644 hadoop-ozone/docs/content/Metrics.md
delete mode 100644 hadoop-ozone/docs/content/Rest.md
delete mode 100644 hadoop-ozone/docs/content/_index.md
delete mode 100755 hadoop-ozone/docs/dev-support/bin/generate-site.sh
delete mode 100644 hadoop-ozone/docs/pom.xml
delete mode 100644 hadoop-ozone/docs/themes/ozonedoc/layouts/_default/single.html
delete mode 100644 hadoop-ozone/docs/themes/ozonedoc/layouts/index.html
delete mode 100644 hadoop-ozone/docs/themes/ozonedoc/layouts/partials/footer.html
delete mode 100644 hadoop-ozone/docs/themes/ozonedoc/layouts/partials/header.html
delete mode 100644 hadoop-ozone/docs/themes/ozonedoc/layouts/partials/navbar.html
delete mode 100644 hadoop-ozone/docs/themes/ozonedoc/layouts/partials/sidebar.html
delete mode 100644 hadoop-ozone/docs/themes/ozonedoc/static/css/ozonedoc.css
delete mode 100644 hadoop-ozone/docs/themes/ozonedoc/static/fonts/glyphicons-halflings-regular.svg
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/container/TestContainerStateManager.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/container/TestContainerStateManagerIntegration.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/pipeline/TestNodeFailure.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/pipeline/TestPipelineStateManager.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/pipeline/TestRatisPipelineProvider.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/pipeline/TestRatisPipelineUtils.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/pipeline/TestSCMPipelineManager.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/pipeline/TestSCMRestart.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/pipeline/TestSimplePipelineProvider.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/pipeline/package-info.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/MiniOzoneHAClusterImpl.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/TestContainerStateMachineIdempotency.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/TestSecureOzoneCluster.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/CertificateClientTestImpl.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/package-info.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rest/TestOzoneRestClient.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rest/package-info.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestBCSID.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestContainerStateMachine.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestContainerStateMachineFailures.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestFailureHandlingByClient.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestHybridPipelineOnDatanode.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestOzoneRpcClientAbstract.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestOzoneRpcClientWithRatis.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestReadRetries.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestSecureOzoneRpcClient.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/TestContainerReplication.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/common/helpers/TestBlockData.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/common/helpers/TestKeyData.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/common/impl/TestCloseContainerHandler.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/common/statemachine/commandhandler/TestDeleteContainerHandler.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/common/statemachine/commandhandler/TestReplicateContainerHandler.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/common/statemachine/commandhandler/package-info.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/ozoneimpl/TestOzoneContainerWithTLS.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/ozoneimpl/TestSecureOzoneContainer.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/server/TestContainerStateMachine.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/server/TestSecureContainerServer.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/freon/TestDataValidate.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/freon/TestFreon.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/genconf/TestGenerateOzoneRequiredConfigurations.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOmAcls.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOmInit.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOmSQLCli.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOzoneManagerConfiguration.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOzoneManagerHA.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestScmChillMode.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/ozShell/TestOzoneDatanodeShell.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/package-info.java
delete mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/scm/TestContainerSQLCli.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/scm/TestSCMNodeManagerMXBean.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/web/client/package-info.java
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/web/package-info.java
create mode 100644 hadoop-ozone/integration-test/src/test/resources/ssl/ca.crt
create mode 100644 hadoop-ozone/integration-test/src/test/resources/ssl/ca.key
create mode 100644 hadoop-ozone/integration-test/src/test/resources/ssl/client.crt
create mode 100644 hadoop-ozone/integration-test/src/test/resources/ssl/client.csr
create mode 100644 hadoop-ozone/integration-test/src/test/resources/ssl/client.key
create mode 100644 hadoop-ozone/integration-test/src/test/resources/ssl/client.pem
create mode 100755 hadoop-ozone/integration-test/src/test/resources/ssl/generate.sh
create mode 100644 hadoop-ozone/integration-test/src/test/resources/ssl/server.crt
create mode 100644 hadoop-ozone/integration-test/src/test/resources/ssl/server.csr
create mode 100644 hadoop-ozone/integration-test/src/test/resources/ssl/server.key
create mode 100644 hadoop-ozone/integration-test/src/test/resources/ssl/server.pem
delete mode 100644 hadoop-ozone/objectstore-service/src/test/java/org/apache/hadoop/ozone/TestErrorCode.java
create mode 100644 hadoop-ozone/objectstore-service/src/test/java/org/apache/hadoop/ozone/web/TestErrorCode.java
create mode 100644 hadoop-ozone/objectstore-service/src/test/java/org/apache/hadoop/ozone/web/package-info.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OMMetadataManager.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OMNodeDetails.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmMetricsInfo.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/S3BucketManager.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/S3BucketManagerImpl.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/S3SecretManager.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/S3SecretManagerImpl.java
delete mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/exceptions/OMException.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/OMRatisHelper.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/OzoneManagerRatisClient.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/OzoneManagerRatisServer.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/OzoneManagerStateMachine.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/package-info.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/protocolPB/OzoneManagerRequestHandler.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/OzoneAddress.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/bucket/BucketCommands.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/bucket/S3BucketMapping.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/keys/KeyCommands.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/keys/RenameKeyHandler.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/s3/GetS3SecretHandler.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/s3/S3Commands.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/s3/package-info.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/token/CancelTokenHandler.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/token/GetTokenHandler.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/token/PrintTokenHandler.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/token/RenewTokenHandler.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/token/TokenCommands.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/token/package-info.java
create mode 100644 hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/web/ozShell/volume/VolumeCommands.java
rename hadoop-ozone/ozone-manager/src/main/{ => resources}/webapps/ozoneManager/index.html (100%)
rename hadoop-ozone/ozone-manager/src/main/{ => resources}/webapps/ozoneManager/main.css (100%)
rename hadoop-ozone/ozone-manager/src/main/{ => resources}/webapps/ozoneManager/main.html (100%)
rename hadoop-ozone/ozone-manager/src/main/{ => resources}/webapps/ozoneManager/om-metrics.html (100%)
rename hadoop-ozone/ozone-manager/src/main/{ => resources}/webapps/ozoneManager/ozoneManager.js (100%)
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/ScmBlockLocationTestIngClient.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/TestKeyDeletingService.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/TestKeyManagerImpl.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/TestOzoneManagerLock.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/TestS3BucketManager.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/ratis/TestOzoneManagerRatisServer.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/security/TestOzoneManagerBlockToken.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/security/TestOzoneTokenIdentifier.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/security/package-info.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/web/ozShell/TestOzoneAddress.java
create mode 100644 hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/web/ozShell/package-info.java
create mode 100644 hadoop-ozone/ozonefs-lib-legacy/pom.xml
create mode 100644 hadoop-ozone/ozonefs-lib-legacy/src/main/resources/ozonefs.txt
create mode 100644 hadoop-ozone/ozonefs-lib/pom.xml
create mode 100644 hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/BasicKeyInfo.java
create mode 100644 hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/FilteredClassLoader.java
create mode 100644 hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/OzoneClientAdapter.java
create mode 100644 hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/OzoneClientAdapterFactory.java
create mode 100644 hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/OzoneClientAdapterImpl.java
create mode 100644 hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/OzoneFSStorageStatistics.java
create mode 100644 hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/Statistic.java
create mode 100644 hadoop-ozone/ozonefs/src/main/resources/META-INF/services/org.apache.hadoop.fs.FileSystem
create mode 100644 hadoop-ozone/ozonefs/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFileSystem.java
create mode 100644 hadoop-ozone/ozonefs/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFsRenameDir.java
create mode 100644 hadoop-ozone/ozonefs/src/test/java/org/apache/hadoop/fs/ozone/package-info.java
create mode 100644 hadoop-ozone/s3gateway/pom.xml
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/CommonHeadersContainerResponseFilter.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/Gateway.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/GatewayApplication.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/HeaderPreprocessor.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/OzoneClientProducer.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/OzoneConfigurationHolder.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/RequestIdentifier.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/S3GatewayConfigKeys.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/S3GatewayHttpServer.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/SignedChunksInputStream.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/VirtualHostStyleFilter.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/commontypes/BucketMetadata.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/commontypes/CommonPrefix.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/commontypes/IsoDateAdapter.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/commontypes/KeyMetadata.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/commontypes/package-info.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/BucketEndpoint.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/CompleteMultipartUploadRequest.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/CompleteMultipartUploadResponse.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/CopyObjectResponse.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/EndpointBase.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/ListBucketResponse.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/ListObjectResponse.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/ListPartsResponse.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/MultiDeleteRequest.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/MultiDeleteRequestUnmarshaller.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/MultiDeleteResponse.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/MultipartUploadInitiateResponse.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/ObjectEndpoint.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/RootEndpoint.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/XmlNamespaceFilter.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/package-info.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/exception/OS3Exception.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/exception/OS3ExceptionMapper.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/exception/S3ErrorTable.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/exception/package-info.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/header/AuthenticationHeaderParser.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/header/AuthorizationHeaderV2.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/header/AuthorizationHeaderV4.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/header/Credential.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/header/package-info.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/io/S3WrapperInputStream.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/io/package-info.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/package-info.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/util/ContinueToken.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/util/RFC1123Util.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/util/RangeHeader.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/util/RangeHeaderParserUtil.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/util/S3Consts.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/util/S3StorageType.java
create mode 100644 hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/util/package-info.java
create mode 100644 hadoop-ozone/s3gateway/src/main/resources/META-INF/beans.xml
create mode 100644 hadoop-ozone/s3gateway/src/main/resources/browser.html
create mode 100644 hadoop-ozone/s3gateway/src/main/resources/webapps/s3gateway/WEB-INF/beans.xml
create mode 100644 hadoop-ozone/s3gateway/src/main/resources/webapps/s3gateway/WEB-INF/web.xml
create mode 100755 hadoop-ozone/s3gateway/src/main/resources/webapps/static/images/ozone.ico
create mode 100644 hadoop-ozone/s3gateway/src/main/resources/webapps/static/index.html
create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/client/ObjectStoreStub.java
create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/client/OzoneBucketStub.java
create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/client/OzoneClientStub.java
create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/client/OzoneOutputStreamStub.java
create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/client/OzoneVolumeStub.java
create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/client/package-info.java
create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/TestSignedChunksInputStream.java
create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/TestVirtualHostStyleFilter.java
create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestAbortMultipartUpload.java
create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestBucketDelete.java
create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestBucketGet.java
create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestBucketHead.java
create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestBucketResponse.java
create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestInitiateMultipartUpload.java
create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestListParts.java
create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestMultiDeleteRequestUnmarshaller.java
create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestMultipartUploadComplete.java
create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestObjectDelete.java
create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestObjectGet.java
create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestObjectHead.java
create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestObjectMultiDelete.java
create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestObjectPut.java
create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestPartUpload.java
create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestRootList.java
create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/package-info.java
create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/exception/TestOS3Exception.java
create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/exception/package-info.java
create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/header/TestAuthorizationHeaderV2.java
create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/header/TestAuthorizationHeaderV4.java
create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/package-info.java
create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/util/TestContinueToken.java
create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/util/TestRFC1123Util.java
create mode 100644 hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/util/TestRangeHeaderParserUtil.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/audit/parser/AuditParser.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/audit/parser/common/DatabaseHelper.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/audit/parser/common/ParserConsts.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/audit/parser/common/package-info.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/audit/parser/handler/LoadCommandHandler.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/audit/parser/handler/QueryCommandHandler.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/audit/parser/handler/TemplateCommandHandler.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/audit/parser/handler/package-info.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/audit/parser/model/AuditEntry.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/audit/parser/model/package-info.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/audit/parser/package-info.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/freon/ProgressBar.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/freon/RandomKeyGenerator.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/fsck/BlockIdDetails.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/fsck/ContainerMapper.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/fsck/package-info.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkBlockManager.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkOMKeyAllocation.java
create mode 100644 hadoop-ozone/tools/src/main/resources/commands.properties
create mode 100644 hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/audit/parser/TestAuditParser.java
create mode 100644 hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/audit/parser/package-info.java
create mode 100644 hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/freon/TestDataValidate.java
create mode 100644 hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/freon/TestFreonWithDatanodeFastRestart.java
create mode 100644 hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/freon/TestFreonWithDatanodeRestart.java
create mode 100644 hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/freon/TestFreonWithPipelineDestroy.java
create mode 100644 hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/freon/TestProgressBar.java
create mode 100644 hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/freon/TestRandomKeyGenerator.java
create mode 100644 hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/freon/package-info.java
create mode 100644 hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/fsck/TestContainerMapper.java
create mode 100644 hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/fsck/package-info.java
create mode 100644 hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/genconf/TestGenerateOzoneRequiredConfigurations.java
create mode 100644 hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/genconf/package-info.java
create mode 100644 hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/om/TestOmSQLCli.java
create mode 100644 hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/om/package-info.java
create mode 100644 hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/scm/TestContainerSQLCli.java
create mode 100644 hadoop-ozone/tools/src/test/java/org/apache/hadoop/ozone/scm/package-info.java
create mode 100644 hadoop-ozone/tools/src/test/resources/commands.properties
create mode 100644 hadoop-ozone/tools/src/test/resources/testaudit.log
create mode 100644 hadoop-tools/hadoop-aliyun/src/main/java/org/apache/hadoop/fs/aliyun/oss/OSS.java
create mode 100644 hadoop-tools/hadoop-aliyun/src/main/java/org/apache/hadoop/fs/aliyun/oss/OSSFileStatus.java
create mode 100644 hadoop-tools/hadoop-aliyun/src/test/java/org/apache/hadoop/fs/aliyun/oss/fileContext/TestOSSFileContext.java
create mode 100644 hadoop-tools/hadoop-aliyun/src/test/java/org/apache/hadoop/fs/aliyun/oss/fileContext/TestOSSFileContextCreateMkdir.java
create mode 100644 hadoop-tools/hadoop-aliyun/src/test/java/org/apache/hadoop/fs/aliyun/oss/fileContext/TestOSSFileContextMainOperations.java
create mode 100644 hadoop-tools/hadoop-aliyun/src/test/java/org/apache/hadoop/fs/aliyun/oss/fileContext/TestOSSFileContextStatistics.java
create mode 100644 hadoop-tools/hadoop-aliyun/src/test/java/org/apache/hadoop/fs/aliyun/oss/fileContext/TestOSSFileContextURI.java
create mode 100644 hadoop-tools/hadoop-aliyun/src/test/java/org/apache/hadoop/fs/aliyun/oss/fileContext/TestOSSFileContextUtil.java
create mode 100644 hadoop-tools/hadoop-aliyun/src/test/java/org/apache/hadoop/fs/aliyun/oss/yarn/TestOSS.java
create mode 100644 hadoop-tools/hadoop-aliyun/src/test/java/org/apache/hadoop/fs/aliyun/oss/yarn/TestOSSMiniYarnCluster.java
delete mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/BasicAWSCredentialsProvider.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/InternalConstants.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/AbstractAWSCredentialProvider.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/AbstractSessionCredentialsProvider.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/IAMInstanceCredentialsProvider.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/MarshalledCredentialBinding.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/MarshalledCredentialProvider.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/MarshalledCredentials.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/NoAwsCredentialsException.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/delegation/AWSPolicyProvider.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/delegation/AbstractDTService.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/delegation/AbstractDelegationTokenBinding.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/delegation/AbstractS3ATokenIdentifier.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/delegation/DelegationConstants.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/delegation/DelegationTokenIOException.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/delegation/EncryptionSecretOperations.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/delegation/EncryptionSecrets.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/delegation/FullCredentialsTokenBinding.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/delegation/FullCredentialsTokenIdentifier.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/delegation/RoleTokenBinding.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/delegation/RoleTokenIdentifier.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/delegation/S3ADelegationTokens.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/delegation/S3ADtFetcher.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/delegation/SessionTokenBinding.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/delegation/SessionTokenIdentifier.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/delegation/package-info.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/ExpirableMetadata.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/S3GuardDataAccessRetryPolicy.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/select/InternalSelectConstants.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/select/SelectBinding.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/select/SelectConstants.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/select/SelectInputStream.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/select/SelectTool.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/select/package-info.java
create mode 100644 hadoop-tools/hadoop-aws/src/main/resources/META-INF/services/org.apache.hadoop.security.token.DtFetcher
create mode 100644 hadoop-tools/hadoop-aws/src/main/resources/META-INF/services/org.apache.hadoop.security.token.TokenIdentifier
create mode 100644 hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/delegation_token_architecture.md
create mode 100644 hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/delegation_tokens.md
create mode 100644 hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/s3_select.md
delete mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3ACredentialsInURL.java
delete mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AEncryptionSSECBlockOutputStream.java
delete mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AEncryptionSSEKMSUserDefinedKeyBlockOutputStream.java
delete mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AEncryptionSSES3BlockOutputStream.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3GuardTtl.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/auth/TestMarshalledCredentials.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/auth/delegation/AbstractDelegationIT.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/auth/delegation/CountInvocationsProvider.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/auth/delegation/Csvout.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/auth/delegation/ILoadTestRoleCredentials.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/auth/delegation/ILoadTestSessionCredentials.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/auth/delegation/ITestDelegatedMRJob.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/auth/delegation/ITestRoleDelegationInFileystem.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/auth/delegation/ITestRoleDelegationTokens.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/auth/delegation/ITestSessionDelegationInFileystem.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/auth/delegation/ITestSessionDelegationTokens.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/auth/delegation/MiniKerberizedHadoopCluster.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/auth/delegation/TestS3ADelegationTokenSupport.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/s3guard/S3GuardToolTestHelper.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/s3guard/TestDynamoDBMiscOperations.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/scale/NanoTimerStats.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/select/AbstractS3SelectTest.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/select/CsvFile.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/select/ITestS3Select.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/select/ITestS3SelectCLI.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/select/ITestS3SelectLandsat.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/select/ITestS3SelectMRJob.java
create mode 100644 hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/mapreduce/MockJob.java
create mode 100644 hadoop-tools/hadoop-azure/src/config/checkstyle-suppressions.xml
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/Abfs.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AbfsConfiguration.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/Abfss.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystem.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/SecureAzureBlobFileSystem.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/AbfsHttpConstants.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/ConfigurationKeys.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/FileSystemConfigurations.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/FileSystemUriSchemes.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/HttpHeaderConfigurations.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/HttpQueryParams.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/annotations/ConfigurationValidationAnnotations.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/annotations/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/diagnostics/ConfigurationValidator.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/diagnostics/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/AbfsRestOperationException.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/AzureBlobFileSystemException.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/ConfigurationPropertyNotFoundException.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/FileSystemOperationUnhandledException.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/InvalidAbfsRestOperationException.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/InvalidAclOperationException.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/InvalidConfigurationValueException.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/InvalidFileSystemPropertyException.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/InvalidUriAuthorityException.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/InvalidUriException.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/KeyProviderException.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/TimeoutException.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/TokenAccessProviderException.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/exceptions/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/services/AzureServiceErrorCode.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/services/ListResultEntrySchema.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/services/ListResultSchema.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/services/ReadBufferStatus.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/contracts/services/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/diagnostics/Base64StringConfigurationBasicValidator.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/diagnostics/BooleanConfigurationBasicValidator.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/diagnostics/ConfigurationBasicValidator.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/diagnostics/IntegerConfigurationBasicValidator.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/diagnostics/LongConfigurationBasicValidator.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/diagnostics/StringConfigurationBasicValidator.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/diagnostics/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/extensions/AbfsAuthorizationException.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/extensions/AbfsAuthorizer.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/extensions/CustomDelegationTokenManager.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/extensions/CustomTokenProviderAdaptee.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/extensions/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/AccessTokenProvider.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/AzureADAuthenticator.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/AzureADToken.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/ClientCredsTokenProvider.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/CustomTokenProviderAdapter.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/IdentityTransformer.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/MsiTokenProvider.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/QueryParams.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/RefreshTokenBasedTokenProvider.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/UserPasswordTokenProvider.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/package.html
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/security/AbfsDelegationTokenIdentifier.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/security/AbfsDelegationTokenManager.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/security/AbfsTokenRenewer.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/security/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsAclHelper.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClientThrottlingAnalyzer.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClientThrottlingIntercept.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsHttpHeader.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsHttpOperation.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsInputStream.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsOutputStream.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsPermission.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsRestOperation.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsRestOperationType.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsUriQueryBuilder.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AuthType.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ExponentialRetryPolicy.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/KeyProvider.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ReadBuffer.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ReadBufferManager.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ReadBufferWorker.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/SharedKeyCredentials.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ShellDecryptionKeyProvider.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/SimpleKeyProvider.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/utils/Base64.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/utils/SSLSocketFactoryEx.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/utils/UriUtils.java
create mode 100644 hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/utils/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/site/markdown/abfs.md
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/ITestNativeFileSystemStatistics.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/TestKeyPageBlobDirectories.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/AbstractAbfsIntegrationTest.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/AbstractAbfsScaleTest.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/AbstractAbfsTestWithTimeout.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAbfsClient.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAbfsIdentityTransformer.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAbfsReadWriteAndSeek.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAbfsRestOperationException.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemAppend.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemAuthorization.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemBackCompat.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemCopy.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemCreate.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemDelete.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemE2E.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemE2EScale.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemFileStatus.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemFinalize.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemFlush.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemInitAndCreate.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemListStatus.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemMkDir.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemOauth.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemPermission.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemRandomRead.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemRename.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemRenameUnicode.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFilesystemAcl.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestClientUrlScheme.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestFileSystemInitialization.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestFileSystemProperties.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestFileSystemRegistration.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestGetNameSpaceEnabled.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestOauthOverAbfsScheme.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestWasbAbfsCompatibility.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/TestAbfsConfigurationFieldsValidation.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/TestAccountConfiguration.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/constants/TestConfigurationKeys.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/constants/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ABFSContractTestBinding.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/AbfsFileSystemContract.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractAppend.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractConcat.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractCreate.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractDelete.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractDistCp.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractGetFileStatus.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractMkdir.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractOpen.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractRename.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractRootDirectory.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractSecureDistCp.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractSeek.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAbfsFileSystemContractSetTimes.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/ITestAzureBlobFileSystemBasics.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/contract/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/diagnostics/TestConfigurationValidators.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/diagnostics/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/extensions/MockAbfsAuthorizer.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/extensions/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestAbfsClient.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestAbfsClientThrottlingAnalyzer.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestQueryParams.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestShellDecryptionKeyProvider.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/utils/AbfsTestUtils.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/utils/AclTestHelpers.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/utils/Parallelized.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/utils/TestUriUtils.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/utils/package-info.java
create mode 100644 hadoop-tools/hadoop-azure/src/test/resources/abfs.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/dev-support/jdiff/Apache_Hadoop_YARN_Client_3.1.2.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/dev-support/jdiff/Apache_Hadoop_YARN_Common_3.1.2.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/CsiAdaptorPB.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/CsiAdaptorPlugin.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/CsiAdaptorProtocol.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetAttributesToNodesRequest.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetAttributesToNodesResponse.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetClusterNodeAttributesRequest.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetClusterNodeAttributesResponse.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetLocalizationStatusesRequest.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetLocalizationStatusesResponse.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetNodesToAttributesRequest.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetNodesToAttributesResponse.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetPluginInfoRequest.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetPluginInfoResponse.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/NodePublishVolumeRequest.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/NodePublishVolumeResponse.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/NodeUnpublishVolumeRequest.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/NodeUnpublishVolumeResponse.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/ValidateVolumeCapabilitiesRequest.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/ValidateVolumeCapabilitiesResponse.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/LocalizationState.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/LocalizationStatus.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/NodeAttribute.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/NodeAttributeInfo.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/NodeAttributeKey.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/NodeAttributeOpCode.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/NodeAttributeType.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/NodeToAttributeValue.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ShellContainerCommand.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/AttributeMappingOperationType.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/NodeToAttributes.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/NodesToAttributesMappingRequest.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/NodesToAttributesMappingResponse.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/util/csi/CsiConfigUtils.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/util/csi/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/YarnCsiAdaptor.proto
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_csi_adaptor.proto
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDSWithMultipleNodeManager.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShellWithNodeLabels.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/resources/a.txt
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/resources/b.txt
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core/src/main/java/org/apache/hadoop/yarn/service/api/records/ComponentContainers.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core/src/main/java/org/apache/hadoop/yarn/service/api/records/LocalizationStatus.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core/src/main/java/org/apache/hadoop/yarn/service/utils/HttpUtil.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core/src/test/java/org/apache/hadoop/yarn/service/component/TestComponentDecommissionInstances.java
copy hadoop-yarn-project/hadoop-yarn/{hadoop-yarn-server/hadoop-yarn-server-timelineservice => hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core}/src/test/resources/log4j.properties (100%)
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core/src/test/resources/org/apache/hadoop/yarn/service/conf/examples/external3.json
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/docker/base/ubuntu-16.04/Dockerfile.cpu.tf_1.8.0
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/docker/base/ubuntu-16.04/Dockerfile.gpu.tf_1.8.0
create mode 100755 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/docker/build-all.sh
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/docker/with-cifar10-models/ubuntu-16.04/Dockerfile.cpu.tf_1.8.0
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/docker/with-cifar10-models/ubuntu-16.04/Dockerfile.gpu.tf_1.8.0
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/docker/with-cifar10-models/ubuntu-16.04/cifar10_estimator_tf_1.8.0/README.md
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/docker/with-cifar10-models/ubuntu-16.04/cifar10_estimator_tf_1.8.0/cifar10.py
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/docker/with-cifar10-models/ubuntu-16.04/cifar10_estimator_tf_1.8.0/cifar10_main.py
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/docker/with-cifar10-models/ubuntu-16.04/cifar10_estimator_tf_1.8.0/cifar10_model.py
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/docker/with-cifar10-models/ubuntu-16.04/cifar10_estimator_tf_1.8.0/cifar10_utils.py
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/docker/with-cifar10-models/ubuntu-16.04/cifar10_estimator_tf_1.8.0/generate_cifar10_tfrecords.py
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/docker/with-cifar10-models/ubuntu-16.04/cifar10_estimator_tf_1.8.0/model_base.py
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/docker/zeppelin-notebook-example/Dockerfile.gpu
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/docker/zeppelin-notebook-example/run_container.sh
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/docker/zeppelin-notebook-example/shiro.ini
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/docker/zeppelin-notebook-example/zeppelin-site.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/param/Localization.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/param/Quicklink.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/main/java/org/apache/hadoop/yarn/submarine/client/cli/param/package-info.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/site/DeveloperGuide.md
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/site/QuickStart.md
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/site/markdown/DeveloperGuide.md
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/site/markdown/Examples.md
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/site/markdown/HowToInstall.md
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/site/markdown/Index.md
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/site/markdown/InstallationGuide.md
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/site/markdown/InstallationGuideChineseVersion.md
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/site/markdown/QuickStart.md
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/site/markdown/RunningDistributedCifar10TFJobs.md
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/site/markdown/RunningZeppelinOnYARN.md
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/site/markdown/TestAndTroubleshooting.md
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/site/markdown/WriteDockerfile.md
copy {hadoop-tools/hadoop-resourceestimator => hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine}/src/site/resources/css/site.css (100%)
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/site/resources/images/job-logs-ui.png
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/site/resources/images/multiple-tensorboard-jobs.png
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/site/resources/images/submarine-installer.gif
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-submarine/src/site/resources/images/tensorboard-service.png
copy hadoop-yarn-project/hadoop-yarn/{hadoop-yarn-site => hadoop-yarn-applications/hadoop-yarn-submarine}/src/site/site.xml (100%)
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/ContainerShellWebSocket.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/impl/AHSv2ClientImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/NodeAttributesCLI.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestAHSv2ClientImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestNodeAttributesCLI.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/CsiAdaptorProtocolPBClientImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/CsiAdaptorProtocolPBServiceImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetAttributesToNodesRequestPBImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetAttributesToNodesResponsePBImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetClusterNodeAttributesRequestPBImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetClusterNodeAttributesResponsePBImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetLocalizationStatusesRequestPBImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetLocalizationStatusesResponsePBImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetNodesToAttributesRequestPBImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetNodesToAttributesResponsePBImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetPluginInfoRequestPBImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetPluginInfoResponsePBImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/NodePublishVolumeRequestPBImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/NodePublishVolumeResponsePBImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/NodeUnpublishVolumeRequestPBImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/NodeUnpublishVolumeResponsePBImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/ValidateVolumeCapabilitiesRequestPBImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/ValidateVolumeCapabilitiesResponsePBImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/LocalizationStatusPBImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/NodeAttributeInfoPBImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/NodeAttributeKeyPBImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/NodeAttributePBImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/NodeToAttributeValuePBImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/client/api/TimelineReaderClient.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/client/api/impl/TimelineReaderClientImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/AbstractLabel.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/AttributeExpressionOperation.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/AttributeValue.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/NodeAttributeStore.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/NodeAttributesManager.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/NodeLabelUtil.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/RMNodeAttribute.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/StringAttributeValue.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/store/AbstractFSNodeStore.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/store/FSStoreOpHandler.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/store/StoreOp.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/store/op/AddClusterLabelOp.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/store/op/AddNodeToAttributeLogOp.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/store/op/FSNodeStoreLogOp.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/store/op/NodeAttributeMirrorOp.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/store/op/NodeLabelMirrorOp.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/store/op/NodeToLabelOp.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/store/op/RemoveClusterLabelOp.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/store/op/RemoveNodeToAttributeLogOp.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/store/op/ReplaceNodeToAttributeLogOp.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/store/op/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/store/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/NodeToAttributesPBImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/NodesToAttributesMappingRequestPBImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/NodesToAttributesMappingResponsePBImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/server/metrics/AppAttemptMetricsConstants.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/server/metrics/ApplicationMetricsConstants.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/server/metrics/ContainerMetricsConstants.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/server/metrics/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/timeline/TimelineEntityV2Converter.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/dt-1.10.18/css/custom_datatable.css
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/dt-1.10.18/css/demo_page.css
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/dt-1.10.18/css/demo_table.css
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/dt-1.10.18/css/jquery.dataTables.css
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/dt-1.10.18/css/jui-dt.css
rename hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/{dt-1.10.7 => dt-1.10.18}/images/Sorting icons.psd (100%)
rename hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/{dt-1.10.7 => dt-1.10.18}/images/back_disabled.jpg (100%)
rename hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/{dt-1.10.7 => dt-1.10.18}/images/back_enabled.jpg (100%)
rename hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/{dt-1.10.7 => dt-1.10.18}/images/favicon.ico (100%)
rename hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/{dt-1.10.7 => dt-1.10.18}/images/forward_disabled.jpg (100%)
rename hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/{dt-1.10.7 => dt-1.10.18}/images/forward_enabled.jpg (100%)
rename hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/{dt-1.10.7 => dt-1.10.18}/images/sort_asc.png (100%)
rename hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/{dt-1.10.7 => dt-1.10.18}/images/sort_asc_disabled.png (100%)
rename hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/{dt-1.10.7 => dt-1.10.18}/images/sort_both.png (100%)
rename hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/{dt-1.10.7 => dt-1.10.18}/images/sort_desc.png (100%)
rename hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/{dt-1.10.7 => dt-1.10.18}/images/sort_desc_disabled.png (100%)
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/dt-1.10.18/js/jquery.dataTables.min.js
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/dt-1.10.7/css/demo_page.css
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/dt-1.10.7/css/demo_table.css
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/dt-1.10.7/css/jui-dt.css
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/dt-1.10.7/js/jquery.dataTables.min.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestTimelineReaderClientImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/nodelabels/TestNodeLabelUtil.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/resource/CustomResourceTypesConfigurationProvider.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/resources/resource-types/resource-types-5.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/resources/resource-types/resource-types-6.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/dev-support/findbugs-exclude.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/pom.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/main/java/org/apache/hadoop/yarn/csi/adaptor/CsiAdaptorFactory.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/main/java/org/apache/hadoop/yarn/csi/adaptor/CsiAdaptorProtocolService.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/main/java/org/apache/hadoop/yarn/csi/adaptor/CsiAdaptorServices.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/main/java/org/apache/hadoop/yarn/csi/adaptor/DefaultCsiAdaptorImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/main/java/org/apache/hadoop/yarn/csi/adaptor/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/main/java/org/apache/hadoop/yarn/csi/client/CsiClient.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/main/java/org/apache/hadoop/yarn/csi/client/CsiClientImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/main/java/org/apache/hadoop/yarn/csi/client/CsiGrpcClient.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/main/java/org/apache/hadoop/yarn/csi/client/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/main/java/org/apache/hadoop/yarn/csi/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/main/java/org/apache/hadoop/yarn/csi/translator/GetPluginInfoResponseProtoTranslator.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/main/java/org/apache/hadoop/yarn/csi/translator/NodePublishVolumeRequestProtoTranslator.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/main/java/org/apache/hadoop/yarn/csi/translator/NodeUnpublishVolumeRequestProtoTranslator.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/main/java/org/apache/hadoop/yarn/csi/translator/ProtoTranslator.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/main/java/org/apache/hadoop/yarn/csi/translator/ProtoTranslatorFactory.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/main/java/org/apache/hadoop/yarn/csi/translator/ValidateVolumeCapabilitiesRequestProtoTranslator.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/main/java/org/apache/hadoop/yarn/csi/translator/ValidationVolumeCapabilitiesResponseProtoTranslator.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/main/java/org/apache/hadoop/yarn/csi/translator/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/main/java/org/apache/hadoop/yarn/csi/utils/GrpcHelper.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/main/java/org/apache/hadoop/yarn/csi/utils/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/main/proto/csi.proto
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/test/java/org/apache/hadoop/yarn/csi/adaptor/MockCsiAdaptor.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/test/java/org/apache/hadoop/yarn/csi/adaptor/TestCsiAdaptorService.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/test/java/org/apache/hadoop/yarn/csi/adaptor/TestGetPluginInfoRequestResponse.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/test/java/org/apache/hadoop/yarn/csi/adaptor/TestNodePublishVolumeRequest.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/test/java/org/apache/hadoop/yarn/csi/adaptor/TestValidateVolumeCapabilityRequest.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/test/java/org/apache/hadoop/yarn/csi/adaptor/TestValidateVolumeCapabilityResponse.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/test/java/org/apache/hadoop/yarn/csi/adaptor/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/test/java/org/apache/hadoop/yarn/csi/client/FakeCsiDriver.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/test/java/org/apache/hadoop/yarn/csi/client/FakeCsiIdentityService.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/test/java/org/apache/hadoop/yarn/csi/client/ICsiClientTest.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/test/java/org/apache/hadoop/yarn/csi/client/TestCsiClient.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/test/java/org/apache/hadoop/yarn/csi/client/package-info.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry/src/main/java/org/apache/hadoop/registry/client/api/RegistryConstants.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry/src/main/java/org/apache/hadoop/registry/client/impl/FSRegistryOperationsService.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry/src/main/java/org/apache/hadoop/registry/server/dns/PrivilegedRegistryDNSStarter.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry/src/main/java/org/apache/hadoop/registry/server/dns/RegistryDNS.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry/src/main/java/org/apache/hadoop/registry/server/dns/RegistryDNSServer.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry/src/main/java/org/apache/hadoop/registry/server/dns/ReverseZoneUtils.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry/src/main/resources/.keep
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry/src/main/tla/yarnregistry.tla
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry/src/test/java/org/apache/hadoop/registry/AbstractZKRegistryTest.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry/src/test/java/org/apache/hadoop/registry/client/impl/TestMicroZookeeperService.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry/src/test/java/org/apache/hadoop/registry/secure/AbstractSecureRegistryTest.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry/src/test/java/org/apache/hadoop/registry/secure/TestSecureLogins.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/metrics/AMRMClientRelayerMetrics.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/metrics/AppAttemptMetricsConstants.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/metrics/ApplicationMetricsConstants.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/metrics/ContainerMetricsConstants.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/metrics/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/AMSecretKeys.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/volume/csi/CsiConstants.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/volume/csi/VolumeCapabilityRange.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/volume/csi/VolumeId.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/volume/csi/VolumeMetaData.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/volume/csi/exception/InvalidVolumeException.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/volume/csi/exception/VolumeException.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/volume/csi/exception/VolumeProvisioningException.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/volume/csi/exception/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/volume/csi/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/LogWebService.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/LogWebServiceUtils.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/test/java/org/apache/hadoop/yarn/server/metrics/TestAMRMClientRelayerMetrics.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/test/java/org/apache/hadoop/yarn/server/webapp/TestLogWebService.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/api/deviceplugin/Device.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/api/deviceplugin/DevicePlugin.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/api/deviceplugin/DevicePluginScheduler.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/api/deviceplugin/DeviceRegisterRequest.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/api/deviceplugin/DeviceRuntimeSpec.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/api/deviceplugin/MountDeviceSpec.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/api/deviceplugin/MountVolumeSpec.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/api/deviceplugin/VolumeSpec.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/api/deviceplugin/YarnRuntimeType.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/api/deviceplugin/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerCleanup.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/docker/DockerExecCommand.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/records/AuxServiceConfiguration.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/records/AuxServiceFile.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/records/AuxServiceRecord.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/records/AuxServiceRecords.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/records/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/deviceframework/AssignedDevice.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/deviceframework/DeviceMappingManager.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/deviceframework/DevicePluginAdapter.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/deviceframework/DeviceResourceHandlerImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/deviceframework/DeviceResourceUpdaterImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/deviceframework/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/gpu/NvidiaDockerV2CommandPlugin.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/volume/csi/ContainerVolumePublisher.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/volume/csi/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/executor/ContainerExecContext.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/AbstractNodeDescriptorsProvider.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/AbstractNodeLabelsProvider.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/ConfigurationNodeAttributesProvider.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/NodeAttributesProvider.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/NodeDescriptorsProvider.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/NodeDescriptorsScriptRunner.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/ScriptBasedNodeAttributesProvider.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/ContainerShellWebSocket.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/ContainerShellWebSocketServlet.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/TerminalServlet.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/dao/AuxiliaryServiceInfo.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/dao/AuxiliaryServicesInfo.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/dao/NMDeviceResourceInfo.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/resources/TERMINAL/css/style.css
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/resources/TERMINAL/terminal.template
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/resources/TERMINAL/xterm/dist/addons/attach/attach.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/resources/TERMINAL/xterm/dist/addons/attach/attach.js.map
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/resources/TERMINAL/xterm/dist/addons/fit/fit.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/resources/TERMINAL/xterm/dist/addons/fit/fit.js.map
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/resources/TERMINAL/xterm/dist/addons/fullscreen/fullscreen.css
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/resources/TERMINAL/xterm/dist/addons/fullscreen/fullscreen.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/resources/TERMINAL/xterm/dist/addons/fullscreen/fullscreen.js.map
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/resources/TERMINAL/xterm/dist/addons/search/search.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/resources/TERMINAL/xterm/dist/addons/search/search.js.map
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/resources/TERMINAL/xterm/dist/addons/terminado/terminado.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/resources/TERMINAL/xterm/dist/addons/terminado/terminado.js.map
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/resources/TERMINAL/xterm/dist/addons/webLinks/webLinks.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/resources/TERMINAL/xterm/dist/addons/webLinks/webLinks.js.map
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/resources/TERMINAL/xterm/dist/addons/winptyCompat/winptyCompat.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/resources/TERMINAL/xterm/dist/addons/winptyCompat/winptyCompat.js.map
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/resources/TERMINAL/xterm/dist/addons/zmodem/zmodem.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/resources/TERMINAL/xterm/dist/addons/zmodem/zmodem.js.map
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/resources/TERMINAL/xterm/dist/xterm.css
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/resources/TERMINAL/xterm/dist/xterm.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/resources/TERMINAL/xterm/dist/xterm.js.map
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeStatusUpdaterForAttributes.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerCleanup.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/MockLinuxContainerRuntime.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceSet.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/deviceframework/FakeTestDevicePlugin1.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/deviceframework/FakeTestDevicePlugin2.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/deviceframework/FakeTestDevicePlugin3.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/deviceframework/FakeTestDevicePlugin4.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/deviceframework/FakeTestDevicePlugin5.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/deviceframework/TestDeviceMappingManager.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/deviceframework/TestDevicePluginAdapter.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/gpu/TestNvidiaDockerV2CommandPlugin.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/TestConfigurationNodeAttributesProvider.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/TestScriptBasedNodeAttributesProvider.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/webapp/ContainerShellClientSocketTest.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/webapp/TestNMContainerWebSocket.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/webapp/TestNMWebServicesAuxServices.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/webapp/TestNMWebTerminal.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/resource-types-pluggable-devices.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/IsResourceManagerActiveServlet.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/FileSystemNodeAttributeStore.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/NodeAttributesManagerImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/NodeAttributesStoreEvent.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/NodeAttributesStoreEventType.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStoreProxyCAEvent.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/QueueMetricsForCustomResources.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/event/NodeAttributesUpdateSchedulerEvent.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/security/ProxyCAManager.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/volume/csi/VolumeBuilder.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/volume/csi/VolumeManager.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/volume/csi/VolumeManagerImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/volume/csi/VolumeStates.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/volume/csi/event/ControllerPublishVolumeEvent.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/volume/csi/event/ValidateVolumeEvent.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/volume/csi/event/VolumeEvent.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/volume/csi/event/VolumeEventType.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/volume/csi/event/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/volume/csi/lifecycle/Volume.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/volume/csi/lifecycle/VolumeImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/volume/csi/lifecycle/VolumeState.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/volume/csi/lifecycle/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/volume/csi/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/volume/csi/processor/VolumeAMSProcessor.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/volume/csi/processor/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/volume/csi/provisioner/VolumeProvisioner.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/volume/csi/provisioner/VolumeProvisioningResults.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/volume/csi/provisioner/VolumeProvisioningTask.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/volume/csi/provisioner/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/volume/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/NodeAttributeInfo.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/NodeAttributesInfo.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/ResourceOptionInfo.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/AppManagerTestBase.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/ApplicationMasterServiceTestBase.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/NodeAttributeTestUtils.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestAppManagerWithFairScheduler.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterService.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterServiceCapacity.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterServiceFair.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterServiceInterceptor.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/TestFileSystemNodeAttributeStore.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/TestNodeAttributesManager.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AppMetricsChecker.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/QueueInfo.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/QueueMetricsTestData.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/ResourceMetricsChecker.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestQueueMetricsForCustomResources.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestCSAllocateCustomResource.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestApplicationMasterServiceWithFS.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestProxyCAManager.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/volume/csi/TestVolumeCapabilityRange.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/volume/csi/TestVolumeLifecycle.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/volume/csi/TestVolumeMetaData.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/volume/csi/TestVolumeProcessor.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesAppAttempts.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesAppCustomResourceTypes.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesCustomResourceTypesCommons.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/fairscheduler/CustomResourceTypesConfigurationProvider.java
rename hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/{ => fairscheduler}/TestRMWebServicesFairScheduler.java (100%)
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/resource-types-test.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/clientrm/ClientMethod.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/clientrm/RouterYarnClientUtils.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/test/java/org/apache/hadoop/yarn/server/router/TestRouter.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/test/java/org/apache/hadoop/yarn/server/router/clientrm/TestRouterYarnClientUtils.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests/src/test/java/org/apache/hadoop/yarn/server/timelineservice/reader/DummyTimelineReaderMetrics.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-client/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/HBaseTimelineSchemaCreator.java
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-client/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/TimelineSchemaCreator.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/metrics/PerNodeAggTimelineCollectorMetrics.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/metrics/TimelineReaderMetrics.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/metrics/package-info.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/NoOpTimelineReaderImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/NoOpTimelineWriterImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/SchemaCreator.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/TimelineSchemaCreator.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/test/java/org/apache/hadoop/yarn/server/timelineservice/collector/TestPerNodeAggTimelineCollectorMetrics.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/test/java/org/apache/hadoop/yarn/server/timelineservice/reader/TestTimelineReaderMetrics.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/test/java/org/apache/hadoop/yarn/server/timelineservice/storage/DummyTimelineSchemaCreator.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/test/java/org/apache/hadoop/yarn/server/timelineservice/storage/TestTimelineSchemaCreator.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/ProxyCA.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/test/java/org/apache/hadoop/yarn/server/webproxy/TestProxyCA.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeAttributes.md
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/registry/index.md
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/registry/registry-configuration.md
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/registry/using-the-yarn-service-registry.md
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/registry/yarn-registry.md
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/yarn-service/RegistryDNS.md
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/adapters/yarn-service.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/models/yarn-service.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/routes/yarn-log-service.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/serializers/yarn-service.js
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/templates/yarn-log-service.hbs
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org
[hadoop] 03/03: HDFS-13794. [PROVIDED Phase 2] Teach
BlockAliasMap.Writer remove method. Contributed by Ewan Higgs
Posted by eh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ehiggs pushed a commit to branch HDFS-12090
in repository https://gitbox.apache.org/repos/asf/hadoop.git
commit 64da7040db3dee4f31e99316b30144c3b76c5433
Author: Virajith Jalaparti <vi...@apache.org>
AuthorDate: Tue Feb 12 13:43:06 2019 -0800
HDFS-13794. [PROVIDED Phase 2] Teach BlockAliasMap.Writer remove method. Contributed by Ewan Higgs
---
.../main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java | 1 -
.../AliasMapProtocolServerSideTranslatorPB.java | 16 ++++++++++++++++
.../InMemoryAliasMapProtocolClientSideTranslatorPB.java | 15 +++++++++++++++
.../hadoop/hdfs/server/aliasmap/InMemoryAliasMap.java | 8 ++++++--
.../hdfs/server/aliasmap/InMemoryAliasMapProtocol.java | 9 +++++++++
.../server/aliasmap/InMemoryLevelDBAliasMapServer.java | 5 +++++
.../hdfs/server/common/blockaliasmap/BlockAliasMap.java | 14 +++++++++++++-
.../impl/InMemoryLevelDBAliasMapClient.java | 7 ++++++-
.../blockaliasmap/impl/LevelDBFileRegionAliasMap.java | 6 ++++++
.../blockaliasmap/impl/TextFileRegionAliasMap.java | 6 ++++++
.../hadoop-hdfs/src/main/proto/AliasMapProtocol.proto | 8 ++++++++
.../hdfs/server/aliasmap/ITestInMemoryAliasMap.java | 10 +++++++---
.../impl/TestInMemoryLevelDBAliasMapClient.java | 13 ++++++++-----
.../impl/TestLevelDbMockAliasMapClient.java | 3 ++-
.../hadoop/hdfs/server/namenode/NullBlockAliasMap.java | 7 ++++++-
15 files changed, 113 insertions(+), 15 deletions(-)
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
index fedfc5a..1e3083d 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
@@ -96,7 +96,6 @@ public class DFSConfigKeys extends CommonConfigurationKeys {
HdfsClientConfigKeys.DFS_PROVIDED_ALIASMAP_INMEMORY_RPC_ADDRESS;
public static final String DFS_PROVIDED_ALIASMAP_INMEMORY_RPC_ADDRESS_DEFAULT = "0.0.0.0:50200";
public static final String DFS_PROVIDED_ALIASMAP_INMEMORY_RPC_BIND_HOST = "dfs.provided.aliasmap.inmemory.rpc.bind-host";
-
public static final String DFS_PROVIDED_ALIASMAP_INMEMORY_LEVELDB_DIR = "dfs.provided.aliasmap.inmemory.leveldb.dir";
public static final String DFS_PROVIDED_ALIASMAP_INMEMORY_BATCH_SIZE = "dfs.provided.aliasmap.inmemory.batch-size";
public static final int DFS_PROVIDED_ALIASMAP_INMEMORY_BATCH_SIZE_DEFAULT = 500;
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/AliasMapProtocolServerSideTranslatorPB.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/AliasMapProtocolServerSideTranslatorPB.java
index 8d89c40..48da058 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/AliasMapProtocolServerSideTranslatorPB.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/AliasMapProtocolServerSideTranslatorPB.java
@@ -57,6 +57,9 @@ public class AliasMapProtocolServerSideTranslatorPB
private static final WriteResponseProto VOID_WRITE_RESPONSE =
WriteResponseProto.newBuilder().build();
+ private static final RemoveResponseProto VOID_REMOVE_RESPONSE =
+ RemoveResponseProto.newBuilder().build();
+
@Override
public WriteResponseProto write(RpcController controller,
WriteRequestProto request) throws ServiceException {
@@ -72,6 +75,19 @@ public class AliasMapProtocolServerSideTranslatorPB
}
@Override
+ public RemoveResponseProto remove(RpcController controller,
+ RemoveRequestProto request) throws ServiceException {
+ try {
+ Block toRemove =
+ PBHelperClient.convert(request.getKey());
+ aliasMap.remove(toRemove);
+ return VOID_REMOVE_RESPONSE;
+ } catch (IOException e) {
+ throw new ServiceException(e);
+ }
+ }
+
+ @Override
public ReadResponseProto read(RpcController controller,
ReadRequestProto request) throws ServiceException {
try {
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/InMemoryAliasMapProtocolClientSideTranslatorPB.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/InMemoryAliasMapProtocolClientSideTranslatorPB.java
index d9e984b..dc5bd3b 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/InMemoryAliasMapProtocolClientSideTranslatorPB.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/InMemoryAliasMapProtocolClientSideTranslatorPB.java
@@ -214,6 +214,21 @@ public class InMemoryAliasMapProtocolClientSideTranslatorPB
}
@Override
+ public void remove(@Nonnull Block block) throws IOException {
+ RemoveRequestProto request =
+ RemoveRequestProto
+ .newBuilder()
+ .setKey(PBHelperClient.convert(block))
+ .build();
+
+ try {
+ rpcProxy.remove(null, request);
+ } catch (ServiceException e) {
+ throw ProtobufHelper.getRemoteException(e);
+ }
+ }
+
+ @Override
public String getBlockPoolId() throws IOException {
try {
BlockPoolResponseProto response = rpcProxy.getBlockPoolId(null,
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/aliasmap/InMemoryAliasMap.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/aliasmap/InMemoryAliasMap.java
index eeee2a3..02c79ff 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/aliasmap/InMemoryAliasMap.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/aliasmap/InMemoryAliasMap.java
@@ -180,6 +180,11 @@ public class InMemoryAliasMap implements InMemoryAliasMapProtocol,
levelDb.put(extendedBlockDbFormat, providedStorageLocationDbFormat);
}
+ public void remove(@Nonnull Block block) throws IOException {
+ byte[] extendedBlockDbFormat = toProtoBufBytes(block);
+ levelDb.delete(extendedBlockDbFormat);
+ }
+
@Override
public String getBlockPoolId() {
return blockPoolID;
@@ -218,8 +223,7 @@ public class InMemoryAliasMap implements InMemoryAliasMapProtocol,
public static byte[] toProtoBufBytes(@Nonnull Block block)
throws IOException {
- BlockProto blockProto =
- PBHelperClient.convert(block);
+ BlockProto blockProto = PBHelperClient.convert(block);
ByteArrayOutputStream blockOutputStream = new ByteArrayOutputStream();
blockProto.writeTo(blockOutputStream);
return blockOutputStream.toByteArray();
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/aliasmap/InMemoryAliasMapProtocol.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/aliasmap/InMemoryAliasMapProtocol.java
index c3824e5..170d1f6 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/aliasmap/InMemoryAliasMapProtocol.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/aliasmap/InMemoryAliasMapProtocol.java
@@ -99,6 +99,15 @@ public interface InMemoryAliasMapProtocol {
throws IOException;
/**
+ * Removed the block and it's associated {@link ProvidedStorageLocation}
+ * from the alias map.
+ * As this is for in memory alias map, we do not require the bpid.
+ * @param block
+ * @throws IOException
+ */
+ void remove(@Nonnull Block block) throws IOException;
+
+ /**
* Get the associated block pool id.
* @return the block pool id associated with the Namenode running
* the in-memory alias map.
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/aliasmap/InMemoryLevelDBAliasMapServer.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/aliasmap/InMemoryLevelDBAliasMapServer.java
index cd2ed82..2fe084a 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/aliasmap/InMemoryLevelDBAliasMapServer.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/aliasmap/InMemoryLevelDBAliasMapServer.java
@@ -127,6 +127,11 @@ public class InMemoryLevelDBAliasMapServer implements InMemoryAliasMapProtocol,
}
@Override
+ public void remove(@Nonnull Block block) throws IOException {
+ aliasMap.remove(block);
+ }
+
+ @Override
public String getBlockPoolId() {
return blockPoolId;
}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/BlockAliasMap.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/BlockAliasMap.java
index 897aefd..9905039 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/BlockAliasMap.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/BlockAliasMap.java
@@ -88,8 +88,20 @@ public abstract class BlockAliasMap<T extends BlockAlias> {
*/
public interface Options { }
- public abstract void store(U token) throws IOException;
+ /**
+ * Store the BlockAlias; this may be a destructive store, replacing old
+ * data.
+ * @param blockAlias
+ * @throws IOException
+ */
+ public abstract void store(U blockAlias) throws IOException;
+ /**
+ * Remove the block from the AliasMap.
+ * @param block
+ * @throws IOException
+ */
+ public abstract void remove(Block block) throws IOException;
}
/**
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/InMemoryLevelDBAliasMapClient.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/InMemoryLevelDBAliasMapClient.java
index fb5ee93..3dac4ec 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/InMemoryLevelDBAliasMapClient.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/InMemoryLevelDBAliasMapClient.java
@@ -145,11 +145,16 @@ public class InMemoryLevelDBAliasMapClient extends BlockAliasMap<FileRegion>
}
@Override
+ public void remove(Block block) throws IOException {
+ aliasMap.remove(block);
+ }
+
+ @Override
public void close() throws IOException {
}
}
- InMemoryLevelDBAliasMapClient() {
+ public InMemoryLevelDBAliasMapClient() {
if (UserGroupInformation.isSecurityEnabled()) {
throw new UnsupportedOperationException("Unable to start "
+ "InMemoryLevelDBAliasMapClient as security is enabled");
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/LevelDBFileRegionAliasMap.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/LevelDBFileRegionAliasMap.java
index 6afe6bb..81756e1 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/LevelDBFileRegionAliasMap.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/LevelDBFileRegionAliasMap.java
@@ -265,6 +265,12 @@ public class LevelDBFileRegionAliasMap
}
@Override
+ public void remove(Block block) throws IOException {
+ byte[] key = toProtoBufBytes(block);
+ db.delete(key);
+ }
+
+ @Override
public void close() throws IOException {
if (db != null) {
db.close();
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/TextFileRegionAliasMap.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/TextFileRegionAliasMap.java
index 4d65142..fb73b41 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/TextFileRegionAliasMap.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/TextFileRegionAliasMap.java
@@ -458,6 +458,12 @@ public class TextFileRegionAliasMap
}
@Override
+ public void remove(Block block) throws IOException {
+ throw new RuntimeException("TextFileWriter does not support " +
+ "block removal");
+ }
+
+ @Override
public void close() throws IOException {
out.close();
}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/proto/AliasMapProtocol.proto b/hadoop-hdfs-project/hadoop-hdfs/src/main/proto/AliasMapProtocol.proto
index 01dd952..077e075 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/proto/AliasMapProtocol.proto
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/proto/AliasMapProtocol.proto
@@ -36,6 +36,13 @@ message WriteRequestProto {
message WriteResponseProto {
}
+message RemoveRequestProto {
+ required BlockProto key = 1;
+}
+
+message RemoveResponseProto {
+}
+
message ReadRequestProto {
required BlockProto key = 1;
}
@@ -62,6 +69,7 @@ message BlockPoolResponseProto {
service AliasMapProtocolService {
rpc write(WriteRequestProto) returns(WriteResponseProto);
+ rpc remove(RemoveRequestProto) returns(RemoveResponseProto);
rpc read(ReadRequestProto) returns(ReadResponseProto);
rpc list(ListRequestProto) returns(ListResponseProto);
rpc getBlockPoolId(BlockPoolRequestProto) returns(BlockPoolResponseProto);
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/aliasmap/ITestInMemoryAliasMap.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/aliasmap/ITestInMemoryAliasMap.java
index 2785f68..a9f5c73 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/aliasmap/ITestInMemoryAliasMap.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/aliasmap/ITestInMemoryAliasMap.java
@@ -64,7 +64,7 @@ public class ITestInMemoryAliasMap {
}
@Test
- public void readNotFoundReturnsNothing() throws IOException {
+ public void testReadNotFoundReturnsNothing() throws IOException {
Block block = new Block(42, 43, 44);
Optional<ProvidedStorageLocation> actualProvidedStorageLocationOpt
@@ -74,7 +74,7 @@ public class ITestInMemoryAliasMap {
}
@Test
- public void readWrite() throws Exception {
+ public void testReadWriteRemove() throws Exception {
Block block = new Block(42, 43, 44);
Path path = new Path("eagle", "mouse");
@@ -92,14 +92,18 @@ public class ITestInMemoryAliasMap {
Optional<ProvidedStorageLocation> actualProvidedStorageLocationOpt
= aliasMap.read(block);
+
assertTrue(actualProvidedStorageLocationOpt.isPresent());
assertEquals(expectedProvidedStorageLocation,
actualProvidedStorageLocationOpt.get());
+ aliasMap.remove(block);
+ actualProvidedStorageLocationOpt = aliasMap.read(block);
+ assertFalse(actualProvidedStorageLocationOpt.isPresent());
}
@Test
- public void list() throws IOException {
+ public void testIteration() throws IOException {
Block block1 = new Block(42, 43, 44);
Block block2 = new Block(43, 44, 45);
Block block3 = new Block(44, 45, 46);
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/TestInMemoryLevelDBAliasMapClient.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/TestInMemoryLevelDBAliasMapClient.java
index 1e42a88..c4fea99 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/TestInMemoryLevelDBAliasMapClient.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/TestInMemoryLevelDBAliasMapClient.java
@@ -28,7 +28,7 @@ import org.apache.hadoop.hdfs.server.aliasmap.InMemoryAliasMap;
import org.apache.hadoop.hdfs.server.aliasmap.InMemoryLevelDBAliasMapServer;
import org.apache.hadoop.hdfs.server.common.blockaliasmap.BlockAliasMap;
import org.apache.hadoop.hdfs.server.common.FileRegion;
-import org.apache.hadoop.test.LambdaTestUtils;
+import org.apache.hadoop.net.NetUtils;
import org.junit.After;
import org.junit.Before;
import org.junit.Rule;
@@ -36,6 +36,7 @@ import org.junit.Test;
import org.junit.rules.ExpectedException;
import static org.apache.hadoop.hdfs.DFSConfigKeys.DFS_NAMENODE_SERVICE_RPC_BIND_HOST_KEY;
+import static org.apache.hadoop.test.LambdaTestUtils.assertOptionalUnset;
import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.Assert.assertArrayEquals;
import static org.junit.Assert.assertEquals;
@@ -70,7 +71,7 @@ public class TestInMemoryLevelDBAliasMapClient {
@Before
public void setUp() throws IOException {
conf = new Configuration();
- int port = 9876;
+ int port = NetUtils.getFreeSocketPort();
conf.set(DFSConfigKeys.DFS_PROVIDED_ALIASMAP_INMEMORY_RPC_ADDRESS,
"localhost:" + port);
@@ -92,7 +93,7 @@ public class TestInMemoryLevelDBAliasMapClient {
}
@Test
- public void writeRead() throws Exception {
+ public void writeReadRemove() throws Exception {
levelDBAliasMapServer.setConf(conf);
levelDBAliasMapServer.start();
inMemoryLevelDBAliasMapClient.setConf(conf);
@@ -110,6 +111,8 @@ public class TestInMemoryLevelDBAliasMapClient {
Optional<FileRegion> fileRegion = reader.resolve(block);
assertEquals(new FileRegion(block, providedStorageLocation),
fileRegion.get());
+ writer.remove(block);
+ assertOptionalUnset("Block should not exist", reader.resolve(block));
}
@Test
@@ -354,7 +357,7 @@ public class TestInMemoryLevelDBAliasMapClient {
@Test
public void testServerBindHost() throws Exception {
conf.set(DFS_NAMENODE_SERVICE_RPC_BIND_HOST_KEY, "0.0.0.0");
- writeRead();
+ writeReadRemove();
}
@Test
@@ -375,7 +378,7 @@ public class TestInMemoryLevelDBAliasMapClient {
BlockAliasMap.Reader<FileRegion> reader =
inMemoryLevelDBAliasMapClient.getReader(null, BPID);
- LambdaTestUtils.assertOptionalUnset("Expected empty BlockAlias",
+ assertOptionalUnset("Expected empty BlockAlias",
reader.resolve(block1));
}
}
\ No newline at end of file
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/TestLevelDbMockAliasMapClient.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/TestLevelDbMockAliasMapClient.java
index 534bc36..ab1fa63 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/TestLevelDbMockAliasMapClient.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/TestLevelDbMockAliasMapClient.java
@@ -26,6 +26,7 @@ import org.apache.hadoop.hdfs.protocol.ProvidedStorageLocation;
import org.apache.hadoop.hdfs.server.aliasmap.InMemoryAliasMap;
import org.apache.hadoop.hdfs.server.aliasmap.InMemoryLevelDBAliasMapServer;
import org.apache.hadoop.hdfs.server.common.FileRegion;
+import org.apache.hadoop.net.NetUtils;
import org.iq80.leveldb.DBException;
import org.junit.After;
import org.junit.Before;
@@ -56,7 +57,7 @@ public class TestLevelDbMockAliasMapClient {
levelDBAliasMapServer = new InMemoryLevelDBAliasMapServer(
(config, blockPoolID) -> aliasMapMock, bpid);
conf = new Configuration();
- int port = 9877;
+ int port = NetUtils.getFreeSocketPort();
conf.set(DFSConfigKeys.DFS_PROVIDED_ALIASMAP_INMEMORY_RPC_ADDRESS,
"localhost:" + port);
diff --git a/hadoop-tools/hadoop-fs2img/src/main/java/org/apache/hadoop/hdfs/server/namenode/NullBlockAliasMap.java b/hadoop-tools/hadoop-fs2img/src/main/java/org/apache/hadoop/hdfs/server/namenode/NullBlockAliasMap.java
index 41e202d..e439bab 100644
--- a/hadoop-tools/hadoop-fs2img/src/main/java/org/apache/hadoop/hdfs/server/namenode/NullBlockAliasMap.java
+++ b/hadoop-tools/hadoop-fs2img/src/main/java/org/apache/hadoop/hdfs/server/namenode/NullBlockAliasMap.java
@@ -74,7 +74,12 @@ public class NullBlockAliasMap extends BlockAliasMap<FileRegion> {
throws IOException {
return new Writer<FileRegion>() {
@Override
- public void store(FileRegion token) throws IOException {
+ public void store(FileRegion blockAlias) throws IOException {
+ // do nothing
+ }
+
+ @Override
+ public void remove(Block block) throws IOException {
// do nothing
}
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org
[hadoop] 01/03: HDFS-13310. The DatanodeProtocol should have a
DNA_BACKUP to backup blocks. Original patch contributed by Ewan Higgs.
Followup work and fixed contributed by Virajith Jalaparthi.
Posted by eh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ehiggs pushed a commit to branch HDFS-12090
in repository https://gitbox.apache.org/repos/asf/hadoop.git
commit ed8d57b55653881ce6747d0e0cf0ca466662c811
Author: Ewan Higgs <ew...@wdc.com>
AuthorDate: Mon Jul 23 13:14:04 2018 +0200
HDFS-13310. The DatanodeProtocol should have a DNA_BACKUP to backup blocks. Original patch contributed by Ewan Higgs. Followup work and fixed contributed by Virajith Jalaparthi.
---
.../protocol/BlockSyncTaskExecutionFeedback.java | 67 +++++++
.../server/protocol/SyncTaskExecutionOutcome.java | 25 +++
.../server/protocol/SyncTaskExecutionResult.java | 46 +++++
.../DatanodeProtocolClientSideTranslatorPB.java | 8 +-
.../DatanodeProtocolServerSideTranslatorPB.java | 6 +-
.../apache/hadoop/hdfs/protocolPB/PBHelper.java | 208 ++++++++++++++++++++-
.../server/blockmanagement/DatanodeManager.java | 4 +-
.../hdfs/server/datanode/BPServiceActor.java | 9 +-
.../hadoop/hdfs/server/namenode/FSNamesystem.java | 8 +-
.../hdfs/server/namenode/NameNodeRpcServer.java | 8 +-
.../hadoop/hdfs/server/protocol/BlockSyncTask.java | 83 ++++++++
.../protocol/BulkSyncTaskExecutionFeedback.java | 36 ++++
.../hdfs/server/protocol/DatanodeProtocol.java | 20 +-
.../hadoop/hdfs/server/protocol/SyncCommand.java | 39 ++++
.../src/main/proto/DatanodeProtocol.proto | 88 ++++++++-
.../blockmanagement/TestDatanodeManager.java | 2 +-
.../TestNameNodePrunesMissingStorages.java | 2 +-
.../server/datanode/InternalDataNodeTestUtils.java | 1 +
.../hdfs/server/datanode/TestBPOfferService.java | 5 +-
.../hdfs/server/datanode/TestBlockRecovery.java | 1 +
.../hdfs/server/datanode/TestDataNodeLifeline.java | 9 +-
.../datanode/TestDatanodeProtocolRetryPolicy.java | 1 +
.../hdfs/server/datanode/TestFsDatasetCache.java | 4 +-
.../hdfs/server/datanode/TestStorageReport.java | 4 +-
.../server/namenode/NNThroughputBenchmark.java | 8 +-
.../hdfs/server/namenode/NameNodeAdapter.java | 5 +-
.../hdfs/server/namenode/TestDeadDatanode.java | 4 +-
27 files changed, 653 insertions(+), 48 deletions(-)
diff --git a/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/protocol/BlockSyncTaskExecutionFeedback.java b/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/protocol/BlockSyncTaskExecutionFeedback.java
new file mode 100644
index 0000000..2e5393e
--- /dev/null
+++ b/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/protocol/BlockSyncTaskExecutionFeedback.java
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdfs.server.protocol;
+
+import java.util.UUID;
+
+/**
+ * Feedback for a BlockSyncTask.
+ */
+public class BlockSyncTaskExecutionFeedback {
+
+ private UUID syncTaskId;
+ private SyncTaskExecutionOutcome outcome;
+ private SyncTaskExecutionResult result;
+ private String syncMountId;
+
+ public BlockSyncTaskExecutionFeedback(UUID syncTaskId,
+ SyncTaskExecutionOutcome outcome, SyncTaskExecutionResult result,
+ String syncMountId) {
+ this.syncTaskId = syncTaskId;
+ this.outcome = outcome;
+ this.result = result;
+ this.syncMountId = syncMountId;
+ }
+
+ public static BlockSyncTaskExecutionFeedback finishedSuccessfully(
+ UUID syncTaskId, String syncMountId, SyncTaskExecutionResult result) {
+ return new BlockSyncTaskExecutionFeedback(syncTaskId,
+ SyncTaskExecutionOutcome.FINISHED_SUCCESSFULLY, result, syncMountId);
+ }
+
+ public static BlockSyncTaskExecutionFeedback failedWithException(
+ UUID syncTaskId, String syncMountId, Exception e) {
+ return new BlockSyncTaskExecutionFeedback(syncTaskId,
+ SyncTaskExecutionOutcome.EXCEPTION, null, syncMountId);
+ }
+
+ public UUID getSyncTaskId() {
+ return syncTaskId;
+ }
+
+ public SyncTaskExecutionOutcome getOutcome() {
+ return outcome;
+ }
+
+ public SyncTaskExecutionResult getResult() {
+ return result;
+ }
+
+ public String getSyncMountId() {
+ return syncMountId;
+ }
+}
diff --git a/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/protocol/SyncTaskExecutionOutcome.java b/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/protocol/SyncTaskExecutionOutcome.java
new file mode 100644
index 0000000..492575b
--- /dev/null
+++ b/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/protocol/SyncTaskExecutionOutcome.java
@@ -0,0 +1,25 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdfs.server.protocol;
+
+/**
+ * SyncTaskExecutionOutcome is whether the SyncTask was successful or not.
+ */
+public enum SyncTaskExecutionOutcome {
+ FINISHED_SUCCESSFULLY,
+ EXCEPTION
+}
diff --git a/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/protocol/SyncTaskExecutionResult.java b/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/protocol/SyncTaskExecutionResult.java
new file mode 100644
index 0000000..b623dc5
--- /dev/null
+++ b/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/protocol/SyncTaskExecutionResult.java
@@ -0,0 +1,46 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdfs.server.protocol;
+
+import java.nio.ByteBuffer;
+
+/**
+ * Result of a SyncTask.
+ */
+public class SyncTaskExecutionResult {
+
+ /** result is the opaque byte stream result of a task. e.g. PartHandle */
+ private ByteBuffer result;
+ private Long numberOfBytes;
+
+ public SyncTaskExecutionResult(ByteBuffer result, Long numberOfBytes) {
+ this.result = result;
+ this.numberOfBytes = numberOfBytes;
+ }
+
+ public static SyncTaskExecutionResult emptyResult() {
+ return new SyncTaskExecutionResult(ByteBuffer.wrap(new byte[0]), 0L);
+ }
+
+ public ByteBuffer getResult() {
+ return result;
+ }
+
+ public Long getNumberOfBytes() {
+ return numberOfBytes;
+ }
+}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/DatanodeProtocolClientSideTranslatorPB.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/DatanodeProtocolClientSideTranslatorPB.java
index e4125dc..20b314c 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/DatanodeProtocolClientSideTranslatorPB.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/DatanodeProtocolClientSideTranslatorPB.java
@@ -48,6 +48,7 @@ import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.StorageBlock
import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.StorageReceivedDeletedBlocksProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.VersionRequestProto;
import org.apache.hadoop.hdfs.server.protocol.BlockReportContext;
+import org.apache.hadoop.hdfs.server.protocol.BulkSyncTaskExecutionFeedback;
import org.apache.hadoop.hdfs.server.protocol.DatanodeCommand;
import org.apache.hadoop.hdfs.server.protocol.DatanodeProtocol;
import org.apache.hadoop.hdfs.server.protocol.DatanodeRegistration;
@@ -138,8 +139,8 @@ public class DatanodeProtocolClientSideTranslatorPB implements
VolumeFailureSummary volumeFailureSummary,
boolean requestFullBlockReportLease,
@Nonnull SlowPeerReports slowPeers,
- @Nonnull SlowDiskReports slowDisks)
- throws IOException {
+ @Nonnull SlowDiskReports slowDisks,
+ BulkSyncTaskExecutionFeedback feedback) throws IOException {
HeartbeatRequestProto.Builder builder = HeartbeatRequestProto.newBuilder()
.setRegistration(PBHelper.convert(registration))
.setXmitsInProgress(xmitsInProgress).setXceiverCount(xceiverCount)
@@ -162,6 +163,9 @@ public class DatanodeProtocolClientSideTranslatorPB implements
if (slowDisks.haveSlowDisks()) {
builder.addAllSlowDisks(PBHelper.convertSlowDiskInfo(slowDisks));
}
+ if(feedback != null && !feedback.getFeedbacks().isEmpty()) {
+ builder.setBulkSyncTaskExecutionFeedback(PBHelper.convert(feedback));
+ }
HeartbeatResponseProto resp;
try {
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/DatanodeProtocolServerSideTranslatorPB.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/DatanodeProtocolServerSideTranslatorPB.java
index 5cba284..a51ce85 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/DatanodeProtocolServerSideTranslatorPB.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/DatanodeProtocolServerSideTranslatorPB.java
@@ -109,6 +109,7 @@ public class DatanodeProtocolServerSideTranslatorPB implements
@Override
public HeartbeatResponseProto sendHeartbeat(RpcController controller,
HeartbeatRequestProto request) throws ServiceException {
+
HeartbeatResponse response;
try {
final StorageReport[] report = PBHelperClient.convertStorageReports(
@@ -122,7 +123,10 @@ public class DatanodeProtocolServerSideTranslatorPB implements
request.getXceiverCount(), request.getFailedVolumes(),
volumeFailureSummary, request.getRequestFullBlockReportLease(),
PBHelper.convertSlowPeerInfo(request.getSlowPeersList()),
- PBHelper.convertSlowDiskInfo(request.getSlowDisksList()));
+ PBHelper.convertSlowDiskInfo(request.getSlowDisksList()),
+ PBHelper.convertBulkSyncTaskExecutionFeedback(
+ request.getBulkSyncTaskExecutionFeedback())
+ );
} catch (IOException e) {
throw new ServiceException(e);
}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java
index baec6fa..a62a00d 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java
@@ -17,6 +17,9 @@
*/
package org.apache.hadoop.hdfs.protocolPB;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.nio.ByteBuffer;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
@@ -24,9 +27,10 @@ import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
+import java.util.UUID;
+import java.util.stream.Collectors;
import com.google.protobuf.ByteString;
-
import org.apache.hadoop.fs.StorageType;
import org.apache.hadoop.ha.HAServiceProtocol.HAServiceState;
import org.apache.hadoop.hdfs.DFSUtilClient;
@@ -43,38 +47,44 @@ import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.BlockCommand
import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.BlockECReconstructionCommandProto;
import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.BlockIdCommandProto;
import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.BlockRecoveryCommandProto;
+import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.BlockReportContextProto;
+import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.BlockSyncTaskProto;
+import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.BulkSyncTaskExecutionFeedbackProto;
import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.DatanodeCommandProto;
import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.DatanodeRegistrationProto;
import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.FinalizeCommandProto;
import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.KeyUpdateCommandProto;
import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.ReceivedDeletedBlockInfoProto;
import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.RegisterCommandProto;
-import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos
- .SlowDiskReportProto;
+import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.SlowDiskReportProto;
import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.SlowPeerReportProto;
+import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.SyncCommandProto;
+import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.SyncTaskExecutionFeedbackProto;
+import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.SyncTaskExecutionOutcomeProto;
+import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.SyncTaskExecutionResultProto;
+import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.SyncTaskIdProto;
import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.VolumeFailureSummaryProto;
-import org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos.BlockReportContextProto;
import org.apache.hadoop.hdfs.protocol.proto.ErasureCodingProtos.BlockECReconstructionInfoProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsProtos;
import org.apache.hadoop.hdfs.protocol.proto.HdfsProtos.BlockProto;
-import org.apache.hadoop.hdfs.protocol.proto.HdfsProtos.ExtendedBlockProto;
-import org.apache.hadoop.hdfs.protocol.proto.HdfsProtos.ProvidedStorageLocationProto;
-import org.apache.hadoop.hdfs.protocol.proto.HdfsProtos.StorageUuidsProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsProtos.DatanodeInfosProto;
+import org.apache.hadoop.hdfs.protocol.proto.HdfsProtos.ExtendedBlockProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsProtos.LocatedBlockProto;
+import org.apache.hadoop.hdfs.protocol.proto.HdfsProtos.ProvidedStorageLocationProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsProtos.StorageTypeProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsProtos.StorageTypesProto;
+import org.apache.hadoop.hdfs.protocol.proto.HdfsProtos.StorageUuidsProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.BlockKeyProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.BlockWithLocationsProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.BlocksWithLocationsProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.CheckpointCommandProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.CheckpointSignatureProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.ExportedBlockKeysProto;
+import org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.NNHAStatusHeartbeatProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.NamenodeCommandProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.NamenodeRegistrationProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.NamenodeRegistrationProto.NamenodeRoleProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.NamespaceInfoProto;
-import org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.NNHAStatusHeartbeatProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.RecoveringBlockProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.RemoteEditLogManifestProto;
import org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.RemoteEditLogProto;
@@ -89,18 +99,23 @@ import org.apache.hadoop.hdfs.server.common.HdfsServerConstants.NodeType;
import org.apache.hadoop.hdfs.server.common.HdfsServerConstants.ReplicaState;
import org.apache.hadoop.hdfs.server.common.StorageInfo;
import org.apache.hadoop.hdfs.server.namenode.CheckpointSignature;
+import org.apache.hadoop.hdfs.server.protocol.BlockSyncTask;
+import org.apache.hadoop.hdfs.server.protocol.BlockSyncTaskExecutionFeedback;
+import org.apache.hadoop.hdfs.server.protocol.SyncTaskExecutionOutcome;
+import org.apache.hadoop.hdfs.server.protocol.SyncTaskExecutionResult;
import org.apache.hadoop.hdfs.server.protocol.BalancerBandwidthCommand;
import org.apache.hadoop.hdfs.server.protocol.BlockCommand;
import org.apache.hadoop.hdfs.server.protocol.BlockECReconstructionCommand;
+import org.apache.hadoop.hdfs.server.protocol.BlockECReconstructionCommand.BlockECReconstructionInfo;
import org.apache.hadoop.hdfs.server.protocol.BlockIdCommand;
import org.apache.hadoop.hdfs.server.protocol.BlockRecoveryCommand;
-import org.apache.hadoop.hdfs.server.protocol.BlockECReconstructionCommand.BlockECReconstructionInfo;
import org.apache.hadoop.hdfs.server.protocol.BlockRecoveryCommand.RecoveringBlock;
import org.apache.hadoop.hdfs.server.protocol.BlockRecoveryCommand.RecoveringStripedBlock;
import org.apache.hadoop.hdfs.server.protocol.BlockReportContext;
import org.apache.hadoop.hdfs.server.protocol.BlocksWithLocations;
import org.apache.hadoop.hdfs.server.protocol.BlocksWithLocations.BlockWithLocations;
import org.apache.hadoop.hdfs.server.protocol.BlocksWithLocations.StripedBlockWithLocations;
+import org.apache.hadoop.hdfs.server.protocol.BulkSyncTaskExecutionFeedback;
import org.apache.hadoop.hdfs.server.protocol.CheckpointCommand;
import org.apache.hadoop.hdfs.server.protocol.DatanodeCommand;
import org.apache.hadoop.hdfs.server.protocol.DatanodeProtocol;
@@ -119,7 +134,9 @@ import org.apache.hadoop.hdfs.server.protocol.RemoteEditLog;
import org.apache.hadoop.hdfs.server.protocol.RemoteEditLogManifest;
import org.apache.hadoop.hdfs.server.protocol.SlowDiskReports;
import org.apache.hadoop.hdfs.server.protocol.SlowPeerReports;
+import org.apache.hadoop.hdfs.server.protocol.SyncCommand;
import org.apache.hadoop.hdfs.server.protocol.VolumeFailureSummary;
+import org.apache.hadoop.ipc.ClientId;
/**
* Utilities for converting protobuf classes to and from implementation classes
@@ -469,11 +486,52 @@ public class PBHelper {
return PBHelper.convert(proto.getBlkIdCmd());
case BlockECReconstructionCommand:
return PBHelper.convert(proto.getBlkECReconstructionCmd());
+ case SyncCommand:
+ return PBHelper.convert(proto.getSyncCommand());
default:
return null;
}
}
-
+
+ private static SyncCommand convert(SyncCommandProto backupCommand) {
+ List<BlockSyncTaskProto> syncTasksProtoList =
+ backupCommand.getSyncTasksList();
+ List<BlockSyncTask> syncTasksList =
+ new ArrayList(syncTasksProtoList.size());
+ for (BlockSyncTaskProto syncTaskProto : syncTasksProtoList) {
+ syncTasksList.add(convertSyncTask(syncTaskProto));
+ }
+
+ return new SyncCommand(DatanodeProtocol.DNA_BACKUP, syncTasksList);
+ }
+
+ private static BlockSyncTask convertSyncTask(
+ BlockSyncTaskProto syncTaskProto) {
+ SyncTaskIdProto syncTaskIdProto = syncTaskProto.getSyncTaskId();
+ UUID syncTaskId = convert(syncTaskIdProto);
+ try {
+ return new BlockSyncTask(syncTaskId,
+ new URI(syncTaskProto.getUri()),
+ PBHelperClient.convertLocatedBlocks(
+ syncTaskProto.getLocatedBlocksList()),
+ syncTaskProto.getPartNumber(),
+ syncTaskProto.getUploadHandle().toByteArray(),
+ syncTaskProto.getOffset(),
+ syncTaskProto.getLength(),
+ syncTaskIdProto.getSyncMountId());
+ } catch (URISyntaxException e) {
+ throw new IllegalArgumentException();
+ }
+ }
+
+ public static UUID convert(SyncTaskIdProto syncTaskIdProto) {
+ byte[] clientId = syncTaskIdProto.getSyncTaskId().toByteArray();
+ long syncTaskIdMsb = ClientId.getMsb(clientId);
+ long syncTaskIdLsb = ClientId.getLsb(clientId);
+ return new UUID(syncTaskIdMsb, syncTaskIdLsb);
+ }
+
+
public static BalancerBandwidthCommandProto convert(
BalancerBandwidthCommand bbCmd) {
return BalancerBandwidthCommandProto.newBuilder()
@@ -603,6 +661,10 @@ public class PBHelper {
.setBlkECReconstructionCmd(
convert((BlockECReconstructionCommand) datanodeCommand));
break;
+ case DatanodeProtocol.DNA_BACKUP:
+ builder.setCmdType(DatanodeCommandProto.Type.SyncCommand)
+ .setSyncCommand(convert((SyncCommand) datanodeCommand));
+ break;
case DatanodeProtocol.DNA_UNKNOWN: //Not expected
default:
builder.setCmdType(DatanodeCommandProto.Type.NullDatanodeCommand);
@@ -1128,4 +1190,130 @@ public class PBHelper {
return new FileRegion(block, providedStorageLocation);
}
+
+ private static SyncCommandProto convert(SyncCommand syncCommand) {
+ SyncCommandProto.Builder builder = SyncCommandProto.newBuilder();
+
+ List<BlockSyncTaskProto> syncTaskProtos = syncCommand.getSyncTasks()
+ .stream()
+ .map(syncTask -> convert(syncTask))
+ .collect(Collectors.toList());
+
+ builder.addAllSyncTasks(syncTaskProtos);
+
+ return builder.build();
+ }
+
+ private static BlockSyncTaskProto convert(BlockSyncTask blockSyncTask) {
+ BlockSyncTaskProto.Builder builder = BlockSyncTaskProto.newBuilder();
+ builder.addAllLocatedBlocks(
+ PBHelperClient.convertLocatedBlocks2(blockSyncTask.getLocatedBlocks()));
+ builder.setUploadHandle(
+ ByteString.copyFrom(blockSyncTask.getUploadHandle()));
+ builder.setPartNumber(blockSyncTask.getPartNumber());
+ builder.setUri(blockSyncTask.getRemoteURI().toString());
+ builder.setOffset(blockSyncTask.getOffset());
+ builder.setLength(blockSyncTask.getLength());
+
+ return builder.build();
+ }
+
+ public static SyncTaskIdProto convert(UUID syncTaskId, String syncMountId) {
+ SyncTaskIdProto.Builder builder = SyncTaskIdProto.newBuilder();
+ ByteBuffer syncTaskIdBytes = ByteBuffer.wrap(new byte[16]);
+ syncTaskIdBytes.putLong(syncTaskId.getMostSignificantBits());
+ syncTaskIdBytes.putLong(syncTaskId.getLeastSignificantBits());
+ builder.setSyncTaskId(ByteString.copyFrom(syncTaskIdBytes.array()));
+ builder.setSyncMountId(syncMountId);
+ return builder.build();
+ }
+
+
+ public static BulkSyncTaskExecutionFeedbackProto convert(
+ BulkSyncTaskExecutionFeedback bulkFeedback) {
+ return BulkSyncTaskExecutionFeedbackProto.newBuilder()
+ .addAllFeedbacks(bulkFeedback.getFeedbacks().stream()
+ .map(f -> convert(f)).collect(Collectors.toList()))
+ .build();
+ }
+
+ public static SyncTaskExecutionFeedbackProto convert(
+ BlockSyncTaskExecutionFeedback feedback) {
+ SyncTaskExecutionFeedbackProto.Builder builder =
+ SyncTaskExecutionFeedbackProto.newBuilder()
+ .setSyncTaskId(
+ convert(feedback.getSyncTaskId(), feedback.getSyncMountId()))
+ .setOutcome(convert(feedback.getOutcome()));
+ if (feedback.getResult() != null) {
+ builder.setResult(convert(feedback.getResult()));
+ }
+ return builder.build();
+ }
+
+ public static SyncTaskExecutionOutcomeProto convert(
+ SyncTaskExecutionOutcome outcome) {
+ switch (outcome) {
+ case FINISHED_SUCCESSFULLY:
+ return SyncTaskExecutionOutcomeProto.FINISHED_SUCCESSFULLY;
+ case EXCEPTION:
+ return SyncTaskExecutionOutcomeProto.EXCEPTION;
+ default:
+ throw new IllegalArgumentException(
+ "Unknown SyncTaskExecutionOutcome: " + outcome);
+ }
+ }
+
+ public static SyncTaskExecutionResultProto convert(
+ SyncTaskExecutionResult result) {
+ SyncTaskExecutionResultProto.Builder builder =
+ SyncTaskExecutionResultProto.newBuilder();
+ if (result.getResult() != null) {
+ builder.setResult(ByteString.copyFrom(result.getResult()));
+ }
+ if (result.getNumberOfBytes() != null) {
+ builder.setNumberOfBytes(result.getNumberOfBytes());
+ }
+ return builder.build();
+ }
+
+ public static BulkSyncTaskExecutionFeedback convertBulkSyncTaskExecutionFeedback(
+ BulkSyncTaskExecutionFeedbackProto bulkSyncTaskExecutionFeedback) {
+ return new BulkSyncTaskExecutionFeedback(
+ bulkSyncTaskExecutionFeedback.getFeedbacksList().stream()
+ .map(feedback -> convert(feedback)).collect(Collectors.toList()));
+ }
+
+ public static BlockSyncTaskExecutionFeedback convert(
+ SyncTaskExecutionFeedbackProto feedback) {
+ return new BlockSyncTaskExecutionFeedback(convert(feedback.getSyncTaskId()),
+ convert(feedback.getOutcome()),
+ feedback.hasResult() ? convert(feedback.getResult()) : null,
+ feedback.getSyncTaskId().getSyncMountId());
+ }
+
+ public static SyncTaskExecutionOutcome convert(
+ SyncTaskExecutionOutcomeProto outcome) {
+ switch (outcome) {
+ case FINISHED_SUCCESSFULLY:
+ return SyncTaskExecutionOutcome.FINISHED_SUCCESSFULLY;
+ case EXCEPTION:
+ return SyncTaskExecutionOutcome.EXCEPTION;
+ default:
+ throw new IllegalArgumentException(
+ "Unknown SyncTaskExecutionOutcomeProto: " + outcome);
+ }
+ }
+
+ public static SyncTaskExecutionResult convert(
+ SyncTaskExecutionResultProto result) {
+ byte[] bytes = null;
+ if (result.getResult() != null) {
+ bytes = result.getResult().toByteArray();
+ }
+
+ ByteBuffer byteBuffer =
+ (bytes == null) ? null : ByteBuffer.wrap(bytes).asReadOnlyBuffer();
+ return new SyncTaskExecutionResult(byteBuffer, result.getNumberOfBytes());
+ }
+
}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java
index 8a3f57b..fd685b8 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java
@@ -1643,7 +1643,9 @@ public class DatanodeManager {
int maxTransfers, int failedVolumes,
VolumeFailureSummary volumeFailureSummary,
@Nonnull SlowPeerReports slowPeers,
- @Nonnull SlowDiskReports slowDisks) throws IOException {
+ @Nonnull SlowDiskReports slowDisks,
+ BulkSyncTaskExecutionFeedback bulkSyncTaskExecutionFeedback)
+ throws IOException {
final DatanodeDescriptor nodeinfo;
try {
nodeinfo = getDatanode(nodeReg);
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPServiceActor.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPServiceActor.java
index f22a586..1dadc41 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPServiceActor.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPServiceActor.java
@@ -50,6 +50,7 @@ import org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolClientSideTranslatorPB;
import org.apache.hadoop.hdfs.server.common.IncorrectVersionException;
import org.apache.hadoop.hdfs.server.namenode.FSNamesystem;
import org.apache.hadoop.hdfs.server.protocol.BlockReportContext;
+import org.apache.hadoop.hdfs.server.protocol.BulkSyncTaskExecutionFeedback;
import org.apache.hadoop.hdfs.server.protocol.DatanodeCommand;
import org.apache.hadoop.hdfs.server.protocol.DatanodeRegistration;
import org.apache.hadoop.hdfs.server.protocol.DatanodeStorage;
@@ -532,6 +533,10 @@ class BPServiceActor implements Runnable {
SlowDiskReports.create(dn.getDiskMetrics().getDiskOutliersStats()) :
SlowDiskReports.EMPTY_REPORT;
+ // TODO - collect feedback from SyncTasks here.
+ BulkSyncTaskExecutionFeedback bulkSyncTaskExecutionFeedback =
+ new BulkSyncTaskExecutionFeedback(Collections.emptyList());
+
HeartbeatResponse response = bpNamenode.sendHeartbeat(bpRegistration,
reports,
dn.getFSDataset().getCacheCapacity(),
@@ -542,13 +547,13 @@ class BPServiceActor implements Runnable {
volumeFailureSummary,
requestBlockReportLease,
slowPeers,
- slowDisks);
+ slowDisks,
+ bulkSyncTaskExecutionFeedback);
if (outliersReportDue) {
// If the report was due and successfully sent, schedule the next one.
scheduler.scheduleNextOutlierReport();
}
-
return response;
}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
index 8659ea4..cc54a96 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
@@ -106,6 +106,7 @@ import org.apache.hadoop.hdfs.protocol.SnapshotDiffReportListing;
import org.apache.hadoop.hdfs.protocol.SnapshotDiffReport;
import org.apache.hadoop.hdfs.server.common.ECTopologyVerifier;
import org.apache.hadoop.hdfs.server.namenode.metrics.ReplicatedBlocksMBean;
+import org.apache.hadoop.hdfs.server.protocol.BulkSyncTaskExecutionFeedback;
import org.apache.hadoop.hdfs.server.protocol.SlowDiskReports;
import org.apache.hadoop.util.Time;
import static org.apache.hadoop.util.Time.now;
@@ -3959,8 +3960,9 @@ public class FSNamesystem implements Namesystem, FSNamesystemMBean,
VolumeFailureSummary volumeFailureSummary,
boolean requestFullBlockReportLease,
@Nonnull SlowPeerReports slowPeers,
- @Nonnull SlowDiskReports slowDisks)
- throws IOException {
+ @Nonnull SlowDiskReports slowDisks,
+ BulkSyncTaskExecutionFeedback bulkSyncTaskExecutionFeedback)
+ throws IOException {
readLock();
try {
//get datanode commands
@@ -3969,7 +3971,7 @@ public class FSNamesystem implements Namesystem, FSNamesystemMBean,
DatanodeCommand[] cmds = blockManager.getDatanodeManager().handleHeartbeat(
nodeReg, reports, getBlockPoolId(), cacheCapacity, cacheUsed,
xceiverCount, maxTransfer, failedVolumes, volumeFailureSummary,
- slowPeers, slowDisks);
+ slowPeers, slowDisks, bulkSyncTaskExecutionFeedback);
long blockReportLeaseId = 0;
if (requestFullBlockReportLease) {
blockReportLeaseId = blockManager.requestBlockReportLeaseId(nodeReg);
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
index f50648d..4be26c4 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
@@ -164,6 +164,7 @@ import org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics;
import org.apache.hadoop.hdfs.server.namenode.sps.StoragePolicySatisfyManager;
import org.apache.hadoop.hdfs.server.protocol.BlockReportContext;
import org.apache.hadoop.hdfs.server.protocol.BlocksWithLocations;
+import org.apache.hadoop.hdfs.server.protocol.BulkSyncTaskExecutionFeedback;
import org.apache.hadoop.hdfs.server.protocol.DatanodeCommand;
import org.apache.hadoop.hdfs.server.protocol.DatanodeProtocol;
import org.apache.hadoop.hdfs.server.protocol.DatanodeRegistration;
@@ -1569,14 +1570,15 @@ public class NameNodeRpcServer implements NamenodeProtocols {
int failedVolumes, VolumeFailureSummary volumeFailureSummary,
boolean requestFullBlockReportLease,
@Nonnull SlowPeerReports slowPeers,
- @Nonnull SlowDiskReports slowDisks)
- throws IOException {
+ @Nonnull SlowDiskReports slowDisks,
+ BulkSyncTaskExecutionFeedback bulkSyncTaskExecutionFeedback)
+ throws IOException {
checkNNStartup();
verifyRequest(nodeReg);
return namesystem.handleHeartbeat(nodeReg, report,
dnCacheCapacity, dnCacheUsed, xceiverCount, xmitsInProgress,
failedVolumes, volumeFailureSummary, requestFullBlockReportLease,
- slowPeers, slowDisks);
+ slowPeers, slowDisks, bulkSyncTaskExecutionFeedback);
}
@Override // DatanodeProtocol
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/BlockSyncTask.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/BlockSyncTask.java
new file mode 100644
index 0000000..875a409
--- /dev/null
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/BlockSyncTask.java
@@ -0,0 +1,83 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdfs.server.protocol;
+import org.apache.hadoop.hdfs.protocol.LocatedBlock;
+import java.net.URI;
+import java.util.List;
+import java.util.UUID;
+
+/**
+ * A BlockSyncTask is an operation that is sent to the datanodes to copy
+ * blocks to an external storage endpoint as a part of an orchestrated
+ * synchronization across multiple datanodes.
+ * BlockSyncTask is intended to be an immutable POJO.
+ */
+public class BlockSyncTask {
+ private final UUID syncTaskId;
+ private final URI remoteURI;
+ private final List<LocatedBlock> locatedBlocks;
+ private String syncMountId;
+ private final int partNumber;
+ private byte[] uploadHandle;
+ private final int offset;
+ private final long length;
+
+ public BlockSyncTask(UUID syncTaskId, URI remoteURI,
+ List<LocatedBlock> locatedBlocks, Integer partNumber, byte[] uploadHandle,
+ int offset, long length, String syncMountId) {
+ this.syncTaskId = syncTaskId;
+ this.remoteURI = remoteURI;
+ this.locatedBlocks = locatedBlocks;
+ this.syncMountId = syncMountId;
+ this.partNumber = partNumber;
+ this.uploadHandle = uploadHandle;
+ this.offset = offset;
+ this.length = length;
+ }
+
+ public int getPartNumber() {
+ return partNumber;
+ }
+
+ public byte[] getUploadHandle() {
+ return uploadHandle;
+ }
+
+ public int getOffset() {
+ return offset;
+ }
+
+ public long getLength() {
+ return length;
+ }
+
+ public UUID getSyncTaskId() {
+ return syncTaskId;
+ }
+
+ public URI getRemoteURI() {
+ return remoteURI;
+ }
+
+ public List<LocatedBlock> getLocatedBlocks() {
+ return locatedBlocks;
+ }
+
+ public String getSyncMountId() {
+ return syncMountId;
+ }
+}
\ No newline at end of file
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/BulkSyncTaskExecutionFeedback.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/BulkSyncTaskExecutionFeedback.java
new file mode 100644
index 0000000..0d459e8
--- /dev/null
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/BulkSyncTaskExecutionFeedback.java
@@ -0,0 +1,36 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdfs.server.protocol;
+
+import java.util.Collection;
+
+/**
+ * Feedback for a collection of {@link BlockSyncTask}s.
+ */
+public class BulkSyncTaskExecutionFeedback {
+
+ private Collection<BlockSyncTaskExecutionFeedback> feedbacks;
+
+ public BulkSyncTaskExecutionFeedback(
+ Collection<BlockSyncTaskExecutionFeedback> feedbacks) {
+ this.feedbacks = feedbacks;
+ }
+
+ public Collection<BlockSyncTaskExecutionFeedback> getFeedbacks() {
+ return feedbacks;
+ }
+}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java
index 5680ef3..692a2e8 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java
@@ -81,6 +81,7 @@ public interface DatanodeProtocol {
final static int DNA_ERASURE_CODING_RECONSTRUCTION = 11; // erasure coding reconstruction command
int DNA_BLOCK_STORAGE_MOVEMENT = 12; // block storage movement command
int DNA_DROP_SPS_WORK_COMMAND = 13; // drop sps work command
+ final static int DNA_BACKUP = 14; // back up data to PROVIDED stores.
/**
* Register Datanode.
@@ -112,20 +113,19 @@ public interface DatanodeProtocol {
* @param slowPeers Details of peer DataNodes that were detected as being
* slow to respond to packet writes. Empty report if no
* slow peers were detected by the DataNode.
+ * @param bulkSyncTaskExecutionFeedback Result of the execution of the
+ * sync tasks.
* @throws IOException on error
*/
@Idempotent
public HeartbeatResponse sendHeartbeat(DatanodeRegistration registration,
- StorageReport[] reports,
- long dnCacheCapacity,
- long dnCacheUsed,
- int xmitsInProgress,
- int xceiverCount,
- int failedVolumes,
- VolumeFailureSummary volumeFailureSummary,
- boolean requestFullBlockReportLease,
- @Nonnull SlowPeerReports slowPeers,
- @Nonnull SlowDiskReports slowDisks)
+ StorageReport[] reports, long cacheCapacity, long cacheUsed,
+ int xmitsInProgress, int xceiverCount, int failedVolumes,
+ VolumeFailureSummary volumeFailureSummary,
+ boolean requestFullBlockReportLease,
+ @Nonnull SlowPeerReports slowPeers,
+ @Nonnull SlowDiskReports slowDisks,
+ BulkSyncTaskExecutionFeedback bulkSyncTaskExecutionFeedback)
throws IOException;
/**
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/SyncCommand.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/SyncCommand.java
new file mode 100644
index 0000000..7e2c242
--- /dev/null
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/SyncCommand.java
@@ -0,0 +1,39 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdfs.server.protocol;
+
+import java.util.List;
+
+/**
+ * A SyncCommand is an instruction to a DataNode to move the
+ * give file to specified target DataNodes provided storage.
+ */
+public class SyncCommand extends DatanodeCommand {
+
+ private final List<BlockSyncTask> syncTasks;
+
+ public SyncCommand(int action, List<BlockSyncTask> syncTasks) {
+ super(action);
+ this.syncTasks = syncTasks;
+ }
+
+ public List<BlockSyncTask> getSyncTasks() {
+ return syncTasks;
+ }
+
+}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/proto/DatanodeProtocol.proto b/hadoop-hdfs-project/hadoop-hdfs/src/main/proto/DatanodeProtocol.proto
index 4a8f9f0..f4e3481 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/proto/DatanodeProtocol.proto
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/proto/DatanodeProtocol.proto
@@ -60,6 +60,7 @@ message DatanodeCommandProto {
NullDatanodeCommand = 7;
BlockIdCommand = 8;
BlockECReconstructionCommand = 9;
+ SyncCommand = 10;
}
required Type cmdType = 1; // Type of the command
@@ -74,6 +75,7 @@ message DatanodeCommandProto {
optional RegisterCommandProto registerCmd = 7;
optional BlockIdCommandProto blkIdCmd = 8;
optional BlockECReconstructionCommandProto blkECReconstructionCmd = 9;
+ optional SyncCommandProto syncCommand = 10;
}
/**
@@ -154,6 +156,89 @@ message BlockECReconstructionCommandProto {
repeated BlockECReconstructionInfoProto blockECReconstructioninfo = 1;
}
+message SyncTaskIdProto {
+ required bytes syncTaskId = 1;
+ required string syncMountId = 2;
+}
+
+/**
+ * Instruct the datanode to perform a backup command
+ */
+message SyncCommandProto {
+ repeated BlockSyncTaskProto syncTasks = 1;
+}
+
+/**
+ * A block synchronization task as part of an orchestrated synchronization
+ * across potentially multiple datanodes (i.e. multipart put part).
+ */
+message BlockSyncTaskProto {
+ required SyncTaskIdProto syncTaskId = 1;
+
+ required bytes uploadHandle = 2;
+ required int32 partNumber = 3;
+ repeated LocatedBlockProto locatedBlocks = 4;
+ required string uri = 5;
+ required int32 offset = 6;
+ required int64 length = 7;
+}
+
+/**
+ * Block storage movement information
+ */
+message BlockMovingInfoProto {
+ required BlockProto block = 1;
+ required DatanodeInfoProto sourceDnInfo = 2;
+ required DatanodeInfoProto targetDnInfo = 3;
+ required StorageTypeProto sourceStorageType = 4;
+ required StorageTypeProto targetStorageType = 5;
+}
+
+/**
+ * Blocks for which storage movements has been attempted and finished
+ * with either success or failure.
+ */
+message BlocksStorageMoveAttemptFinishedProto {
+ repeated BlockProto blocks = 1;
+}
+
+/**
+ * A collection of feedbacks for a collection of sync tasks.
+ */
+message BulkSyncTaskExecutionFeedbackProto {
+ repeated SyncTaskExecutionFeedbackProto feedbacks = 1;
+}
+
+/**
+ * Feedback for a sync task that has been executed.
+ * syncTaskId - identifier for the task.
+ * outcome - success/error.
+ * operation - the type of operation.
+ * result - if the outcome is successful, the results of the sync task.
+ */
+message SyncTaskExecutionFeedbackProto {
+ required SyncTaskIdProto syncTaskId = 1;
+ required SyncTaskExecutionOutcomeProto outcome = 2;
+ optional SyncTaskExecutionResultProto result = 3;
+}
+
+/**
+ * Success of failure indication of a sync task.
+ */
+enum SyncTaskExecutionOutcomeProto {
+ FINISHED_SUCCESSFULLY = 0;
+ EXCEPTION = 1;
+}
+
+/**
+ * result - the opaque result data from the sync task.
+ * numberOfBytes - the number of bytes copied.
+ */
+message SyncTaskExecutionResultProto {
+ optional bytes result = 1;
+ optional int64 numberOfBytes = 2;
+}
+
/**
* registration - Information of the datanode registering with the namenode
*/
@@ -211,6 +296,7 @@ message HeartbeatRequestProto {
optional bool requestFullBlockReportLease = 9 [ default = false ];
repeated SlowPeerReportProto slowPeers = 10;
repeated SlowDiskReportProto slowDisks = 11;
+ optional BulkSyncTaskExecutionFeedbackProto bulkSyncTaskExecutionFeedback = 12;
}
/**
@@ -276,7 +362,7 @@ message StorageBlockReportProto {
*/
message BlockReportResponseProto {
optional DatanodeCommandProto cmd = 1;
-}
+}
/**
* registration - datanode registration information
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestDatanodeManager.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestDatanodeManager.java
index aa7f4d2..bd610bd 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestDatanodeManager.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestDatanodeManager.java
@@ -590,7 +590,7 @@ public class TestDatanodeManager {
Mockito.when(dm.getDatanode(dnReg)).thenReturn(nodeInfo);
DatanodeCommand[] cmds = dm.handleHeartbeat(
dnReg, new StorageReport[1], "bp-123", 0, 0, 10, maxTransfers, 0, null,
- SlowPeerReports.EMPTY_REPORT, SlowDiskReports.EMPTY_REPORT);
+ SlowPeerReports.EMPTY_REPORT, SlowDiskReports.EMPTY_REPORT, null);
long expectedNumCmds = Arrays.stream(
new int[]{numReplicationTasks, numECTasks})
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestNameNodePrunesMissingStorages.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestNameNodePrunesMissingStorages.java
index f64767a..d8e53ae 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestNameNodePrunesMissingStorages.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestNameNodePrunesMissingStorages.java
@@ -117,7 +117,7 @@ public class TestNameNodePrunesMissingStorages {
cluster.stopDataNode(0);
cluster.getNameNodeRpc().sendHeartbeat(dnReg, prunedReports, 0L, 0L, 0, 0,
0, null, true, SlowPeerReports.EMPTY_REPORT,
- SlowDiskReports.EMPTY_REPORT);
+ SlowDiskReports.EMPTY_REPORT, null);
// Check that the missing storage was pruned.
assertThat(dnDescriptor.getStorageInfos().length, is(expectedStoragesAfterTest));
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/InternalDataNodeTestUtils.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/InternalDataNodeTestUtils.java
index 97302b5..5586f89 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/InternalDataNodeTestUtils.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/InternalDataNodeTestUtils.java
@@ -163,6 +163,7 @@ public class InternalDataNodeTestUtils {
Mockito.anyInt(), Mockito.any(),
Mockito.anyBoolean(),
Mockito.any(),
+ Mockito.any(),
Mockito.any())).thenReturn(
new HeartbeatResponse(new DatanodeCommand[0], new NNHAStatusHeartbeat(
HAServiceState.ACTIVE, 1), null, ThreadLocalRandom.current()
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBPOfferService.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBPOfferService.java
index 87bbd73..37f1a95 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBPOfferService.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBPOfferService.java
@@ -18,6 +18,8 @@
package org.apache.hadoop.hdfs.server.datanode;
import static org.apache.hadoop.hdfs.DFSConfigKeys.DFS_DATANODE_DATA_DIR_KEY;
+
+import org.apache.hadoop.hdfs.server.protocol.BulkSyncTaskExecutionFeedback;
import org.apache.hadoop.hdfs.server.protocol.SlowDiskReports;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNull;
@@ -155,7 +157,8 @@ public class TestBPOfferService {
Mockito.any(VolumeFailureSummary.class),
Mockito.anyBoolean(),
Mockito.any(SlowPeerReports.class),
- Mockito.any(SlowDiskReports.class));
+ Mockito.any(SlowDiskReports.class),
+ Mockito.any(BulkSyncTaskExecutionFeedback.class));
mockHaStatuses[nnIdx] = new NNHAStatusHeartbeat(HAServiceState.STANDBY, 0);
datanodeCommands[nnIdx] = new DatanodeCommand[0];
return mock;
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockRecovery.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockRecovery.java
index f12285c..fb8768d 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockRecovery.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockRecovery.java
@@ -228,6 +228,7 @@ public class TestBlockRecovery {
Mockito.any(),
Mockito.anyBoolean(),
Mockito.any(),
+ Mockito.any(),
Mockito.any()))
.thenReturn(new HeartbeatResponse(
new DatanodeCommand[0],
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeLifeline.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeLifeline.java
index dcfbf02..7c62edc 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeLifeline.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeLifeline.java
@@ -25,13 +25,14 @@ import static org.apache.hadoop.hdfs.DFSConfigKeys.DFS_NAMENODE_HEARTBEAT_RECHEC
import static org.apache.hadoop.hdfs.DFSConfigKeys.DFS_NAMENODE_LIFELINE_RPC_ADDRESS_KEY;
import static org.apache.hadoop.hdfs.DFSConfigKeys.DFS_NAMENODE_STALE_DATANODE_INTERVAL_KEY;
+import org.apache.hadoop.hdfs.server.protocol.BulkSyncTaskExecutionFeedback;
import org.apache.hadoop.hdfs.server.protocol.SlowDiskReports;
import static org.apache.hadoop.test.MetricsAsserts.getLongCounter;
import static org.apache.hadoop.test.MetricsAsserts.getMetrics;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNotNull;
import static org.junit.Assert.assertTrue;
-import static org.mockito.Mockito.any;
+import static org.mockito.Matchers.any;
import static org.mockito.Mockito.anyBoolean;
import static org.mockito.Mockito.anyInt;
import static org.mockito.Mockito.anyLong;
@@ -172,7 +173,8 @@ public class TestDataNodeLifeline {
any(),
anyBoolean(),
any(SlowPeerReports.class),
- any(SlowDiskReports.class));
+ any(SlowDiskReports.class),
+ any(BulkSyncTaskExecutionFeedback.class));
// Intercept lifeline to trigger latch count-down on each call.
doAnswer(new LatchCountingAnswer<Void>(lifelinesSent))
@@ -241,7 +243,8 @@ public class TestDataNodeLifeline {
any(),
anyBoolean(),
any(SlowPeerReports.class),
- any(SlowDiskReports.class));
+ any(SlowDiskReports.class),
+ any(BulkSyncTaskExecutionFeedback.class));
// While waiting on the latch for the expected number of heartbeat messages,
// poll DataNode tracking information. We expect that the DataNode always
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDatanodeProtocolRetryPolicy.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDatanodeProtocolRetryPolicy.java
index 93f9f44..bb4c125 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDatanodeProtocolRetryPolicy.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDatanodeProtocolRetryPolicy.java
@@ -216,6 +216,7 @@ public class TestDatanodeProtocolRetryPolicy {
Mockito.any(),
Mockito.anyBoolean(),
Mockito.any(),
+ Mockito.any(),
Mockito.any());
dn = new DataNode(conf, locations, null, null) {
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestFsDatasetCache.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestFsDatasetCache.java
index eccc237..c00aea0 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestFsDatasetCache.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestFsDatasetCache.java
@@ -18,6 +18,7 @@
package org.apache.hadoop.hdfs.server.datanode;
import net.jcip.annotations.NotThreadSafe;
+import org.apache.hadoop.hdfs.server.protocol.BulkSyncTaskExecutionFeedback;
import org.apache.hadoop.hdfs.server.protocol.SlowDiskReports;
import static org.apache.hadoop.test.MetricsAsserts.getMetrics;
import static org.junit.Assert.assertEquals;
@@ -208,7 +209,8 @@ public class TestFsDatasetCache {
(StorageReport[]) any(), anyLong(), anyLong(),
anyInt(), anyInt(), anyInt(), (VolumeFailureSummary) any(),
anyBoolean(), any(SlowPeerReports.class),
- any(SlowDiskReports.class));
+ any(SlowDiskReports.class),
+ any(BulkSyncTaskExecutionFeedback.class));
} finally {
lock.writeLock().unlock();
}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestStorageReport.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestStorageReport.java
index 9df6209..875cef7 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestStorageReport.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestStorageReport.java
@@ -29,6 +29,7 @@ import org.apache.hadoop.hdfs.HdfsConfiguration;
import org.apache.hadoop.hdfs.MiniDFSCluster;
import org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolClientSideTranslatorPB;
import org.apache.hadoop.hdfs.server.namenode.NameNode;
+import org.apache.hadoop.hdfs.server.protocol.BulkSyncTaskExecutionFeedback;
import org.apache.hadoop.hdfs.server.protocol.DatanodeRegistration;
import org.apache.hadoop.hdfs.server.protocol.DatanodeStorage;
import org.apache.hadoop.hdfs.server.protocol.SlowDiskReports;
@@ -110,7 +111,8 @@ public class TestStorageReport {
anyLong(), anyLong(), anyInt(), anyInt(), anyInt(),
any(), Mockito.anyBoolean(),
Mockito.any(SlowPeerReports.class),
- Mockito.any(SlowDiskReports.class));
+ Mockito.any(SlowDiskReports.class),
+ Mockito.any(BulkSyncTaskExecutionFeedback.class));
StorageReport[] reports = captor.getValue();
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java
index e5d9826..9a121a1 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java
@@ -959,8 +959,8 @@ public class NNThroughputBenchmark implements Tool {
DF_CAPACITY, DF_USED, DF_CAPACITY - DF_USED, DF_USED, 0L) };
DatanodeCommand[] cmds = dataNodeProto.sendHeartbeat(dnRegistration, rep,
0L, 0L, 0, 0, 0, null, true,
- SlowPeerReports.EMPTY_REPORT, SlowDiskReports.EMPTY_REPORT)
- .getCommands();
+ SlowPeerReports.EMPTY_REPORT, SlowDiskReports.EMPTY_REPORT,
+ null).getCommands();
if(cmds != null) {
for (DatanodeCommand cmd : cmds ) {
if(LOG.isDebugEnabled()) {
@@ -1010,8 +1010,8 @@ public class NNThroughputBenchmark implements Tool {
false, DF_CAPACITY, DF_USED, DF_CAPACITY - DF_USED, DF_USED, 0) };
DatanodeCommand[] cmds = dataNodeProto.sendHeartbeat(dnRegistration,
rep, 0L, 0L, 0, 0, 0, null, true,
- SlowPeerReports.EMPTY_REPORT, SlowDiskReports.EMPTY_REPORT)
- .getCommands();
+ SlowPeerReports.EMPTY_REPORT, SlowDiskReports.EMPTY_REPORT,
+ null).getCommands();
if (cmds != null) {
for (DatanodeCommand cmd : cmds) {
if (cmd.getAction() == DatanodeProtocol.DNA_TRANSFER) {
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NameNodeAdapter.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NameNodeAdapter.java
index ebd5faf..9d3e14a 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NameNodeAdapter.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NameNodeAdapter.java
@@ -19,11 +19,13 @@ package org.apache.hadoop.hdfs.server.namenode;
import org.apache.hadoop.ha.HAServiceProtocol.HAServiceState;
import org.apache.hadoop.hdfs.server.blockmanagement.BlockManager;
+import org.apache.hadoop.hdfs.server.protocol.BulkSyncTaskExecutionFeedback;
import org.apache.hadoop.hdfs.server.protocol.SlowDiskReports;
import static org.mockito.Mockito.spy;
import java.io.File;
import java.io.IOException;
+import java.util.ArrayList;
import java.util.concurrent.locks.ReentrantReadWriteLock;
import org.apache.commons.lang3.reflect.FieldUtils;
@@ -132,7 +134,8 @@ public class NameNodeAdapter {
return namesystem.handleHeartbeat(nodeReg,
BlockManagerTestUtil.getStorageReportsForDatanode(dd),
dd.getCacheCapacity(), dd.getCacheRemaining(), 0, 0, 0, null, true,
- SlowPeerReports.EMPTY_REPORT, SlowDiskReports.EMPTY_REPORT);
+ SlowPeerReports.EMPTY_REPORT, SlowDiskReports.EMPTY_REPORT,
+ new BulkSyncTaskExecutionFeedback(new ArrayList<>()));
}
public static boolean setReplication(final FSNamesystem ns,
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java
index 46b8db1..6212292 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java
@@ -140,8 +140,8 @@ public class TestDeadDatanode {
false, 0, 0, 0, 0, 0) };
DatanodeCommand[] cmd =
dnp.sendHeartbeat(reg, rep, 0L, 0L, 0, 0, 0, null, true,
- SlowPeerReports.EMPTY_REPORT, SlowDiskReports.EMPTY_REPORT)
- .getCommands();
+ SlowPeerReports.EMPTY_REPORT, SlowDiskReports.EMPTY_REPORT,
+ null).getCommands();
assertEquals(1, cmd.length);
assertEquals(cmd[0].getAction(), RegisterCommand.REGISTER
.getAction());
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org
[hadoop] 02/03: HDFS-13421. [PROVIDED Phase 2] Implement DNA_BACKUP
command in Datanode. Contributed by Ewan Higgs.
Posted by eh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ehiggs pushed a commit to branch HDFS-12090
in repository https://gitbox.apache.org/repos/asf/hadoop.git
commit f482c677d299db81e90e27ecd3685622627a190e
Author: Virajith Jalaparti <vi...@apache.org>
AuthorDate: Wed Aug 1 12:13:31 2018 -0700
HDFS-13421. [PROVIDED Phase 2] Implement DNA_BACKUP command in Datanode. Contributed by Ewan Higgs.
---
.../org/apache/hadoop/hdfs/BlockInputStream.java | 52 +++++++++
.../hdfs/server/datanode/BPOfferService.java | 6 +
.../hadoop/hdfs/server/datanode/DataNode.java | 18 +++
.../SyncServiceSatisfierDatanodeWorker.java | 97 ++++++++++++++++
.../SyncTaskExecutionFeedbackCollector.java | 54 +++++++++
.../executor/BlockSyncOperationExecutor.java | 122 +++++++++++++++++++++
.../executor/BlockSyncReaderFactory.java | 92 ++++++++++++++++
.../syncservice/executor/BlockSyncTaskRunner.java | 69 ++++++++++++
.../apache/hadoop/hdfs/TestBlockInputStream.java | 84 ++++++++++++++
.../executor/TestBlockSyncOperationExecutor.java | 94 ++++++++++++++++
10 files changed, 688 insertions(+)
diff --git a/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/BlockInputStream.java b/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/BlockInputStream.java
new file mode 100644
index 0000000..152f83e
--- /dev/null
+++ b/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/BlockInputStream.java
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdfs;
+
+import java.io.IOException;
+import java.io.InputStream;
+
+/**
+ * Facade around BlockReader that indeed implements the InputStream interface.
+ */
+public class BlockInputStream extends InputStream {
+ private final BlockReader blockReader;
+
+ public BlockInputStream(BlockReader blockReader) {
+ this.blockReader = blockReader;
+ }
+
+ @Override
+ public int read() throws IOException {
+ byte[] b = new byte[1];
+ int c = blockReader.read(b, 0, b.length);
+ if (c > 0) {
+ return b[0];
+ } else {
+ return -1;
+ }
+ }
+
+ @Override
+ public int read(byte b[], int off, int len) throws IOException {
+ return blockReader.read(b, off, len);
+ }
+
+ @Override
+ public long skip(long n) throws IOException {
+ return blockReader.skip(n);
+ }
+}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPOfferService.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPOfferService.java
index 3233e2c..1d63fa9 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPOfferService.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPOfferService.java
@@ -797,6 +797,12 @@ class BPOfferService {
((BlockECReconstructionCommand) cmd).getECTasks();
dn.getErasureCodingWorker().processErasureCodingTasks(ecTasks);
break;
+ case DatanodeProtocol.DNA_BACKUP:
+ LOG.info("DatanodeCommand action: DNA_BACKUP");
+ Collection<BlockSyncTask> backupTasks =
+ ((SyncCommand) cmd).getSyncTasks();
+ dn.getSyncServiceSatisfierDatanodeWorker().processSyncTasks(backupTasks);
+ break;
default:
LOG.warn("Unknown DatanodeCommand action: " + cmd.getAction());
}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
index e926b6a..11fb2f0 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
@@ -388,6 +388,7 @@ public class DataNode extends ReconfigurableBase
private String dnUserName = null;
private BlockRecoveryWorker blockRecoveryWorker;
private ErasureCodingWorker ecWorker;
+ private SyncServiceSatisfierDatanodeWorker syncServiceSatisfierDatanodeWorker;
private final Tracer tracer;
private final TracerConfigurationManager tracerConfigurationManager;
private static final int NUM_CORES = Runtime.getRuntime()
@@ -1436,6 +1437,9 @@ public class DataNode extends ReconfigurableBase
ecWorker = new ErasureCodingWorker(getConf(), this);
blockRecoveryWorker = new BlockRecoveryWorker(this);
+ syncServiceSatisfierDatanodeWorker =
+ new SyncServiceSatisfierDatanodeWorker(getConf(), this);
+ syncServiceSatisfierDatanodeWorker.start();
blockPoolManager = new BlockPoolManager(this);
blockPoolManager.refreshNamenodes(getConf());
@@ -1988,6 +1992,11 @@ public class DataNode extends ReconfigurableBase
}
}
+ // stop syncServiceSatisfierDatanodeWorker
+ if (syncServiceSatisfierDatanodeWorker != null) {
+ syncServiceSatisfierDatanodeWorker.stop();
+ }
+
List<BPOfferService> bposArray = (this.blockPoolManager == null)
? new ArrayList<BPOfferService>()
: this.blockPoolManager.getAllNamenodeThreads();
@@ -2140,6 +2149,11 @@ public class DataNode extends ReconfigurableBase
notifyAll();
}
tracer.close();
+
+ // Waiting to finish backup SPS worker thread.
+ if (syncServiceSatisfierDatanodeWorker != null) {
+ syncServiceSatisfierDatanodeWorker.waitToFinishWorkerThread();
+ }
}
/**
@@ -3633,4 +3647,8 @@ public class DataNode extends ReconfigurableBase
}
return this.diskBalancer;
}
+
+ public SyncServiceSatisfierDatanodeWorker getSyncServiceSatisfierDatanodeWorker() {
+ return syncServiceSatisfierDatanodeWorker;
+ }
}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/SyncServiceSatisfierDatanodeWorker.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/SyncServiceSatisfierDatanodeWorker.java
new file mode 100644
index 0000000..7216e8f
--- /dev/null
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/SyncServiceSatisfierDatanodeWorker.java
@@ -0,0 +1,97 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdfs.server.datanode;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hdfs.server.datanode.syncservice.SyncTaskExecutionFeedbackCollector;
+import org.apache.hadoop.hdfs.server.datanode.syncservice.executor.BlockSyncOperationExecutor;
+import org.apache.hadoop.hdfs.server.datanode.syncservice.executor.BlockSyncReaderFactory;
+import org.apache.hadoop.hdfs.server.datanode.syncservice.executor.BlockSyncTaskRunner;
+import org.apache.hadoop.hdfs.server.protocol.BlockSyncTask;
+import org.apache.hadoop.util.concurrent.HadoopExecutors;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.Collection;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.RejectedExecutionException;
+import java.util.concurrent.TimeUnit;
+
+public class SyncServiceSatisfierDatanodeWorker {
+ private static final Logger LOG = LoggerFactory
+ .getLogger(SyncServiceSatisfierDatanodeWorker.class);
+
+ private ExecutorService executorService;
+ private BlockSyncOperationExecutor syncOperationExecutor;
+ private SyncTaskExecutionFeedbackCollector syncTaskExecutionFeedbackCollector;
+
+ public SyncServiceSatisfierDatanodeWorker(Configuration conf, DataNode dataNode) throws IOException {
+ this.executorService = HadoopExecutors.newFixedThreadPool(4);
+ this.syncOperationExecutor =
+ BlockSyncOperationExecutor.createOnDataNode(conf,
+ (locatedBlock, config) -> {
+ try {
+ return BlockSyncReaderFactory.createBlockReader(dataNode, locatedBlock, config);
+ } catch (IOException e) {
+ throw new RuntimeException(e);
+ }
+ }
+ );
+ this.syncTaskExecutionFeedbackCollector = new SyncTaskExecutionFeedbackCollector();
+ }
+
+
+ public void start() {
+ this.executorService = HadoopExecutors.newFixedThreadPool(4);
+ }
+
+ public void stop() {
+ this.executorService.shutdown();
+ }
+
+ public void waitToFinishWorkerThread() {
+ try {
+ this.executorService.awaitTermination(3, TimeUnit.MINUTES);
+ } catch (InterruptedException e) {
+ LOG.warn("SyncServiceSatisfierDatanodeWorker interrupted during waiting for finalization.");
+ Thread.currentThread().interrupt();
+ }
+ }
+
+ public void processSyncTasks(Collection<BlockSyncTask> blockSyncTasks) {
+
+ LOG.debug("Received SyncTasks: {}", blockSyncTasks);
+ for (BlockSyncTask blockSyncTask : blockSyncTasks) {
+ try {
+ executorService.submit(new BlockSyncTaskRunner(blockSyncTask,
+ syncOperationExecutor,
+ syncTaskExecutionFeedback -> syncTaskExecutionFeedbackCollector
+ .addFeedback(syncTaskExecutionFeedback)));
+ } catch (RejectedExecutionException e) {
+ LOG.warn("BlockSyncTask {} for {} was rejected: {}",
+ blockSyncTask.getSyncTaskId(), blockSyncTask.getRemoteURI(),
+ e.getCause());
+ }
+ }
+ }
+
+ public SyncTaskExecutionFeedbackCollector getSyncTaskExecutionFeedbackCollector() {
+ return syncTaskExecutionFeedbackCollector;
+ }
+
+}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/syncservice/SyncTaskExecutionFeedbackCollector.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/syncservice/SyncTaskExecutionFeedbackCollector.java
new file mode 100644
index 0000000..41cd441
--- /dev/null
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/syncservice/SyncTaskExecutionFeedbackCollector.java
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdfs.server.datanode.syncservice;
+
+import com.google.common.collect.Lists;
+import org.apache.hadoop.hdfs.server.protocol.BlockSyncTaskExecutionFeedback;
+import org.apache.hadoop.hdfs.server.protocol.BulkSyncTaskExecutionFeedback;
+
+import java.util.List;
+
+/**
+ * DatanodeSyncTaskExecutionFeedbackCollector collects feedback for the
+ * sync service tracker to determine what has happened and report statistics.
+ */
+public class SyncTaskExecutionFeedbackCollector {
+
+ private List<BlockSyncTaskExecutionFeedback> collectedFeedback;
+
+ public SyncTaskExecutionFeedbackCollector() {
+ this.collectedFeedback = Lists.newArrayList();
+ }
+
+ public void addFeedback(BlockSyncTaskExecutionFeedback feedback) {
+ synchronized (this) {
+ collectedFeedback.add(feedback);
+ }
+ }
+
+ public BulkSyncTaskExecutionFeedback packageFeedbackForHeartbeat() {
+
+ List<BlockSyncTaskExecutionFeedback> feedbackForHeartbeat;
+
+ synchronized (this) {
+ feedbackForHeartbeat = collectedFeedback;
+ collectedFeedback = Lists.newArrayList();
+ }
+ return new BulkSyncTaskExecutionFeedback(feedbackForHeartbeat);
+
+ }
+}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/syncservice/executor/BlockSyncOperationExecutor.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/syncservice/executor/BlockSyncOperationExecutor.java
new file mode 100644
index 0000000..7fde230
--- /dev/null
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/syncservice/executor/BlockSyncOperationExecutor.java
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdfs.server.datanode.syncservice.executor;
+
+import com.google.common.annotations.VisibleForTesting;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.BBUploadHandle;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.MultipartUploader;
+import org.apache.hadoop.fs.MultipartUploaderFactory;
+import org.apache.hadoop.fs.PartHandle;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hdfs.BlockInputStream;
+import org.apache.hadoop.hdfs.BlockReader;
+import org.apache.hadoop.hdfs.DFSInputStream;
+import org.apache.hadoop.hdfs.protocol.LocatedBlock;
+import org.apache.hadoop.hdfs.server.protocol.BlockSyncTask;
+import org.apache.hadoop.hdfs.server.protocol.SyncTaskExecutionResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.ByteArrayInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.SequenceInputStream;
+import java.net.URI;
+import java.nio.ByteBuffer;
+import java.util.function.BiFunction;
+import java.util.function.Function;
+import java.util.Enumeration;
+import java.util.List;
+import java.util.Vector;
+
+/**
+ * BlockSyncOperationExecutor writes the blocks to the sync service remote
+ * endpoint.
+ */
+public class BlockSyncOperationExecutor {
+
+ public static final Logger LOG =
+ LoggerFactory.getLogger(BlockSyncOperationExecutor.class);
+
+ private Configuration conf;
+ private BiFunction<LocatedBlock, Configuration, BlockReader>
+ createBlockReader;
+ private Function<FileSystem, MultipartUploader> multipartUploaderSupplier;
+
+ @VisibleForTesting
+ BlockSyncOperationExecutor(Configuration conf,
+ BiFunction<LocatedBlock, Configuration, BlockReader> createBlockReader,
+ Function<FileSystem, MultipartUploader> multipartUploaderSupplier) {
+ this.conf = conf;
+ this.createBlockReader = createBlockReader;
+ this.multipartUploaderSupplier = multipartUploaderSupplier;
+ }
+
+ public static BlockSyncOperationExecutor createOnDataNode(Configuration conf,
+ BiFunction<LocatedBlock, Configuration, BlockReader> createBlockReader) {
+ return new BlockSyncOperationExecutor(conf,
+ createBlockReader,
+ fs -> {
+ try {
+ return MultipartUploaderFactory.get(fs, conf);
+ } catch (IOException e) {
+ throw new RuntimeException(e);
+ }
+ });
+ }
+
+ public SyncTaskExecutionResult execute(BlockSyncTask blockSyncTask)
+ throws Exception {
+ LOG.info("Executing MetadataSyncTask {} (on {})",
+ blockSyncTask.getSyncTaskId(), blockSyncTask.getRemoteURI());
+
+ return doMultiPartPart(
+ blockSyncTask.getRemoteURI(),
+ blockSyncTask.getLocatedBlocks(),
+ blockSyncTask.getPartNumber(),
+ blockSyncTask.getUploadHandle(),
+ blockSyncTask.getOffset(),
+ blockSyncTask.getLength());
+ }
+
+ private SyncTaskExecutionResult doMultiPartPart(URI uri,
+ List<LocatedBlock> locatedBlocks, int partNumber, byte[] uploadHandle,
+ int offset, long length) throws IOException {
+ FileSystem fs = FileSystem.get(uri, conf);
+ Path filePath = new Path(uri);
+ Vector<InputStream> inputStreams = new Vector<>(locatedBlocks.size());
+ for (int i = 0; i < locatedBlocks.size(); ++i) {
+ LocatedBlock locatedBlock = locatedBlocks.get(i);
+ BlockReader reader = createBlockReader.apply(locatedBlock, conf);
+ if (i == 0) {
+ reader.skip(offset);
+ }
+ BlockInputStream inputStream = new BlockInputStream(reader);
+ inputStreams.add(inputStream);
+ }
+ Enumeration<InputStream> streamEnumeration = inputStreams.elements();
+ SequenceInputStream inputStream =
+ new SequenceInputStream(streamEnumeration);
+ MultipartUploader mpu = multipartUploaderSupplier.apply(fs);
+ PartHandle partHandle = mpu.putPart(filePath, inputStream,
+ partNumber, BBUploadHandle.from(ByteBuffer.wrap(uploadHandle)), length);
+ return new SyncTaskExecutionResult(partHandle.bytes(), length);
+ }
+}
+
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/syncservice/executor/BlockSyncReaderFactory.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/syncservice/executor/BlockSyncReaderFactory.java
new file mode 100644
index 0000000..cc5eb5c
--- /dev/null
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/syncservice/executor/BlockSyncReaderFactory.java
@@ -0,0 +1,92 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdfs.server.datanode.syncservice.executor;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FsTracer;
+import org.apache.hadoop.hdfs.BlockReader;
+import org.apache.hadoop.hdfs.ClientContext;
+import org.apache.hadoop.hdfs.DFSUtilClient;
+import org.apache.hadoop.hdfs.client.impl.BlockReaderFactory;
+import org.apache.hadoop.hdfs.client.impl.DfsClientConf;
+import org.apache.hadoop.hdfs.net.Peer;
+import org.apache.hadoop.hdfs.protocol.DatanodeInfo;
+import org.apache.hadoop.hdfs.protocol.HdfsConstants;
+import org.apache.hadoop.hdfs.protocol.LocatedBlock;
+import org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier;
+import org.apache.hadoop.hdfs.server.datanode.CachingStrategy;
+import org.apache.hadoop.hdfs.server.datanode.DataNode;
+import org.apache.hadoop.net.NetUtils;
+import org.apache.hadoop.security.token.Token;
+
+import java.io.IOException;
+import java.net.InetSocketAddress;
+import java.net.Socket;
+import java.util.EnumSet;
+
+/**
+ * BlockSyncReaderFactory constructs a block reader in the Datanode for the
+ * Sync Command to read blocks that will be written to the synchronization
+ * remote endpoint.
+ */
+public class BlockSyncReaderFactory {
+
+ public static BlockReader createBlockReader(DataNode dataNode,
+ LocatedBlock locatedBlock, Configuration conf) throws IOException {
+ ClientContext clientContext = ClientContext.getFromConf(conf);
+ Token<BlockTokenIdentifier> accessToken = dataNode.getBlockAccessToken(
+ locatedBlock.getBlock(),
+ EnumSet.of(BlockTokenIdentifier.AccessMode.READ),
+ locatedBlock.getStorageTypes(), locatedBlock.getStorageIDs());
+
+ DatanodeInfo datanodeInfo = locatedBlock.getLocations()[0];
+
+ Socket socked = NetUtils.getDefaultSocketFactory(conf).createSocket();
+ InetSocketAddress resolvedAddress =
+ datanodeInfo.getResolvedAddress();
+ socked.connect(resolvedAddress);
+
+ return new BlockReaderFactory(new DfsClientConf(conf))
+ .setConfiguration(conf)
+ .setBlock(locatedBlock.getBlock())
+ .setBlockToken(accessToken)
+ .setStartOffset(0)
+ .setLength(locatedBlock.getBlock().getNumBytes())
+ .setInetSocketAddress(datanodeInfo.getResolvedAddress())
+ .setVerifyChecksum(true)
+ .setDatanodeInfo(datanodeInfo)
+ .setClientName("BlockSyncOperationExecutor")
+ .setCachingStrategy(CachingStrategy.newDefaultStrategy())
+ .setRemotePeerFactory((addr, blockToken, datanodeId) -> {
+ Peer peer = null;
+ Socket sock = NetUtils.getDefaultSocketFactory(conf).createSocket();
+ try {
+ sock.connect(addr, HdfsConstants.READ_TIMEOUT);
+ sock.setSoTimeout(HdfsConstants.READ_TIMEOUT);
+ peer = DFSUtilClient.peerFromSocket(sock);
+ } finally {
+ if (peer == null) {
+ IOUtils.closeQuietly(sock);
+ }
+ }
+ return peer;
+ })
+ .setClientCacheContext(clientContext)
+ .build();
+ }
+}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/syncservice/executor/BlockSyncTaskRunner.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/syncservice/executor/BlockSyncTaskRunner.java
new file mode 100644
index 0000000..660e39e
--- /dev/null
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/syncservice/executor/BlockSyncTaskRunner.java
@@ -0,0 +1,69 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdfs.server.datanode.syncservice.executor;
+
+import org.apache.hadoop.hdfs.server.protocol.BlockSyncTask;
+import org.apache.hadoop.hdfs.server.protocol.BlockSyncTaskExecutionFeedback;
+import org.apache.hadoop.hdfs.server.protocol.SyncTaskExecutionResult;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * BlockSyncTaskRunner glues together the sync task and the feedback reporting.
+ */
+import java.util.function.Consumer;
+
+public class BlockSyncTaskRunner implements Runnable {
+
+ public static final Logger LOG =
+ LoggerFactory.getLogger(BlockSyncTaskRunner.class);
+
+ private BlockSyncTask blockSyncTask;
+ private BlockSyncOperationExecutor syncOperationExecutor;
+
+ private Consumer<BlockSyncTaskExecutionFeedback> publishOutcomeCallback;
+
+ public BlockSyncTaskRunner(BlockSyncTask blockSyncTask,
+ BlockSyncOperationExecutor syncOperationExecutor,
+ Consumer<BlockSyncTaskExecutionFeedback> publishOutcomeCallback) {
+ this.blockSyncTask = blockSyncTask;
+ this.syncOperationExecutor = syncOperationExecutor;
+ this.publishOutcomeCallback = publishOutcomeCallback;
+ }
+
+ @Override
+ public void run() {
+ LOG.info("Executing BlockyncTask {} (on {})",
+ blockSyncTask.getSyncTaskId(), blockSyncTask.getRemoteURI());
+ try {
+ SyncTaskExecutionResult result =
+ syncOperationExecutor.execute(blockSyncTask);
+ publishOutcomeCallback.accept(BlockSyncTaskExecutionFeedback
+ .finishedSuccessfully(blockSyncTask.getSyncTaskId(),
+ blockSyncTask.getSyncMountId(),
+ result));
+ } catch (Exception e) {
+ LOG.error(
+ String.format("Exception executing MetadataSyncTask %s (on %s)",
+ blockSyncTask.getSyncTaskId(), blockSyncTask.getRemoteURI()), e);
+ publishOutcomeCallback.accept(BlockSyncTaskExecutionFeedback
+ .failedWithException(blockSyncTask.getSyncTaskId(),
+ blockSyncTask.getSyncMountId(), e));
+ }
+ }
+
+}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockInputStream.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockInputStream.java
new file mode 100644
index 0000000..43d4881
--- /dev/null
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockInputStream.java
@@ -0,0 +1,84 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdfs;
+
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.mockito.Mock;
+import org.mockito.runners.MockitoJUnitRunner;
+
+import java.io.IOException;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.fail;
+import static org.mockito.Matchers.any;
+import static org.mockito.Matchers.eq;
+import static org.mockito.Mockito.times;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+/**
+ * Test the BlockInputStream facade.
+ */
+@RunWith(MockitoJUnitRunner.class)
+public class TestBlockInputStream {
+ @Mock
+ private BlockReader blockReaderMock;
+
+ @Test
+ public void testBlockInputStreamReadChar() {
+ BlockInputStream is = new BlockInputStream(blockReaderMock);
+
+ try {
+ when(blockReaderMock.read(any(), eq(0), eq(1)))
+ .thenReturn(32);
+ // Making the mock perform the side effect of writing to buf is nasty.
+ is.read();
+ verify(blockReaderMock, times(1)).read(any(), eq(0), eq(1));
+ } catch (IOException e) {
+ fail("Could not even mock out read function.");
+ }
+ }
+
+ @Test
+ public void testBlockInputStreamReadBuf() {
+ BlockInputStream is = new BlockInputStream(blockReaderMock);
+
+ try {
+ byte[] buf = new byte[1024];
+ when(blockReaderMock.read(buf, 0, buf.length)).thenReturn(1024);
+ is.read(buf, 0, buf.length);
+ verify(blockReaderMock, times(1)).read(buf, 0, buf.length);
+ } catch (IOException e) {
+ fail("Could not even mock out read function.");
+ }
+ }
+
+ @Test
+ public void testBlockInputStreamSkip() {
+ BlockInputStream is = new BlockInputStream(blockReaderMock);
+
+ try {
+ when(blockReaderMock.skip(10)).thenReturn(10L);
+ long ret = is.skip(10);
+ assertEquals(10, ret);
+ verify(blockReaderMock, times(1)).skip(10L);
+ } catch (IOException e) {
+ fail("Could not even mock out skip function.");
+ }
+ }
+}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/syncservice/executor/TestBlockSyncOperationExecutor.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/syncservice/executor/TestBlockSyncOperationExecutor.java
new file mode 100644
index 0000000..e16d086
--- /dev/null
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/syncservice/executor/TestBlockSyncOperationExecutor.java
@@ -0,0 +1,94 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdfs.server.datanode.syncservice.executor;
+
+import com.google.common.collect.Lists;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.BBPartHandle;
+import org.apache.hadoop.fs.MultipartUploader;
+import org.apache.hadoop.fs.PartHandle;
+import org.apache.hadoop.hdfs.BlockReader;
+import org.apache.hadoop.hdfs.protocol.Block;
+import org.apache.hadoop.hdfs.protocol.ExtendedBlock;
+import org.apache.hadoop.hdfs.protocol.LocatedBlock;
+import org.apache.hadoop.hdfs.server.protocol.BlockSyncTask;
+import org.apache.hadoop.hdfs.server.protocol.SyncTaskExecutionResult;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.mockito.Mock;
+import org.mockito.runners.MockitoJUnitRunner;
+
+import java.net.URI;
+import java.nio.ByteBuffer;
+import java.util.List;
+import java.util.UUID;
+
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.mockito.Matchers.any;
+import static org.mockito.Matchers.anyInt;
+import static org.mockito.Matchers.anyLong;
+import static org.mockito.Mockito.times;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+@RunWith(MockitoJUnitRunner.class)
+public class TestBlockSyncOperationExecutor {
+
+ @Mock
+ private BlockReader blockReaderMock;
+
+ @Mock
+ private MultipartUploader multipartUploaderMock;
+
+ @Test
+ public void executeMultipartPutFileSyncTask() throws Exception {
+ long blockLength = 42L;
+ Configuration conf = new Configuration();
+ BlockSyncOperationExecutor blockSyncOperationExecutor =
+ new BlockSyncOperationExecutor(conf,
+ (locatedBlock, config) -> blockReaderMock,
+ fs -> multipartUploaderMock);
+ String uploadHandleStr = "uploadHandle";
+ byte[] uploadHandle = uploadHandleStr.getBytes();
+ ByteBuffer byteBuffer = ByteBuffer.wrap(uploadHandle);
+ PartHandle partHandle = BBPartHandle.from(byteBuffer);
+ when(multipartUploaderMock.putPart(any(), any(), anyInt(), any(),
+ anyLong())).thenReturn(partHandle);
+ UUID syncTaskId = UUID.randomUUID();
+ URI remoteUri = new URI("remoteUri");
+ String syncMountId = "syncMountId";
+ Block block = new Block(42L, blockLength, 44L);
+ ExtendedBlock extendedBlock1 = new ExtendedBlock("poolId", block);
+ LocatedBlock locatedBlock = new LocatedBlock(extendedBlock1, null);
+ List<LocatedBlock> locatedBlocks = Lists.newArrayList(locatedBlock);
+ Integer partNumber = 85;
+ final int offset = 0;
+ final long length = locatedBlock.getBlockSize();
+
+
+ BlockSyncTask blockSyncTask = new BlockSyncTask(syncTaskId, remoteUri,
+ locatedBlocks, partNumber, uploadHandle, offset, length, syncMountId);
+
+ SyncTaskExecutionResult result =
+ blockSyncOperationExecutor.execute(blockSyncTask);
+
+ assertThat(result).isNotNull();
+ Long actualLength = result.getNumberOfBytes();
+ assertThat(actualLength).isEqualTo(blockLength);
+ assertThat(result.getResult()).isEqualTo(partHandle.bytes());
+ }
+}
\ No newline at end of file
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org