You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@ozone.apache.org by fe...@apache.org on 2022/08/25 03:55:24 UTC
[ozone] branch HDDS-5713 updated (8e31b09c63 -> 09a0b8e1b7)
This is an automated email from the ASF dual-hosted git repository.
ferhui pushed a change to branch HDDS-5713
in repository https://gitbox.apache.org/repos/asf/ozone.git
omit 8e31b09c63 HDDS-7106. [DiskBalancer] Client-SCM interface (#3663)
add 3b89d4daac HDDS-5564. feature/PrefixFSO.md translation (#2582)
add ccf1b7c18f HDDS-6951. Replace bucket.listKeys() with bucket.listStatus() in OmBucketReadWriteKeyOps (#3554)
add 2396611e3c HDDS-7044. Ignore pr_title_check for selective checks (#3620)
add fa61f9e663 HDDS-7077. EC: Fix block deletion not allowed due to missing pipelineID (#3644)
add 15c9a5bca3 HDDS-7081. EC: ReplicationManager - UnderRep handler should handle duplicate indexes (#3649)
add 348fcb6157 HDDS-7051. Fix offset Condition in ECKeyOutputStream (#3623)
add ef218d8b68 HDDS-6662. Provide basic hash code generator for OBSKeyPathLockStrategy (#3358)
add 4459f47c11 HDDS-7063. Increment createContainer metric when container is created while writing chunk (#3643)
add 8f23e6bd54 HDDS-4970. Significant overhead when DataNode is over-subscribed (#3654)
add 51897d4aa1 HDDS-7023. Add CLI for reseting the expired deleted block txn (#3615)
add 192eff11a9 HDDS-7087. Manage RocksObjects to detect leaks (#3656)
add 8e11723984 HDDS-7085. Update gRPC to 1.48.1 to address OOM bug (#3659)
add 7d50d29d51 HDDS-7066. Include number of tests requested in Freon output (#3638)
add 8240341efd HDDS-6997. Add support for "echo" lookup to measure raw performance in OM (#3614)
add 1afaa652c1 HDDS-7047. Add Recon to ozone-topology sample compose environment (#3641)
add b696473437 HDDS-7109. SCM should close LayoutVersionManager in stopping stage (#3666)
add fded237013 HDDS-7053. Add client-side pipelines distribution metrics (#3626)
add dd699d952d HDDS-7043. Disallow ellipsis in PR title (#3651)
add 9583e47331 HDDS-6581. Introduce KEY_PATH_LOCK in OMKeyCreateRequest class (#3560)
add 64032cc223 HDDS-7022. EC: Open EC container are not closed when SCM container was already closed. (#3668)
add 658e139b45 HDDS-7116. Avoid leaking RocksObject from DBProfile (#3673)
add 13ccc5f8d3 HDDS-5504. Refactor namespace summaries (du, dist & counts) for upcoming legacy FS buckets support (#3309)
add 5adbef2135 HDDS-6127. file checksum to support both CRC32 and CRC32C. (#3570)
add 5ba6728f7c HDDS-6726. Close RocksObject in Recon and tools (#3678)
add f9bd49e043 HDDS-7102. Exclude unnecessary RangerClient dependencies (#3669)
add 2554228edc HDDS-7033. Include committed space in log for disk out of space (#3636)
add ec3d55f6b5 HDDS-7127. Fix Transparent Data Encryption documentation (#3685)
add 5f84fd1c9f HDDS-7074. DN EndpointStateMachineMBean to add getType() (#3650)
add 636d23615c HDDS-7120. Prometheus displays only one volume_io_stats per datanode (#3682)
add ab608c5d46 HDDS-7045. Election info is out of date in Recon (#3677)
add 82b6441c6f HDDS-7076. Log container file path when container cannot be written. (#3684)
add be2fe71d69 HDDS-6458. Add ReplicationConfig options for bucketArgs in MiniOzoneChaosCluster (#3686)
add 5d8573a970 HDDS-7134. NPE when Ranger client throws RangerServiceException without Status. (#3689)
add 3efb039091 HDDS-7113. Support overriding response header values (#3680)
add 749b526503 HDDS-7147. DirectoryDeletingService interval unit mismatch (#3695)
add 01afc1ff33 HDDS-7136. Memory leak due to ChunkInputStream.close() not releasing buffer (#3692)
add 20863d3148 HDDS-7139. INVALID_TOKEN is handled as internal error in S3G (#3698)
add e9689764d3 HDDS-7108. Include supported EC data-parity combinations in error message (#3665)
add 4fd698f05e Multiple RPC clients per test run. (#3696)
add 38c767fa5f HDDS-7135. ofs file input stream should support StreamCapabilities interface (#3694)
add b11c5f12f8 HDDS-5779. Create s3g bucket creation performance test (#3629)
add b2b19de090 HDDS-7154. Fixed code issues in org.apache.hadoop.hdds.client reported by sonar (#3700)
add 9fc0f9801d HDDS-7150. Freon fail-at-end option caused not shutdown (#3697)
add a1ba9eaa55 HDDS-7108. Make ReplicationConfig validation error message type-specific (#3702)
add ac45a349ef HDDS-7159. Upgrade ubuntu version from 18.04 to 20.04 (#3707)
add 5ea8e6ab3f HDDS-7157. Log the error msg explicitly in BlockDeletingService (#3704)
add df9ed54ba1 HDDS-7159. Use Java 8 for integration tests (#3708)
add 8bb3d6e7af HDDS-7151. Avoid using GeneratedMessage in non-generated code (#3699)
add 132bec2363 HDDS-7153. User defined override of ozone shell hardcoded values for log files (#3706)
add 59a795b2cc HDDS-7164. Skip installing awscli if already present (#3712)
add a0b996651d HDDS-6456. Update RocksDB Version to 7.x (#3690)
add a828dba870 HDDS-7104. Remove WARNINGs while running ozone fs/CLI commands. (#3671)
add 5c18a5624f HDDS-7162. Add Double in ConfigType (#3710)
add 09a0b8e1b7 HDDS-7106. [DiskBalancer] Client-SCM interface (#3663)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (8e31b09c63)
\
N -- N -- N refs/heads/HDDS-5713 (09a0b8e1b7)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/workflows/close-pending.yaml | 2 +-
.github/workflows/comments.yaml | 2 +-
.github/workflows/post-commit.yml | 26 +-
dev-support/ci/pr_title_check.bats | 8 +
dev-support/ci/pr_title_check.sh | 2 +
dev-support/ci/selective_ci_checks.bats | 12 +
dev-support/ci/selective_ci_checks.sh | 10 +-
.../hadoop/hdds/scm/ContainerClientMetrics.java | 138 +++
.../hadoop/hdds/scm/storage/BlockOutputStream.java | 10 +-
.../hadoop/hdds/scm/storage/ChunkInputStream.java | 1 +
.../hdds/scm/storage/ECBlockOutputStream.java | 6 +-
.../hdds/scm/storage/RatisBlockOutputStream.java | 7 +-
.../hdds/scm/TestContainerClientMetrics.java | 116 ++
.../storage/TestBlockOutputStreamCorrectness.java | 4 +-
.../hdds/client/DefaultReplicationConfig.java | 2 +-
.../hadoop/hdds/client/ECReplicationConfig.java | 2 +-
.../org/apache/hadoop/hdds/client/OzoneQuota.java | 9 -
.../org/apache/hadoop/hdds/client/QuotaList.java | 19 +-
.../hadoop/hdds/client/ReplicationConfig.java | 4 +-
.../hdds/client/ReplicationConfigValidator.java | 32 +-
.../apache/hadoop/hdds/recon/ReconConfigKeys.java | 2 +-
.../apache/hadoop/hdds/scm/client/ScmClient.java | 7 +
.../protocol/StorageContainerLocationProtocol.java | 9 +
.../org/apache/hadoop/ozone/audit/SCMAction.java | 3 +-
.../hdds/client/TestECReplicationConfig.java | 29 +-
.../hadoop/hdds/client/TestReplicationConfig.java | 36 +-
.../client/TestReplicationConfigValidator.java | 14 +-
.../hadoop/hdds/conf/SimpleConfiguration.java | 13 +
.../hadoop/hdds/conf/TestOzoneConfiguration.java | 10 +
.../hadoop/hdds/scm/pipeline/MockPipeline.java | 2 +
.../org/apache/hadoop/hdds/conf/ConfigType.java | 3 +-
.../hdds/conf/ConfigurationReflectionUtil.java | 9 +
.../hadoop/hdds/conf/ConfigurationTarget.java | 4 +
.../hadoop/hdds/conf/ConfigurationExample.java | 13 +
.../hdds/conf/TestConfigurationReflectionUtil.java | 4 +
.../datanode/metadata/DatanodeCRLStoreImpl.java | 13 +-
.../container/common/impl/HddsDispatcher.java | 9 +-
.../container/common/report/ReportManager.java | 4 +-
.../container/common/report/ReportPublisher.java | 6 +-
.../common/report/ReportPublisherFactory.java | 6 +-
.../common/statemachine/EndpointStateMachine.java | 16 +-
.../statemachine/EndpointStateMachineMBean.java | 2 +
.../common/statemachine/StateContext.java | 56 +-
.../states/endpoint/HeartbeatEndpointTask.java | 6 +-
.../common/utils/db/DatanodeDBProfile.java | 43 +-
.../common/volume/AvailableSpaceFilter.java | 84 ++
.../volume/CapacityVolumeChoosingPolicy.java | 42 +-
.../ozone/container/common/volume/HddsVolume.java | 3 +-
.../volume/RoundRobinVolumeChoosingPolicy.java | 22 +-
.../common/volume/VolumeChoosingUtil.java | 49 +
.../container/common/volume/VolumeIOStats.java | 10 +-
.../ECReconstructionCoordinator.java | 8 +-
.../container/keyvalue/KeyValueContainer.java | 9 +-
.../container/keyvalue/helpers/ChunkUtils.java | 50 +-
.../background/BlockDeletingService.java | 46 +-
.../container/metadata/AbstractDatanodeStore.java | 13 +-
.../metadata/DatanodeSchemaOneDBDefinition.java | 9 +-
.../metadata/DatanodeSchemaThreeDBDefinition.java | 5 +-
.../protocol/commands/CommandForDatanode.java | 4 +-
.../hadoop/ozone/protocol/commands/SCMCommand.java | 4 +-
.../common/report/TestReportPublisher.java | 6 +-
.../common/statemachine/TestStateContext.java | 22 +-
.../volume/TestCapacityVolumeChoosingPolicy.java | 21 +-
.../volume/TestRoundRobinVolumeChoosingPolicy.java | 33 +-
.../TestVolumeIOStatsWithPrometheusSink.java | 92 ++
.../container/keyvalue/TestKeyValueContainer.java | 36 -
.../container/keyvalue/helpers/TestChunkUtils.java | 22 +-
hadoop-hdds/docs/content/concept/Recon.md | 2 +-
hadoop-hdds/docs/content/concept/Recon.zh.md | 2 +-
hadoop-hdds/docs/content/feature/PrefixFSO.zh.md | 69 ++
hadoop-hdds/docs/content/security/SecuringTDE.md | 12 +-
.../docs/content/security/SecuringTDE.zh.md | 6 +-
...inerLocationProtocolClientSideTranslatorPB.java | 13 +
.../java/org/apache/hadoop/hdds/utils/HAUtils.java | 30 +-
.../apache/hadoop/hdds/utils/db/CodecRegistry.java | 22 +-
.../hdds/utils/db/DBColumnFamilyDefinition.java | 8 +-
.../hadoop/hdds/utils/db/DBConfigFromFile.java | 8 +-
.../org/apache/hadoop/hdds/utils/db/DBProfile.java | 61 +-
.../hadoop/hdds/utils/db/DBStoreBuilder.java | 119 +-
.../hadoop/hdds/utils/db/RDBBatchOperation.java | 12 +-
.../hadoop/hdds/utils/db/RDBCheckpointManager.java | 8 +-
.../hadoop/hdds/utils/db/RDBSstFileLoader.java | 10 +-
.../hadoop/hdds/utils/db/RDBSstFileWriter.java | 28 +-
.../org/apache/hadoop/hdds/utils/db/RDBStore.java | 35 +-
.../hadoop/hdds/utils/db/RDBStoreIterator.java | 30 +-
.../apache/hadoop/hdds/utils/db/RocksDatabase.java | 176 +--
.../apache/hadoop/hdds/utils/db/TableConfig.java | 27 +-
.../db/managed/ManagedBlockBasedTableConfig.java | 74 ++
.../hdds/utils/db/managed/ManagedBloomFilter.java | 28 +-
.../hdds/utils/db/managed/ManagedCheckpoint.java | 33 +
.../db/managed/ManagedColumnFamilyOptions.java | 83 ++
.../hdds/utils/db/managed/ManagedDBOptions.java | 28 +-
.../hdds/utils/db/managed/ManagedEnvOptions.java | 27 +-
.../hdds/utils/db/managed/ManagedFlushOptions.java | 28 +-
.../managed/ManagedIngestExternalFileOptions.java | 29 +-
.../hdds/utils/db/managed/ManagedLRUCache.java | 31 +-
.../hdds/utils/db/managed/ManagedObject.java | 43 +-
.../hdds/utils/db/managed/ManagedOptions.java | 28 +-
.../hdds/utils/db/managed/ManagedReadOptions.java | 31 +-
.../hdds/utils/db/managed/ManagedRocksDB.java | 81 ++
.../utils/db/managed/ManagedRocksIterator.java | 34 +
.../db/managed/ManagedRocksObjectMetrics.java | 60 +
.../utils/db/managed/ManagedRocksObjectUtils.java | 52 +
.../utils/db/managed/ManagedSstFileWriter.java | 34 +-
.../hdds/utils/db/managed/ManagedStatistics.java | 27 +-
.../db/managed/ManagedTransactionLogIterator.java | 31 +-
.../hdds/utils/db/managed/ManagedWriteBatch.java | 36 +-
.../hdds/utils/db/managed/ManagedWriteOptions.java | 31 +-
.../hadoop/hdds/utils/db/managed/package-info.java | 35 +
.../hadoop/hdds/utils/db/TestDBStoreBuilder.java | 8 +-
.../apache/hadoop/hdds/utils/db/TestRDBStore.java | 18 +-
.../hadoop/hdds/utils/db/TestRDBStoreIterator.java | 31 +-
.../hadoop/hdds/utils/db/TestRDBTableStore.java | 15 +-
.../hdds/utils/db/TestTypedRDBTableStore.java | 13 +-
.../src/main/proto/ScmAdminProtocol.proto | 18 +-
.../hadoop/hdds/scm/block/DeletedBlockLog.java | 9 +
.../hadoop/hdds/scm/block/DeletedBlockLogImpl.java | 20 +
.../scm/block/DeletedBlockLogStateManager.java | 4 +
.../scm/block/DeletedBlockLogStateManagerImpl.java | 33 +
.../replication/ECUnderReplicationHandler.java | 52 +-
.../replication/LegacyReplicationManager.java | 4 +-
.../replication/OverReplicatedProcessor.java | 2 +-
.../container/replication/ReplicationManager.java | 77 ++
.../replication/UnderReplicatedProcessor.java | 2 +-
.../hadoop/hdds/scm/ha/BackgroundSCMService.java | 7 +-
.../hdds/scm/ha/SCMHADBTransactionBufferImpl.java | 3 +
.../apache/hadoop/hdds/scm/ha/StatefulService.java | 12 +-
.../apache/hadoop/hdds/scm/ha/io/CodecFactory.java | 11 +-
.../apache/hadoop/hdds/scm/ha/io/EnumCodec.java | 2 +-
.../hdds/scm/ha/io/GeneratedMessageCodec.java | 12 +-
.../apache/hadoop/hdds/scm/ha/io/ListCodec.java | 2 +-
...inerLocationProtocolServerSideTranslatorPB.java | 18 +
.../hadoop/hdds/scm/server/SCMCertStore.java | 5 +-
.../hdds/scm/server/SCMClientProtocolServer.java | 18 +
.../scm/server/SCMDatanodeHeartbeatDispatcher.java | 4 +-
.../hdds/scm/server/StorageContainerManager.java | 3 +
.../hadoop/hdds/scm/block/TestDeletedBlockLog.java | 47 +
.../replication/TestECUnderReplicationHandler.java | 24 +-
.../hdds/scm/cli/ContainerOperationClient.java | 5 +
.../client/checksum/BaseFileChecksumHelper.java | 40 +-
.../checksum/ReplicatedFileChecksumHelper.java | 4 +-
.../ozone/client/io/BlockOutputStreamEntry.java | 19 +-
.../client/io/BlockOutputStreamEntryPool.java | 20 +-
.../ozone/client/io/ECBlockOutputStreamEntry.java | 16 +-
.../client/io/ECBlockOutputStreamEntryPool.java | 7 +-
.../hadoop/ozone/client/io/ECKeyOutputStream.java | 6 +-
.../hadoop/ozone/client/io/KeyOutputStream.java | 28 +-
.../apache/hadoop/ozone/client/rpc/RpcClient.java | 7 +-
.../hadoop/ozone/client/TestOzoneECClient.java | 65 +-
.../checksum/TestReplicatedFileChecksumHelper.java | 11 +-
.../main/java/org/apache/hadoop/ozone/OmUtils.java | 1 +
.../hadoop/ozone/om/lock/OzoneManagerLock.java | 18 +-
.../ozone/om/protocol/OzoneManagerProtocol.java | 14 +
...OzoneManagerProtocolClientSideTranslatorPB.java | 20 +
.../hadoop/ozone/om/lock/TestKeyPathLock.java | 347 ++++++
.../hadoop/ozone/om/lock/TestOzoneManagerLock.java | 508 +++-----
.../compose/ozone-topology/docker-compose.yaml | 15 +
.../src/main/compose/ozone-topology/docker-config | 5 +
.../dist/src/main/compose/ozone-topology/test.sh | 2 +
hadoop-ozone/dist/src/main/license/bin/LICENSE.txt | 6 +
hadoop-ozone/dist/src/main/license/jar-report.txt | 6 -
.../src/main/smoketest/freon/echoRPCLoad.robot | 42 +
.../dist/src/main/smoketest/recon/recon-api.robot | 14 +
.../main/smoketest/recon/recon-fso-nssummary.robot | 146 +++
.../dist/src/main/smoketest/s3/commonawslib.robot | 2 +
.../dist/src/main/smoketest/s3/freon.robot | 44 +
.../dist/src/main/smoketest/s3/objectputget.robot | 6 +
hadoop-ozone/dist/src/shell/ozone/ozone | 3 +-
.../hadoop/ozone/MiniOzoneLoadGenerator.java | 22 +-
.../hadoop/ozone/TestMiniChaosOzoneCluster.java | 28 +-
.../hadoop/fs/ozone/TestRootedOzoneFileSystem.java | 17 +
.../ozone/client/rpc/TestECKeyOutputStream.java | 48 +-
.../client/rpc/read/TestChunkInputStream.java | 20 +
.../ozone/container/TestECContainerRecovery.java | 84 +-
.../commandhandler/TestBlockDeletion.java | 101 +-
.../container/metrics/TestContainerMetrics.java | 11 +-
.../TestResetDeletedBlockRetryCountShell.java | 169 +++
.../src/main/proto/OmClientProtocol.proto | 19 +
hadoop-ozone/ozone-manager/pom.xml | 16 +
.../org/apache/hadoop/ozone/om/KeyManagerImpl.java | 2 +-
.../org/apache/hadoop/ozone/om/OzoneManager.java | 58 +-
.../om/hashcodegenerator/OMHashCodeGenerator.java | 23 +-
.../StringOMHashCodeGeneratorImpl.java | 21 +-
.../ozone/om/hashcodegenerator/package-info.java} | 20 +-
.../ozone/om/lock/OBSKeyPathLockStrategy.java | 41 +-
.../hadoop/ozone/om/lock/OzoneLockStrategy.java | 15 +-
.../ozone/om/lock/RegularBucketLockStrategy.java | 8 +-
.../RangerClientMultiTenantAccessController.java | 27 +-
.../om/ratis/utils/OzoneManagerRatisUtils.java | 3 +
.../hadoop/ozone/om/request/OMEchoRPCRequest.java} | 28 +-
.../ozone/om/request/key/OMKeyCreateRequest.java | 11 +-
.../hadoop/ozone/om/request/key/OMKeyRequest.java | 6 +
.../protocolPB/OzoneManagerRequestHandler.java | 29 +
.../om/request/key/TestOMKeyCreateRequest.java | 47 +-
.../request/key/TestOMKeyCreateRequestWithFSO.java | 5 +
.../ozone/om/response/TestCleanupTableInfo.java | 3 +
.../fs/ozone/BasicRootedOzoneFileSystem.java | 8 +-
.../hadoop/fs/ozone/RootedOzoneFileSystem.java | 6 +
.../hadoop/fs/ozone/RootedOzoneFileSystem.java | 6 +
.../hadoop/ozone/recon/ReconControllerModule.java | 4 +-
.../hadoop/ozone/recon/ReconRestServletModule.java | 2 +-
.../hadoop/ozone/recon/api/NSSummaryEndpoint.java | 929 +--------------
.../hadoop/ozone/recon/api/PipelineEndpoint.java | 57 +-
.../recon/api/handlers/BucketEntityHandler.java | 152 +++
.../ozone/recon/api/handlers/BucketHandler.java | 207 ++++
.../recon/api/handlers/DirectoryEntityHandler.java | 150 +++
.../ozone/recon/api/handlers/EntityHandler.java | 342 ++++++
.../ozone/recon/api/handlers/FSOBucketHandler.java | 279 +++++
.../ozone/recon/api/handlers/KeyEntityHandler.java | 93 ++
.../recon/api/handlers/RootEntityHandler.java | 176 +++
.../recon/api/handlers/UnknownEntityHandler.java | 82 ++
.../recon/api/handlers/VolumeEntityHandler.java | 160 +++
.../EntityType.java => handlers/package-info.java} | 13 +-
.../hadoop/ozone/recon/api/types/EntityType.java | 85 +-
.../hadoop/ozone/recon/api/types/NSSummary.java | 7 +-
.../hadoop/ozone/recon/scm/ReconNodeManager.java | 18 +-
.../scm/ReconStorageContainerManagerFacade.java | 11 +-
.../spi/impl/OzoneManagerServiceProviderImpl.java | 8 +-
.../impl/ReconContainerMetadataManagerImpl.java | 191 +--
.../ozone/recon/spi/impl/ReconDBProvider.java | 11 +-
.../ozone/recon/tasks/ContainerKeyMapperTask.java | 89 +-
.../hadoop/ozone/recon/tasks/NSSummaryTask.java | 206 +---
.../ozone/recon/tasks/NSSummaryTaskWithFSO.java | 217 ++++
.../ozone/recon/tasks/OMDBUpdatesHandler.java | 4 +-
.../src/views/pipelines/pipelines.tsx | 22 +-
.../ozone/recon/OMMetadataManagerTestUtils.java | 44 +-
.../ozone/recon/api/TestNSSummaryEndpoint.java | 663 -----------
.../recon/api/TestNSSummaryEndpointWithFSO.java | 1249 ++++++++++++++++++++
.../impl/TestOzoneManagerServiceProviderImpl.java | 21 +-
.../ozone/recon/tasks/TestNSSummaryTask.java | 541 ---------
.../recon/tasks/TestNSSummaryTaskWithFSO.java | 604 ++++++++++
.../ozone/recon/tasks/TestOMDBUpdatesHandler.java | 9 +-
.../hadoop/ozone/s3/endpoint/BucketEndpoint.java | 12 +-
.../hadoop/ozone/s3/endpoint/EndpointBase.java | 7 +
.../hadoop/ozone/s3/endpoint/ObjectEndpoint.java | 71 +-
.../hadoop/ozone/s3/endpoint/TestObjectGet.java | 115 +-
.../ozone/s3/metrics/TestS3GatewayMetrics.java | 10 +
.../scm/ResetDeletedBlockRetryCountSubcommand.java | 73 ++
.../apache/hadoop/ozone/admin/scm/ScmAdmin.java | 3 +-
.../org/apache/hadoop/ozone/debug/DBScanner.java | 48 +-
.../org/apache/hadoop/ozone/debug/DropTable.java | 6 +-
.../hadoop/ozone/freon/BaseFreonGenerator.java | 10 +-
.../java/org/apache/hadoop/ozone/freon/Freon.java | 5 +-
.../ozone/freon/FreonReplicationOptions.java | 3 +-
.../ozone/freon/OmBucketReadWriteKeyOps.java | 14 +-
.../hadoop/ozone/freon/OmRPCLoadGenerator.java | 120 ++
.../hadoop/ozone/freon/S3BucketGenerator.java | 73 ++
.../hadoop/ozone/freon/S3EntityGenerator.java | 65 +
.../apache/hadoop/ozone/freon/S3KeyGenerator.java | 47 +-
.../freon/containergenerator/GeneratorOm.java | 14 +-
pom.xml | 54 +-
251 files changed, 8917 insertions(+), 4165 deletions(-)
create mode 100644 hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/ContainerClientMetrics.java
create mode 100644 hadoop-hdds/client/src/test/java/org/apache/hadoop/hdds/scm/TestContainerClientMetrics.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/AvailableSpaceFilter.java
create mode 100644 hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/VolumeChoosingUtil.java
create mode 100644 hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/volume/TestVolumeIOStatsWithPrometheusSink.java
create mode 100644 hadoop-hdds/docs/content/feature/PrefixFSO.zh.md
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/managed/ManagedBlockBasedTableConfig.java
copy hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/types/EntityType.java => hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/managed/ManagedBloomFilter.java (62%)
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/managed/ManagedCheckpoint.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/managed/ManagedColumnFamilyOptions.java
copy hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/types/EntityType.java => hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/managed/ManagedDBOptions.java (63%)
copy hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/types/EntityType.java => hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/managed/ManagedEnvOptions.java (63%)
copy hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/types/EntityType.java => hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/managed/ManagedFlushOptions.java (62%)
copy hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/types/EntityType.java => hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/managed/ManagedIngestExternalFileOptions.java (59%)
copy hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/types/EntityType.java => hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/managed/ManagedLRUCache.java (59%)
copy hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/types/EntityType.java => hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/managed/ManagedObject.java (50%)
copy hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/types/EntityType.java => hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/managed/ManagedOptions.java (63%)
copy hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/types/EntityType.java => hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/managed/ManagedReadOptions.java (60%)
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/managed/ManagedRocksDB.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/managed/ManagedRocksIterator.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/managed/ManagedRocksObjectMetrics.java
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/managed/ManagedRocksObjectUtils.java
copy hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/types/EntityType.java => hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/managed/ManagedSstFileWriter.java (52%)
copy hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/types/EntityType.java => hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/managed/ManagedStatistics.java (63%)
copy hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/types/EntityType.java => hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/managed/ManagedTransactionLogIterator.java (53%)
copy hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/types/EntityType.java => hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/managed/ManagedWriteBatch.java (57%)
copy hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/types/EntityType.java => hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/managed/ManagedWriteOptions.java (59%)
create mode 100644 hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/managed/package-info.java
create mode 100644 hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/om/lock/TestKeyPathLock.java
create mode 100644 hadoop-ozone/dist/src/main/smoketest/freon/echoRPCLoad.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/recon/recon-fso-nssummary.robot
create mode 100644 hadoop-ozone/dist/src/main/smoketest/s3/freon.robot
create mode 100644 hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/shell/TestResetDeletedBlockRetryCountShell.java
copy hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/EndpointStateMachineMBean.java => hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/hashcodegenerator/OMHashCodeGenerator.java (70%)
copy hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/EndpointStateMachineMBean.java => hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/hashcodegenerator/StringOMHashCodeGeneratorImpl.java (69%)
copy hadoop-ozone/{recon/src/main/java/org/apache/hadoop/ozone/recon/api/types/EntityType.java => ozone-manager/src/main/java/org/apache/hadoop/ozone/om/hashcodegenerator/package-info.java} (74%)
copy hadoop-ozone/{recon/src/main/java/org/apache/hadoop/ozone/recon/api/types/EntityType.java => ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/OMEchoRPCRequest.java} (54%)
create mode 100644 hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/handlers/BucketEntityHandler.java
create mode 100644 hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/handlers/BucketHandler.java
create mode 100644 hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/handlers/DirectoryEntityHandler.java
create mode 100644 hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/handlers/EntityHandler.java
create mode 100644 hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/handlers/FSOBucketHandler.java
create mode 100644 hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/handlers/KeyEntityHandler.java
create mode 100644 hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/handlers/RootEntityHandler.java
create mode 100644 hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/handlers/UnknownEntityHandler.java
create mode 100644 hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/handlers/VolumeEntityHandler.java
copy hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/{types/EntityType.java => handlers/package-info.java} (81%)
create mode 100644 hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/NSSummaryTaskWithFSO.java
delete mode 100644 hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/api/TestNSSummaryEndpoint.java
create mode 100644 hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/api/TestNSSummaryEndpointWithFSO.java
delete mode 100644 hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/tasks/TestNSSummaryTask.java
create mode 100644 hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/tasks/TestNSSummaryTaskWithFSO.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/scm/ResetDeletedBlockRetryCountSubcommand.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/freon/OmRPCLoadGenerator.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/freon/S3BucketGenerator.java
create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/freon/S3EntityGenerator.java
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@ozone.apache.org
For additional commands, e-mail: commits-help@ozone.apache.org