You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@ambari.apache.org by ji...@apache.org on 2026/09/14 02:26:26 UTC
(ambari) branch trunk updated: AMBARI-26656: Implement isolated multi-cluster management and recoverable HBase dependencies (#4216)
This is an automated email from the ASF dual-hosted git repository.
JiaLiangC pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/ambari.git
The following commit(s) were added to refs/heads/trunk by this push:
new 72ff1a4734 AMBARI-26656: Implement isolated multi-cluster management and recoverable HBase dependencies (#4216)
72ff1a4734 is described below
commit 72ff1a47345cb83b50fb902d5973181c30d1023d
Author: jialiang <ji...@gmail.com>
AuthorDate: Mon Sep 14 02:26:19 2026 +0000
AMBARI-26656: Implement isolated multi-cluster management and recoverable HBase dependencies (#4216)
* AMBARI-26656: Add unified multi-cluster management foundation
* AMBARI-26656: Harden managed dependency protocol validation
* AMBARI-26656: Align dependency command fixtures
* AMBARI-26656: Validate agent task ownership before report side effects
Validate reports against persisted task, stage, host, role and cluster identities before either heartbeat path applies state or credential side effects. Include mixed-report and persisted-command regressions.
* AMBARI-26656: Persist managed deployment and credential recovery lineage
Commit deployment state, exact task associations and credential producer evidence together; reconcile terminal results and restart recovery through production callers. Enforce managed Blueprint handoff and lifecycle boundaries.
Schema, DAO, command publication, runtime planning, credential callbacks and REST changes share one contract and remain atomic to avoid intermediate publication without lineage. Includes focused state, authorization, migration and real H2 transaction tests.
* AMBARI-26656: Encode cluster and resource API path segments
Use one raw-identity segment encoder at the existing cluster and resource request callers. Include encoded transport regressions.
* AMBARI-26656: Preserve shared workflow references during sanitization
Treat only ancestor references as cycles, preserving repeated handoff/config objects while applying credential redaction at every occurrence. Cover actual cycles, shared references and the existing reload-before-resume persistence contract.
* AMBARI-26656: Render durable deployments and integrate dependency lifecycle
Render backend-owned installation, dependency readiness, start and service-check progress. Preserve complete wizard checkpoints and acknowledged deployment identity across retry and refresh. Integrate provider impact confirmation and binding actions with scoped rendering. Keep endpoint types and UI consumers together.
* AMBARI-26656: Consolidate multi-cluster review and remediation evidence
Preserve the original review and map each repair to production code, focused tests and remaining acceptance gates. Merge interaction design into the main design and update the React baseline.
Combined validation: Java compile and 205 tests, 91 frontend tests, and 31 Python tests passed. TypeScript retains 49 baseline diagnostics versus 52 at the reviewed HEAD; real clusters, KDC, browser and process-kill acceptance remain unexecuted.
* AMBARI-26656: Reload Add Service data for each recovery generation
* AMBARI-26656: Restore React type checking for packaged builds
* AMBARI-26656: Defer managed security collaborators to avoid startup cycles
* AMBARI-26656: Observe heartbeat timeout window after server restart
* AMBARI-26656: Publish managed ownership and task lineage with production constraints
* AMBARI-26656: Use typed SDK observations for managed dependency operations
* AMBARI-26656: Restore native Bigtop HBase lifecycle and structured service checks
* AMBARI-26656: Recover owned dependency deployment attempts and guard detach
* AMBARI-26656: Preserve nested JSON routes and map dependency conflicts at REST boundary
* AMBARI-26656: Start Admin views before publishing shared sessions
* AMBARI-26656: Retain installation draft identity across wizard navigation
* AMBARI-26656: Enter the current cluster and expose explicit management navigation
* AMBARI-26656: Expand scoped Admin cluster management and repository navigation
* AMBARI-26656: Align review evidence and require authoritative state observations
* AMBARI-26656: Stop Registry DNS supervision before terminating its daemon
* AMBARI-26656: Scope HBase hook initialization to command configuration
* AMBARI-26656: Record independent cluster recovery and runtime corrections
* AMBARI-26656: Record final packaged independent cluster acceptance
* AMBARI-26656: Implement isolated multi-cluster management and recoverable HBase dependencies
* AMBARI-26656: Fix precommit and advisor configuration failures
* AMBARI-26656: Correct managed dependency Kerberos plans
* AMBARI-26656: Fix managed dependency deployment contracts
* AMBARI-26656: Fix managed dependency persistence contracts
* AMBARI-26656: Fix server lifecycle concurrency
* AMBARI-26656: Isolate and stabilize Java tests
* AMBARI-26656: Run Java tests in isolated parallel forks
* AMBARI-26656: Stabilize React UI tests
* AMBARI-26656: Make deployment signature test deterministic
* AMBARI-26656: Stabilize ResourceManager HA navigation test
* AMBARI-26656: Stabilize parallel React UI tests
* AMBARI-26656: Set compatible Capacity Scheduler CI worker bounds
* AMBARI-26656: Synchronize ResourceManager HA menu interaction test
* AMBARI-26656: Create alert DAO test clusters through the lifecycle API
---
AGENTS.md | 25 +
Jenkinsfile | 11 +-
.../src/main/resources/ui/ambari-admin/src/App.tsx | 154 +-
.../main/resources/ui/ambari-admin/src/NavBar.tsx | 20 +-
.../main/resources/ui/ambari-admin/src/SideBar.tsx | 11 +-
.../resources/ui/ambari-admin/src/SideItemList.tsx | 50 +-
.../src/__mocks__/mockClusterBluePrintInfo.ts | 7 +-
.../ui/ambari-admin/src/api/clusterApi.ts | 6 +-
.../ui/ambari-admin/src/api/clusterManagement.ts | 53 +
.../ui/ambari-admin/src/api/configs/axiosConfig.ts | 5 +-
.../resources/ui/ambari-admin/src/api/logout.ts | 3 +-
.../resources/ui/ambari-admin/src/api/versions.ts | 10 +-
.../ui/ambari-admin/src/components/SidebarItem.tsx | 56 +-
.../src/components/SidebarItemCollapsed.tsx | 28 +-
.../ambari-admin/src/context/ManagementContext.tsx | 49 +
.../ui/ambari-admin/src/hooks/useAdminResource.ts | 39 +
.../ui/ambari-admin/src/router/Routes.tsx | 4 +-
.../ui/ambari-admin/src/router/RoutesList.tsx | 16 +-
.../screens/ClusterManagement/ClusterCreate.tsx | 54 +
.../ClusterManagement/ClusterDeleteDialog.tsx | 62 +
.../screens/ClusterManagement/ClusterDetails.tsx | 84 +
.../ClusterManagement/ClusterInformation/index.tsx | 76 +-
.../screens/ClusterManagement/ClusterOverview.tsx | 68 +
.../ClusterManagement/ClusterPermissions.tsx | 81 +
.../screens/ClusterManagement/Dasboard/index.tsx | 11 +-
.../screens/ClusterManagement/HostResources.tsx | 63 +
.../screens/ClusterManagement/ManagementShared.tsx | 43 +
.../ClusterManagement/RemoteClusters/index.tsx | 1 +
.../ClusterManagement/StackVersions/List.tsx | 88 +-
.../ClusterInformation/ClusterIsInstalled.test.tsx | 65 +-
.../ClusterIsNotInstalled.test.tsx | 180 +-
.../src/tests/EditRemoteCluster.test.tsx | 34 +-
.../src/tests/RegisterRemoteCluster.test.tsx | 1 -
.../src/tests/clusterManagement.test.tsx | 76 +
.../RemoteCluster/EditRemoteCluster.test.tsx | 34 +-
.../RemoteCluster/RegisterRemoteCluster.test.tsx | 1 -
.../ui/ambari-admin/src/tests/navigation.test.ts | 8 +
.../ambari-admin/src/tests/stackVersions.test.tsx | 97 +
.../main/resources/ui/ambari-admin/src/types.ts | 1 +
.../ui/ambari-admin/src/utils/navigation.ts | 20 +
.../resources/ui/ambari-admin/src/utils/session.ts | 38 +-
.../tools/hadoop/ManagedDependencyClient.java | 290 +++
.../ambari/tools/zk/ManagedDependencyZk.java | 424 ++++
.../org/apache/ambari/tools/zk/ZkConnection.java | 59 +-
.../org/apache/ambari/tools/zk/ZkMigrator.java | 5 +
.../ambari_agent/CustomServiceOrchestrator.py | 26 +
.../tools/hadoop/ManagedDependencyClientTest.java | 126 +
.../ambari/tools/zk/ManagedDependencyZkTest.java | 153 ++
.../apache/ambari/tools/zk/ZkConnectionTest.java | 91 +
.../ambari_agent/TestCustomServiceOrchestrator.py | 59 +
.../resource_management/TestManagedDependency.py | 855 +++++++
.../libraries/functions/managed_dependency.py | 1633 ++++++++++++
.../functions/managed_dependency_client.py | 106 +
ambari-server/pom.xml | 18 +-
.../server/actionmanager/ActionDBAccessorImpl.java | 68 +-
.../ambari/server/actionmanager/ActionManager.java | 84 +-
.../server/actionmanager/HostRoleCommand.java | 9 +
.../ambari/server/agent/ExecutionCommand.java | 13 +
.../ambari/server/agent/HeartBeatHandler.java | 5 +-
.../ambari/server/agent/HeartbeatMonitor.java | 18 +-
.../ambari/server/agent/HeartbeatProcessor.java | 26 +-
.../agent/stomp/PrometheusTargetDiscovery.java | 10 +-
.../server/agent/stomp/dto/AlertGroupUpdate.java | 5 +
.../server/api/ContentTypeOverrideFilter.java | 14 +-
.../ambari/server/api/handlers/DeleteHandler.java | 1 +
.../ambari/server/api/services/BaseService.java | 8 +
.../api/services/ManagedDependencyApiSupport.java | 317 +++
.../services/ManagedServiceDependencyService.java | 234 ++
.../services/ManagedServiceDependentsService.java | 54 +
.../server/api/services/PersistKeyValueImpl.java | 788 +++++-
.../api/services/PersistKeyValueService.java | 91 +-
.../api/services/ScopedWorkflowDraftSummary.java | 54 +
.../server/api/services/ScopedWorkflowState.java | 99 +
.../server/api/services/ScopedWorkflowUpdate.java | 63 +
.../services/ServiceDependenciesApiService.java | 146 ++
.../ambari/server/api/services/ServiceService.java | 18 +
.../api/services/metrics/PrometheusApiService.java | 14 +-
.../services/stackadvisor/StackAdvisorRequest.java | 17 +-
.../stackadvisor/commands/StackAdvisorCommand.java | 25 +
.../api/stomp/ApiStompAuthorizationService.java | 174 ++
.../server/api/stomp/ApiStompDestinations.java | 98 +
.../stomp/ApiStompEventMessageHeaders.java} | 28 +-
.../server/api/stomp/ApiStompEventProjector.java | 247 ++
.../stomp/ApiStompInboundChannelInterceptor.java | 86 +
.../stomp/ApiStompOutboundChannelInterceptor.java | 91 +
.../server/api/stomp/ApiStompSessionRegistry.java | 220 ++
.../api/stomp/NamedTasksSubscribeListener.java | 25 +-
.../server/api/stomp/NamedTasksSubscriptions.java | 8 +-
.../ambari/server/configuration/Configuration.java | 4 +
.../configuration/spring/ApiStompConfig.java | 68 +-
.../AmbariCustomCommandExecutionHelper.java | 125 +-
.../server/controller/AmbariHandlerList.java | 24 +-
.../controller/AmbariManagementControllerImpl.java | 117 +-
.../ambari/server/controller/AmbariServer.java | 10 +-
.../ambari/server/controller/ClusterRequest.java | 21 +
.../ambari/server/controller/ControllerModule.java | 4 +
.../ambari/server/controller/KerberosHelper.java | 26 +
.../server/controller/KerberosHelperImpl.java | 377 ++-
.../ManagedDependencyBlueprintPlan.java | 94 +
.../dependencies/ManagedDependencyCommand.java | 1285 ++++++++++
.../ManagedDependencyCommandBundle.java | 249 ++
.../ManagedDependencyConfigPolicy.java | 157 ++
.../ManagedDependencyCredentialManager.java | 284 +++
.../ManagedDependencyDeploymentCoordinator.java | 634 +++++
.../dependencies/ManagedDependencyDescriptor.java | 226 ++
.../ManagedDependencyDescriptorResolver.java | 1325 ++++++++++
.../dependencies/ManagedDependencyErrorCode.java | 59 +
.../dependencies/ManagedDependencyIdentity.java | 131 +
.../ManagedDependencyIntegrationException.java} | 32 +-
.../ManagedDependencyLifecyclePolicy.java | 467 ++++
.../dependencies/ManagedDependencyNamespace.java | 113 +
.../ManagedDependencyOperationDispatcher.java | 590 +++++
.../ManagedDependencyPairSecurityProof.java | 81 +
.../ManagedDependencyProviderSecurityProof.java | 82 +
.../ManagedDependencyReadinessPolicy.java | 222 ++
.../ManagedDependencyRequestPersistence.java | 53 +
.../ManagedDependencyRuntimePlanner.java | 1080 ++++++++
.../ManagedDependencySecurityMode.java | 10 +-
.../ManagedDependencySecurityPolicyKind.java | 29 +-
...ManagedDependencySecurityValidationContext.java | 44 +
.../dependencies/ManagedDependencyServiceKey.java} | 31 +-
.../dependencies/ManagedDependencySnapshot.java | 170 ++
.../ManagedDependencySnapshotValidator.java | 1035 ++++++++
.../ManagedDependencyStackAdvisorPlanner.java | 369 +++
.../ManagedDependencyTaskResultProcessor.java | 733 ++++++
.../dependencies/ManagedDependencyType.java | 27 +-
.../dependencies/ManagedDependencyVersion.java | 104 +
.../ManagedHBaseConsumerLocalMapping.java | 162 ++
.../ManagedServiceDependencyCoordinator.java | 2624 ++++++++++++++++++++
...sistedManagedHBaseKerberosLivePlanProvider.java | 187 ++
.../security/ManagedHBaseKerberosCalculation.java | 102 +
.../ManagedHBaseKerberosDescriptorOverlay.java | 524 ++++
.../security/ManagedHBaseKerberosLivePlan.java | 38 +
.../ManagedHBaseKerberosLivePlanProvider.java | 39 +
.../security/ManagedHBaseKerberosOverlaySpec.java | 193 ++
.../ManagedHBaseSecurityDescriptorAdapter.java | 324 +++
.../security/ManagedHdfsAuthToLocalVerifier.java | 611 +++++
.../ManagedZooKeeperSaslPolicyVerifier.java | 222 ++
.../internal/AlertGroupResourceProvider.java | 3 +
.../internal/BlueprintConfigurationProcessor.java | 12 +
.../internal/ClusterResourceProvider.java | 5 +-
.../controller/internal/HostResourceProvider.java | 13 +-
.../controller/internal/ProvisionAction.java | 7 +
.../internal/ProvisionClusterRequest.java | 11 +
.../internal/RecommendationResourceProvider.java | 6 +
.../RepositoryVersionResourceProvider.java | 7 +-
.../internal/RequestResourceProvider.java | 84 +-
.../controller/internal/RequestStageContainer.java | 40 +-
.../internal/ServiceResourceProvider.java | 96 +-
.../internal/StackAdvisorResourceProvider.java | 243 +-
.../controller/internal/StageResourceProvider.java | 204 +-
.../controller/internal/TaskResourceProvider.java | 324 ++-
.../internal/ValidationResourceProvider.java | 6 +
.../VersionDefinitionResourceProvider.java | 195 +-
.../server/events/AlertGroupsUpdateEvent.java | 27 +-
.../server/events/ClusterConfigFinishedEvent.java | 12 +
.../server/events/DefaultMessageEmitter.java | 11 +-
.../ambari/server/events/RequestUpdateEvent.java | 6 +
.../alerts/AlertServiceStateListener.java | 8 +
.../listeners/requests/STOMPUpdateListener.java | 12 +-
.../events/listeners/tasks/TaskStatusListener.java | 40 +-
.../events/publishers/AgentCommandsPublisher.java | 7 +
.../events/publishers/STOMPUpdatePublisher.java | 4 +
.../events/publishers/TaskEventPublisher.java | 2 +-
.../ambari/server/hooks/AmbariEventFactory.java | 2 +-
.../server/hooks/users/UserCreatedEvent.java | 8 +-
.../ambari/server/hooks/users/UserHookService.java | 59 +-
.../server/orm/AmbariJpaLocalTxnInterceptor.java | 85 +-
.../ambari/server/orm/dao/AlertDispatchDAO.java | 22 +-
.../apache/ambari/server/orm/dao/ClusterDAO.java | 169 ++
.../ambari/server/orm/dao/ConfigGroupDAO.java | 29 +
.../server/orm/dao/ConfigGroupHostMappingDAO.java | 13 +
.../org/apache/ambari/server/orm/dao/HostDAO.java | 65 +-
.../apache/ambari/server/orm/dao/KeyValueDAO.java | 24 +
.../server/orm/dao/ScopedWorkflowStateDAO.java | 83 +
.../server/orm/dao/ServiceDependencyDAO.java | 1931 ++++++++++++++
.../orm/dao/ServiceDependencyDeploymentDAO.java | 113 +
.../server/orm/dao/ServiceDesiredStateDAO.java | 30 +
.../ambari/server/orm/dao/TopologyRequestDAO.java | 9 +-
.../ambari/server/orm/entities/ClusterEntity.java | 34 +-
.../ambari/server/orm/entities/HostEntity.java | 4 +-
.../orm/entities/ScopedWorkflowStateEntity.java | 125 +
.../entities/ServiceDependencyBindingEntity.java | 215 ++
.../ServiceDependencyDeploymentEntity.java | 80 +
.../orm/entities/ServiceDependencyFenceEntity.java | 99 +
.../ServiceDependencyHostResultEntity.java | 202 ++
.../ServiceDependencyHostResultEntityPK.java | 73 +
.../entities/ServiceDependencyOperationEntity.java | 93 +
.../entities/ServiceDependencySnapshotEntity.java | 101 +
.../ServiceDependencySnapshotEntityPK.java | 63 +
.../server/orm/entities/TopologyRequestEntity.java | 44 +
.../server/orm/helpers/dbms/GenericDbmsHelper.java | 9 +
.../ambari/server/orm/helpers/dbms/H2Helper.java | 10 +-
.../authorization/AmbariAuthorizationFilter.java | 3 +
.../metrics/BuiltinDatasourceProvisioner.java | 16 +
.../server/service/metrics/DatasourceService.java | 20 +
.../service/metrics/MetricsScopeException.java | 25 +-
.../service/metrics/PrometheusQueryClient.java | 55 +-
.../org/apache/ambari/server/state/Cluster.java | 20 +
.../server/state/ClusterCreationContext.java | 49 +-
.../org/apache/ambari/server/state/Clusters.java | 20 +-
.../org/apache/ambari/server/state/ConfigImpl.java | 107 +-
.../ambari/server/state/ServiceComponentImpl.java | 20 +-
.../apache/ambari/server/state/ServiceImpl.java | 12 +-
.../ambari/server/state/cluster/ClusterImpl.java | 169 +-
.../ambari/server/state/cluster/ClustersImpl.java | 427 +++-
.../server/state/configgroup/ConfigGroupImpl.java | 174 +-
.../apache/ambari/server/state/host/HostImpl.java | 12 +-
.../AbstractKerberosDescriptorContainer.java | 4 +-
.../svccomphost/ServiceComponentHostImpl.java | 72 +-
.../ambari/server/topology/AmbariContext.java | 64 +-
.../server/topology/BlueprintValidatorImpl.java | 5 +
.../apache/ambari/server/topology/HostRequest.java | 36 +
.../ambari/server/topology/LogicalRequest.java | 7 +
.../ambari/server/topology/PersistedState.java | 14 +
.../ambari/server/topology/PersistedStateImpl.java | 143 +-
.../server/topology/PersistedTopologyRequest.java | 29 +
.../ambari/server/topology/TopologyManager.java | 810 +++++-
.../addservice/AddServiceOrchestrator.java | 9 +-
.../topology/tasks/ConfigureClusterTask.java | 8 +-
.../tasks/RegisterWithConfigGroupTask.java | 7 +
.../upgrade/ClusterCreationSchemaUpgrade.java | 129 +
.../upgrade/HostMembershipSchemaUpgrade.java | 106 +
.../ambari/server/upgrade/SchemaUpgradeHelper.java | 54 +-
.../upgrade/ScopedWorkflowSchemaUpgrade.java | 98 +
.../upgrade/ServiceDependencySchemaUpgrade.java | 294 +++
.../upgrade/TopologyProvisioningSchemaUpgrade.java | 67 +
.../src/main/resources/Ambari-DDL-Derby-CREATE.sql | 158 +-
.../src/main/resources/Ambari-DDL-H2-CREATE.sql | 158 +-
.../src/main/resources/Ambari-DDL-MySQL-CREATE.sql | 105 +-
.../main/resources/Ambari-DDL-Oracle-CREATE.sql | 105 +-
.../main/resources/Ambari-DDL-Postgres-CREATE.sql | 105 +-
.../resources/Ambari-DDL-SQLAnywhere-CREATE.sql | 105 +-
.../main/resources/Ambari-DDL-SQLServer-CREATE.sql | 112 +-
.../src/main/resources/META-INF/persistence.xml | 7 +
.../stack-hooks/before-ANY/scripts/params.py | 12 +-
.../before-ANY/scripts/shared_initialization.py | 2 +-
.../BIGTOP/3.2.0/properties/stack_packages.json | 13 +-
.../BIGTOP/3.2.0/services/HBASE/metainfo.xml | 64 +
.../3.2.0/services/HBASE/package/scripts/hbase.py | 28 +-
.../services/HBASE/package/scripts/hbase_client.py | 6 +-
.../HBASE/package/scripts/hbase_decommission.py | 4 +-
.../services/HBASE/package/scripts/hbase_master.py | 33 +-
.../HBASE/package/scripts/hbase_regionserver.py | 33 +-
.../HBASE/package/scripts/hbase_service.py | 2 +-
.../services/HBASE/package/scripts/hbase_thrift.py | 33 +-
.../package/scripts/managed_hbase_dependency.py | 1443 +++++++++++
.../services/HBASE/package/scripts/params_linux.py | 34 +-
.../HBASE/package/scripts/service_check.py | 74 +-
.../services/HBASE/package/scripts/upgrade.py | 15 +-
.../services/HDFS/configuration/hdfs-site.xml | 1 -
.../stacks/BIGTOP/3.2.0/services/HDFS/metainfo.xml | 36 +
.../package/scripts/managed_hdfs_dependency.py | 390 +++
.../services/HDFS/package/scripts/namenode.py | 26 +
.../BIGTOP/3.2.0/services/HDFS/service_advisor.py | 4 -
.../KERBEROS/package/scripts/kerberos_client.py | 5 +
.../VICTORIAMETRICS/package/scripts/params.py | 1 +
.../package/templates/promscrape.yml.j2 | 1 +
.../3.2.0/services/YARN/package/scripts/service.py | 18 +-
.../BIGTOP/3.2.0/services/YARN/service_advisor.py | 8 +-
.../BIGTOP/3.2.0/services/ZOOKEEPER/metainfo.xml | 38 +
.../scripts/managed_zookeeper_dependency.py | 457 ++++
.../ZOOKEEPER/package/scripts/zookeeper_server.py | 32 +
.../BIGTOP/3.3.0/services/HBASE/metainfo.xml | 6 +
.../src/main/resources/stacks/stack_advisor.py | 47 +-
.../actionmanager/ActionManagerTestHelper.java | 11 +
.../actionmanager/TestActionDBAccessorImpl.java | 12 +-
.../server/actionmanager/TestActionManager.java | 23 +-
.../server/actionmanager/TestActionScheduler.java | 8 +-
.../server/agent/CommandReportOwnershipTest.java | 241 ++
.../server/agent/HeartbeatMonitorRecoveryTest.java | 99 +
.../server/agent/HeartbeatProcessorTest.java | 12 +-
.../ambari/server/agent/TestHeartbeatHandler.java | 3 +
.../agent/stomp/PrometheusTargetDiscoveryTest.java | 2 +
.../server/api/ContentTypeOverrideFilterTest.java | 50 +
.../server/api/handlers/DeleteHandlerTest.java | 17 +
.../server/api/services/BaseServiceTest.java | 16 +
.../services/ManagedDependencyApiSupportTest.java | 118 +
.../ManagedServiceDependencyServiceTest.java | 129 +
.../api/services/PersistKeyValueImplTest.java | 526 +++-
.../server/api/services/PersistServiceTest.java | 105 +-
.../ServiceDependenciesApiServiceTest.java | 145 ++
.../services/metrics/PrometheusApiServiceTest.java | 45 +
.../commands/StackAdvisorCommandTest.java | 99 +
.../stomp/ApiStompAuthorizationServiceTest.java | 81 +
.../api/stomp/ApiStompBrokerIsolationTest.java | 126 +
.../api/stomp/ApiStompEventProjectorTest.java | 169 ++
.../ApiStompInboundChannelInterceptorTest.java | 136 +
.../ApiStompOutboundChannelInterceptorTest.java | 85 +
.../api/stomp/ApiStompSessionRegistryTest.java | 161 ++
.../api/stomp/NamedTasksSubscribeListenerTest.java | 80 +
.../api/stomp/NamedTasksSubscriptionsTest.java | 3 +
.../spring/StompWebSocketConfigurationTest.java | 68 +
.../AmbariCustomCommandExecutionHelperTest.java | 76 +
.../server/controller/AmbariHandlerListTest.java | 58 +-
.../AmbariManagementControllerImplTest.java | 117 +
.../controller/AmbariManagementControllerTest.java | 24 +-
.../server/controller/KerberosHelperTest.java | 4 +
.../ManagedHBaseKerberosCalculationTest.java | 634 +++++
.../ManagedDependencyBlueprintPlanTest.java | 107 +
.../dependencies/ManagedDependencyCommandTest.java | 696 ++++++
.../ManagedDependencyConfigPolicyTest.java | 137 +
.../ManagedDependencyCredentialManagerTest.java | 163 ++
...ManagedDependencyDeploymentCoordinatorTest.java | 404 +++
.../ManagedDependencyDescriptorResolverTest.java | 969 ++++++++
.../ManagedDependencyLifecyclePolicyTest.java | 262 ++
.../ManagedDependencyOperationDispatcherTest.java | 394 +++
.../ManagedDependencyReadinessPolicyTest.java | 190 ++
.../ManagedDependencyRuntimePlannerTest.java | 713 ++++++
.../ManagedDependencySnapshotValidatorTest.java | 711 ++++++
.../ManagedDependencyStackAdvisorPlannerTest.java | 164 ++
.../ManagedDependencyTaskResultProcessorTest.java | 264 ++
.../ManagedHBaseConsumerLocalMappingTest.java | 143 ++
.../ManagedServiceDependencyCoordinatorTest.java | 1030 ++++++++
...edManagedHBaseKerberosLivePlanProviderTest.java | 366 +++
.../ManagedHBaseKerberosDescriptorOverlayTest.java | 324 +++
.../ManagedHBaseKerberosOverlaySpecTest.java | 118 +
.../ManagedHBaseSecurityDescriptorAdapterTest.java | 558 +++++
.../ManagedHdfsAuthToLocalVerifierTest.java | 325 +++
.../ManagedZooKeeperSaslPolicyVerifierTest.java | 109 +
.../internal/AlertGroupResourceProviderTest.java | 57 +-
.../internal/ClusterResourceProviderTest.java | 25 +
.../internal/HostResourceProviderTest.java | 36 +
.../PreUpgradeCheckResourceProviderTest.java | 4 +
.../internal/ProvisionClusterRequestTest.java | 11 +
.../RepositoryVersionResourceProviderTest.java | 7 +-
.../internal/RequestResourceProviderTest.java | 107 +-
.../internal/ServiceResourceProviderTest.java | 150 ++
.../internal/StackAdvisorResourceProviderTest.java | 273 ++
.../internal/StageResourceProviderTest.java | 162 ++
.../internal/TaskResourceProviderTest.java | 331 ++-
.../VersionDefinitionResourceProviderTest.java | 196 +-
.../server/credentialapi/CredentialUtilTest.java | 8 +-
.../listeners/tasks/TaskStatusListenerTest.java | 92 +-
.../server/hooks/users/UserHookServiceTest.java | 67 +-
.../dispatchers/AmbariSNMPDispatcherTest.java | 4 +-
.../dispatchers/SNMPDispatcherTest.java | 3 +-
.../server/orm/dao/AlertDispatchDAOTest.java | 6 +-
.../dao/ServiceDependencyDAOIntegrationTest.java | 1691 +++++++++++++
.../server/orm/dao/ServiceDependencyDAOTest.java | 138 +
.../server/security/TestAuthenticationFactory.java | 10 +
.../AbstractPrepareKerberosServerActionTest.java | 4 +
.../upgrades/PreconfigureKerberosActionTest.java | 4 +
.../metrics/BuiltinDatasourceProvisionerTest.java | 50 +
.../service/metrics/PrometheusQueryClientTest.java | 73 +
.../metrics/PrometheusScrapeContractTest.java | 6 +
.../ambari/server/state/cluster/ClusterTest.java | 2 +-
.../ambari/server/state/cluster/ClustersTest.java | 564 ++++-
.../ambari/server/topology/AmbariContextTest.java | 41 +-
.../topology/ClusterDeployWithStartOnlyTest.java | 15 +-
...terInstallWithoutStartOnComponentLevelTest.java | 9 +-
.../topology/ClusterInstallWithoutStartTest.java | 9 +-
.../ambari/server/topology/HostRequestTest.java | 122 +
.../server/topology/PersistedStateImplTest.java | 85 +
.../server/topology/TopologyManagerTest.java | 419 +++-
.../ambari/server/update/HostUpdateHelperTest.java | 23 +-
.../upgrade/ClusterCreationSchemaUpgradeTest.java | 127 +
.../upgrade/HostMembershipSchemaUpgradeTest.java | 96 +
.../server/upgrade/SchemaUpgradeHelperTest.java | 216 ++
.../upgrade/ScopedWorkflowSchemaUpgradeTest.java | 101 +
.../ServiceDependencySchemaUpgradeTest.java | 148 ++
.../TopologyProvisioningSchemaUpgradeTest.java | 74 +
.../src/test/python/TestHbaseBigtopLifecycle.py | 3 +-
.../src/test/python/TestHbaseBigtopWorkflows.py | 79 +-
ambari-server/src/test/python/TestHdfsBigtop.py | 17 +
.../src/test/python/TestKerberosBigtop.py | 28 +
.../python/TestManagedHbaseDependencyBigtop.py | 1892 ++++++++++++++
.../test/python/TestManagedHdfsDependencyBigtop.py | 454 ++++
.../python/TestManagedZookeeperDependencyBigtop.py | 359 +++
.../src/test/python/TestYarnBigtopLifecycle.py | 30 +-
.../src/test/python/TestYarnBigtopWorkflows.py | 27 +-
.../stack-hooks/before-ANY/test_before_any.py | 31 +
.../test_managed_dependency_stack_advisor.py | 180 ++
ambari-web/classic/app/assets/test/tests.js | 1 +
.../global/wizard_watcher_controller.js | 74 +-
ambari-web/classic/app/controllers/installer.js | 85 +-
.../app/controllers/wizard/step8_controller.js | 279 +--
.../app/mappers/repository_version_mapper.js | 3 +-
ambari-web/classic/app/mappers/stack_mapper.js | 3 +-
ambari-web/classic/app/messages.js | 17 +
.../classic/app/mixins/common/localStorage.js | 3 +-
ambari-web/classic/app/models/cluster_states.js | 131 +-
ambari-web/classic/app/models/repository.js | 2 +
ambari-web/classic/app/utils.js | 1 +
ambari-web/classic/app/utils/ajax/ajax.js | 19 +-
ambari-web/classic/app/utils/db.js | 327 ++-
.../app/utils/scoped_workflow_persistence.js | 782 ++++++
.../global/wizard_watcher_controller_test.js | 88 +-
.../classic/test/controllers/installer_test.js | 120 +
.../classic/test/controllers/wizard/step8_test.js | 320 +--
.../classic/test/models/cluster_states_test.js | 67 +
ambari-web/classic/test/utils/db_test.js | 106 +-
.../test/utils/scoped_workflow_persistence_test.js | 402 +++
ambari-web/latest/package-lock.json | 15 +
ambari-web/latest/package.json | 1 +
ambari-web/latest/src/AppLoader.test.tsx | 104 +-
ambari-web/latest/src/AppLoader.tsx | 179 +-
ambari-web/latest/src/Utils/adminViewRedirect.ts | 11 +-
ambari-web/latest/src/Utils/authNavigation.ts | 29 +-
ambari-web/latest/src/Utils/authPolicy.test.ts | 11 +
ambari-web/latest/src/Utils/authPolicy.ts | 8 +
ambari-web/latest/src/Utils/cacheUtils.ts | 6 +
.../src/Utils/centralizedServiceStateUtils.ts | 3 +-
ambari-web/latest/src/Utils/clusterEvents.test.ts | 41 +
ambari-web/latest/src/Utils/clusterEvents.ts | 87 +
ambari-web/latest/src/Utils/clusterRoute.test.ts | 70 +
ambari-web/latest/src/Utils/clusterRoute.ts | 85 +
ambari-web/latest/src/Utils/db.ts | 2 +-
.../src/Utils/managedDependencyPresentation.ts | 56 +
.../latest/src/Utils/runtimeIdentity.test.ts | 33 +-
.../runtimeIdentity.ts} | 35 +-
ambari-web/latest/src/Utils/scopedWorkflow.test.ts | 500 ++++
ambari-web/latest/src/Utils/scopedWorkflow.ts | 458 ++++
.../latest/src/Utils/serviceNavigation.test.ts | 23 +
.../latest/src/Utils/uuid.ts | 25 +-
.../latest/src/Utils/workflowReturnPath.test.ts | 121 +
ambari-web/latest/src/Utils/workflowReturnPath.ts | 76 +
.../src/Utils/workflowSelectionIntent.test.ts | 91 +
.../latest/src/Utils/workflowSelectionIntent.ts | 70 +
ambari-web/latest/src/api/actionsApi.ts | 13 +-
ambari-web/latest/src/api/adminApi.ts | 13 +-
ambari-web/latest/src/api/alertsApi.ts | 35 +-
ambari-web/latest/src/api/apiPath.test.ts | 37 +
.../latest/src/api/apiPath.ts | 8 +-
ambari-web/latest/src/api/cachedServiceApi.test.ts | 105 +
ambari-web/latest/src/api/cachedServiceApi.ts | 80 +-
.../src/api/centralizedServiceStateApi.test.ts | 83 +
.../latest/src/api/centralizedServiceStateApi.ts | 89 +-
ambari-web/latest/src/api/chooseServicesApi.ts | 5 +-
ambari-web/latest/src/api/clusterApi.ts | 43 +-
ambari-web/latest/src/api/clusterDeployment.ts | 11 +-
ambari-web/latest/src/api/componentApi.ts | 3 +-
ambari-web/latest/src/api/configGroupApi.ts | 13 +-
ambari-web/latest/src/api/configHistoryApi.ts | 7 +-
ambari-web/latest/src/api/configsApi.ts | 37 +-
ambari-web/latest/src/api/hostsApi.ts | 87 +-
ambari-web/latest/src/api/kerberosApi.ts | 3 +-
ambari-web/latest/src/api/logApi.ts | 3 +-
ambari-web/latest/src/api/metricsApi.test.ts | 13 +-
ambari-web/latest/src/api/metricsApi.ts | 7 +
ambari-web/latest/src/api/nameNodeFederationApi.ts | 7 +-
ambari-web/latest/src/api/observerNameNodeApi.ts | 3 +-
ambari-web/latest/src/api/quicklinksApi.ts | 5 +-
ambari-web/latest/src/api/requestApi.ts | 41 +-
ambari-web/latest/src/api/serviceApi.ts | 27 +-
ambari-web/latest/src/api/serviceConfigApi.ts | 5 +-
.../latest/src/api/serviceDependenciesApi.test.ts | 265 ++
.../latest/src/api/serviceDependenciesApi.ts | 564 +++++
ambari-web/latest/src/api/servicesApi.ts | 3 +-
ambari-web/latest/src/api/versionsApi.ts | 31 +-
ambari-web/latest/src/api/workflowStateApi.test.ts | 58 +
ambari-web/latest/src/api/workflowStateApi.ts | 98 +
.../latest/src/components/AdminRouteGuard.tsx | 4 +-
.../latest/src/components/AssignMasters.test.tsx | 399 +++
ambari-web/latest/src/components/AssignMasters.tsx | 404 +--
.../src/components/AssignMastersAddable.test.tsx | 573 ++++-
.../latest/src/components/AssignMastersAddable.tsx | 269 +-
ambari-web/latest/src/components/AuthGuard.tsx | 6 +-
.../latest/src/components/ClusterSwitchMenu.tsx | 92 +
.../latest/src/components/FeatureRouteGuard.tsx | 4 +-
ambari-web/latest/src/components/Navbar.tsx | 34 +-
.../latest/src/components/NotificationDropdown.tsx | 12 +-
.../latest/src/components/ScopedLink.tsx | 24 +-
.../latest/src/components/ScopedNavigate.tsx | 16 +-
.../src/components/ServiceOperationRouteGuard.tsx | 4 +-
.../latest/src/components/Sidebar/SidebarItem.tsx | 10 +-
.../components/Sidebar/SidebarItemCollapsed.tsx | 4 +-
ambari-web/latest/src/components/UpgradeGuard.tsx | 4 +-
ambari-web/latest/src/hooks/useAuth.ts | 40 +-
ambari-web/latest/src/hooks/useClusterNavigate.ts | 56 +
.../latest/src/hooks/useClusterPath.ts | 18 +-
.../src/hooks/useClusterWorkflowPersistence.ts | 114 +
ambari-web/latest/src/hooks/useEnhancedConfigs.ts | 262 +-
.../latest/src/hooks/useHDFSConfigUpdater.ts | 6 +-
.../latest/src/hooks/useHbaseConfigUpdater.ts | 6 +-
.../latest/src/hooks/useHiveConfigUpdater.ts | 6 +-
.../latest/src/hooks/useKerberosConfigUpdater.ts | 6 +-
.../latest/src/hooks/useKyuubiConfigUpdater.ts | 7 +-
.../latest/src/hooks/useMapReduce2ConfigUpdater.ts | 6 +-
.../latest/src/hooks/usePinotConfigUpdater.ts | 6 +-
.../latest/src/hooks/useRangerConfigUpdater.ts | 4 +-
.../latest/src/hooks/useRangerKMSConfigUpdater.ts | 7 +-
ambari-web/latest/src/hooks/useSSMConfigUpdater.ts | 7 +-
ambari-web/latest/src/hooks/useServiceCache.ts | 11 +-
ambari-web/latest/src/hooks/useServiceDeletion.tsx | 4 +-
.../latest/src/hooks/useSpark3ConfigUpdater.ts | 6 +-
.../latest/src/hooks/useSqoopConfigUpdater.ts | 6 +-
ambari-web/latest/src/hooks/useStackServices.ts | 7 +-
.../src/hooks/useStepWizard.navigation.test.tsx | 52 +
ambari-web/latest/src/hooks/useStepWizard.ts | 10 +-
ambari-web/latest/src/hooks/useTezConfigUpdater.ts | 6 +-
.../latest/src/hooks/useTrinoConfigUpdater.tsx | 4 +-
.../src/hooks/useTrinoGatewayConfigUpdater.ts | 7 +-
ambari-web/latest/src/hooks/useUpgrade.test.tsx | 17 +-
ambari-web/latest/src/hooks/useUpgrade.ts | 52 +-
.../latest/src/hooks/useYarnConfigUpdater.ts | 8 +-
ambari-web/latest/src/hooks/useZkConfigUpdater.ts | 6 +-
ambari-web/latest/src/layout/Main.tsx | 2 +-
ambari-web/latest/src/locales/en/translation.json | 243 ++
ambari-web/latest/src/locales/zh/translation.json | 243 ++
ambari-web/latest/src/router/RoutesList.test.tsx | 32 +-
ambari-web/latest/src/router/RoutesList.tsx | 42 +-
.../latest/src/screens/Alerts/ActionsButton.tsx | 5 +-
.../src/screens/Alerts/AlertDefinitionWizard.tsx | 5 +-
.../src/screens/Alerts/AlertInstancesTable.tsx | 10 +-
ambari-web/latest/src/screens/Alerts/Alerts.tsx | 6 +-
.../latest/src/screens/Alerts/HeaderSection.tsx | 10 +-
.../latest/src/screens/Alerts/HostAlerts.tsx | 10 +-
.../Authentication/AdminViewRedirect.test.ts | 13 +-
.../screens/Authentication/AdminViewRedirect.tsx | 2 +-
.../BackgroundOperations.test.tsx | 10 +-
.../src/screens/BackgroundOperations/index.tsx | 4 +-
.../ClusterAdmin/StackAndVersions/ListStack.tsx | 20 +-
.../ClusterAdmin/StackAndVersions/ListVersion.tsx | 168 +-
.../StackAndVersions/ListVersionRecovery.test.ts | 53 +
.../StackAndVersions/PreUpgradeCheckItem.tsx | 4 +-
.../StackAndVersions/StackAndVersions.tsx | 5 +-
.../ClusterAdmin/StackAndVersions/Upgrade.tsx | 32 +-
.../ManagedDependencySelector.test.tsx | 487 ++++
.../ClusterWizard/ManagedDependencySelector.tsx | 633 +++++
.../ManagedDependencySettings.test.tsx | 64 +
.../ClusterWizard/ManagedDependencySettings.tsx | 121 +
.../ClusterWizard/ManagedDeploymentProgress.tsx | 231 ++
.../latest/src/screens/ClusterWizard/Step1.tsx | 50 +-
.../src/screens/ClusterWizard/Step10.test.tsx | 13 +
.../latest/src/screens/ClusterWizard/Step10.tsx | 21 +-
.../src/screens/ClusterWizard/Step4.test.tsx | 417 +++-
.../latest/src/screens/ClusterWizard/Step4.tsx | 452 +++-
.../src/screens/ClusterWizard/Step5.test.tsx | 443 +++-
.../latest/src/screens/ClusterWizard/Step5.tsx | 143 +-
.../src/screens/ClusterWizard/Step6.test.tsx | 176 ++
.../latest/src/screens/ClusterWizard/Step6.tsx | 171 +-
.../ClusterWizard/Step7/RestAllTabs.test.tsx | 341 +++
.../screens/ClusterWizard/Step7/RestAllTabs.tsx | 23 +
.../src/screens/ClusterWizard/Step7/index.test.tsx | 326 ++-
.../src/screens/ClusterWizard/Step7/index.tsx | 521 +++-
.../src/screens/ClusterWizard/Step8.test.tsx | 1177 ++++++++-
.../latest/src/screens/ClusterWizard/Step8.tsx | 1417 ++++++++++-
.../src/screens/ClusterWizard/Step9.test.tsx | 170 ++
.../latest/src/screens/ClusterWizard/Step9.tsx | 103 +-
.../ClusterWizard/clusterConfigPayload.test.ts | 100 +
.../screens/ClusterWizard/clusterConfigPayload.ts | 38 +-
.../ClusterWizard/clusterStore/context.test.tsx | 408 +++
.../screens/ClusterWizard/clusterStore/context.tsx | 280 ++-
.../ClusterWizard/deploymentInputRecovery.test.ts | 182 ++
.../ClusterWizard/deploymentInputRecovery.ts | 210 ++
.../ClusterWizard/installationProgress.test.ts | 16 +
.../screens/ClusterWizard/installationProgress.ts | 42 +-
.../ClusterWizard/managedDependencyAdvisor.test.ts | 176 ++
.../ClusterWizard/managedDependencyAdvisor.ts | 230 ++
.../ClusterWizard/managedDependencyConfig.test.ts | 130 +
.../ClusterWizard/managedDependencyConfig.ts | 137 +
.../managedDependencyMaterialization.test.ts | 372 +++
.../managedDependencyMaterialization.ts | 235 ++
.../managedDependencySelection.test.ts | 68 +
.../ClusterWizard/managedDependencySelection.ts | 72 +
.../repositoryVersionResolution.test.ts | 174 ++
.../ClusterWizard/repositoryVersionResolution.ts | 255 ++
.../ClusterWizard/types/AssignMastersTypes.ts | 3 +
.../latest/src/screens/ClusterWizard/types/Os.ts | 8 +-
.../ClusterWizard/types/VersionsDefinition.ts | 1 +
.../latest/src/screens/CommonConfigs/Config.tsx | 73 +-
.../latest/src/screens/Dashboard/ConfigHistory.tsx | 4 +-
ambari-web/latest/src/screens/Dashboard/Index.tsx | 5 +-
.../screens/Directories/ClusterDirectory.test.tsx | 116 +
.../src/screens/Directories/ClusterDirectory.tsx | 320 +++
.../screens/Directories/ClusterTasksRoute.test.tsx | 87 +
.../src/screens/Directories/ClusterTasksRoute.tsx | 49 +
.../screens/Directories/DirectoryPagination.tsx | 60 +
.../screens/Directories/DirectorySortControls.tsx | 72 +
.../Directories/GlobalDirectoryLayout.test.tsx | 56 +
.../screens/Directories/GlobalDirectoryLayout.tsx | 54 +
.../screens/Directories/ServiceDirectory.test.tsx | 466 ++++
.../src/screens/Directories/ServiceDirectory.tsx | 556 +++++
.../src/screens/Directories/directories.scss | 129 +
.../src/screens/Directories/directoryUtils.test.ts | 99 +
.../src/screens/Directories/directoryUtils.ts | 161 ++
.../latest/src/screens/Experimental/index.tsx | 4 +-
.../AddHostWizard/wizardDataStore/context.test.tsx | 125 +-
.../AddHostWizard/wizardDataStore/context.tsx | 94 +-
.../latest/src/screens/Hosts/HostLogMetrics.tsx | 6 +-
.../latest/src/screens/Hosts/HostSummary.tsx | 11 +-
ambari-web/latest/src/screens/Hosts/HostsList.tsx | 14 +-
ambari-web/latest/src/screens/Hosts/details.tsx | 7 +-
.../src/screens/Hosts/hostsFilterNavigation.ts | 4 +-
ambari-web/latest/src/screens/Hosts/index.tsx | 6 +-
.../screens/KerberosWizard/ConfigureKerberos.tsx | 12 +-
.../KerberosWizard/ConfirmConfiguration.tsx | 6 +-
.../screens/KerberosWizard/EnableKerberos.test.tsx | 35 +-
.../src/screens/KerberosWizard/EnableKerberos.tsx | 36 +-
.../src/screens/KerberosWizard/GetStarted.tsx | 6 +-
.../screens/KerberosWizard/KerberizeCluster.tsx | 6 +-
.../KerberosWizard/KerberosStore/context.test.tsx | 94 +-
.../KerberosWizard/KerberosStore/context.tsx | 162 +-
.../KerberosWizard/StartAndTestKerberosClient.tsx | 6 +-
.../KerberosWizard/StartAndTestServices.tsx | 11 +-
.../src/screens/KerberosWizard/StopServices.tsx | 6 +-
.../screens/Monitoring/Dashboard/DashboardPage.tsx | 10 +-
.../Monitoring/Dashboard/DashboardPanel.tsx | 3 +-
.../Dashboard/renderers/HexbinRenderer.tsx | 20 +-
.../Dashboard/renderers/TableRenderer.tsx | 20 +-
.../latest/src/screens/Monitoring/Dashboards.tsx | 7 +-
.../latest/src/screens/Monitoring/Explorer.tsx | 88 +-
.../src/screens/Monitoring/MonitoringLayout.tsx | 9 +-
.../src/screens/Monitoring/ScopedMetadata.test.tsx | 246 ++
.../latest/src/screens/Monitoring/Targets.tsx | 53 +-
.../latest/src/screens/ServiceConfigs/index.tsx | 9 +-
.../latest/src/screens/Services/Actions.test.tsx | 22 +-
ambari-web/latest/src/screens/Services/Actions.tsx | 27 +-
.../wizardDataStore/context.test.tsx | 650 +++++
.../AddServiceWizard/wizardDataStore/context.tsx | 424 +++-
.../src/screens/Services/DependencyActions.tsx | 130 +
.../latest/src/screens/Services/FlumeSummary.tsx | 5 +-
.../screens/Services/HDFSFederationTopology.tsx | 6 +-
.../latest/src/screens/Services/ServiceAlerts.tsx | 4 +-
.../src/screens/Services/ServiceComponents.tsx | 32 +-
.../src/screens/Services/ServiceDashboard.tsx | 39 +-
.../screens/Services/ServiceDependencies.test.tsx | 173 ++
.../src/screens/Services/ServiceDependencies.tsx | 232 ++
.../screens/Services/ServiceDependents.test.tsx | 201 ++
.../src/screens/Services/ServiceDependents.tsx | 146 ++
.../src/screens/Services/ServiceRestartModal.tsx | 7 +-
.../Federation/PersistedWorkflowContext.test.tsx | 85 +
.../Federation/PersistedWorkflowContext.tsx | 173 +-
.../Services/highAvailibility/Federation/Step4.tsx | 4 +-
.../Services/highAvailibility/Federation/index.tsx | 5 +-
.../highAvailibility/Federation/store/context.tsx | 157 +-
.../Federation/validateEnablement.tsx | 19 +-
.../highAvailibility/HawqStandby/Steps.tsx | 4 +-
.../highAvailibility/HawqStandby/index.tsx | 2 +-
.../highAvailibility/RouterFederation/Steps.tsx | 4 +-
.../highAvailibility/RouterFederation/index.tsx | 2 +-
.../Services/highAvailibility/WorkflowActions.tsx | 4 +-
.../highAvailibility/journalNode/Step7.tsx | 4 +-
.../journalNode/ValidateEnablement.tsx | 21 +-
.../highAvailibility/journalNode/index.tsx | 5 +-
.../journalNode/store/context.test.tsx | 40 +-
.../highAvailibility/journalNode/store/context.tsx | 138 +-
.../Services/highAvailibility/nameNode/Step9.tsx | 5 +-
.../nameNode/ValidateEnablement.tsx | 19 +-
.../Services/highAvailibility/nameNode/index.tsx | 5 +-
.../nameNode/store/context.test.tsx | 61 +-
.../highAvailibility/nameNode/store/context.tsx | 134 +-
.../highAvailibility/observerNameNode/Step4.tsx | 5 +-
.../highAvailibility/observerNameNode/index.tsx | 5 +-
.../observerNameNode/store/context.tsx | 181 +-
.../observerNameNode/validateEnablement.tsx | 19 +-
.../highAvailibility/rangerAdmin/Step4.tsx | 4 +-
.../rangerAdmin/ValidateEnablement.test.tsx | 11 +-
.../rangerAdmin/ValidateEnablement.tsx | 16 +-
.../highAvailibility/rangerAdmin/index.tsx | 5 +-
.../rangerAdmin/store/context.test.tsx | 38 +-
.../highAvailibility/rangerAdmin/store/context.tsx | 140 +-
.../highAvailibility/resourceManager/Step4.tsx | 5 +-
.../resourceManager/ValidateEnablement.test.tsx | 12 +-
.../resourceManager/ValidateEnablement.tsx | 27 +-
.../resourceManager/index.test.tsx | 51 +-
.../highAvailibility/resourceManager/index.tsx | 5 +-
.../resourceManager/store/context.test.tsx | 52 +-
.../resourceManager/store/context.tsx | 138 +-
.../latest/src/screens/Services/reassign/Step1.tsx | 4 +-
.../latest/src/screens/Services/reassign/Step2.tsx | 6 +-
.../latest/src/screens/Services/reassign/Step3.tsx | 6 +-
.../latest/src/screens/Services/reassign/Step4.tsx | 12 +-
.../latest/src/screens/Services/reassign/Step6.tsx | 4 +-
.../src/screens/Services/reassign/ValidateMove.tsx | 27 +-
.../latest/src/screens/Services/reassign/index.tsx | 5 +-
.../Services/reassign/store/context.test.tsx | 83 +
.../screens/Services/reassign/store/context.tsx | 136 +-
.../src/screens/Services/serviceRestartUtils.ts | 12 +-
.../screens/Services/useDependencyImpact.test.tsx | 48 +
.../src/screens/Services/useDependencyImpact.tsx | 82 +
ambari-web/latest/src/store/AlertsContext.tsx | 10 +-
ambari-web/latest/src/store/ServiceContext.tsx | 27 +-
ambari-web/latest/src/store/UserContext.test.tsx | 119 +
ambari-web/latest/src/store/UserContext.tsx | 116 +-
ambari-web/latest/src/store/context.test.tsx | 66 +-
ambari-web/latest/src/store/context.tsx | 241 +-
ambari-web/latest/src/test/setup.ts | 10 +
ambari-web/latest/src/types/auth.ts | 20 +-
docs/design/multi-cluster-management.md | 679 +++++
.../design/multi-cluster-review-and-remediation.md | 1515 +++++++++++
.../react-current/04-services-configs-gap.md | 18 +-
.../react-current/07-cluster-installation-gap.md | 134 +-
.../react-current/10-rm-ranger-ha-gap.md | 2 +-
docs/frontend-refactor/react-current/README.md | 5 +
.../react-current/multi-cluster-foundation.md | 137 +
687 files changed, 83286 insertions(+), 4833 deletions(-)
diff --git a/AGENTS.md b/AGENTS.md
index 00140974eb..ba5bd44e99 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -56,6 +56,31 @@ include:
- Do not claim a test passed unless the command was actually run successfully.
Report skipped tests and existing failures explicitly.
+## Reliable State And Result Checks
+
+- Do not infer business state, readiness, success, failure, permissions,
+ ownership, versions, or operation lineage by matching log messages, parsing
+ human-readable CLI output, searching for keywords, or using fuzzy strings.
+ This rule applies to production code, deployment tooling, acceptance scripts,
+ and tests. Logs and human-readable output are diagnostic evidence only.
+- Use authoritative API fields, typed client results, documented status or
+ error codes, and exact persistent identifiers. Exact comparisons of enum
+ values or identifiers defined by a machine-readable contract are valid;
+ inferring their meaning from arbitrary text is not.
+- When a client library is needed, expose its typed observations through a
+ defined, versioned machine-readable schema. Validate the schema, required
+ fields, types, and operation identity before consuming the result. Wrapping
+ parsed CLI text in JSON does not satisfy this requirement.
+- A successful process exit alone does not prove a business operation
+ completed. Confirm the authoritative result and its request/task or
+ binding/operation/epoch lineage before advancing persistent workflow state.
+ Missing, malformed, ambiguous, or mismatched results must not become success;
+ preserve an explicit failure or unresolved state for recovery.
+- Regression tests must exercise structured observations and failure paths,
+ including diagnostic noise, missing or invalid results, and stale or foreign
+ operation identities. Do not prove correctness only by mocking expected
+ output strings or checking that a command was invoked.
+
## Frontend Refactor Baseline
`docs/frontend-refactor` is temporary working material for the migration from
diff --git a/Jenkinsfile b/Jenkinsfile
index 7f879c8319..2a0cbec3e4 100644
--- a/Jenkinsfile
+++ b/Jenkinsfile
@@ -119,7 +119,8 @@ pipeline {
stage('Ambari Java Tests') {
steps {
- sh 'mvn -am test -pl ambari-server,ambari-funtest -DskipPythonTests -DskipFunctionalTests=false -Dmaven.artifact.threads=10 -Drat.skip -DskipAdminWebTests=true -DskipUiBuild=true'
+ // Isolated forks keep static state and embedded databases from leaking between test classes.
+ sh 'mvn -B -am test -pl ambari-server,ambari-funtest -DforkCount=8 -DreuseForks=false -DskipPythonTests -DskipFunctionalTests=false -Dmaven.artifact.threads=10 -Drat.skip -DskipAdminWebTests=true -DskipUiBuild=true'
}
}
@@ -139,7 +140,7 @@ pipeline {
sh 'flock "$HOME/.m2/.frontend-install.lock" mvn -B -pl ambari-admin com.github.eirslett:frontend-maven-plugin:1.11.0:install-node-and-npm@install-node-and-npm-react'
dir('ambari-admin/src/main/resources/ui/ambari-admin') {
sh './node/node ./node/node_modules/npm/bin/npm-cli.js ci --no-audit --no-fund'
- sh './node/node ./node/node_modules/npm/bin/npm-cli.js test -- --run'
+ sh './node/node ./node/node_modules/npm/bin/npm-cli.js test -- --run --maxWorkers=4'
sh './node/node ./node/node_modules/npm/bin/npm-cli.js run build'
}
}
@@ -149,7 +150,7 @@ pipeline {
sh 'flock "$HOME/.m2/.frontend-install.lock" mvn -B -pl ambari-web com.github.eirslett:frontend-maven-plugin:1.11.0:install-node-and-npm@install-node-and-npm-react'
dir('ambari-web/latest') {
sh './node/node ./node/node_modules/npm/bin/npm-cli.js ci --no-audit --no-fund'
- sh './node/node ./node/node_modules/npm/bin/npm-cli.js test'
+ sh './node/node ./node/node_modules/npm/bin/npm-cli.js test -- --maxWorkers=4'
sh './node/node ./node/node_modules/npm/bin/npm-cli.js run build'
}
}
@@ -159,7 +160,7 @@ pipeline {
sh 'flock "$HOME/.m2/.frontend-install.lock" mvn -B -f contrib/views/files/pom.xml com.github.eirslett:frontend-maven-plugin:1.11.0:install-node-and-npm@install-node-and-npm'
dir('contrib/views/files/src/main/resources/ui') {
sh './node/node ./node/node_modules/npm/bin/npm-cli.js ci --no-audit --no-fund'
- sh './node/node ./node/node_modules/npm/bin/npm-cli.js test'
+ sh './node/node ./node/node_modules/npm/bin/npm-cli.js test -- --maxWorkers=4'
sh './node/node ./node/node_modules/npm/bin/npm-cli.js run build'
}
}
@@ -169,7 +170,7 @@ pipeline {
sh 'flock "$HOME/.m2/.frontend-install.lock" mvn -B -f contrib/views/capacity-scheduler/pom.xml com.github.eirslett:frontend-maven-plugin:1.11.0:install-node-and-npm@install-node-and-npm'
dir('contrib/views/capacity-scheduler/src/main/resources/ui') {
sh './node/node ./node/node_modules/npm/bin/npm-cli.js ci --no-audit --no-fund'
- sh './node/node ./node/node_modules/npm/bin/npm-cli.js test'
+ sh './node/node ./node/node_modules/npm/bin/npm-cli.js test -- --minWorkers=1 --maxWorkers=4'
sh './node/node ./node/node_modules/npm/bin/npm-cli.js run build'
}
}
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/App.tsx b/ambari-admin/src/main/resources/ui/ambari-admin/src/App.tsx
index a78e258fe5..851b1c38a1 100644
--- a/ambari-admin/src/main/resources/ui/ambari-admin/src/App.tsx
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/App.tsx
@@ -15,42 +15,42 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
+import { clearClientSession } from "./utils/session";
import Routes from "./router/Routes";
import { SideItemLabels } from "./SideItemList";
import SideBar from "./SideBar";
-import { Container, Card } from "react-bootstrap";
+import { Container, Card, Alert, Button } from "react-bootstrap";
import { useEffect, useState } from "react";
+import { ManagementProvider } from "./context/ManagementContext";
import NavBar from "./NavBar";
import AppContent from "./context/AppContext";
import { HostCluster } from "./types";
import { get } from "lodash";
import { Toaster } from "react-hot-toast";
-import { HashRouter, Route } from "react-router-dom";
+import { HashRouter } from "react-router-dom";
import ClusterApi from "./api/clusterApi";
import Spinner from "./components/Spinner";
-import InstallClusterButton from "./components/InstallClusterButton.tsx";
-import { Form } from "react-bootstrap";
-import ClusterInformationNavigate from "./components/ClusterInformationNavigate.tsx";
import usePolling from "./hooks/usePolling.ts";
import clusterApi from "./api/clusterApi";
import InactivityTimeout from "./InactivityTimeout.tsx";
-import InstallBox from "./assets/img/install-box.svg"
import { latestAmbariUrl } from "./utils/navigation.ts";
function App() {
const [clusterInfo, setClusterInfo] = useState<HostCluster>(
{} as HostCluster
);
- const [loading, setLoading] = useState(false);
+ const [loading, setLoading] = useState(true);
+ const [availableClusters, setAvailableClusters] = useState<HostCluster[]>([]);
+ const [clusterError, setClusterError] = useState("");
+ const [clusterLoadAttempt, setClusterLoadAttempt] = useState(0);
const [selectedOption, setSelectedOption] = useState<string>(
- SideItemLabels.CLUSTERINFORMATION
+ SideItemLabels.CLUSTEROVERVIEW
);
const [rbacData, setRbacData] = useState({});
const [ambariVersion, setAmbariVersion] = useState<string>("");
const [permissionLabelList, setPermissionLabelList] = useState<string[]>([]);
const [clusterExists, setClusterExists] = useState(false);
const [clusterInfoLoading, setClusterInfoLoading] = useState(true);
- const [isInstallWizardLaunched, setInstallWizardLaunched] = useState(false);
const [isSidebarCollapsed, setIsSidebarCollapsed] = useState(false);
const [userSessiontTimeout, setUserSessiontTimeout] = useState<number>();
@@ -71,58 +71,71 @@ function App() {
async function pollNoopUserTimeout() {
try {
- const response = await clusterApi.noopPolling();
- if (response.status === 403) {
- localStorage.clear();
+ await clusterApi.noopPolling();
+ } catch (error) {
+ if ((error as { response?: { status?: number } }).response?.status === 401) {
+ clearClientSession();
window.location.replace(latestAmbariUrl("/login"));
}
- } catch (error) {
- console.error("Error in noop polling", error);
- localStorage.clear();
- window.location.replace(latestAmbariUrl("/login"));
}
}
+
usePolling(pollNoopUserTimeout, 10000);
useEffect(() => {
- async function getClusterInfoData() {
+ let active = true;
+ async function loadClusters() {
setLoading(true);
- const data = await ClusterApi.hostClustersInfo();
- const hostClusterInfo = get(data, "items[0].Clusters", "");
- setClusterInfo(hostClusterInfo);
- setLoading(false);
- }
- getClusterInfoData();
- }, []);
-
- useEffect(() => {
- async function checkClusterExists() {
- console.log("checking cluster exists");
- const response = await ClusterApi.clusterInfo(
- "Clusters/provisioning_state,Clusters/security_type,Clusters/version,Clusters/cluster_id"
- );
- if (
- response &&
- response.items &&
- response.items.length > 0 &&
- response.items[0].Clusters.provisioning_state === "INSTALLED"
- ) {
- setClusterExists(true);
+ setClusterError("");
+ try {
+ const data = await ClusterApi.hostClustersInfo();
+ if (!Array.isArray(data?.items)) throw new Error("Invalid cluster response");
+ const clusters: HostCluster[] = data.items.map((item: { Clusters: HostCluster }) => item.Clusters);
+ const requestedName = new URLSearchParams(window.location.search).get("cluster");
+ const selected = requestedName
+ ? clusters.find((item) => item.cluster_name === requestedName)
+ : clusters.length === 1 ? clusters[0] : undefined;
+ if (requestedName && !selected) throw new Error("Cluster access unavailable");
+ if (!active) return;
+ setAvailableClusters(clusters);
+ setClusterInfo(selected || {} as HostCluster);
+ setClusterExists(selected?.provisioning_state === "INSTALLED");
setClusterInfoLoading(false);
- } else {
- setClusterExists(false);
- setClusterInfoLoading(true);
+ } catch {
+ if (active) setClusterError("Unable to load the selected cluster. It may be unavailable or your access may have changed.");
+ } finally {
+ if (active) setLoading(false);
}
}
- checkClusterExists();
- }, []);
+ void loadClusters();
+ return () => { active = false; };
+ }, [clusterLoadAttempt]);
+
+ const selectCluster = (clusterName: string) => {
+ const target = new URL(window.location.href);
+ target.searchParams.set("cluster", clusterName);
+ target.hash = "/clusterInformation";
+ window.location.assign(target.href);
+ };
+
+ const updateClusterInfo = (next: HostCluster) => {
+ setClusterInfo(next);
+ setAvailableClusters((items) => items.map((item) => item.cluster_id === next.cluster_id ? next : item));
+ const target = new URL(window.location.href);
+ target.searchParams.set("cluster", next.cluster_name);
+ window.history.replaceState(null, "", target.href);
+ };
if (loading) {
return <Spinner />;
}
- const handleRedirectToInstallCluster = () => {
- setInstallWizardLaunched(true);
- };
+ if (clusterError) {
+ return <Alert variant="danger" className="m-4">
+ <p>{clusterError}</p>
+ <Button onClick={() => setClusterLoadAttempt((value) => value + 1)}>Retry</Button>
+ <Button variant="link" href={latestAmbariUrl("/")}>Return to Ambari</Button>
+ </Alert>;
+ }
return (
<HashRouter>
<AppContent.Provider
@@ -130,7 +143,9 @@ function App() {
selectedOption,
setSelectedOption,
cluster: clusterInfo,
- setClusterInfo,
+ setClusterInfo: updateClusterInfo,
+ availableClusters,
+ selectCluster,
rbacData,
setRbacData,
permissionLabelList,
@@ -141,6 +156,7 @@ function App() {
setAmbariVersion,
}}
>
+ <ManagementProvider>
<Toaster />
<div className="d-flex h-100" style={{ maxHeight: "100vh" }}>
<SideBar
@@ -164,50 +180,12 @@ function App() {
subPath={selectedOption}
clusterName={get(clusterInfo, "cluster_name", "")}
/>
- <Container className="mt-4">
- {clusterInfoLoading && !isInstallWizardLaunched && !clusterExists &&
- (window.location.hash.endsWith('/clusterInformation') || window.location.hash.endsWith('/')) ? (
- <>
- <Card.Title className="text-center text-dark mt-4">
- Welcome to Apache Ambari
- </Card.Title>
- <Card.Text className="text-center text-muted mb-10 mt-3">
- Provision a cluster, manage who can access the cluster, and customize views for Ambari users.
- </Card.Text>
- </>
- ) : null}
- <Card className="p-4 rounded-0">
- {clusterInfoLoading && !isInstallWizardLaunched && !clusterExists &&
- (window.location.hash.endsWith('/clusterInformation') || window.location.hash.endsWith('/')) ? (
- <Form.Group className="d-flex flex-column justify-content-center align-items-center text-center">
- <Card.Title>Create a Cluster</Card.Title>
- <Card.Text className="text-muted">
- Use the Install Wizard to select services and configure
- your cluster
- </Card.Text>
- <Card.Img
- variant="middle"
- src={InstallBox}
- width="100"
- height="100"
- alt="Install Box"
- />
- <InstallClusterButton
- onButtonClick={handleRedirectToInstallCluster}
- setInstallWizardLaunched={setInstallWizardLaunched}
- />
- </Form.Group>
- ) : null}
- <Route path="/clusterInformation">
- <ClusterInformationNavigate
- setInstallWizardLaunched={setInstallWizardLaunched}
- />
- </Route>
- <Routes />
- </Card>
+ <Container fluid className="mt-4 px-4">
+ <Card className="p-4 rounded-0"><Routes /></Card>
</Container>
</div>
</div>
+ </ManagementProvider>
</AppContent.Provider>
<InactivityTimeout timeout={userSessiontTimeout ?? 900} />
</HashRouter>
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/NavBar.tsx b/ambari-admin/src/main/resources/ui/ambari-admin/src/NavBar.tsx
index 7a8a364f60..e0dcf32790 100644
--- a/ambari-admin/src/main/resources/ui/ambari-admin/src/NavBar.tsx
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/NavBar.tsx
@@ -17,7 +17,8 @@
*/
import { faUser } from "@fortawesome/free-solid-svg-icons";
import { FontAwesomeIcon } from "@fortawesome/react-fontawesome";
-import {useEffect, useState} from "react";
+import AppContent from "./context/AppContext";
+import {useContext, useEffect, useState} from "react";
import {
Container,
Navbar,
@@ -40,6 +41,7 @@ type NavBarProps = {
};
export default function NavBar({ subPath, clusterName }: NavBarProps) {
+ const { availableClusters, selectCluster } = useContext(AppContent);
const [showAmbariAboutModal, setShowAmbariAboutModal] = useState(false);
const [loginUserName, setLoginUserName] = useState("");
const [ambariLsVal, setAmbariLsVal] = useState(null);
@@ -81,9 +83,19 @@ export default function NavBar({ subPath, clusterName }: NavBarProps) {
</div>
</Navbar.Brand>
<div className="d-flex align-items-center ">
- <Nav.Link className="navbar-text navbar-size">
- {clusterName}
- </Nav.Link>
+ <Dropdown>
+ <Dropdown.Toggle variant="outline-secondary" title="Select the cluster to manage">
+ {clusterName ? `Cluster: ${clusterName}` : "Choose a cluster"}
+ </Dropdown.Toggle>
+ <Dropdown.Menu align="end">
+ {(availableClusters || []).map((item: { cluster_id: number; cluster_name: string }) => (
+ <Dropdown.Item key={item.cluster_id} active={item.cluster_name === clusterName}
+ onClick={() => selectCluster(item.cluster_name)}>
+ {item.cluster_name === clusterName ? "✓ " : ""}{item.cluster_name}
+ </Dropdown.Item>
+ ))}
+ </Dropdown.Menu>
+ </Dropdown>
<Dropdown>
<Dropdown.Toggle
variant="transparent"
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/SideBar.tsx b/ambari-admin/src/main/resources/ui/ambari-admin/src/SideBar.tsx
index 10ebc88944..5ef29b9e86 100644
--- a/ambari-admin/src/main/resources/ui/ambari-admin/src/SideBar.tsx
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/SideBar.tsx
@@ -17,6 +17,7 @@
*/
/* eslint-disable @typescript-eslint/no-explicit-any */
import { Collapse } from "react-bootstrap";
+import { useLocation, matchPath } from "react-router-dom";
import SideItem from "./SideItem";
import { useContext, useEffect, useState } from "react";
import { SideItemLabels } from "./SideItemList";
@@ -39,6 +40,7 @@ type SideBarProps = {
};
const SideBar = ({ clusterExists, isSidebarCollapsed,setIsSidebarCollapsed }: SideBarProps) => {
+ const location = useLocation();
const SideItemList: SideItem[] = getSideItemList(clusterExists ?? false);
const [openOptions, setOpenOptions] = useState<string[]>([
SideItemLabels.CLUSTERMANAGEMENT,
@@ -49,15 +51,14 @@ const SideBar = ({ clusterExists, isSidebarCollapsed,setIsSidebarCollapsed }: Si
};
useEffect(() => {
- const currentHash = window.location.hash;
- const currentPath = currentHash.replace("#", "");
+ const currentPath = location.pathname;
const matchedRoute = RoutesList.find(
- (route: any) => route.path === currentPath
+ (route: any) => matchPath(currentPath, { path: route.path, exact: true })
);
if (matchedRoute) {
setSelectedOption(matchedRoute.name);
}
- }, []);
+ }, [location.pathname]);
const handleSideItemClick = (itemId: string) => {
if (isElementOpen(itemId)) {
@@ -91,7 +92,7 @@ const SideBar = ({ clusterExists, isSidebarCollapsed,setIsSidebarCollapsed }: Si
{ele.children.map((child) => {
return (
<SidebarItem
- isSelected={selectedOption === child.id}
+ isSelected={selectedOption === child.id || (child.id === SideItemLabels.CLUSTEROVERVIEW && location.pathname === "/clusterInformation")}
ele={child}
key={child.id}
onClick={() => {
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/SideItemList.tsx b/ambari-admin/src/main/resources/ui/ambari-admin/src/SideItemList.tsx
index 3103eb328e..5d74f7f2c2 100644
--- a/ambari-admin/src/main/resources/ui/ambari-admin/src/SideItemList.tsx
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/SideItemList.tsx
@@ -30,44 +30,17 @@ export enum SideItemLabels {
LOGO = "logo",
DASHBOARD = "dashboard",
CLUSTERMANAGEMENT = "Cluster Management",
- CLUSTERINFORMATION = "Cluster Information",
+ CLUSTERINFORMATION = "Cluster Details",
+ CLUSTEROVERVIEW = "Cluster Overview",
+ CREATECLUSTER = "Create Cluster",
+ HOSTRESOURCES = "Host Resources",
+ CLUSTERPERMISSIONS = "Cluster Permissions",
VERSIONS = "Versions",
REMOTECLUSTERS = "Remote Clusters",
USERS = "Users",
VIEWS = "Views",
}
-// START GENAI@CHATGPT4
-const getSideItemList = (clusterExists: boolean): SideItem[] => {
- const baseList: SideItem[] = [...SideItemList];
-
- const clusterMgmtItem = baseList.find(
- (item) => item.id === SideItemLabels.CLUSTERMANAGEMENT
- );
-
- if (clusterMgmtItem) {
- const versionsChildIndex = clusterMgmtItem.children.findIndex(child => child.id === SideItemLabels.VERSIONS);
-
- if (clusterExists) {
- // If cluster exists, add the VERSIONS child to CLUSTERMANAGEMENT
- if (versionsChildIndex === -1) {
- clusterMgmtItem.children.push({
- id: SideItemLabels.VERSIONS,
- icon: null,
- name: "Versions",
- path: "/stackVersions",
- children: [],
- });
- }
- } else {
- // If cluster does not exist, remove the VERSIONS child from CLUSTERMANAGEMENT
- if (versionsChildIndex !== -1) {
- clusterMgmtItem.children.splice(versionsChildIndex, 1);
- }
- }
- }
-
- return baseList;
-};
+const getSideItemList = (_clusterExists: boolean): SideItem[] => SideItemList;
const SideItemList: SideItem[] = [
{
@@ -93,16 +66,19 @@ const SideItemList: SideItem[] = [
name: "Cluster Management",
children: [
{
- id: SideItemLabels.CLUSTERINFORMATION,
+ id: SideItemLabels.CLUSTEROVERVIEW,
icon: null,
- name: "Cluster Information",
- path: "/clusterInformation",
+ name: "Cluster Overview",
+ path: "/clusters",
children: [],
},
+ { id: SideItemLabels.CREATECLUSTER, icon: null, name: "Create Cluster", path: "/clusters/create", children: [] },
+ { id: SideItemLabels.HOSTRESOURCES, icon: null, name: "Host Resources", path: "/hostResources", children: [] },
+ { id: SideItemLabels.CLUSTERPERMISSIONS, icon: null, name: "Cluster Permissions", path: "/clusterPermissions", children: [] },
{
id: SideItemLabels.VERSIONS,
icon: null,
- name: "Versions",
+ name: "Versions & Repositories",
path: "/stackVersions",
children: [],
},
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/__mocks__/mockClusterBluePrintInfo.ts b/ambari-admin/src/main/resources/ui/ambari-admin/src/__mocks__/mockClusterBluePrintInfo.ts
index e1f2432d0d..ab0d16ab90 100644
--- a/ambari-admin/src/main/resources/ui/ambari-admin/src/__mocks__/mockClusterBluePrintInfo.ts
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/__mocks__/mockClusterBluePrintInfo.ts
@@ -15,5 +15,8 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-const mockClusterBluePrintInfo = {};
-export default mockClusterBluePrintInfo;
\ No newline at end of file
+const mockClusterBluePrintInfo = {
+ configurations: [],
+ host_groups: [],
+};
+export default mockClusterBluePrintInfo;
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/api/clusterApi.ts b/ambari-admin/src/main/resources/ui/ambari-admin/src/api/clusterApi.ts
index 12ca2ba01f..53b67dd304 100644
--- a/ambari-admin/src/main/resources/ui/ambari-admin/src/api/clusterApi.ts
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/api/clusterApi.ts
@@ -29,7 +29,7 @@ const ClusterApi = {
return response.data;
},
hostClustersInfo: async function () {
- const url = `/clusters?fields=Clusters/provisioning_state`;
+ const url = `/clusters?fields=Clusters/cluster_id,Clusters/cluster_name,Clusters/provisioning_state`;
const response = await adminApi.request({
url: url,
method: "GET",
@@ -48,7 +48,7 @@ const ClusterApi = {
clusterName: string,
format: string = "blueprint"
) {
- const url = `/clusters/${clusterName}?format=${format}`;
+ const url = `/clusters/${encodeURIComponent(clusterName)}?format=${format}`;
const response = await adminApi.request({
url: url,
method: "GET",
@@ -56,7 +56,7 @@ const ClusterApi = {
return response.data;
},
updateClusterName: async function (clusterName: string,updatedClusterName:string) {
- const url = `/clusters/${clusterName}`;
+ const url = `/clusters/${encodeURIComponent(clusterName)}`;
const response = await adminApi.request({
url: url,
method: "PUT",
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/api/clusterManagement.ts b/ambari-admin/src/main/resources/ui/ambari-admin/src/api/clusterManagement.ts
new file mode 100644
index 0000000000..18c5f840f4
--- /dev/null
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/api/clusterManagement.ts
@@ -0,0 +1,53 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import { adminApi } from "./configs/axiosConfig";
+
+export type ClusterSummary = {
+ Clusters: { cluster_id: number; cluster_name: string; provisioning_state: string; version?: string;
+ security_type?: string; health_report?: Record<string, number> };
+ hosts?: { Hosts: { host_name: string } }[];
+ services?: { ServiceInfo: { service_name: string; state?: string } }[];
+};
+export type HostSummary = { Hosts: { host_name: string; cluster_name?: string; ip?: string; host_status?: string } };
+export type Grant = { PrivilegeInfo: { privilege_id: number; principal_name: string;
+ principal_type: string; permission_name: string; permission_label?: string } };
+export type Role = { PermissionInfo: { permission_name: string; permission_label: string; resource_name: string } };
+export type Authorization = { authorization_id: string; resource_type: string; cluster_name?: string };
+export const clusterEndpoint = (name: string) => `/clusters/${encodeURIComponent(name)}`;
+export const clusterListPath = "/clusters?fields=Clusters/cluster_id,Clusters/cluster_name,Clusters/provisioning_state,Clusters/version,Clusters/security_type,Clusters/health_report,hosts/Hosts/host_name,services/ServiceInfo/service_name";
+export const hostsPath = "/hosts?fields=Hosts/host_name,Hosts/cluster_name,Hosts/ip,Hosts/host_status";
+
+export async function readResource<T>(path: string, signal?: AbortSignal): Promise<T> {
+ const response = await adminApi.get(path, { signal });
+ if (!response.data || typeof response.data !== "object") throw new Error("Invalid server response");
+ return response.data;
+}
+export function items<T>(data: { items?: T[] } | undefined): T[] {
+ if (!data) return [];
+ if (!Array.isArray(data.items)) throw new Error("The server did not return a resource collection");
+ return data.items;
+}
+export function errorMessage(error: unknown): string {
+ const response = (error as { response?: { status?: number; data?: { message?: string; code?: string } } })?.response;
+ return response?.data?.message || (response?.status ? `Request failed (HTTP ${response.status}). Refresh to reconcile the current server state.` : "The response was lost or the server is unavailable. Refresh before retrying the operation.");
+}
+export function hasPermission(grants: Authorization[], permission: string, clusterName?: string) {
+ return grants.some((grant) => grant.authorization_id === permission
+ && (grant.resource_type === "AMBARI" || (Boolean(clusterName) && grant.resource_type === "CLUSTER" && grant.cluster_name === clusterName)));
+}
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/api/configs/axiosConfig.ts b/ambari-admin/src/main/resources/ui/ambari-admin/src/api/configs/axiosConfig.ts
index a9b46902e3..5c3e9ddb0d 100644
--- a/ambari-admin/src/main/resources/ui/ambari-admin/src/api/configs/axiosConfig.ts
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/api/configs/axiosConfig.ts
@@ -17,6 +17,7 @@
*/
import axios from "axios";
+import { ambariApplicationRoot } from "../../utils/navigation";
import { toast } from "react-hot-toast";
import { get } from "lodash";
@@ -26,11 +27,13 @@ const createAxiosInstance = (baseURL: string, headers = {}) => {
withCredentials: true,
headers: {
"Content-Type": "application/json",
+ "X-Requested-By": "ambari-admin",
...headers,
},
});
instance.interceptors.response.use(undefined, (error) => {
+ if (axios.isCancel(error)) return Promise.reject(error);
const responseMessage = get(error, "response.data.message", undefined);
if (responseMessage) {
toast.error(responseMessage);
@@ -43,4 +46,4 @@ const createAxiosInstance = (baseURL: string, headers = {}) => {
return instance;
};
-export const adminApi = createAxiosInstance("/api/v1");
\ No newline at end of file
+export const adminApi = createAxiosInstance(`${ambariApplicationRoot(window.location.pathname)}api/v1`);
\ No newline at end of file
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/api/logout.ts b/ambari-admin/src/main/resources/ui/ambari-admin/src/api/logout.ts
index a0e4c42603..5d8fb710fb 100644
--- a/ambari-admin/src/main/resources/ui/ambari-admin/src/api/logout.ts
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/api/logout.ts
@@ -15,6 +15,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
+import { clearClientSession } from "../utils/session";
import {encryptData, decryptData, getFromLocalStorage, parseJSONData, setInLocalStorage} from "./Utility.ts";
import {adminApi} from "./configs/axiosConfig.ts";
import { AxiosError } from 'axios';
@@ -44,7 +45,7 @@ const signOut = async () => {
method: 'GET',
headers: headers
});
- localStorage.clear();
+ clearClientSession();
window.location.replace(latestAmbariUrl("/login"));
} catch (error) {
const axiosError = error as AxiosError;
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/api/versions.ts b/ambari-admin/src/main/resources/ui/ambari-admin/src/api/versions.ts
index 1077f20888..027fb9a7c0 100644
--- a/ambari-admin/src/main/resources/ui/ambari-admin/src/api/versions.ts
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/api/versions.ts
@@ -17,19 +17,21 @@
*/
import { adminApi } from "./configs/axiosConfig";
const VersionsApi = {
- versionsList: async function (repoVersion: string,clusterName:string) {
- const url = `/clusters/${clusterName}/stack_versions?fields=*&ClusterStackVersions/repository_version=${repoVersion}`;
+ versionsList: async function (repoVersion: string, clusterName: string, signal?: AbortSignal) {
+ const url = `/clusters/${encodeURIComponent(clusterName)}/stack_versions?fields=*&ClusterStackVersions/repository_version=${encodeURIComponent(repoVersion)}`;
const response = await adminApi.request({
url: url,
method: "GET",
+ signal,
});
return response.data;
},
- getRepos: async function () {
+ getRepos: async function (signal?: AbortSignal) {
const url = `/stacks?fields=versions/repository_versions/RepositoryVersions`;
const response = await adminApi.request({
url: url,
method: "GET",
+ signal,
});
return response.data;
},
@@ -114,4 +116,4 @@ const VersionsApi = {
}
};
-export default VersionsApi;
\ No newline at end of file
+export default VersionsApi;
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/components/SidebarItem.tsx b/ambari-admin/src/main/resources/ui/ambari-admin/src/components/SidebarItem.tsx
index 74c2b86691..09ee9ced79 100644
--- a/ambari-admin/src/main/resources/ui/ambari-admin/src/components/SidebarItem.tsx
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/components/SidebarItem.tsx
@@ -3,7 +3,7 @@
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
+ * to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
@@ -15,52 +15,18 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
+
import { faChevronDown, faChevronRight } from "@fortawesome/free-solid-svg-icons";
import { FontAwesomeIcon } from "@fortawesome/react-fontawesome";
import { MouseEventHandler } from "react";
import { Link } from "react-router-dom";
-const SidebarItem = ({
- ele,
- onClick,
- isSelected,
- isOpen = false,
- hasChildren = false,
- }: {
- ele: any;
- onClick?: MouseEventHandler<HTMLDivElement>;
- isSelected: boolean;
- isOpen?: boolean;
- hasChildren?: boolean;
- }) => {
- return (
- <Link to={ele.path} className="text-decoration-none">
- <div
- className={`d-flex justify-content-between ${
- ele.className
- } sideitem align-items-center ${isSelected ? "selected-item" : ""}`}
- style={{
- ...(ele.style as any),
- fontSize: 14,
- cursor: "pointer",
- padding: "10px 5px 10px 20px",
- position: "relative",
- }}
- onClick={onClick}
- >
- <div className="d-flex">
- <div>{ele.icon}</div>
- <div className="ms-2">{ele.name}</div>
- </div>
- {hasChildren ? (
- <FontAwesomeIcon
- icon={isOpen ? faChevronDown : faChevronRight}
- className="me-1"
- />
- ) : null}
- </div>
- </Link>
- );
- };
-
- export default SidebarItem
\ No newline at end of file
+export default function SidebarItem({ ele, onClick, isSelected, isOpen = false, hasChildren = false }: {
+ ele: any; onClick?: MouseEventHandler<HTMLElement>; isSelected: boolean; isOpen?: boolean; hasChildren?: boolean;
+}) {
+ const className = `text-decoration-none d-flex justify-content-between ${ele.className || ""} sideitem align-items-center ${isSelected ? "selected-item" : ""}`;
+ const style = { ...ele.style, fontSize: 14, cursor: "pointer", padding: "10px 5px 10px 20px", width: "100%" };
+ const content = <><span className="d-flex"><span>{ele.icon}</span><span className="ms-2">{ele.name}</span></span>{hasChildren && <FontAwesomeIcon icon={isOpen ? faChevronDown : faChevronRight} className="me-1" />}</>;
+ return hasChildren ? <button type="button" className={`${className} border-0 bg-transparent text-start`} style={style} aria-expanded={isOpen} onClick={onClick}>{content}</button>
+ : <Link to={ele.path} className={className} style={style} onClick={onClick}>{content}</Link>;
+}
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/components/SidebarItemCollapsed.tsx b/ambari-admin/src/main/resources/ui/ambari-admin/src/components/SidebarItemCollapsed.tsx
index 560e983317..cfadba1115 100644
--- a/ambari-admin/src/main/resources/ui/ambari-admin/src/components/SidebarItemCollapsed.tsx
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/components/SidebarItemCollapsed.tsx
@@ -15,6 +15,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
+import { Link } from "react-router-dom";
import { MouseEventHandler, useState } from "react";
import { Dropdown } from "react-bootstrap";
import SidebarItem from "./SidebarItem";
@@ -41,12 +42,14 @@ const SidebarItemCollapsed = ({
return (
<Dropdown
drop="end"
+ show={showDropdown}
+ onToggle={setShowDropdown}
className="collapsed-sidebar-item"
onMouseLeave={() => setShowDropdown(false)}
onMouseOver={() => setShowDropdown(true)}
// style={{ width: "166px" }}
>
- <Dropdown.Toggle as="div" className="main-style" id="dropdown-basic">
+ <Dropdown.Toggle variant="link" className="main-style" id={`collapsed-${ele.id}`} aria-label={String(ele.name)}>
<div
className={`d-flex justify-content-between ${
ele.className
@@ -85,24 +88,11 @@ const SidebarItemCollapsed = ({
</Dropdown>
);
} else {
- return (
- <div
- className={`d-flex justify-content-between ${
- ele.className
- } sideitem align-items-center ${isSelected ? "selected-item" : ""}`}
- style={{
- ...(ele.style as any),
- cursor: "pointer",
- padding: "10px 5px 10px 20px",
- position: "relative",
- }}
- onClick={() => {
- setSelectedOption(ele.id);
- }}
- >
- <div style={{ fontSize: 20 }}>{ele.icon}</div>
- </div>
- );
+ return <Link to={ele.path} title={typeof ele.name === "string" ? ele.name : "Ambari"}
+ className={`d-flex justify-content-center sideitem ${isSelected ? "selected-item" : ""}`}
+ style={{ ...ele.style, padding: "10px 5px", fontSize: 20 }}
+ onClick={() => setSelectedOption(ele.id)}>{ele.icon}</Link>;
+
}
};
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/context/ManagementContext.tsx b/ambari-admin/src/main/resources/ui/ambari-admin/src/context/ManagementContext.tsx
new file mode 100644
index 0000000000..56c0f268b9
--- /dev/null
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/context/ManagementContext.tsx
@@ -0,0 +1,49 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import { createContext, useContext, useEffect, useState, ReactNode } from "react";
+import { adminApi } from "../api/configs/axiosConfig";
+import { Authorization, hasPermission } from "../api/clusterManagement";
+
+const ManagementContext = createContext({ loading: true, error: "", can: (_permission: string, _cluster?: string) => false });
+export const useManagement = () => useContext(ManagementContext);
+export function ManagementProvider({ children }: { children: ReactNode }) {
+ const [grants, setGrants] = useState<Authorization[]>([]);
+ const [loading, setLoading] = useState(true);
+ const [error, setError] = useState("");
+ useEffect(() => {
+ const controller = new AbortController();
+ async function load() {
+ try {
+ const session = await adminApi.get("/clusters?fields=Clusters/cluster_id", { signal: controller.signal });
+ const user = session.headers.user;
+ if (typeof user !== "string" || !user) throw new Error("Missing authenticated user");
+ const response = await adminApi.get(`/users/${encodeURIComponent(user)}/authorizations?fields=*`, { signal: controller.signal });
+ if (!Array.isArray(response.data?.items)) throw new Error("Invalid authorization response");
+ if (!controller.signal.aborted) setGrants(response.data.items.map((item: { AuthorizationInfo: Authorization }) => item.AuthorizationInfo));
+ } catch {
+ if (!controller.signal.aborted) setError("Unable to load your permissions. Management actions are disabled; reload this page to retry.");
+ } finally {
+ if (!controller.signal.aborted) setLoading(false);
+ }
+ }
+ void load();
+ return () => controller.abort();
+ }, []);
+ return <ManagementContext.Provider value={{ loading, error, can: (permission, cluster) => hasPermission(grants, permission, cluster) }}>{children}</ManagementContext.Provider>;
+}
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/hooks/useAdminResource.ts b/ambari-admin/src/main/resources/ui/ambari-admin/src/hooks/useAdminResource.ts
new file mode 100644
index 0000000000..4e17859174
--- /dev/null
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/hooks/useAdminResource.ts
@@ -0,0 +1,39 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import { useEffect, useState } from "react";
+import { errorMessage, readResource } from "../api/clusterManagement";
+
+export default function useAdminResource<T>(path: string | null, collection = false) {
+ const [revision, setRevision] = useState(0);
+ const [state, setState] = useState<{ path: string | null; data?: T; error?: string; loading: boolean }>({ path: null, loading: false });
+ useEffect(() => {
+ if (!path) return;
+ const controller = new AbortController();
+ setState({ path, loading: true });
+ readResource<T>(path, controller.signal).then((data) => {
+ if (collection && !Array.isArray((data as { items?: unknown }).items)) throw new Error("Invalid resource collection");
+ if (!controller.signal.aborted) setState({ path, data, loading: false });
+ }).catch((error) => {
+ if (!controller.signal.aborted) setState({ path, error: errorMessage(error), loading: false });
+ });
+ return () => controller.abort();
+ }, [path, revision, collection]);
+ return { ...(state.path === path ? state : { loading: Boolean(path), data: undefined, error: undefined }),
+ reload: () => setRevision((value) => value + 1) };
+}
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/router/Routes.tsx b/ambari-admin/src/main/resources/ui/ambari-admin/src/router/Routes.tsx
index bd7f7ed25c..6b4b0013f2 100644
--- a/ambari-admin/src/main/resources/ui/ambari-admin/src/router/Routes.tsx
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/router/Routes.tsx
@@ -21,8 +21,8 @@ import routes from "./RoutesList.tsx";
export default function Routes() {
return (
<Switch>
- {routes.map(({ path, Element }, key) => (
- <Route path={path} key={key} render={() => <Element />}></Route>
+ {routes.map(({ path, exact, Element }, key) => (
+ <Route exact={exact} path={path} key={key} render={() => <Element />}></Route>
))}
</Switch>
);
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/router/RoutesList.tsx b/ambari-admin/src/main/resources/ui/ambari-admin/src/router/RoutesList.tsx
index 888ba5dbfb..f4f3c63dbc 100644
--- a/ambari-admin/src/main/resources/ui/ambari-admin/src/router/RoutesList.tsx
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/router/RoutesList.tsx
@@ -15,7 +15,11 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-import ClusterInformation from "../screens/ClusterManagement/ClusterInformation";
+import ClusterDetails from "../screens/ClusterManagement/ClusterDetails";
+import ClusterOverview from "../screens/ClusterManagement/ClusterOverview";
+import ClusterCreate from "../screens/ClusterManagement/ClusterCreate";
+import HostResources from "../screens/ClusterManagement/HostResources";
+import ClusterPermissions from "../screens/ClusterManagement/ClusterPermissions";
import RemoteClusters from "../screens/ClusterManagement/RemoteClusters";
import RegisterRemoteCluster from "../screens/ClusterManagement/RemoteClusters/RegisterRemoteCluster";
import EditRemoteCluster from "../screens/ClusterManagement/RemoteClusters/EditRemoteCluster";
@@ -32,6 +36,10 @@ import { Register, VersionsList } from "../screens/ClusterManagement/StackVersio
export default [
+ { path: "/clusters/create", exact: true, Element: ClusterCreate, name: "Create Cluster" },
+ { path: "/clusters", exact: true, Element: ClusterOverview, name: "Cluster Overview" },
+ { path: "/hostResources", exact: true, Element: HostResources, name: "Host Resources" },
+ { path: "/clusterPermissions", exact: true, Element: ClusterPermissions, name: "Cluster Permissions" },
{
path: "/main/dashboard",
exact: true,
@@ -47,8 +55,8 @@ export default [
{
path: "/clusterInformation",
exact: true,
- Element: () => <ClusterInformation />,
- name: "Cluster Information",
+ Element: () => <ClusterDetails />,
+ name: "Cluster Details",
},
{
path: "/stackVersions/create",
@@ -125,7 +133,7 @@ export default [
{
path: '/',
exact: true,
- Element: () => <Redirect to="/clusterInformation" />,
+ Element: () => <Redirect to="/clusters" />,
},
];
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/ClusterCreate.tsx b/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/ClusterCreate.tsx
new file mode 100644
index 0000000000..3be1fa1e23
--- /dev/null
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/ClusterCreate.tsx
@@ -0,0 +1,54 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import { useState } from "react";
+import { Alert, Button, Table } from "react-bootstrap";
+import { useManagement } from "../../context/ManagementContext";
+import useAdminResource from "../../hooks/useAdminResource";
+import { items } from "../../api/clusterManagement";
+import { latestAmbariUrl } from "../../utils/navigation";
+import { LoadError, PageHeading } from "./ManagementShared";
+
+type Draft = { draft_id: string; phase: string; revision: number; cluster_name?: string };
+export default function ClusterCreate() {
+ const { can, loading } = useManagement();
+ const allowed = can("AMBARI.ADD_DELETE_CLUSTERS");
+ const drafts = useAdminResource<{ items: Draft[] }>(allowed ? "/persist/scopes/drafts" : null, true);
+ const [launchError, setLaunchError] = useState("");
+ function launch() {
+ try {
+ const bytes = crypto.getRandomValues(new Uint8Array(16));
+ bytes[6] = (bytes[6] & 15) | 64; bytes[8] = (bytes[8] & 63) | 128;
+ const hex = Array.from(bytes, (v) => v.toString(16).padStart(2, "0")).join("");
+ const id = `${hex.slice(0,8)}-${hex.slice(8,12)}-${hex.slice(12,16)}-${hex.slice(16,20)}-${hex.slice(20)}`;
+ window.location.assign(latestAmbariUrl(`/installer/step0?draft=${id}`));
+ } catch { setLaunchError("A secure installation identifier could not be generated. Reload the page before trying again."); }
+ }
+ return <>
+ <PageHeading title="Create Cluster" description="Create an independent cluster using the existing installation wizard, or continue one of your saved installations." />
+ <Alert variant="info">Choose the Stack and repository, select unassigned hosts, then configure services and component placement. For an independent HBase deployment, select this cluster's HDFS and ZooKeeper.</Alert>
+ {launchError && <Alert variant="danger">{launchError}</Alert>}
+ {!loading && !allowed ? <Alert variant="warning">You do not have permission to create clusters.</Alert> : <Button disabled={!allowed} onClick={launch}>Start installation wizard</Button>}
+ <h4 className="mt-4">Your saved installations</h4>
+ <LoadError error={drafts.error} retry={drafts.reload} />
+ {drafts.loading ? <p>Loading installations…</p> : <Table responsive><thead><tr><th>Cluster</th><th>Phase</th><th>Installation ID</th><th /></tr></thead><tbody>
+ {items(drafts.data).map((draft) => <tr key={draft.draft_id}><td>{draft.cluster_name || "Not named yet"}</td><td>{draft.phase}</td><td><code>{draft.draft_id}</code></td><td><Button size="sm" href={latestAmbariUrl(`/installer/step0?draft=${encodeURIComponent(draft.draft_id)}`)}>Continue installation</Button></td></tr>)}
+ </tbody></Table>}
+ {allowed && !drafts.loading && !drafts.error && !items(drafts.data).length && <p>No saved installations for your account.</p>}
+ </>;
+}
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/ClusterDeleteDialog.tsx b/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/ClusterDeleteDialog.tsx
new file mode 100644
index 0000000000..27f728fb76
--- /dev/null
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/ClusterDeleteDialog.tsx
@@ -0,0 +1,62 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import { useState } from "react";
+import { Alert, Button, Form, Modal } from "react-bootstrap";
+import { adminApi } from "../../api/configs/axiosConfig";
+import { clusterEndpoint, ClusterSummary, errorMessage, readResource } from "../../api/clusterManagement";
+import useAdminResource from "../../hooks/useAdminResource";
+import { LoadError } from "./ManagementShared";
+
+type Preview = Omit<ClusterSummary, "hosts"> & { hosts?: { Hosts: { host_name: string }; host_components?: { HostRoles: { state: string; desired_state?: string } }[] }[] };
+const removableStates = new Set(["INIT", "INSTALLED", "INSTALL_FAILED", "UNINSTALLED", "DISABLED"]);
+export default function ClusterDeleteDialog({ cluster, close, completed }: { cluster: ClusterSummary; close: () => void; completed: () => void }) {
+ const name = cluster.Clusters.cluster_name;
+ const path = clusterEndpoint(name);
+ const previewPath = `${path}?fields=Clusters/cluster_id,hosts/Hosts/host_name,services/ServiceInfo/service_name,hosts/host_components/HostRoles/state,hosts/host_components/HostRoles/desired_state`;
+ const preview = useAdminResource<Preview>(previewPath);
+ const [confirmation, setConfirmation] = useState("");
+ const [busy, setBusy] = useState(false);
+ const [error, setError] = useState("");
+ function blocked(data?: Preview) {
+ return !data || data.Clusters.cluster_id !== cluster.Clusters.cluster_id || !Array.isArray(data.hosts)
+ || data.hosts.some((host) => !Array.isArray(host.host_components) || host.host_components.some((item) => !removableStates.has(item.HostRoles.state) || !removableStates.has(item.HostRoles.desired_state || "")));
+ }
+ async function remove() {
+ setBusy(true); setError("");
+ try {
+ const current = await readResource<Preview>(previewPath);
+ if (blocked(current)) { setError("The cluster identity or component states changed. Refresh the deletion review."); return; }
+ let mutationError: unknown;
+ try { await adminApi.delete(path); } catch (e) { mutationError = e; }
+ try { await readResource(path); } catch (e) {
+ if ((e as { response?: { status?: number } }).response?.status === 404) { completed(); return; }
+ throw e;
+ }
+ throw mutationError || new Error("Deletion is not confirmed; the cluster still exists");
+ } catch (e) { setError(errorMessage(e)); }
+ finally { setBusy(false); }
+ }
+ return <Modal show onHide={() => { if (!busy) close(); }} size="lg"><Modal.Header closeButton={!busy}><Modal.Title>Delete {name}</Modal.Title></Modal.Header><Modal.Body>
+ <Alert variant="warning">This removes the cluster's Ambari configuration, service definitions and permissions. Its hosts are released from cluster membership. This does not erase Hadoop data from disks.</Alert>
+ <LoadError error={preview.error} retry={preview.reload} />
+ {preview.loading ? <p>Loading deletion review…</p> : preview.data && <><p>Cluster ID: {preview.data.Clusters.cluster_id} · Hosts: {preview.data.hosts?.length ?? "Unavailable"} · Services: {preview.data.services?.map((s) => s.ServiceInfo.service_name).join(", ") || "None"}</p><Alert variant={blocked(preview.data) ? "danger" : "info"}>{blocked(preview.data) ? "Deletion is blocked while components are running, changing state, or their state cannot be confirmed." : "Component states perm [...]
+ {error && <Alert variant="danger">{error}</Alert>}
+ <Form.Label htmlFor="confirm-cluster-deletion">Type {name} to confirm</Form.Label><Form.Control id="confirm-cluster-deletion" value={confirmation} onChange={(e) => setConfirmation(e.target.value)} disabled={busy} />
+ </Modal.Body><Modal.Footer><Button variant="secondary" disabled={busy} onClick={close}>Cancel</Button><Button variant="danger" disabled={busy || preview.loading || Boolean(preview.error) || blocked(preview.data) || confirmation !== name} onClick={() => void remove()}>Delete cluster</Button></Modal.Footer></Modal>;
+}
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/ClusterDetails.tsx b/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/ClusterDetails.tsx
new file mode 100644
index 0000000000..9c1c26af27
--- /dev/null
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/ClusterDetails.tsx
@@ -0,0 +1,84 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import { useContext, useState } from "react";
+import { Alert, Button, Nav, Table } from "react-bootstrap";
+import { Link, useHistory, useLocation } from "react-router-dom";
+import AppContent from "../../context/AppContext";
+import { clusterEndpoint, ClusterSummary, HostSummary, items } from "../../api/clusterManagement";
+import useAdminResource from "../../hooks/useAdminResource";
+import { useManagement } from "../../context/ManagementContext";
+import { clusterOperationUrl, latestClusterDashboardUrl } from "../../utils/navigation";
+import ClusterInformation from "./ClusterInformation";
+import { PermissionEditor } from "./ClusterPermissions";
+import { ClusterSelect, LoadError, PageHeading } from "./ManagementShared";
+
+type Service = { ServiceInfo: { service_name: string; state: string } };
+type Operation = { Requests: { id: number; request_context: string; request_status: string; progress_percent?: number; start_time?: number } };
+const tabs = [ ["general", "Basic information"], ["resources", "Services & hosts"], ["permissions", "Access permissions"], ["operations", "Operation history"], ["export", "Configuration & export"] ];
+
+function ServiceHosts({ name }: { name: string }) {
+ const services = useAdminResource<{ items: Service[] }>(`${clusterEndpoint(name)}/services?fields=ServiceInfo/service_name,ServiceInfo/state`, true);
+ const hosts = useAdminResource<{ items: HostSummary[] }>(`${clusterEndpoint(name)}/hosts?fields=Hosts/host_name,Hosts/host_status,Hosts/ip`, true);
+ const { can } = useManagement();
+ return <>
+ <div className="d-flex gap-2 mb-3"><Button variant="outline-secondary" onClick={() => { services.reload(); hosts.reload(); }}>Refresh</Button>
+ {can("SERVICE.ADD_DELETE_SERVICES", name) && <Button href={clusterOperationUrl(name, "/main/service/add/step1")}>Add service</Button>}
+ {can("HOST.ADD_DELETE_HOSTS", name) && <Button href={clusterOperationUrl(name, "/main/host/add/step1")}>Add hosts</Button>}
+ </div>
+ <LoadError error={services.error || hosts.error} retry={() => { services.reload(); hosts.reload(); }} />
+ <h5>Services</h5>{services.loading ? <p>Loading services…</p> : <Table responsive><thead><tr><th>Service</th><th>State</th><th>Configuration</th></tr></thead><tbody>{items(services.data).map(({ ServiceInfo: s }) => <tr key={s.service_name}><td><a href={clusterOperationUrl(name, `/main/services/${encodeURIComponent(s.service_name)}/summary`)}>{s.service_name}</a></td><td>{s.state || "Unknown"}</td><td><a href={clusterOperationUrl(name, `/main/services/${encodeURIComponent(s.service_na [...]
+ <h5>Hosts</h5>{hosts.loading ? <p>Loading hosts…</p> : <Table responsive><thead><tr><th>Host</th><th>IP</th><th>Health</th></tr></thead><tbody>{items(hosts.data).map(({ Hosts: host }) => <tr key={host.host_name}><td><a href={clusterOperationUrl(name, `/main/hosts/${encodeURIComponent(host.host_name)}/summary`)}>{host.host_name}</a></td><td>{host.ip || "Unavailable"}</td><td>{host.host_status || "Unknown"}</td></tr>)}</tbody></Table>}
+ </>;
+}
+function Operations({ name }: { name: string }) {
+ const [page, setPage] = useState(0);
+ const { can } = useManagement();
+ const permitted = ["CLUSTER.VIEW_STATUS_INFO", "HOST.VIEW_STATUS_INFO", "SERVICE.VIEW_STATUS_INFO"].some((permission) => can(permission, name));
+ const result = useAdminResource<{ items: Operation[] }>(permitted ? `${clusterEndpoint(name)}/requests?fields=Requests/id,Requests/request_context,Requests/request_status,Requests/progress_percent,Requests/start_time&sortBy=Requests/id.desc&page_size=20&from=${page * 20}` : null, true);
+ if (!permitted) return <Alert variant="info">You do not have permission to view this cluster's operation history.</Alert>;
+ return <>
+ <p>Server request records for {name}. Open a request to inspect its tasks and failure details. Continue installation or service workflows through their original wizard.</p>
+ <Button variant="outline-secondary" className="mb-3" onClick={result.reload}>Refresh</Button><LoadError error={result.error} retry={result.reload} />
+ {result.loading ? <p>Loading requests…</p> : <Table responsive><thead><tr><th>Request</th><th>Operation</th><th>Status</th><th>Progress</th><th>Started</th></tr></thead><tbody>{items(result.data).map(({ Requests: request }) => <tr key={request.id}><td><a href={clusterOperationUrl(name, `/main/requests?requestId=${request.id}`)}>#{request.id}</a></td><td>{request.request_context}</td><td>{request.request_status}</td><td>{typeof request.progress_percent === "number" ? `${Math.round(req [...]
+ <div className="d-flex gap-2"><Button variant="outline-secondary" disabled={page === 0 || result.loading} onClick={() => setPage((v) => v - 1)}>Newer</Button><span className="align-self-center">Page {page + 1}</span><Button variant="outline-secondary" disabled={result.loading || items(result.data).length < 20} onClick={() => setPage((v) => v + 1)}>Older</Button></div>
+ </>;
+}
+export default function ClusterDetails() {
+ const { cluster } = useContext(AppContent);
+ const name: string = cluster?.cluster_name || "";
+ const location = useLocation(); const history = useHistory();
+ const requestedTab = new URLSearchParams(location.search).get("tab") || "general";
+ const tab = tabs.some(([key]) => key === requestedTab) ? requestedTab : "general";
+ const result = useAdminResource<ClusterSummary>(name ? `${clusterEndpoint(name)}?fields=Clusters/cluster_id,Clusters/cluster_name,Clusters/provisioning_state,Clusters/version,Clusters/security_type,hosts/Hosts/host_name,services/ServiceInfo/service_name` : null);
+ const c = result.data?.Clusters;
+ return <>
+ <PageHeading title="Cluster Details" description="Manage one explicitly selected cluster. Dashboard and service actions keep this cluster's identity." />
+ <div className="d-flex gap-2 mb-3"><Link to="/clusters">All clusters</Link>{name && <a href={latestClusterDashboardUrl(name)}>Dashboard: {name}</a>}</div>
+ <ClusterSelect route="/clusterInformation" />
+ {name && <><h4 className="mb-3">{name}</h4><Nav variant="tabs" activeKey={tab} onSelect={(key) => history.replace({ pathname: location.pathname, search: `?tab=${key}` })}>{tabs.map(([key, label]) => <Nav.Item key={key}><Nav.Link eventKey={key}>{label}</Nav.Link></Nav.Item>)}</Nav>
+ <div className="pt-4" key={`${cluster.cluster_id}:${tab}`}>
+ {tab === "general" && <><LoadError error={result.error} retry={result.reload} />{result.loading ? <p>Loading cluster details…</p> : c && <Table><tbody>{[["Cluster name", c.cluster_name], ["Stable cluster ID", c.cluster_id], ["Installation state", c.provisioning_state], ["Stack version", c.version || "Not selected"], ["Security", c.security_type || "Unavailable"], ["Hosts", result.data?.hosts?.length ?? "Unavailable"], ["Services", result.data?.services?.map((s) => s.ServiceInfo.s [...]
+ {tab === "resources" && <ServiceHosts name={name} />}
+ {tab === "permissions" && <PermissionEditor clusterName={name} />}
+ {tab === "operations" && <Operations name={name} />}
+ {tab === "export" && <ClusterInformation />}
+ </div>
+ </>}
+ </>;
+}
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/ClusterInformation/index.tsx b/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/ClusterInformation/index.tsx
index ed5ae4b7db..9705735b02 100644
--- a/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/ClusterInformation/index.tsx
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/ClusterInformation/index.tsx
@@ -16,18 +16,21 @@
* limitations under the License.
*/
/* eslint-disable @typescript-eslint/no-explicit-any */
+import { useManagement } from "../../../context/ManagementContext";
import { useEffect, useState } from "react";
-import {Form, OverlayTrigger, Tooltip} from "react-bootstrap";
+import {Button, Form, OverlayTrigger, Tooltip} from "react-bootstrap";
import DefaultButton from "../../../components/DefaultButton";
import AppContent from "../../../context/AppContext";
import ClusterApi from "../../../api/clusterApi";
import Spinner from "../../../components/Spinner";
import toast from "react-hot-toast";
-import { cloneDeep } from "lodash";
import ConfirmationModal from "../../../components/ConfirmationModal";
import { useContext } from "react";
export default function ClusterInformation() {
+ const { can } = useManagement();
+ const [loadError, setLoadError] = useState(false);
+ const [saving, setSaving] = useState(false);
const [infoData, setInfoData] = useState({});
const [loading, setLoading] = useState(false);
const {
@@ -38,14 +41,13 @@ export default function ClusterInformation() {
const [clusterNameInput, setClusterNameInput] = useState(clusterName);
const [clusterNameError, setClusterNameError] = useState("");
const [showConfirmationModal, setShowConfirmationModal] = useState(false);
- console.log("Clutser is", cluster);
- const { clusterExists, setSelectedOption } = useContext(AppContent);
+ const { setSelectedOption, availableClusters, selectCluster } = useContext(AppContent);
const [showTooltip, setShowTooltip] = useState(false);
const handleFocus = () => setShowTooltip(true);
const handleBlur = () => setShowTooltip(false);
useEffect(() => {
- setSelectedOption("Cluster Information");
+ setSelectedOption("Cluster Details");
}, []);
useEffect(() => {
@@ -68,15 +70,20 @@ export default function ClusterInformation() {
}
}, [clusterNameInput]);
- async function getClusterInfoData(requiredClusterName: string = clusterName) {
- setLoading(true);
- const data = await ClusterApi.blueprintInfo(requiredClusterName);
- setInfoData(data as any);
- setLoading(false);
- }
-
useEffect(() => {
- if (clusterName) getClusterInfoData();
+ if (!clusterName) return;
+ let active = true;
+ setInfoData({});
+ setLoadError(false);
+ setLoading(true);
+ ClusterApi.blueprintInfo(clusterName).then((data) => {
+ if (active) setInfoData(data);
+ }).catch(() => {
+ if (active) { setLoadError(true); toast.error("Could not load the cluster blueprint"); }
+ }).finally(() => {
+ if (active) setLoading(false);
+ });
+ return () => { active = false; };
}, [clusterName]);
function downloadBlueprint() {
@@ -96,20 +103,38 @@ export default function ClusterInformation() {
};
const saveNewClusterName = async () => {
+ setSaving(true);
try {
- await ClusterApi.updateClusterName(clusterName, clusterNameInput);
- const clusterInfoCopy = cloneDeep(cluster);
- clusterInfoCopy.cluster_name = clusterNameInput;
- setClusterInfo(clusterInfoCopy);
- } catch (err) {
- console.log("Error is", err);
- toast.error("Could not update cluster name");
+ try { await ClusterApi.updateClusterName(clusterName, clusterNameInput); }
+ catch { /* Reconcile a possibly lost response by stable cluster identity. */ }
+ const current = await ClusterApi.hostClustersInfo();
+ const renamed = current.items?.find((item: { Clusters: { cluster_id: number; cluster_name: string } }) =>
+ item.Clusters.cluster_id === cluster.cluster_id && item.Clusters.cluster_name === clusterNameInput);
+ if (!renamed) throw new Error("Cluster rename could not be confirmed");
+ setClusterInfo({ ...cluster, ...renamed.Clusters });
+ } catch {
+ toast.error("Could not confirm the cluster name change. Refresh cluster overview before retrying.");
} finally {
+ setSaving(false);
setShowConfirmationModal(false);
}
};
- return clusterExists ? (
+ if (!clusterName && availableClusters?.length > 1) {
+ return <div>
+ <h4>Choose a cluster</h4>
+ <p>Select a cluster to view its details or return to its Dashboard.</p>
+ <div className="d-flex flex-wrap gap-2">
+ {availableClusters.map((item: { cluster_id: number; cluster_name: string }) => (
+ <Button key={item.cluster_id} variant="outline-primary" onClick={() => selectCluster(item.cluster_name)}>
+ {item.cluster_name}
+ </Button>
+ ))}
+ </div>
+ </div>;
+ }
+
+ return clusterName ? (
<div>
<ConfirmationModal
successCallback={saveNewClusterName}
@@ -122,7 +147,8 @@ export default function ClusterInformation() {
/>
<Form
className="p-2 m-2 d-flex flex-column"
- onSubmit={() => {
+ onSubmit={(event) => {
+ event.preventDefault();
if (!clusterNameError && clusterNameInput !== clusterName)
setShowConfirmationModal(true);
}}
@@ -144,6 +170,7 @@ export default function ClusterInformation() {
>
<Form.Control
type="input"
+ disabled={!can("AMBARI.RENAME_CLUSTER") || saving}
value={clusterNameInput}
placeholder="ClusterName"
className="me-2"
@@ -156,11 +183,11 @@ export default function ClusterInformation() {
<div className="text-danger">{clusterNameError}</div>
)}
</div>
- {clusterNameInput !== clusterName && (
+ {can("AMBARI.RENAME_CLUSTER") && clusterNameInput !== clusterName && (
<DefaultButton
type="submit"
variant="primary"
- disabled={clusterNameError || clusterNameInput.length > 80}
+ disabled={saving || clusterNameError || clusterNameInput.length > 80}
>
Save
</DefaultButton>
@@ -172,6 +199,7 @@ export default function ClusterInformation() {
<DefaultButton
variant="primary"
className="pull-right"
+ disabled={loading || loadError || !Object.keys(infoData).length}
onClick={downloadBlueprint}
>
Download
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/ClusterOverview.tsx b/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/ClusterOverview.tsx
new file mode 100644
index 0000000000..49d4b60f66
--- /dev/null
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/ClusterOverview.tsx
@@ -0,0 +1,68 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import { useState } from "react";
+import { Badge, Button, Dropdown, Form, Table } from "react-bootstrap";
+import { Link } from "react-router-dom";
+import { ClusterSummary, clusterListPath, items } from "../../api/clusterManagement";
+import useAdminResource from "../../hooks/useAdminResource";
+import { adminClusterUrl, latestClusterDashboardUrl } from "../../utils/navigation";
+import { useManagement } from "../../context/ManagementContext";
+import { LoadError, PageHeading } from "./ManagementShared";
+import ClusterDeleteDialog from "./ClusterDeleteDialog";
+
+export default function ClusterOverview() {
+ const result = useAdminResource<{ items: ClusterSummary[] }>(clusterListPath, true);
+ const { can } = useManagement();
+ const [query, setQuery] = useState("");
+ const [deleting, setDeleting] = useState<ClusterSummary>();
+ const clusters = items(result.data).filter((item) => item.Clusters.cluster_name.toLowerCase().includes(query.toLowerCase()));
+ return <>
+ <PageHeading title="Cluster Overview" description="All clusters you can access, with their own hosts, services and management actions." />
+ <div className="d-flex gap-2 mb-3 flex-wrap">
+ <Form.Control aria-label="Find cluster" placeholder="Find a cluster" value={query} onChange={(e) => setQuery(e.target.value)} style={{ maxWidth: 320 }} />
+ <Button variant="outline-secondary" onClick={result.reload}>Refresh</Button>
+ {can("AMBARI.ADD_DELETE_CLUSTERS") && <Link className="btn btn-primary" to="/clusters/create">Create cluster</Link>}
+ </div>
+ <LoadError error={result.error} retry={result.reload} />
+ {result.loading ? <p role="status">Loading clusters…</p> : !result.error && <Table responsive hover>
+ <thead><tr><th>Cluster</th><th>Installation</th><th>Host health</th><th>Hosts</th><th>Services</th><th>Stack</th><th>Actions</th></tr></thead>
+ <tbody>{clusters.map((item) => {
+ const c = item.Clusters;
+ const health = c.health_report;
+ return <tr key={c.cluster_id}>
+ <td><a href={adminClusterUrl(c.cluster_name)}>{c.cluster_name}</a><div className="small text-muted">ID {c.cluster_id}</div></td>
+ <td><Badge bg={c.provisioning_state === "INSTALLED" ? "success" : "secondary"}>{c.provisioning_state || "Unknown"}</Badge></td>
+ <td>{typeof health?.["Host/host_status/HEALTHY"] === "number" ? `${health["Host/host_status/HEALTHY"]} healthy / ${item.hosts?.length ?? "?"}` : "Unavailable"}</td>
+ <td>{item.hosts?.length ?? "Unavailable"}</td><td>{Array.isArray(item.services) ? item.services.map((s) => s.ServiceInfo.service_name).join(", ") || "No services" : "Unavailable"}</td><td>{c.version || "Not selected"}</td>
+ <td><div className="d-flex flex-wrap gap-2">
+ <Button size="sm" variant="outline-primary" href={latestClusterDashboardUrl(c.cluster_name)}>Dashboard</Button>
+ <Button size="sm" href={adminClusterUrl(c.cluster_name)}>Manage</Button>
+ <Dropdown><Dropdown.Toggle size="sm" variant="outline-secondary">More</Dropdown.Toggle><Dropdown.Menu>
+ {can("AMBARI.RENAME_CLUSTER") && <Dropdown.Item href={adminClusterUrl(c.cluster_name, "/clusterInformation?tab=export")}>Rename / export Blueprint</Dropdown.Item>}
+ {can("AMBARI.ADD_DELETE_CLUSTERS") && <Dropdown.Item onClick={() => setDeleting(item)}>Delete cluster…</Dropdown.Item>}
+ <Dropdown.Item href={adminClusterUrl(c.cluster_name, "/clusterInformation?tab=operations")}>Operation history</Dropdown.Item>
+ </Dropdown.Menu></Dropdown>
+ </div></td>
+ </tr>;
+ })}</tbody>
+ </Table>}
+ {!result.loading && !result.error && !clusters.length && <p>No matching clusters.</p>}
+ {deleting && <ClusterDeleteDialog cluster={deleting} close={() => setDeleting(undefined)} completed={() => { const target = new URL(window.location.href); target.searchParams.delete("cluster"); target.hash = "/clusters"; window.location.assign(target.href); }} />}
+ </>;
+}
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/ClusterPermissions.tsx b/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/ClusterPermissions.tsx
new file mode 100644
index 0000000000..ffedf3c02f
--- /dev/null
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/ClusterPermissions.tsx
@@ -0,0 +1,81 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import { useContext, useState } from "react";
+import { Alert, Button, Form, Modal, Table } from "react-bootstrap";
+import AppContent from "../../context/AppContext";
+import { adminApi } from "../../api/configs/axiosConfig";
+import { clusterEndpoint, Grant, Role, items, errorMessage, readResource } from "../../api/clusterManagement";
+import useAdminResource from "../../hooks/useAdminResource";
+import { useManagement } from "../../context/ManagementContext";
+import { ClusterSelect, LoadError, PageHeading } from "./ManagementShared";
+
+export function PermissionEditor({ clusterName }: { clusterName: string }) {
+ const { can } = useManagement();
+ const allowed = can("AMBARI.ASSIGN_ROLES");
+ const path = `${clusterEndpoint(clusterName)}/privileges`;
+ const result = useAdminResource<{ items: Grant[] }>(allowed ? `${path}?fields=PrivilegeInfo/*` : null, true);
+ const roles = useAdminResource<{ items: Role[] }>(allowed ? "/permissions?PermissionInfo/resource_name=CLUSTER&fields=PermissionInfo/*" : null, true);
+ const [kind, setKind] = useState("USER");
+ const [principal, setPrincipal] = useState("");
+ const [role, setRole] = useState("");
+ const [busy, setBusy] = useState(false);
+ const [error, setError] = useState("");
+ const [message, setMessage] = useState("");
+ const [remove, setRemove] = useState<Grant>();
+ const principals = useAdminResource<{ items: { Users?: { user_name: string }; Groups?: { group_name: string } }[] }>(allowed ? (kind === "USER" ? "/users?fields=Users/user_name" : "/groups?fields=Groups/group_name") : null, true);
+ async function mutate(grant?: Grant) {
+ setBusy(true); setError(""); setMessage("");
+ const name = principal.trim();
+ let mutationError: unknown;
+ try {
+ if (grant) await adminApi.delete(`${path}/${grant.PrivilegeInfo.privilege_id}`);
+ else await adminApi.post(path, [{ PrivilegeInfo: { principal_type: kind, principal_name: name, permission_name: role } }]);
+ } catch (e) { mutationError = e; }
+ try {
+ const current = items(await readResource<{ items: Grant[] }>(`${path}?fields=PrivilegeInfo/*`));
+ const applied = grant ? !current.some((g) => g.PrivilegeInfo.privilege_id === grant.PrivilegeInfo.privilege_id)
+ : current.some(({ PrivilegeInfo: g }) => g.principal_type === kind && g.principal_name === name && g.permission_name === role);
+ if (!applied) throw mutationError || new Error("The requested grant was not confirmed");
+ setMessage(grant ? "Grant removed and confirmed by the server." : "Grant saved and confirmed by the server.");
+ setRemove(undefined); if (!grant) setPrincipal("");
+ } catch (e) { setError(errorMessage(e)); }
+ finally { setBusy(false); result.reload(); }
+ }
+ if (!allowed) return <Alert variant="info">Cluster grants require Ambari role-assignment permission.</Alert>;
+ return <>
+ <p>Explicit user and group grants for <strong>{clusterName}</strong>. Group members inherit their group's grants. Ambari administrator access applies to all clusters and is managed under Users.</p>
+ <LoadError error={result.error || roles.error || principals.error} retry={() => { result.reload(); roles.reload(); principals.reload(); }} />
+ {error && <Alert variant="danger">{error}</Alert>}{message && <Alert variant="success">{message}</Alert>}
+ <Form className="d-flex flex-wrap gap-2 mb-3" onSubmit={(e) => { e.preventDefault(); void mutate(); }}>
+ <Form.Select aria-label="Principal type" value={kind} onChange={(e) => { setKind(e.target.value); setPrincipal(""); }} disabled={busy} style={{ width: 140 }}><option>USER</option><option>GROUP</option></Form.Select>
+ <Form.Select aria-label="User or group" value={principal} onChange={(e) => setPrincipal(e.target.value)} disabled={busy || principals.loading || Boolean(principals.error)} style={{ maxWidth: 260 }} required>
+ <option value="">Select {kind === "USER" ? "user" : "group"}</option>{items(principals.data).map((p) => { const name = kind === "USER" ? p.Users?.user_name : p.Groups?.group_name; return name ? <option key={name}>{name}</option> : null; })}
+ </Form.Select>
+ <Form.Select aria-label="Cluster role" value={role} onChange={(e) => setRole(e.target.value)} disabled={busy} style={{ maxWidth: 260 }} required><option value="">Select cluster role</option>{items(roles.data).map(({ PermissionInfo: p }) => <option key={p.permission_name} value={p.permission_name}>{p.permission_label}</option>)}</Form.Select>
+ <Button type="submit" disabled={busy || !principal || !role || result.loading || Boolean(result.error)}>Grant role</Button>
+ </Form>
+ {result.loading ? <p>Loading grants…</p> : <Table responsive><thead><tr><th>Principal</th><th>Type</th><th>Role</th><th /></tr></thead><tbody>{items(result.data).map((grant) => <tr key={grant.PrivilegeInfo.privilege_id}><td>{grant.PrivilegeInfo.principal_name}</td><td>{grant.PrivilegeInfo.principal_type}</td><td>{grant.PrivilegeInfo.permission_label || grant.PrivilegeInfo.permission_name}</td><td><Button size="sm" variant="outline-danger" disabled={busy} onClick={() => setRemove(gran [...]
+ {!result.loading && !result.error && !items(result.data).length && <p>No explicit grants on this cluster.</p>}
+ <Modal show={Boolean(remove)} onHide={() => { if (!busy) setRemove(undefined); }}><Modal.Header closeButton={!busy}><Modal.Title>Remove cluster grant</Modal.Title></Modal.Header><Modal.Body>{error && <Alert variant="danger">{error}</Alert>}Remove {remove?.PrivilegeInfo.permission_label || remove?.PrivilegeInfo.permission_name} from {remove?.PrivilegeInfo.principal_name} on {clusterName}? Other grants and group memberships will remain in effect.</Modal.Body><Modal.Footer><Button varia [...]
+ </>;
+}
+export default function ClusterPermissions() {
+ const { cluster } = useContext(AppContent);
+ return <><PageHeading title="Cluster Permissions" description="Manage access independently for each cluster." /><ClusterSelect route="/clusterPermissions" />{cluster?.cluster_name && <PermissionEditor key={cluster.cluster_id} clusterName={cluster.cluster_name} />}</>;
+}
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/Dasboard/index.tsx b/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/Dasboard/index.tsx
index 5f254c54dd..4500fbee6d 100644
--- a/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/Dasboard/index.tsx
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/Dasboard/index.tsx
@@ -15,9 +15,16 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-import { latestAmbariUrl } from "../../../utils/navigation";
+import { latestClusterDashboardUrl } from "../../../utils/navigation";
+
+import { useContext, useEffect } from "react";
+import AppContent from "../../../context/AppContext";
export default function Dashboard() {
- window.location.replace(latestAmbariUrl("/main/dashboard"));
+ const { cluster } = useContext(AppContent);
+ const clusterName = cluster?.cluster_name;
+ useEffect(() => {
+ window.location.replace(latestClusterDashboardUrl(clusterName));
+ }, [clusterName]);
return null;
}
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/HostResources.tsx b/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/HostResources.tsx
new file mode 100644
index 0000000000..105627f338
--- /dev/null
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/HostResources.tsx
@@ -0,0 +1,63 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import { useContext, useState } from "react";
+import { Alert, Button, Form, Table } from "react-bootstrap";
+import AppContent from "../../context/AppContext";
+import { HostSummary, hostsPath, items } from "../../api/clusterManagement";
+import useAdminResource from "../../hooks/useAdminResource";
+import { adminClusterUrl, clusterOperationUrl } from "../../utils/navigation";
+import { useManagement } from "../../context/ManagementContext";
+import { LoadError, PageHeading } from "./ManagementShared";
+
+export default function HostResources() {
+ const result = useAdminResource<{ items: HostSummary[] }>(hostsPath, true);
+ const { availableClusters } = useContext(AppContent);
+ const { can } = useManagement();
+ const [filter, setFilter] = useState("all");
+ const [query, setQuery] = useState("");
+ const [target, setTarget] = useState("");
+ const hosts = items(result.data);
+ const visible = hosts.filter(({ Hosts: host }) => (filter === "all" || (filter === "free" ? !host.cluster_name : host.cluster_name === filter)) && host.host_name.toLowerCase().includes(query.toLowerCase()));
+ return <>
+ <PageHeading title="Host Resources" description="Registered hosts across clusters. Ownership changes are performed through the target cluster's Add Host wizard." />
+ <Alert variant="info">Unassigned hosts can join a new or existing cluster. Hosts that belong to a cluster cannot be reassigned directly to another cluster.</Alert>
+ <div className="d-flex flex-wrap gap-2 mb-3">
+ <Form.Control aria-label="Find host" placeholder="Find a host" value={query} onChange={(e) => setQuery(e.target.value)} style={{ maxWidth: 280 }} />
+ <Form.Select aria-label="Filter host ownership" value={filter} onChange={(e) => setFilter(e.target.value)} style={{ maxWidth: 260 }}>
+ <option value="all">All registered hosts</option><option value="free">Unassigned hosts</option>
+ {availableClusters.map((c: { cluster_name: string; cluster_id: number }) => <option key={c.cluster_id} value={c.cluster_name}>{c.cluster_name}</option>)}
+ </Form.Select><Button variant="outline-secondary" onClick={result.reload}>Refresh</Button>
+ </div>
+ <div className="d-flex flex-wrap gap-2 mb-3 align-items-center">
+ <Form.Select aria-label="Target cluster for Add Host" value={target} onChange={(e) => setTarget(e.target.value)} style={{ maxWidth: 280 }}>
+ <option value="">Select target cluster</option>
+ {availableClusters.filter((c: { cluster_name: string; provisioning_state: string }) => c.provisioning_state === "INSTALLED" && can("HOST.ADD_DELETE_HOSTS", c.cluster_name)).map((c: { cluster_name: string; cluster_id: number }) => <option key={c.cluster_id} value={c.cluster_name}>{c.cluster_name}</option>)}
+ </Form.Select>
+ <Button disabled={!target} href={target ? clusterOperationUrl(target, "/main/host/add/step1") : undefined}>Open Add Host wizard</Button>
+ </div>
+ <LoadError error={result.error} retry={result.reload} />
+ {result.loading ? <p>Loading registered hosts…</p> : !result.error && <>
+ <p>{hosts.length} registered · {hosts.filter((h) => !h.Hosts.cluster_name).length} unassigned</p>
+ <Table responsive hover><thead><tr><th>Host</th><th>IP</th><th>Agent health</th><th>Cluster ownership</th></tr></thead><tbody>{visible.map(({ Hosts: host }) => <tr key={host.host_name}>
+ <td>{host.cluster_name ? <a href={clusterOperationUrl(host.cluster_name, `/main/hosts/${encodeURIComponent(host.host_name)}/summary`)}>{host.host_name}</a> : host.host_name}</td><td>{host.ip || "Unavailable"}</td><td>{host.host_status || "Unknown"}</td>
+ <td>{host.cluster_name ? <a href={adminClusterUrl(host.cluster_name)}>{host.cluster_name}</a> : <strong>Unassigned</strong>}</td>
+ </tr>)}</tbody></Table>{!visible.length && <p>No matching hosts.</p>}
+ </>}
+ </>;
+}
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/ManagementShared.tsx b/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/ManagementShared.tsx
new file mode 100644
index 0000000000..42deb253ee
--- /dev/null
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/ManagementShared.tsx
@@ -0,0 +1,43 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import { Alert, Button, Form } from "react-bootstrap";
+import { useContext, useEffect } from "react";
+import AppContent from "../../context/AppContext";
+import { adminClusterUrl } from "../../utils/navigation";
+import { useManagement } from "../../context/ManagementContext";
+
+export function PageHeading({ title, description }: { title: string; description: string }) {
+ const { setSelectedOption } = useContext(AppContent);
+ const { error } = useManagement();
+ useEffect(() => { setSelectedOption(title); }, [title, setSelectedOption]);
+ return <><h3>{title}</h3><p className="text-muted">{description}</p>{error && <Alert variant="warning">{error}</Alert>}</>;
+}
+export function LoadError({ error, retry }: { error?: string; retry: () => void }) {
+ return error ? <Alert variant="danger">{error} <Button variant="outline-danger" size="sm" onClick={retry}>Refresh</Button></Alert> : null;
+}
+export function ClusterSelect({ route }: { route: string }) {
+ const { cluster, availableClusters } = useContext(AppContent);
+ return <Form.Group className="mb-3" controlId="management-cluster-select">
+ <Form.Label>Cluster</Form.Label>
+ <Form.Select value={cluster?.cluster_name || ""} onChange={(event) => { if (event.target.value) window.location.assign(adminClusterUrl(event.target.value, route)); }}>
+ <option value="" disabled>Select a cluster</option>
+ {availableClusters.map((item: { cluster_id: number; cluster_name: string }) => <option key={item.cluster_id} value={item.cluster_name}>{item.cluster_name}</option>)}
+ </Form.Select>
+ </Form.Group>;
+}
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/RemoteClusters/index.tsx b/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/RemoteClusters/index.tsx
index 6c25f19c0c..b09703090b 100644
--- a/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/RemoteClusters/index.tsx
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/RemoteClusters/index.tsx
@@ -97,6 +97,7 @@ export default function RemoteClusters() {
}
return (
<>
+ <p className="text-muted">Remote registrations connect external clusters. Clusters installed and managed by this Ambari server are listed under Cluster Overview.</p>
<div className="d-flex justify-content-end pb-3">
<DefaultButton
onClick={() => {
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/StackVersions/List.tsx b/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/StackVersions/List.tsx
index 033348e139..8e35bb49b6 100644
--- a/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/StackVersions/List.tsx
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/screens/ClusterManagement/StackVersions/List.tsx
@@ -29,7 +29,10 @@ import Spinner from "../../../components/Spinner";
import ComboSearch from "../../../components/ComboSearch";
import Paginator from "../../../components/Paginator";
import Table from "../../../components/Table";
-import { latestAmbariUrl } from "../../../utils/navigation";
+import { clusterOperationUrl } from "../../../utils/navigation";
+import { errorMessage } from "../../../api/clusterManagement";
+import { useManagement } from "../../../context/ManagementContext";
+import { ClusterSelect, LoadError } from "../ManagementShared";
enum RepoStatus {
CURRENT = "CURRENT",
@@ -37,16 +40,24 @@ enum RepoStatus {
}
const StackVersionsList = () => {
+ const { cluster } = useContext(AppContent);
+ return <ScopedStackVersionsList key={cluster.cluster_name || ""} />;
+};
+
+const ScopedStackVersionsList = () => {
const [repos, setRepos] = useState<
unknown[] | ((prevState: never[]) => never[])
>([]);
- const [loading, setLoading] = useState(false);
+ const [loading, setLoading] = useState(true);
+ const [error, setError] = useState("");
+ const [revision, setRevision] = useState(0);
+ const [saving, setSaving] = useState(false);
+ const { can } = useManagement();
const [showFilters, setShowFilters] = useState(false);
const [filteredRepos, setFilteredRepos] = useState<
unknown[] | ((prevState: never[]) => never[])
>([]);
- const [clusterInformation, setClusterInformation] = useState({});
const {
cluster: { cluster_name: clusterName },
setSelectedOption,
@@ -60,15 +71,14 @@ const StackVersionsList = () => {
setItemsPerPage,
} = usePagination(filteredRepos);
useEffect(() => {
+ const controller = new AbortController();
setSelectedOption("Versions");
async function getReposData() {
setLoading(true);
let tempRepos: unknown[] | ((prevState: never[]) => never[]) = [];
- const allRepos = await VersionsApi.getRepos();
-
- const clusterData = await VersionsApi.getClusterInfo();
- const clusterInfo = clusterData.items?.[0];
- setClusterInformation(clusterInfo);
+ setError("");
+ const allRepos = await VersionsApi.getRepos(controller.signal);
+ if (!Array.isArray(allRepos?.items)) throw new Error("Invalid repository collection");
const data = allRepos.items;
for (const stack of data) {
const stackVersions = stack.versions;
@@ -77,10 +87,10 @@ const StackVersionsList = () => {
}
}
for (const repo of tempRepos as any) {
- const stackVersion = await VersionsApi.versionsList(
- repo.RepositoryVersions.id,
- clusterName
- );
+ const stackVersion = clusterName ? await VersionsApi.versionsList(
+ String(repo.RepositoryVersions.id), clusterName, controller.signal
+ ) : { items: [] };
+ if (!Array.isArray(stackVersion?.items)) throw new Error("Invalid cluster version collection");
const repoStackVersion = stackVersion?.items?.[0];
repo.isPatch = repo.RepositoryVersions.type === "PATCH";
repo.isMaint = repo.RepositoryVersions.type === "MAINT";
@@ -100,15 +110,22 @@ const StackVersionsList = () => {
repo.cluster =
repo.status === RepoStatus.CURRENT ||
repo.status === RepoStatus.INSTALLED
- ? clusterInfo?.Clusters?.cluster_name
+ ? clusterName
: "";
}
- setLoading(false);
+ if (controller.signal.aborted) return;
setRepos(tempRepos);
setFilteredRepos(tempRepos);
}
- if (clusterName) getReposData();
- }, [clusterName]);
+ void getReposData().catch((failure) => {
+ if (!controller.signal.aborted) {
+ setRepos([]);
+ setFilteredRepos([]);
+ setError(errorMessage(failure));
+ }
+ }).finally(() => { if (!controller.signal.aborted) setLoading(false); });
+ return () => controller.abort();
+ }, [clusterName, revision, setSelectedOption]);
const columns = [
{
header: "Stack",
@@ -152,7 +169,7 @@ const StackVersionsList = () => {
) : (
<a
className="custom-link"
- href={latestAmbariUrl("/main/admin/stack/versions")}
+ href={clusterOperationUrl(row.original.cluster, "/main/admin/stack/versions")}
>
{row.original.cluster}
</a>
@@ -181,6 +198,7 @@ const StackVersionsList = () => {
);
}
}
+ if (!clusterName) return <span>Select a cluster to view installation status</span>;
return (
<Dropdown>
<Dropdown.Toggle
@@ -188,6 +206,7 @@ const StackVersionsList = () => {
variant="light"
size="sm"
style={{ fontSize: 12 }}
+ disabled={!can("CLUSTER.UPGRADE_DOWNGRADE_STACK", clusterName)}
>
INSTALL ON
</Dropdown.Toggle>
@@ -196,12 +215,11 @@ const StackVersionsList = () => {
<div
onClick={() => {
window.location.replace(
- latestAmbariUrl("/main/admin/stack/versions")
+ clusterOperationUrl(clusterName, "/main/admin/stack/versions")
);
}}
>
- {((clusterInformation as any)?.Clusters
- ?.cluster_name as string) || ""}
+ {clusterName}
</div>
</Dropdown.Item>
</Dropdown.Menu>
@@ -219,7 +237,7 @@ const StackVersionsList = () => {
<Form.Check
inline
className="hiddencheckbox"
- disabled={row.original.cluster}
+ disabled={Boolean(row.original.cluster) || saving || !can("AMBARI.MANAGE_STACK_VERSIONS")}
label=""
onChange={(e) => {
toggleHiddenFor(row.original, e.target.checked);
@@ -235,19 +253,15 @@ const StackVersionsList = () => {
];
const toggleHiddenFor = async (repo: any, checked: boolean) => {
- const repoIndex = (repos as any[]).findIndex(
- (repository) =>
- repository.RepositoryVersions.id === repo.RepositoryVersions.id
- );
- const newRepos: any[] = [...(repos as any[])];
- newRepos[repoIndex].RepositoryVersions.hidden = checked;
- setRepos(newRepos);
- await VersionsApi.saveRepoVersions(
- repo.stackName,
- repo.RepositoryVersions.repository_version,
- repo.RepositoryVersions.id,
- { RepositoryVersions: { hidden: checked } }
- );
+ setSaving(true);
+ try {
+ await VersionsApi.saveRepoVersions(
+ repo.RepositoryVersions.stack_name, repo.RepositoryVersions.stack_version,
+ repo.RepositoryVersions.id, { RepositoryVersions: { hidden: checked } }
+ );
+ setRevision((value) => value + 1);
+ } catch (failure) { setError(errorMessage(failure)); }
+ finally { setSaving(false); }
};
if (loading) {
@@ -255,6 +269,8 @@ const StackVersionsList = () => {
}
return (
<div>
+ <ClusterSelect route="/stackVersions" />
+ <LoadError error={error} retry={() => setRevision((value) => value + 1)} />
<div className="d-flex w-100 justify-content-end">
<DefaultButton
onClick={() => {
@@ -264,9 +280,9 @@ const StackVersionsList = () => {
>
<FontAwesomeIcon className="text-muted" icon={faFilter} />
</DefaultButton>
- <Link to="/stackVersions/create">
+ {can("AMBARI.MANAGE_STACK_VERSIONS") && <Link to="/stackVersions/create">
<DefaultButton className="ms-3">REGISTER VERSION</DefaultButton>
- </Link>
+ </Link>}
</div>
{showFilters ? (
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/tests/ClusterInformation/ClusterIsInstalled.test.tsx b/ambari-admin/src/main/resources/ui/ambari-admin/src/tests/ClusterInformation/ClusterIsInstalled.test.tsx
index f736745685..f6000057b2 100644
--- a/ambari-admin/src/main/resources/ui/ambari-admin/src/tests/ClusterInformation/ClusterIsInstalled.test.tsx
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/tests/ClusterInformation/ClusterIsInstalled.test.tsx
@@ -15,8 +15,8 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-import { describe, it, beforeEach, expect} from "vitest";
-import {render, screen, waitFor} from "@testing-library/react";
+import { describe, it, beforeEach, expect, vi} from "vitest";
+import {fireEvent, render, screen, waitFor} from "@testing-library/react";
import {Router} from "react-router-dom";
import AppContent from "../../../src/context/AppContext";
import "@testing-library/jest-dom/vitest";
@@ -131,7 +131,7 @@ describe('Cluster is Installed', () => {
});
});
- it('should call the download methods the correct number of times', () => {
+ it('downloads the loaded blueprint as blueprint.json', async () => {
render(
<Router history={createMemoryHistory()}>
<AppContent.Provider value={mockData}>
@@ -139,43 +139,24 @@ describe('Cluster is Installed', () => {
</AppContent.Provider>
</Router>
);
- // Save original methods
- const originalCreateElement = document.createElement;
- const originalAppendChild = document.body.appendChild;
-
- // Setup our mock methods with counters
- let createElementCounter = 0;
- let setAttributeCounter = 0;
- let appendChildCounter = 0;
- let clickCounter = 0;
- let removeCounter = 0;
-
- // Mock the relevant methods to increment counters
- document.createElement = (() => {
- createElementCounter++;
- return {
- setAttribute: () => setAttributeCounter++,
- click: () => clickCounter++,
- remove: () => removeCounter++
- };
- }) as any;
-
- document.body.appendChild = (() => appendChildCounter++) as any;
-
- // Simulate click event on the download button
- const downloadButton = screen.getByText(/Download/i);
- console.log("download button is ", downloadButton);
- downloadButton.click();
-
- // Check that the methods were called
- expect(createElementCounter).toBe(1);
- expect(setAttributeCounter).toBe(2);
- expect(appendChildCounter).toBe(1);
- expect(clickCounter).toBe(1);
- expect(removeCounter).toBe(1);
-
- // Restore original methods
- document.createElement = originalCreateElement;
- document.body.appendChild = originalAppendChild;
+ const downloadButton = await screen.findByRole('button', {name: /download/i});
+ await waitFor(() => expect(downloadButton).not.toBeDisabled());
+
+ const anchor = document.createElement('a');
+ const click = vi.spyOn(anchor, 'click').mockImplementation(() => undefined);
+ const remove = vi.spyOn(anchor, 'remove').mockImplementation(() => undefined);
+ const originalCreateElement = document.createElement.bind(document);
+ const createElement = vi.spyOn(document, 'createElement').mockImplementation((tagName, options) =>
+ tagName.toLowerCase() === 'a' ? anchor : originalCreateElement(tagName, options));
+ const appendChild = vi.spyOn(document.body, 'appendChild');
+
+ fireEvent.click(downloadButton);
+
+ expect(createElement).toHaveBeenCalledWith('a');
+ expect(anchor.getAttribute('download')).toBe('blueprint.json');
+ expect(anchor.getAttribute('href')).toContain('data:text/json;charset=utf-8,');
+ expect(appendChild).toHaveBeenCalledWith(anchor);
+ expect(click).toHaveBeenCalledOnce();
+ expect(remove).toHaveBeenCalledOnce();
});
-});
\ No newline at end of file
+});
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/tests/ClusterInformation/ClusterIsNotInstalled.test.tsx b/ambari-admin/src/main/resources/ui/ambari-admin/src/tests/ClusterInformation/ClusterIsNotInstalled.test.tsx
index 7560b40a7d..481e24fb16 100644
--- a/ambari-admin/src/main/resources/ui/ambari-admin/src/tests/ClusterInformation/ClusterIsNotInstalled.test.tsx
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/tests/ClusterInformation/ClusterIsNotInstalled.test.tsx
@@ -15,152 +15,56 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-import { describe, it, beforeEach, expect} from "vitest";
-import {render, screen, waitFor} from "@testing-library/react";
-import {Router} from "react-router-dom";
-import App from "../../../src/App";
+import { cleanup, fireEvent, render, screen } from "@testing-library/react";
+import { afterEach, describe, expect, it, vi } from "vitest";
import AppContent from "../../../src/context/AppContext";
import "@testing-library/jest-dom/vitest";
-import userEvent from '@testing-library/user-event';
-import { createMemoryHistory, MemoryHistory } from "history";
-import ClusterApi from "../../../src/api/clusterApi";
-import InstallBox from "../../../src/assets/img/install-box.svg";
-import mockClusterInfo from "../../__mocks__/mockClusterInfo";
-import mockHostClusterInfo from "../../__mocks__/mockHostClusterInfo";
import ClusterInformation from "../../screens/ClusterManagement/ClusterInformation";
-describe('Cluster is not installed', () => {
- //The mockData consists of variables and values only with initial state
- //once the apis update the necessary variables the state changes accordingly
- const mockData = {
- clusterInfoLoading: false,
- isInstallWizardLaunched: false,
- clusterExists: false,
- selectedOption: '',
- cluster: {},
- setClusterInfo: () => {},
- rbacData: {},
- setRbacData: () => {},
- permissionLabelList: [],
- setPermissionLabelList: () => {},
- setSelectedOption: () => "Versions",
- };
- const history: MemoryHistory = createMemoryHistory();
- // let originalLocationHref: string;
- // let originalLocationHash: string;
- // beforeAll(() => {
- // // Save the original location
- // originalLocationHref = global.window.location.href;
- // originalLocationHash = global.window.location.hash;
- // Object.defineProperty(window, 'matchMedia', {
- // writable: true,
- // value: (query) => ({
- // matches: false,
- // media: query,
- // onchange: null,
- // addListener: () => {}, // Deprecated
- // removeListener: () => {}, // Deprecated
- // addEventListener: () => {},
- // removeEventListener: () => {},
- // dispatchEvent: () => {},
- // }),
- // });
- // });
- //
- // afterAll(() => {
- // // Restore the original location properties
- // global.window.location.href = originalLocationHref;
- // global.window.location.hash = originalLocationHash;
- // });
-
- beforeAll(() => {
- window.matchMedia = window.matchMedia || function() {
- return {
- matches: false,
- addListener: function() {},
- removeListener: function() {}
- };
- };
- });
- beforeEach(() => {
- // history = createMemoryHistory();
- //Mock window object
- const { location } = window;
- //delete global.window.location;
- global.window.location = { ...location,
- //@ts-ignore
- replace: () => {},
- hash: '/clusterInformation' // add this line to set the hash
- };
- const url = "http://localhost";
- global.window.location.href = url;
- ClusterApi.clusterInfo = async () => mockClusterInfo;
- ClusterApi.hostClustersInfo = async () => mockHostClusterInfo;
- });
- it('should render without crashing', () => {
- render(
- <Router history={createMemoryHistory()}>
- <AppContent.Provider value={mockData}>
- <ClusterInformation />
- </AppContent.Provider>
- </Router>
- );
- });
+describe("Cluster is not selected", () => {
+ afterEach(cleanup);
- const texts = [
- 'Welcome to Apache Ambari',
- 'Provision a cluster, manage who can access the cluster, and customize views for Ambari users.',
- 'Create a Cluster',
- 'Use the Install Wizard to select services and configure your cluster'
- ];
+ it("offers an explicit choice when multiple clusters are available", () => {
+ const selectCluster = vi.fn();
+ render(
+ <AppContent.Provider value={{
+ cluster: {},
+ setClusterInfo: vi.fn(),
+ setSelectedOption: vi.fn(),
+ availableClusters: [
+ { cluster_id: 11, cluster_name: "analytics" },
+ { cluster_id: 22, cluster_name: "warehouse" },
+ ],
+ selectCluster,
+ }}>
+ <ClusterInformation />
+ </AppContent.Provider>,
+ );
- it.each(texts)('displays the text "%s"', async (text) => {
- render(
- <Router history={history}>
- <AppContent.Provider value={mockData}>
- <App />
- </AppContent.Provider>
- </Router>
- );
- await waitFor(async () => {
- const textElement = await screen.findByText(new RegExp(text, 'i'));
- console.log("text element is ", textElement.textContent);
- expect(textElement).toBeInTheDocument();
- });
- });
+ expect(screen.getByRole("heading", { name: "Choose a cluster" })).toBeInTheDocument();
+ expect(screen.getByText("Select a cluster to view its details or return to its Dashboard."))
+ .toBeInTheDocument();
- it('renders the launch install wizard button and navigates to the installer route on click', async () => {
- render(
- <Router history={history}>
- <AppContent.Provider value={mockData}>
- <App />
- </AppContent.Provider>
- </Router>
- );
- const launchInstallWizardButton = await screen.findByRole(
- 'button',
- {name: /launch install wizard/i},
- {timeout: 5000},
- );
- expect(launchInstallWizardButton).toBeInTheDocument();
+ fireEvent.click(screen.getByRole("button", { name: "warehouse" }));
- await userEvent.click(launchInstallWizardButton);
+ expect(selectCluster).toHaveBeenCalledOnce();
+ expect(selectCluster).toHaveBeenCalledWith("warehouse");
+ });
- expect(window.location.href).toBe('/latest/#/installer/step0');
- });
+ it("does not render cluster-specific controls without a selected cluster", () => {
+ const { container } = render(
+ <AppContent.Provider value={{
+ cluster: {},
+ setClusterInfo: vi.fn(),
+ setSelectedOption: vi.fn(),
+ availableClusters: [],
+ selectCluster: vi.fn(),
+ }}>
+ <ClusterInformation />
+ </AppContent.Provider>,
+ );
- it('renders installBox image when conditions are met', async () => {
- render(
- <Router history={history}>
- <AppContent.Provider value={mockData}>
- <App />
- </AppContent.Provider>
- </Router>
- );
- await waitFor(() => {
- const installBoxImgElement = screen.getByAltText('Install Box') as HTMLImageElement;
- expect(installBoxImgElement).toBeInTheDocument();
- expect(installBoxImgElement).toHaveAttribute("src", InstallBox);
- })
- });
+ expect(container).toBeEmptyDOMElement();
+ expect(screen.queryByRole("button", { name: /download/i })).not.toBeInTheDocument();
+ });
});
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/tests/EditRemoteCluster.test.tsx b/ambari-admin/src/main/resources/ui/ambari-admin/src/tests/EditRemoteCluster.test.tsx
index 4c59f2b042..8eee0c83d0 100644
--- a/ambari-admin/src/main/resources/ui/ambari-admin/src/tests/EditRemoteCluster.test.tsx
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/tests/EditRemoteCluster.test.tsx
@@ -64,14 +64,23 @@ describe("EditRemoteCluster component", () => {
beforeEach(() => {
mockToastErrorMessage = "";
mockToastSuccessMessage = "";
+ RemoteClusterApi.getRemoteClusterByName = async () =>
+ mockClusterDataForEdit;
});
- it("renders without crashing", () => {
+ it("renders without crashing", async () => {
renderEditRemoteCluster();
+
+ await waitFor(() => {
+ expect(getClusterNameInput()).toHaveValue(
+ mockClusterDataForEdit.ClusterInfo.name
+ );
+ });
});
- it("shows loading spinner when data is being fetched.", async () => {
- RemoteClusterApi.getRemoteClusterByName = async () => [];
+ it("shows loading spinner when data is being fetched.", () => {
+ RemoteClusterApi.getRemoteClusterByName = () =>
+ new Promise<never>(() => undefined);
renderEditRemoteCluster();
const spinner = screen.getByTestId("admin-spinner");
@@ -83,13 +92,14 @@ describe("EditRemoteCluster component", () => {
mockClusterDataForEdit;
renderEditRemoteCluster();
- await waitFor(() => {});
- expect(getClusterNameInput()).toHaveValue(
- mockClusterDataForEdit.ClusterInfo.name
- );
- expect(getclusterUrlInput()).toHaveValue(
- mockClusterDataForEdit.ClusterInfo.url
- );
+ await waitFor(() => {
+ expect(getClusterNameInput()).toHaveValue(
+ mockClusterDataForEdit.ClusterInfo.name
+ );
+ expect(getclusterUrlInput()).toHaveValue(
+ mockClusterDataForEdit.ClusterInfo.url
+ );
+ });
});
it("should redirect to /remoteCluster on clicking cancel button", async () => {
@@ -103,12 +113,10 @@ describe("EditRemoteCluster component", () => {
</AppContent.Provider>
);
- await waitFor(() => {});
- const cancelButton = screen.getByRole("button", { name: /cancel/i });
+ const cancelButton = await screen.findByRole("button", { name: /cancel/i });
expect(cancelButton).toBeInTheDocument();
fireEvent.click(cancelButton);
- await waitFor(() => {});
await waitFor(() => {
expect(window.location.hash).toBe("#/remoteClusters");
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/tests/RegisterRemoteCluster.test.tsx b/ambari-admin/src/main/resources/ui/ambari-admin/src/tests/RegisterRemoteCluster.test.tsx
index d8167e0bff..b013790954 100644
--- a/ambari-admin/src/main/resources/ui/ambari-admin/src/tests/RegisterRemoteCluster.test.tsx
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/tests/RegisterRemoteCluster.test.tsx
@@ -83,7 +83,6 @@ describe("RegisterRemoteCluster component", () => {
expect(cancelButton).toBeInTheDocument();
fireEvent.click(cancelButton);
- await waitFor(() => {});
await waitFor(() => {
expect(window.location.hash).toBe("#/remoteClusters");
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/tests/clusterManagement.test.tsx b/ambari-admin/src/main/resources/ui/ambari-admin/src/tests/clusterManagement.test.tsx
new file mode 100644
index 0000000000..c2451d96b2
--- /dev/null
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/tests/clusterManagement.test.tsx
@@ -0,0 +1,76 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import { act, fireEvent, render, renderHook, screen, waitFor, cleanup } from "@testing-library/react";
+import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
+import { hasPermission, clusterEndpoint } from "../api/clusterManagement";
+import useAdminResource from "../hooks/useAdminResource";
+import { PermissionEditor } from "../screens/ClusterManagement/ClusterPermissions";
+
+const transport = vi.hoisted(() => ({ get: vi.fn(), post: vi.fn(), delete: vi.fn() }));
+vi.mock("../api/configs/axiosConfig", () => ({ adminApi: transport }));
+vi.mock("../context/ManagementContext", () => ({ useManagement: () => ({ can: () => true }) }));
+afterEach(cleanup);
+beforeEach(() => { vi.resetAllMocks(); });
+
+describe("cluster management boundaries", () => {
+ it("does not apply cluster A permissions to cluster B or global management", () => {
+ const grants = [{ authorization_id: "HOST.ADD_DELETE_HOSTS", resource_type: "CLUSTER", cluster_name: "alpha" }];
+ expect(hasPermission(grants, "HOST.ADD_DELETE_HOSTS", "alpha")).toBe(true);
+ expect(hasPermission(grants, "HOST.ADD_DELETE_HOSTS", "beta")).toBe(false);
+ expect(hasPermission(grants, "HOST.ADD_DELETE_HOSTS")).toBe(false);
+ expect(clusterEndpoint("east / prod")).toBe("/clusters/east%20%2F%20prod");
+ });
+
+ it("discards a late response from the previously selected cluster", async () => {
+ let finishAlpha!: (value: unknown) => void;
+ transport.get.mockImplementation((path: string) => path === "/clusters/alpha"
+ ? new Promise((resolve) => { finishAlpha = resolve; })
+ : Promise.resolve({ data: { Clusters: { cluster_id: 2 } } }));
+ const hook = renderHook(({ path }) => useAdminResource<{ Clusters: { cluster_id: number } }>(path), { initialProps: { path: "/clusters/alpha" } });
+ hook.rerender({ path: "/clusters/beta" });
+ await waitFor(() => expect(hook.result.current.data?.Clusters.cluster_id).toBe(2));
+ await act(async () => finishAlpha({ data: { Clusters: { cluster_id: 1 } } }));
+ expect(hook.result.current.data?.Clusters.cluster_id).toBe(2);
+ });
+
+ it("does not treat a malformed collection as an empty successful result", async () => {
+ transport.get.mockResolvedValue({ data: {} });
+ const hook = renderHook(() => useAdminResource("/hosts", true));
+ await waitFor(() => expect(hook.result.current.error).toBeTruthy());
+ expect(hook.result.current.data).toBeUndefined();
+ });
+
+ it("reconciles a lost grant response using the exact server grant", async () => {
+ let saved = false;
+ transport.get.mockImplementation((path: string) => Promise.resolve({ data: { items:
+ path.startsWith("/permissions") ? [{ PermissionInfo: { resource_name: "CLUSTER", permission_name: "CLUSTER.USER", permission_label: "Cluster User" } }]
+ : path.startsWith("/users") ? [{ Users: { user_name: "operator" } }]
+ : saved ? [{ PrivilegeInfo: { privilege_id: 7, principal_type: "USER", principal_name: "operator", permission_name: "CLUSTER.USER" } }] : []
+ } }));
+ transport.post.mockImplementation(async () => { saved = true; throw new Error("Response lost"); });
+ render(<PermissionEditor clusterName="alpha" />);
+ await screen.findByRole("option", { name: "operator" });
+ fireEvent.change(screen.getByLabelText("User or group"), { target: { value: "operator" } });
+ fireEvent.change(screen.getByLabelText("Cluster role"), { target: { value: "CLUSTER.USER" } });
+ fireEvent.click(screen.getByRole("button", { name: "Grant role" }));
+ expect(await screen.findByText("Grant saved and confirmed by the server.")).toBeTruthy();
+ expect(transport.post).toHaveBeenCalledTimes(1);
+ expect(transport.post.mock.calls[0][0]).toBe("/clusters/alpha/privileges");
+ });
+});
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/tests/components/ClusterManagement/RemoteCluster/EditRemoteCluster.test.tsx b/ambari-admin/src/main/resources/ui/ambari-admin/src/tests/components/ClusterManagement/RemoteCluster/EditRemoteCluster.test.tsx
index 2a1428b6b5..dc48d6436b 100644
--- a/ambari-admin/src/main/resources/ui/ambari-admin/src/tests/components/ClusterManagement/RemoteCluster/EditRemoteCluster.test.tsx
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/tests/components/ClusterManagement/RemoteCluster/EditRemoteCluster.test.tsx
@@ -65,14 +65,23 @@ describe("EditRemoteCluster component", () => {
beforeEach(() => {
mockToastErrorMessage = "";
mockToastSuccessMessage = "";
+ RemoteClusterApi.getRemoteClusterByName = async () =>
+ mockClusterDataForEdit;
});
- it("renders without crashing", () => {
+ it("renders without crashing", async () => {
renderEditRemoteCluster();
+
+ await waitFor(() => {
+ expect(getClusterNameInput()).toHaveValue(
+ mockClusterDataForEdit.ClusterInfo.name
+ );
+ });
});
- it("shows loading spinner when data is being fetched.", async () => {
- RemoteClusterApi.getRemoteClusterByName = async () => [];
+ it("shows loading spinner when data is being fetched.", () => {
+ RemoteClusterApi.getRemoteClusterByName = () =>
+ new Promise<never>(() => undefined);
renderEditRemoteCluster();
const spinner = screen.getByTestId("admin-spinner");
@@ -84,13 +93,14 @@ describe("EditRemoteCluster component", () => {
mockClusterDataForEdit;
renderEditRemoteCluster();
- await waitFor(() => {});
- expect(getClusterNameInput()).toHaveValue(
- mockClusterDataForEdit.ClusterInfo.name
- );
- expect(getclusterUrlInput()).toHaveValue(
- mockClusterDataForEdit.ClusterInfo.url
- );
+ await waitFor(() => {
+ expect(getClusterNameInput()).toHaveValue(
+ mockClusterDataForEdit.ClusterInfo.name
+ );
+ expect(getclusterUrlInput()).toHaveValue(
+ mockClusterDataForEdit.ClusterInfo.url
+ );
+ });
});
it("should redirect to /remoteCluster on clicking cancel button", async () => {
@@ -105,12 +115,10 @@ describe("EditRemoteCluster component", () => {
</AppContent.Provider>
);
- await waitFor(() => {});
- const cancelButton = screen.getByRole("button", { name: /cancel/i });
+ const cancelButton = await screen.findByRole("button", { name: /cancel/i });
expect(cancelButton).toBeInTheDocument();
fireEvent.click(cancelButton);
- await waitFor(() => {});
await waitFor(() => {
expect(history.location.pathname).toBe("/remoteClusters");
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/tests/components/ClusterManagement/RemoteCluster/RegisterRemoteCluster.test.tsx b/ambari-admin/src/main/resources/ui/ambari-admin/src/tests/components/ClusterManagement/RemoteCluster/RegisterRemoteCluster.test.tsx
index 71c8880498..42e4b4e537 100644
--- a/ambari-admin/src/main/resources/ui/ambari-admin/src/tests/components/ClusterManagement/RemoteCluster/RegisterRemoteCluster.test.tsx
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/tests/components/ClusterManagement/RemoteCluster/RegisterRemoteCluster.test.tsx
@@ -85,7 +85,6 @@ describe("RegisterRemoteCluster component", () => {
expect(cancelButton).toBeInTheDocument();
fireEvent.click(cancelButton);
- await waitFor(() => {});
await waitFor(() => {
expect(history.location.pathname).toBe("/remoteClusters");
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/tests/navigation.test.ts b/ambari-admin/src/main/resources/ui/ambari-admin/src/tests/navigation.test.ts
index 7e8fada83d..9884db7b58 100644
--- a/ambari-admin/src/main/resources/ui/ambari-admin/src/tests/navigation.test.ts
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/tests/navigation.test.ts
@@ -20,6 +20,7 @@ import { describe, expect, it } from "vitest";
import {
ambariApplicationRoot,
latestAmbariUrl,
+ latestClusterDashboardUrl,
latestShortViewUrl,
latestViewInstanceUrl,
} from "../utils/navigation";
@@ -46,4 +47,11 @@ describe("Ambari experience navigation", () => {
expect(latestShortViewUrl("CAPACITY-SCHEDULER", "queue editor", path))
.toBe("/latest/#/main/view/CAPACITY-SCHEDULER/queue%20editor");
});
+ it("returns from Admin to the explicit cluster dashboard", () => {
+ expect(latestClusterDashboardUrl("east / prod", "/gateway/ambari/views/ADMIN_VIEW/3/INSTANCE/latest/"))
+ .toBe("/gateway/ambari/latest/#/clusters/east%20%2F%20prod/main/dashboard/metrics");
+ expect(latestClusterDashboardUrl(undefined, "/views/ADMIN_VIEW/3/INSTANCE/latest/"))
+ .toBe("/latest/#/");
+ });
+
});
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/tests/stackVersions.test.tsx b/ambari-admin/src/main/resources/ui/ambari-admin/src/tests/stackVersions.test.tsx
new file mode 100644
index 0000000000..aff054aef2
--- /dev/null
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/tests/stackVersions.test.tsx
@@ -0,0 +1,97 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import { act, cleanup, render, screen } from "@testing-library/react";
+import { MemoryRouter } from "react-router-dom";
+import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
+import AppContent from "../context/AppContext";
+import StackVersionsList from "../screens/ClusterManagement/StackVersions/List";
+
+const transport = vi.hoisted(() => ({ request: vi.fn() }));
+vi.mock("../api/configs/axiosConfig", () => ({ adminApi: transport }));
+vi.mock("../context/ManagementContext", () => ({ useManagement: () => ({ can: () => true }) }));
+afterEach(cleanup);
+beforeEach(() => vi.resetAllMocks());
+const catalog = { items: [{ versions: [{ repository_versions: [{ RepositoryVersions: {
+ id: 1, display_name: "Test repository", type: "STANDARD", stack_name: "BIGTOP",
+ stack_version: "3.3.0", repository_version: "3.3.0", hidden: false,
+} }] }] }] };
+const status = (count: number) => ({ items: [{ ClusterStackVersions: {
+ id: 10, state: "CURRENT", host_states: { CURRENT: Array.from({ length: count }, (_, i) => `host${i}`) },
+} }] });
+const setSelectedOption = vi.fn();
+function page(name: string) {
+ return <MemoryRouter><AppContent.Provider value={{ cluster: { cluster_name: name },
+ availableClusters: [{ cluster_id: 1, cluster_name: "alpha" }, { cluster_id: 2, cluster_name: "east / prod" }], setSelectedOption }}>
+ <StackVersionsList />
+ </AppContent.Provider></MemoryRouter>;
+}
+
+describe("Admin repository cluster ownership", () => {
+ it("renders status and encoded navigation for the selected cluster", async () => {
+ transport.request.mockImplementation(async ({ url }) => {
+ if (url.startsWith("/stacks?")) return { data: structuredClone(catalog) };
+ if (url === "/clusters/east%20%2F%20prod/stack_versions?fields=*&ClusterStackVersions/repository_version=1") return { data: status(2) };
+ throw new Error(`Unexpected request: ${url}`);
+ });
+ render(page("east / prod"));
+ const link = await screen.findByRole("link", { name: "east / prod" });
+ expect(link.getAttribute("href")).toBe("/latest/#/clusters/east%20%2F%20prod/main/admin/stack/versions");
+ expect(screen.getByText("CURRENT: 2/2")).toBeTruthy();
+ expect(screen.queryByRole("link", { name: "alpha" })).toBeNull();
+ });
+
+ it("loads the global repository catalog without inventing a current cluster", async () => {
+ transport.request.mockImplementation(async ({ url }) => {
+ if (url.startsWith("/stacks?")) return { data: structuredClone(catalog) };
+ throw new Error(`Unexpected cluster request: ${url}`);
+ });
+ render(page(""));
+ expect(await screen.findByRole("link", { name: "Test repository" })).toBeTruthy();
+ expect(screen.getByText("Select a cluster to view installation status")).toBeTruthy();
+ expect(screen.queryByText("INSTALL ON")).toBeNull();
+ });
+
+ it("ignores a delayed status response after switching clusters", async () => {
+ let resolveAlpha!: (value: unknown) => void;
+ const alphaPending = new Promise((resolve) => { resolveAlpha = resolve; });
+ transport.request.mockImplementation(async ({ url }) => {
+ if (url.startsWith("/stacks?")) return { data: structuredClone(catalog) };
+ if (url.startsWith("/clusters/alpha/")) return alphaPending;
+ return { data: status(2) };
+ });
+ const view = render(page("alpha"));
+ await act(async () => {});
+ view.rerender(page("east / prod"));
+ await screen.findByRole("link", { name: "east / prod" });
+ await act(async () => { resolveAlpha({ data: status(5) }); });
+ expect(screen.queryByRole("link", { name: "alpha" })).toBeNull();
+ expect(screen.getByText("CURRENT: 2/2")).toBeTruthy();
+ });
+
+ it("reports a failed cluster status load instead of showing a usable empty status", async () => {
+ transport.request.mockImplementation(async ({ url }) => {
+ if (url.startsWith("/stacks?")) return { data: structuredClone(catalog) };
+ throw new Error("Status unavailable");
+ });
+ render(page("alpha"));
+ expect(await screen.findByRole("alert")).toBeTruthy();
+ expect(screen.queryByText("INSTALL ON")).toBeNull();
+ expect(screen.getByRole("button", { name: "Refresh" })).toBeTruthy();
+ });
+});
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/types.ts b/ambari-admin/src/main/resources/ui/ambari-admin/src/types.ts
index fbf635a369..c4dfbd78d9 100644
--- a/ambari-admin/src/main/resources/ui/ambari-admin/src/types.ts
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/types.ts
@@ -27,6 +27,7 @@ interface ClusterInfoType {
};
interface HostCluster{
+ cluster_id?: number;
cluster_name: string;
provisioning_state: string;
}
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/utils/navigation.ts b/ambari-admin/src/main/resources/ui/ambari-admin/src/utils/navigation.ts
index 5e9284589e..fb165e865a 100644
--- a/ambari-admin/src/main/resources/ui/ambari-admin/src/utils/navigation.ts
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/utils/navigation.ts
@@ -33,6 +33,15 @@ export function latestAmbariUrl(
return `${ambariApplicationRoot(documentPath)}latest/#${normalizedRoute}`;
}
+export function latestClusterDashboardUrl(
+ clusterName: string | undefined,
+ documentPath = window.location.pathname,
+): string {
+ return latestAmbariUrl(clusterName
+ ? `/clusters/${encodeURIComponent(clusterName)}/main/dashboard/metrics`
+ : "/", documentPath);
+}
+
export function latestViewInstanceUrl(
viewName: string,
version: string,
@@ -55,3 +64,14 @@ export function latestShortViewUrl(
.join("/");
return latestAmbariUrl(`/main/view/${route}`, documentPath);
}
+
+export function adminClusterUrl(clusterName: string, route = "/clusterInformation"): string {
+ const target = new URL(window.location.href);
+ target.searchParams.set("cluster", clusterName);
+ target.hash = route;
+ return target.href;
+}
+
+export function clusterOperationUrl(clusterName: string, route: string): string {
+ return latestAmbariUrl(`/clusters/${encodeURIComponent(clusterName)}${route}`);
+}
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/topology/PersistedTopologyRequest.java b/ambari-admin/src/main/resources/ui/ambari-admin/src/utils/session.ts
similarity index 56%
copy from ambari-server/src/main/java/org/apache/ambari/server/topology/PersistedTopologyRequest.java
copy to ambari-admin/src/main/resources/ui/ambari-admin/src/utils/session.ts
index 8c09eee75d..09d2c93d9d 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/topology/PersistedTopologyRequest.java
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/utils/session.ts
@@ -1,9 +1,9 @@
-/*
+/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
+ * to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
@@ -16,26 +16,18 @@
* limitations under the License.
*/
-package org.apache.ambari.server.topology;
-
-/**
- * Wrapper around a TopologyRequest which adds an id that can be used
- * to refer to the persisted entity.
- */
-public class PersistedTopologyRequest {
- private final long id;
- private final TopologyRequest request;
-
- public PersistedTopologyRequest(long id, TopologyRequest request) {
- this.id = id;
- this.request = request;
- }
-
- public long getId() {
- return id;
- }
-
- public TopologyRequest getRequest() {
- return request;
+export function clearClientSession(): void {
+ for (const key of Object.keys(localStorage)) {
+ let navigationPreference = false;
+ if (key.startsWith("ambari.navigation.v1:")) {
+ try {
+ const value = JSON.parse(localStorage.getItem(key) || "null");
+ navigationPreference = typeof value === "number" && Number.isSafeInteger(value) && value > 0;
+ } catch {
+ // Invalid values cannot survive session cleanup.
+ }
+ }
+ if (!navigationPreference) localStorage.removeItem(key);
}
+ sessionStorage.clear();
}
diff --git a/ambari-agent/src/main/java/org/apache/ambari/tools/hadoop/ManagedDependencyClient.java b/ambari-agent/src/main/java/org/apache/ambari/tools/hadoop/ManagedDependencyClient.java
new file mode 100644
index 0000000000..6865ceffd7
--- /dev/null
+++ b/ambari-agent/src/main/java/org/apache/ambari/tools/hadoop/ManagedDependencyClient.java
@@ -0,0 +1,290 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ambari.tools.hadoop;
+
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.net.URI;
+import java.nio.charset.StandardCharsets;
+import java.security.MessageDigest;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.UUID;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FSDataInputStream;
+import org.apache.hadoop.fs.FSDataOutputStream;
+import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.fs.permission.AclEntry;
+import org.apache.hadoop.fs.permission.FsPermission;
+import org.apache.hadoop.security.AccessControlException;
+import org.apache.hadoop.security.UserGroupInformation;
+import org.apache.hadoop.util.VersionInfo;
+
+/** Stateless client observations. The Agent journal and Server own workflow state. */
+public final class ManagedDependencyClient {
+ private static final ObjectMapper JSON = new ObjectMapper();
+ static final int PROBE_BYTES = 128 * 1024 + 1;
+ private static final int MAX_MARKER_BYTES = 8192;
+
+ private ManagedDependencyClient() {
+ }
+
+ public static void main(String[] args) throws Exception {
+ if (args.length != 4) {
+ throw new IllegalArgumentException("Expected operation, configuration directory, identity, payload");
+ }
+ String operation = args[0];
+ JsonNode identity = JSON.readTree(args[2]);
+ validateIdentity(identity);
+ JsonNode payload = JSON.readTree(args[3]);
+ Map<String, Object> response = new LinkedHashMap<>();
+ response.put("schemaVersion", 1);
+ response.put("identity", identity);
+ response.put("operation", operation);
+ try {
+ Map<String, Object> result;
+ if ("VERSION".equals(operation)) {
+ String kind = requiredText(payload, "kind");
+ // HBase is supplied by the installed client, not bundled into the Agent.
+ String version = "HADOOP_CLIENT".equals(kind) ? VersionInfo.getVersion()
+ : "HBASE_CLIENT".equals(kind)
+ ? (String) Class.forName("org.apache.hadoop.hbase.util.VersionInfo")
+ .getMethod("getVersion").invoke(null)
+ : null;
+ if (version == null) {
+ throw new IllegalArgumentException("Unsupported client software kind");
+ }
+ result = Map.of("kind", kind, "version", version);
+ } else {
+ Configuration config = new Configuration(false);
+ config.addResource(new Path(args[1], "core-site.xml"));
+ config.addResource(new Path(args[1], "hdfs-site.xml"));
+ UserGroupInformation.setConfiguration(config);
+ URI uri = URI.create(requiredText(payload, "path"));
+ if (!"hdfs".equals(uri.getScheme()) || uri.getAuthority() == null
+ || uri.getQuery() != null || uri.getFragment() != null || uri.getUserInfo() != null) {
+ throw new IllegalArgumentException("Expected an explicit HDFS authority");
+ }
+ try (FileSystem fs = FileSystem.newInstance(uri, config)) {
+ result = execute(fs, operation, identity, payload);
+ }
+ }
+ response.put("status", "SUCCEEDED");
+ response.put("result", result);
+ } catch (ProbeFailure failure) {
+ response.put("status", "FAILED");
+ response.put("errorCode", failure.code);
+ } catch (AccessControlException failure) {
+ response.put("status", "FAILED");
+ response.put("errorCode", "DEPENDENCY_AUTHORIZATION_FAILED");
+ } catch (Exception failure) {
+ response.put("status", "FAILED");
+ response.put("errorCode", "DEPENDENCY_CLIENT_OBSERVATION_FAILED");
+ }
+ System.out.println(JSON.writeValueAsString(response));
+ }
+
+ static Map<String, Object> execute(FileSystem fs, String operation,
+ JsonNode identity, JsonNode payload) throws Exception {
+ Path path = new Path(requiredText(payload, "path"));
+ switch (operation) {
+ case "STAT":
+ try {
+ FileStatus status = fs.getFileLinkStatus(path);
+ return Map.of("exists", true, "type", status.isSymlink() ? "SYMLINK"
+ : status.isDirectory() ? "DIRECTORY" : "FILE",
+ "owner", status.getOwner(), "group", status.getGroup(),
+ "mode", (int) status.getPermission().toShort());
+ } catch (FileNotFoundException absent) {
+ return Map.of("exists", false);
+ }
+ case "ACL":
+ List<Map<String, Object>> entries = new ArrayList<>();
+ for (AclEntry entry : fs.getAclStatus(path).getEntries()) {
+ entries.add(Map.of("scope", entry.getScope().name(), "type", entry.getType().name(),
+ "name", entry.getName() == null ? "" : entry.getName(),
+ "permission", entry.getPermission().ordinal()));
+ }
+ return Map.of("entries", entries);
+ case "COUNT":
+ // Listing only the first child avoids a recursive traversal of provider data.
+ return Map.of("empty", !fs.listStatusIterator(path).hasNext());
+ case "MKDIR":
+ requireMutationPath(path, identity, true);
+ if (!fs.getFileStatus(path.getParent()).isDirectory()) {
+ throw new IOException("Managed parent is not a directory");
+ }
+ FsPermission permission = permission(payload);
+ fs.getConf().set("fs.permissions.umask-mode", "000");
+ if (!fs.mkdirs(path, permission)) {
+ throw new IOException("Managed directory creation failed");
+ }
+ return Map.of("created", true);
+ case "SET_OWNER":
+ requireMutationPath(path, identity, false);
+ fs.setOwner(path, requiredText(payload, "owner"), requiredText(payload, "group"));
+ return Map.of("applied", true);
+ case "SET_PERMISSION":
+ requireMutationPath(path, identity, false);
+ fs.setPermission(path, permission(payload));
+ return Map.of("applied", true);
+ case "READ_MARKER":
+ requireMarkerPath(path, identity);
+ try (FSDataInputStream input = fs.open(path)) {
+ byte[] bytes = input.readNBytes(MAX_MARKER_BYTES + 1);
+ if (bytes.length > MAX_MARKER_BYTES) {
+ throw new IOException("Oversized binding marker");
+ }
+ JsonNode marker = JSON.readTree(bytes);
+ if (!marker.isObject()) {
+ throw new IOException("Invalid binding marker");
+ }
+ return Map.of("marker", marker);
+ }
+ case "WRITE_MARKER":
+ requireMarkerPath(path, identity);
+ byte[] marker = JSON.writeValueAsBytes(payload.required("marker"));
+ if (marker.length > MAX_MARKER_BYTES || !payload.get("marker").isObject()) {
+ throw new IOException("Invalid binding marker");
+ }
+ try (FSDataOutputStream output = fs.create(path, new FsPermission((short) 0600),
+ false, 4096, fs.getDefaultReplication(path), fs.getDefaultBlockSize(path), null)) {
+ output.write(marker);
+ }
+ return Map.of("written", true);
+ case "PROBE":
+ requireMutationPath(path, identity, false);
+ if (!path.toUri().getPath().endsWith("/root")) {
+ throw new IllegalArgumentException("Probe must be inside the binding root");
+ }
+ return probe(fs, path, identity);
+ default:
+ throw new IllegalArgumentException("Unsupported client operation");
+ }
+ }
+
+ private static Map<String, Object> probe(FileSystem fs, Path root, JsonNode identity)
+ throws Exception {
+ Path path = new Path(root, ".ambari-managed-probe-" + requiredText(identity, "operationId"));
+ byte[] digest = MessageDigest.getInstance("SHA-256")
+ .digest(requiredText(identity, "immutableRequestHash").getBytes(StandardCharsets.UTF_8));
+ byte[] expected = new byte[PROBE_BYTES];
+ for (int i = 0; i < expected.length; i++) {
+ expected[i] = digest[i % digest.length];
+ }
+ boolean owned = false;
+ try {
+ if (!fs.exists(path)) {
+ try (FSDataOutputStream output = fs.create(path, false)) {
+ owned = true;
+ output.write(expected);
+ }
+ }
+ byte[] observed;
+ try (FSDataInputStream input = fs.open(path)) {
+ observed = input.readNBytes(PROBE_BYTES + 1);
+ }
+ if (!Arrays.equals(observed, expected)) {
+ throw new ProbeFailure("DEPENDENCY_DATA_INTEGRITY_FAILED");
+ }
+ // Only an exact replay's verified content authorizes cleaning a preexisting probe.
+ owned = true;
+ } catch (ProbeFailure failure) {
+ throw failure;
+ } catch (AccessControlException failure) {
+ throw failure;
+ } catch (IOException failure) {
+ throw new ProbeFailure("DEPENDENCY_DATANODE_READ_WRITE_FAILED");
+ } finally {
+ if (owned) {
+ try {
+ if (!fs.delete(path, false)) {
+ throw new IOException("Probe was not deleted");
+ }
+ } catch (IOException failure) {
+ throw new ProbeFailure("DEPENDENCY_PROBE_CLEANUP_FAILED");
+ }
+ }
+ }
+ return Map.of("readWriteVerified", true, "bytes", PROBE_BYTES, "cleaned", true);
+ }
+
+ private static void requireMutationPath(Path path, JsonNode identity, boolean parents) {
+ String binding = requiredText(identity, "bindingId");
+ String actual = path.toUri().getPath();
+ String base = "/apps/ambari-managed/hbase/" + binding;
+ if (actual.equals(base) || actual.equals(base + "/root") || actual.equals(base + "/wal")
+ || (parents && List.of("/apps", "/apps/ambari-managed",
+ "/apps/ambari-managed/hbase", "/apps/ambari-managed/.bindings").contains(actual))) {
+ return;
+ }
+ throw new IllegalArgumentException("Path is outside the exact managed namespace");
+ }
+
+ private static void requireMarkerPath(Path path, JsonNode identity) {
+ if (!path.toUri().getPath().equals("/apps/ambari-managed/.bindings/"
+ + requiredText(identity, "bindingId") + ".json")) {
+ throw new IllegalArgumentException("Path is outside the exact binding marker");
+ }
+ }
+
+ private static FsPermission permission(JsonNode payload) {
+ JsonNode mode = payload.required("mode");
+ if (!mode.isInt() || mode.intValue() < 0 || mode.intValue() > 0777) {
+ throw new IllegalArgumentException("Invalid permission bits");
+ }
+ return new FsPermission((short) mode.intValue());
+ }
+
+ private static void validateIdentity(JsonNode identity) {
+ for (String name : List.of("bindingId", "operationId")) {
+ String value = requiredText(identity, name);
+ if (!UUID.fromString(value).toString().equals(value)) {
+ throw new IllegalArgumentException("Invalid operation identity");
+ }
+ }
+ if (!identity.path("epoch").isIntegralNumber() || identity.get("epoch").longValue() <= 0
+ || !identity.path("immutableRequestHash").asText().matches("sha256:[0-9a-f]{64}")) {
+ throw new IllegalArgumentException("Invalid operation lineage");
+ }
+ }
+
+ private static String requiredText(JsonNode node, String name) {
+ JsonNode value = node.required(name);
+ if (!value.isTextual() || value.textValue().isEmpty()) {
+ throw new IllegalArgumentException("Missing text field");
+ }
+ return value.textValue();
+ }
+
+ private static final class ProbeFailure extends IOException {
+ private final String code;
+
+ private ProbeFailure(String code) {
+ this.code = code;
+ }
+ }
+}
diff --git a/ambari-agent/src/main/java/org/apache/ambari/tools/zk/ManagedDependencyZk.java b/ambari-agent/src/main/java/org/apache/ambari/tools/zk/ManagedDependencyZk.java
new file mode 100644
index 0000000000..9d7c73d765
--- /dev/null
+++ b/ambari-agent/src/main/java/org/apache/ambari/tools/zk/ManagedDependencyZk.java
@@ -0,0 +1,424 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ambari.tools.zk;
+
+import java.nio.charset.StandardCharsets;
+import java.util.List;
+import java.util.UUID;
+import java.util.regex.Pattern;
+
+import org.apache.commons.cli.CommandLine;
+import org.apache.commons.cli.GnuParser;
+import org.apache.commons.cli.Option;
+import org.apache.commons.cli.Options;
+import org.apache.zookeeper.CreateMode;
+import org.apache.zookeeper.KeeperException;
+import org.apache.zookeeper.ZooDefs;
+import org.apache.zookeeper.ZooKeeper;
+import org.apache.zookeeper.data.ACL;
+import org.apache.zookeeper.data.Id;
+import org.apache.zookeeper.data.Stat;
+
+/** Nonrecursive ZooKeeper operations reserved for managed HBase dependencies. */
+public final class ManagedDependencyZk {
+ static final String PARENT = "/ambari-managed-hbase";
+ private static final int SESSION_TIMEOUT_MILLIS = 5000;
+ private static final int CONNECTION_TIMEOUT_MILLIS = 30000;
+ private static final Pattern OWNER = Pattern.compile("[A-Za-z_][A-Za-z0-9._-]{0,127}");
+ private static final Pattern POSITIVE_INTEGER = Pattern.compile("[1-9][0-9]{0,18}");
+
+ private ManagedDependencyZk() {
+ }
+
+ public static void main(String[] args) throws Exception {
+ // HBase's client classpath can supply Commons CLI 1.2 before this helper JAR.
+ CommandLine command = new GnuParser().parse(options(), args);
+ String result = execute(command);
+ System.out.println(result);
+ }
+
+ static String execute(CommandLine command) throws Exception {
+ Request request = Request.parse(command);
+ ZooKeeper client = ZkConnection.open(
+ request.connectionString(), SESSION_TIMEOUT_MILLIS, CONNECTION_TIMEOUT_MILLIS);
+ try {
+ switch (request.operation()) {
+ case "PREPARE_INSECURE":
+ prepareInsecure(client, request);
+ return "{\"connected\":true,\"namespaceExists\":true,"
+ + "\"parentAclPolicy\":\"INSECURE_PROVIDER_PREPARED\","
+ + "\"subtreeAclPolicy\":\"INSECURE_BINDING_SCOPED\"}";
+ case "VERIFY_INSECURE":
+ verifyInsecure(client, request);
+ return "{\"connected\":true,\"privateZnodeVerified\":true}";
+ case "PREPARE_SECURE_LEDGER":
+ prepareSecureLedger(client, request);
+ return "{\"connected\":true,\"ledgerPrepared\":true}";
+ case "CREATE_SECURE_CONTAINER":
+ boolean created = createSecureContainer(client, request);
+ return created
+ ? "{\"connected\":true,\"containerCreated\":true,\"nodeExists\":false}"
+ : "{\"connected\":true,\"containerCreated\":false,\"nodeExists\":true}";
+ case "VERIFY_SECURE":
+ verifySecure(client, request);
+ return "{\"connected\":true,\"consumerSaslId\":\""
+ + request.consumerSaslId()
+ + "\",\"hbaseZnodeVerified\":true,\"privateContainerVerified\":true,"
+ + "\"siblingAuthorityDenied\":true}";
+ default:
+ throw new IllegalArgumentException("Unsupported managed dependency ZooKeeper operation");
+ }
+ } finally {
+ client.close();
+ }
+ }
+
+ private static void prepareInsecure(ZooKeeper client, Request request)
+ throws KeeperException, InterruptedException {
+ ensureExactNode(client, PARENT, parentMarker(), ZooDefs.Ids.OPEN_ACL_UNSAFE);
+ ensureExactNode(client, request.container(), bindingMarker(request), ZooDefs.Ids.OPEN_ACL_UNSAFE);
+ ensureExactNode(client, request.znode(), new byte[0], ZooDefs.Ids.OPEN_ACL_UNSAFE);
+ }
+
+ private static void verifyInsecure(ZooKeeper client, Request request)
+ throws KeeperException, InterruptedException {
+ requireExactNode(client, PARENT, parentMarker(), ZooDefs.Ids.OPEN_ACL_UNSAFE);
+ requireExactNode(client, request.container(), bindingMarker(request), ZooDefs.Ids.OPEN_ACL_UNSAFE);
+ requireExactNode(client, request.znode(), new byte[0], ZooDefs.Ids.OPEN_ACL_UNSAFE);
+ String probe = request.znode() + "/.ambari-managed-probe-" + request.probeId();
+ boolean created = false;
+ try {
+ client.create(probe, request.probeId().toString().getBytes(StandardCharsets.UTF_8),
+ ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL);
+ created = true;
+ byte[] observed = client.getData(probe, false, new Stat());
+ if (!java.util.Arrays.equals(observed,
+ request.probeId().toString().getBytes(StandardCharsets.UTF_8))) {
+ throw new IllegalStateException("Managed dependency ZooKeeper probe data mismatch");
+ }
+ } finally {
+ if (created) {
+ client.delete(probe, -1);
+ }
+ }
+ }
+
+ private static void prepareSecureLedger(ZooKeeper client, Request request)
+ throws KeeperException, InterruptedException {
+ List<ACL> providerAcl = privateAcl(request.providerSaslId());
+ ensureExactNode(client, PARENT, parentMarker(), providerAcl);
+ ensureExactNode(client, PARENT + "/.bindings", ledgerParentMarker(), providerAcl);
+ ensureExactNode(client, request.ledger(), ledgerMarker(request), providerAcl);
+ }
+
+ private static boolean createSecureContainer(ZooKeeper client, Request request)
+ throws KeeperException, InterruptedException {
+ try {
+ client.create(request.container(), bindingMarker(request),
+ privateAcl(request.consumerSaslId()), CreateMode.PERSISTENT);
+ return true;
+ } catch (KeeperException.NodeExistsException expected) {
+ return false;
+ }
+ }
+
+ private static void verifySecure(ZooKeeper client, Request request)
+ throws KeeperException, InterruptedException {
+ List<ACL> consumerAcl = privateAcl(request.consumerSaslId());
+ Stat containerStat = requireExactNode(
+ client, request.container(), bindingMarker(request), consumerAcl);
+ ensureSupportedHBaseBaseNode(client, request);
+ verifyProbe(client, request, consumerAcl);
+ requireSiblingAuthorityDenied(client, request, consumerAcl, containerStat.getVersion());
+ }
+
+ private static void verifyProbe(ZooKeeper client, Request request, List<ACL> acl)
+ throws KeeperException, InterruptedException {
+ String probe = request.znode() + "/.ambari-managed-probe-" + request.probeId();
+ boolean created = false;
+ try {
+ client.create(probe, request.probeId().toString().getBytes(StandardCharsets.UTF_8),
+ acl, CreateMode.EPHEMERAL);
+ created = true;
+ byte[] observed = client.getData(probe, false, new Stat());
+ if (!java.util.Arrays.equals(observed,
+ request.probeId().toString().getBytes(StandardCharsets.UTF_8))) {
+ throw new IllegalStateException("Managed dependency ZooKeeper probe data mismatch");
+ }
+ } finally {
+ if (created) {
+ client.delete(probe, -1);
+ }
+ }
+ }
+
+ private static void requireSiblingAuthorityDenied(ZooKeeper client, Request request,
+ List<ACL> acl, int containerVersion) throws KeeperException, InterruptedException {
+ String forbidden = PARENT + "/.ambari-managed-denied-" + request.probeId();
+ boolean created = false;
+ try {
+ client.create(forbidden, new byte[0], acl, CreateMode.EPHEMERAL);
+ created = true;
+ } catch (KeeperException.NoAuthException expected) {
+ // The provider-only parent must deny consumer sibling creation.
+ } finally {
+ if (created) {
+ try {
+ client.delete(forbidden, -1);
+ } catch (KeeperException.NoNodeException ignored) {
+ // The helper cleans only its own UUID probe; session expiry may race the delete.
+ }
+ }
+ }
+ if (created) {
+ throw new IllegalStateException(
+ "Managed dependency consumer unexpectedly has sibling creation authority");
+ }
+ try {
+ client.delete(request.container(), Math.addExact(containerVersion, 1));
+ throw new IllegalStateException(
+ "Managed dependency consumer unexpectedly deleted its binding container");
+ } catch (KeeperException.NoAuthException expected) {
+ // DELETE is also checked on the provider-only parent ACL.
+ } catch (KeeperException.BadVersionException | KeeperException.NotEmptyException authorized) {
+ throw new IllegalStateException(
+ "Managed dependency consumer unexpectedly has sibling deletion authority", authorized);
+ }
+ }
+
+ private static void ensureExactNode(ZooKeeper client, String path, byte[] data,
+ List<ACL> acls) throws KeeperException, InterruptedException {
+ if (client.exists(path, false) == null) {
+ try {
+ client.create(path, data, acls, CreateMode.PERSISTENT);
+ } catch (KeeperException.NodeExistsException ignored) {
+ // A concurrent exact retry may have created it; validation below decides.
+ }
+ }
+ requireExactNode(client, path, data, acls);
+ }
+
+ private static void ensureSupportedHBaseBaseNode(ZooKeeper client, Request request)
+ throws KeeperException, InterruptedException {
+ List<ACL> privateAcl = privateAcl(request.consumerSaslId());
+ if (client.exists(request.znode(), false) == null) {
+ try {
+ client.create(request.znode(), new byte[0], privateAcl, CreateMode.PERSISTENT);
+ } catch (KeeperException.NodeExistsException ignored) {
+ // A concurrent HBase create is accepted only if its final policy is canonical.
+ }
+ }
+ Stat dataStat = new Stat();
+ byte[] actualData = client.getData(request.znode(), false, dataStat);
+ Stat aclStat = new Stat();
+ List<ACL> actualAcls = client.getACL(request.znode(), aclStat);
+ if (actualData.length != 0 || dataStat.getEphemeralOwner() != 0
+ || dataStat.getVersion() != 0
+ || !isSupportedHBaseBaseAcl(
+ actualAcls, request.consumerSaslId(), aclStat.getAversion())) {
+ throw new IllegalStateException(
+ "Managed dependency HBase base znode metadata conflicts");
+ }
+ }
+
+ static boolean isSupportedHBaseBaseAcl(List<ACL> actual, String consumerSaslId,
+ int aclVersion) {
+ List<ACL> privateAcl = privateAcl(consumerSaslId);
+ if (aclVersion == 0 && sameAclSet(actual, privateAcl)) {
+ return true;
+ }
+ List<ACL> hbaseNativeAcl = List.of(
+ new ACL(ZooDefs.Perms.ALL, new Id("sasl", consumerSaslId)),
+ new ACL(ZooDefs.Perms.READ, ZooDefs.Ids.ANYONE_ID_UNSAFE));
+ return aclVersion >= 0 && sameAclSet(actual, hbaseNativeAcl);
+ }
+
+ private static boolean sameAclSet(List<ACL> actual, List<ACL> expected) {
+ return actual.size() == expected.size()
+ && actual.containsAll(expected)
+ && expected.containsAll(actual);
+ }
+
+ private static Stat requireExactNode(ZooKeeper client, String path, byte[] expectedData,
+ List<ACL> expectedAcls) throws KeeperException, InterruptedException {
+ Stat dataStat = new Stat();
+ byte[] actualData = client.getData(path, false, dataStat);
+ Stat aclStat = new Stat();
+ List<ACL> actualAcls = client.getACL(path, aclStat);
+ if (!java.util.Arrays.equals(expectedData, actualData)
+ || !expectedAcls.equals(actualAcls)
+ || dataStat.getEphemeralOwner() != 0
+ || dataStat.getVersion() != 0
+ || aclStat.getAversion() != 0) {
+ throw new IllegalStateException("Managed dependency ZooKeeper node metadata conflicts");
+ }
+ return dataStat;
+ }
+
+ private static byte[] parentMarker() {
+ return "ambari-managed-hbase-parent:v1".getBytes(StandardCharsets.UTF_8);
+ }
+
+ private static byte[] ledgerParentMarker() {
+ return "ambari-managed-hbase-ledger:v1".getBytes(StandardCharsets.UTF_8);
+ }
+
+ private static byte[] ledgerMarker(Request request) {
+ String marker = "ambari-managed-hbase-binding-ledger:v1\n"
+ + "bindingId=" + request.bindingId() + "\n"
+ + "providerClusterId=" + request.providerClusterId() + "\n"
+ + "container=" + request.container() + "\n"
+ + "hbaseZnode=" + request.znode() + "\n"
+ + "consumerSaslId=" + request.consumerSaslId() + "\n";
+ return marker.getBytes(StandardCharsets.UTF_8);
+ }
+
+ private static List<ACL> privateAcl(String saslId) {
+ return List.of(new ACL(ZooDefs.Perms.ALL, new Id("sasl", saslId)));
+ }
+
+ private static byte[] bindingMarker(Request request) {
+ String marker = "ambari-managed-hbase-binding:v1\n"
+ + "bindingId=" + request.bindingId() + "\n"
+ + "providerClusterId=" + request.providerClusterId() + "\n"
+ + "ownerUser=" + request.ownerUser() + "\n";
+ return marker.getBytes(StandardCharsets.UTF_8);
+ }
+
+ private static Options options() {
+ return new Options()
+ .addOption(required("o", "operation", "managed dependency operation"))
+ .addOption(required("c", "connection-string", "ZooKeeper connection string"))
+ .addOption(required("p", "parent", "reserved parent znode"))
+ .addOption(required("n", "container", "binding handoff container"))
+ .addOption(required("z", "znode", "binding znode"))
+ .addOption(required("b", "binding-id", "binding UUID"))
+ .addOption(required("i", "provider-cluster-id", "provider cluster ID"))
+ .addOption(required("u", "owner-user", "consumer short user"))
+ .addOption(optional("l", "ledger", "provider-only binding ledger znode"))
+ .addOption(optional("a", "provider-sasl-id", "provider ZooKeeper SASL authorization ID"))
+ .addOption(optional("s", "consumer-sasl-id", "consumer ZooKeeper SASL authorization ID"))
+ .addOption(optional("r", "probe-id", "VERIFY operation UUID"));
+ }
+
+ private static Option required(String shortName, String longName, String description) {
+ Option option = optional(shortName, longName, description);
+ option.setRequired(true);
+ return option;
+ }
+
+ private static Option optional(String shortName, String longName, String description) {
+ Option option = new Option(shortName, longName, true, description);
+ option.setArgName(longName);
+ return option;
+ }
+
+ record Request(String operation, String connectionString, String container, String znode,
+ String ledger, UUID bindingId, long providerClusterId, String ownerUser,
+ String providerSaslId, String consumerSaslId, UUID probeId) {
+ static Request parse(CommandLine command) {
+ String operation = command.getOptionValue("operation");
+ if (!List.of("PREPARE_INSECURE", "VERIFY_INSECURE", "PREPARE_SECURE_LEDGER",
+ "CREATE_SECURE_CONTAINER", "VERIFY_SECURE").contains(operation)) {
+ throw new IllegalArgumentException("Unsupported managed dependency ZooKeeper operation");
+ }
+ String connection = bounded(command.getOptionValue("connection-string"), 4096,
+ "connection-string");
+ if (!PARENT.equals(command.getOptionValue("parent"))) {
+ throw new IllegalArgumentException("Managed dependency ZooKeeper parent is not reserved");
+ }
+ UUID bindingId = canonicalUuid(command.getOptionValue("binding-id"), "binding-id");
+ String container = command.getOptionValue("container");
+ if (!(PARENT + "/" + bindingId).equals(container)) {
+ throw new IllegalArgumentException("Managed dependency ZooKeeper container is not derived from binding-id");
+ }
+ String znode = command.getOptionValue("znode");
+ if (!(container + "/hbase").equals(znode)) {
+ throw new IllegalArgumentException("Managed dependency ZooKeeper znode is not derived from binding-id");
+ }
+ String clusterId = command.getOptionValue("provider-cluster-id");
+ if (!POSITIVE_INTEGER.matcher(clusterId).matches()) {
+ throw new IllegalArgumentException("provider-cluster-id must be a positive integer");
+ }
+ long providerClusterId;
+ try {
+ providerClusterId = Long.parseLong(clusterId);
+ } catch (NumberFormatException e) {
+ throw new IllegalArgumentException("provider-cluster-id exceeds the supported range", e);
+ }
+ String owner = command.getOptionValue("owner-user");
+ if (owner == null || !OWNER.matcher(owner).matches()) {
+ throw new IllegalArgumentException("owner-user is invalid");
+ }
+ UUID probeId = null;
+ boolean secureProvider = List.of("PREPARE_SECURE_LEDGER", "CREATE_SECURE_CONTAINER")
+ .contains(operation);
+ boolean secureConsumer = "VERIFY_SECURE".equals(operation);
+ boolean verify = "VERIFY_INSECURE".equals(operation) || secureConsumer;
+ String ledger = command.getOptionValue("ledger");
+ String providerSaslId = command.getOptionValue("provider-sasl-id");
+ String consumerSaslId = command.getOptionValue("consumer-sasl-id");
+ if (secureProvider) {
+ if (!(PARENT + "/.bindings/" + bindingId).equals(ledger)
+ || !validIdentity(providerSaslId) || !validIdentity(consumerSaslId)
+ || !owner.equals(consumerSaslId)) {
+ throw new IllegalArgumentException("Secure provider handoff arguments are invalid");
+ }
+ } else if (secureConsumer) {
+ if (ledger != null || providerSaslId != null || !validIdentity(consumerSaslId)
+ || !owner.equals(consumerSaslId)) {
+ throw new IllegalArgumentException("Secure consumer verification arguments are invalid");
+ }
+ } else if (ledger != null || providerSaslId != null || consumerSaslId != null) {
+ throw new IllegalArgumentException("Secure arguments are allowed only for secure operations");
+ }
+ if (verify) {
+ probeId = canonicalUuid(command.getOptionValue("probe-id"), "probe-id");
+ } else if (command.hasOption("probe-id")) {
+ throw new IllegalArgumentException("probe-id is allowed only for verification");
+ }
+ return new Request(operation, connection, container, znode, ledger, bindingId,
+ providerClusterId, owner, providerSaslId, consumerSaslId, probeId);
+ }
+
+ private static boolean validIdentity(String value) {
+ return value != null && OWNER.matcher(value).matches();
+ }
+
+ private static String bounded(String value, int maximum, String field) {
+ if (value == null || value.isBlank() || value.length() > maximum
+ || value.chars().anyMatch(character -> Character.isISOControl(character)
+ || Character.isWhitespace(character))) {
+ throw new IllegalArgumentException(field + " is missing, unsafe, or too long");
+ }
+ return value;
+ }
+
+ private static UUID canonicalUuid(String value, String field) {
+ try {
+ UUID parsed = UUID.fromString(value);
+ if (!parsed.toString().equals(value)) {
+ throw new IllegalArgumentException(field + " must be a canonical UUID");
+ }
+ return parsed;
+ } catch (RuntimeException e) {
+ throw new IllegalArgumentException(field + " must be a canonical UUID", e);
+ }
+ }
+ }
+}
diff --git a/ambari-agent/src/main/java/org/apache/ambari/tools/zk/ZkConnection.java b/ambari-agent/src/main/java/org/apache/ambari/tools/zk/ZkConnection.java
index ae74b4bb3c..c9458d6dd4 100644
--- a/ambari-agent/src/main/java/org/apache/ambari/tools/zk/ZkConnection.java
+++ b/ambari-agent/src/main/java/org/apache/ambari/tools/zk/ZkConnection.java
@@ -33,14 +33,52 @@ import java.util.concurrent.CountDownLatch;
*/
public class ZkConnection {
+ interface PendingConnection {
+ ZooKeeper client();
+
+ boolean isConnected();
+
+ void close() throws InterruptedException;
+ }
+
+ @FunctionalInterface
+ interface ConnectionFactory {
+ PendingConnection create(String serverAddress, int sessionTimeoutMillis, Watcher watcher)
+ throws IOException;
+ }
+
/**
* Opens a connection to zookeeper and waits until the connection established
*/
public static ZooKeeper open(String serverAddress, int sessionTimeoutMillis, int connectionTimeoutMillis)
throws IOException, InterruptedException, IllegalStateException
{
+ return open(serverAddress, sessionTimeoutMillis, connectionTimeoutMillis,
+ (address, timeout, watcher) -> {
+ ZooKeeper client = new ZooKeeper(address, timeout, watcher);
+ return new PendingConnection() {
+ @Override
+ public ZooKeeper client() {
+ return client;
+ }
+
+ @Override
+ public boolean isConnected() {
+ return client.getState().isConnected();
+ }
+
+ @Override
+ public void close() throws InterruptedException {
+ client.close();
+ }
+ };
+ });
+ }
+
+ static ZooKeeper open(String serverAddress, int sessionTimeoutMillis, int connectionTimeoutMillis,
+ ConnectionFactory factory) throws IOException, InterruptedException, IllegalStateException {
final CountDownLatch connSignal = new CountDownLatch(1);
- ZooKeeper zooKeeper = new ZooKeeper(serverAddress, sessionTimeoutMillis, new Watcher() {
+ PendingConnection pending = factory.create(serverAddress, sessionTimeoutMillis, new Watcher() {
@Override
public void process(WatchedEvent event) {
if (event.getState() == SyncConnected) {
@@ -48,7 +86,22 @@ public class ZkConnection {
}
}
});
- connSignal.await(connectionTimeoutMillis, MILLISECONDS);
- return zooKeeper;
+ boolean connected = false;
+ try {
+ if (!connSignal.await(connectionTimeoutMillis, MILLISECONDS)
+ || !pending.isConnected()) {
+ throw new IllegalStateException("ZooKeeper connection timed out before SyncConnected");
+ }
+ connected = true;
+ return pending.client();
+ } finally {
+ if (!connected) {
+ try {
+ pending.close();
+ } catch (InterruptedException closeInterrupted) {
+ Thread.currentThread().interrupt();
+ }
+ }
+ }
}
}
diff --git a/ambari-agent/src/main/java/org/apache/ambari/tools/zk/ZkMigrator.java b/ambari-agent/src/main/java/org/apache/ambari/tools/zk/ZkMigrator.java
index e4b33eba0c..ce7e575b01 100644
--- a/ambari-agent/src/main/java/org/apache/ambari/tools/zk/ZkMigrator.java
+++ b/ambari-agent/src/main/java/org/apache/ambari/tools/zk/ZkMigrator.java
@@ -19,6 +19,7 @@
package org.apache.ambari.tools.zk;
import java.io.IOException;
+import java.util.Arrays;
import org.apache.commons.cli.CommandLine;
import org.apache.commons.cli.DefaultParser;
@@ -39,6 +40,10 @@ public class ZkMigrator {
private static final int CONNECTION_TIMEOUT_MILLIS = 30000;
public static void main(String[] args) throws Exception {
+ if (args.length > 0 && "--managed-dependency".equals(args[0])) {
+ ManagedDependencyZk.main(Arrays.copyOfRange(args, 1, args.length));
+ return;
+ }
CommandLine cli = new DefaultParser().parse(options(), args);
if (cli.hasOption("connection-string") && cli.hasOption("znode")) {
if (cli.hasOption("acl") && !cli.hasOption("delete")) {
diff --git a/ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py b/ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py
index 6ba4c1a9fe..5329da2268 100644
--- a/ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py
+++ b/ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py
@@ -721,6 +721,7 @@ class CustomServiceOrchestrator(object):
del command_dict["clusterHostInfo"]
command = Utils.update_nested(Utils.get_mutable_copy(command_dict), command_header)
+ self._apply_configuration_type_overrides(command, command_header)
# topology needs to be decompressed if and only if it originates from command header
if "clusterHostInfo" in command_header and command_header["clusterHostInfo"]:
@@ -730,6 +731,31 @@ class CustomServiceOrchestrator(object):
return command
+ @staticmethod
+ def _apply_configuration_type_overrides(command, command_header):
+ override_types = command_header.get("configurationTypeOverrides", [])
+ if not override_types:
+ return
+ command_params = command_header.get("commandParams", {})
+ if (
+ command_header.get("serviceName") != "HBASE"
+ or not command_params.get("managed_dependency_commands")
+ ):
+ raise AgentException(
+ "Configuration type replacement is reserved for managed HBase dependency commands"
+ )
+ allowed = {"core-site", "hdfs-site"}
+ if not isinstance(override_types, list) or not set(override_types).issubset(allowed):
+ raise AgentException("Managed HBase configuration type replacement is invalid")
+ command_configurations = command_header.get("configurations", {})
+ for config_type in override_types:
+ replacement = command_configurations.get(config_type)
+ if not isinstance(replacement, dict):
+ raise AgentException(
+ f"Managed HBase replacement for {config_type} is missing"
+ )
+ command["configurations"][config_type] = Utils.get_mutable_copy(replacement)
+
def requestComponentStatus(self, command_header, command_name="STATUS"):
"""
Component status is determined by exit code, returned by runCommand().
diff --git a/ambari-agent/src/test/java/org/apache/ambari/tools/hadoop/ManagedDependencyClientTest.java b/ambari-agent/src/test/java/org/apache/ambari/tools/hadoop/ManagedDependencyClientTest.java
new file mode 100644
index 0000000000..80c514d70b
--- /dev/null
+++ b/ambari-agent/src/test/java/org/apache/ambari/tools/hadoop/ManagedDependencyClientTest.java
@@ -0,0 +1,126 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ambari.tools.hadoop;
+
+import static org.junit.Assert.assertArrayEquals;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertTrue;
+import static org.junit.Assert.fail;
+
+import java.io.File;
+import java.io.IOException;
+import java.net.URI;
+import java.nio.charset.StandardCharsets;
+import java.util.Map;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FSDataOutputStream;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.fs.RawLocalFileSystem;
+import org.junit.Before;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.rules.TemporaryFolder;
+
+public class ManagedDependencyClientTest {
+ private static final ObjectMapper JSON = new ObjectMapper();
+ private static final String BINDING = "59646bd5-39eb-414e-a07e-f5469d369687";
+ private static final String OPERATION = "43d3dad3-d0b0-4ba2-a9ae-2ccf7ab758f2";
+ private static final String ROOT = "/apps/ambari-managed/hbase/" + BINDING + "/root";
+ @Rule
+ public TemporaryFolder directory = new TemporaryFolder();
+ private RawLocalFileSystem fs;
+ private JsonNode identity;
+
+ @Before
+ public void setup() throws Exception {
+ File base = directory.getRoot();
+ fs = new RawLocalFileSystem() {
+ @Override
+ public File pathToFile(Path path) {
+ return new File(base, path.toUri().getPath().replaceFirst("^/", ""));
+ }
+ };
+ fs.initialize(URI.create("file:///"), new Configuration());
+ identity = JSON.valueToTree(Map.of("bindingId", BINDING, "operationId", OPERATION,
+ "epoch", 1, "immutableRequestHash", "sha256:" + "a".repeat(64)));
+ fs.mkdirs(new Path(ROOT));
+ }
+
+ @Test
+ public void observesRealMetadataAndNonemptyRoundTripWithoutLeavingProbe() throws Exception {
+ Map<String, Object> status = execute("STAT", ROOT);
+ assertEquals(true, status.get("exists"));
+ assertEquals("DIRECTORY", status.get("type"));
+ assertEquals(System.getProperty("user.name"), status.get("owner"));
+ assertTrue(status.get("mode") instanceof Integer);
+ Map<String, Object> result = execute("PROBE", ROOT);
+ assertEquals(Map.of("readWriteVerified", true, "cleaned", true,
+ "bytes", ManagedDependencyClient.PROBE_BYTES), result);
+ assertFalse(fs.exists(new Path(ROOT, ".ambari-managed-probe-" + OPERATION)));
+ assertEquals(result, execute("PROBE", ROOT));
+ }
+
+ @Test
+ public void absentMetadataIsDistinctFromReadOrAuthorizationFailure() throws Exception {
+ assertEquals(Map.of("exists", false), execute("STAT", ROOT + "/absent"));
+ try {
+ execute("COUNT", ROOT + "/absent");
+ fail("An observation failure cannot become an empty directory");
+ } catch (IOException expected) {
+ assertFalse(fs.exists(new Path(ROOT + "/absent")));
+ }
+ }
+
+ @Test
+ public void mismatchedExistingProbeIsPreservedAndCannotBecomeSuccess() throws Exception {
+ Path probe = new Path(ROOT, ".ambari-managed-probe-" + OPERATION);
+ byte[] foreign = "foreign data".getBytes(StandardCharsets.UTF_8);
+ try (FSDataOutputStream output = fs.create(probe, false)) {
+ output.write(foreign);
+ }
+ try {
+ execute("PROBE", ROOT);
+ fail("Unexpected data must not be accepted or removed");
+ } catch (IOException expected) {
+ try (var input = fs.open(probe)) {
+ assertArrayEquals(foreign, input.readAllBytes());
+ }
+ }
+ }
+
+ @Test
+ public void foreignBindingMutationIsRejectedBeforeAnyWrite() throws Exception {
+ String foreign = ROOT.replace(BINDING, OPERATION);
+ try {
+ ManagedDependencyClient.execute(fs, "MKDIR", identity,
+ JSON.valueToTree(Map.of("path", foreign, "mode", 0700)));
+ fail("Foreign binding mutation must be denied");
+ } catch (IllegalArgumentException expected) {
+ assertFalse(fs.exists(new Path(foreign)));
+ }
+ }
+
+ private Map<String, Object> execute(String operation, String path) throws Exception {
+ return ManagedDependencyClient.execute(fs, operation, identity,
+ JSON.valueToTree(Map.of("path", path)));
+ }
+}
diff --git a/ambari-agent/src/test/java/org/apache/ambari/tools/zk/ManagedDependencyZkTest.java b/ambari-agent/src/test/java/org/apache/ambari/tools/zk/ManagedDependencyZkTest.java
new file mode 100644
index 0000000000..676a54887b
--- /dev/null
+++ b/ambari-agent/src/test/java/org/apache/ambari/tools/zk/ManagedDependencyZkTest.java
@@ -0,0 +1,153 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ambari.tools.zk;
+
+import static org.junit.Assert.assertArrayEquals;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertNotNull;
+import static org.junit.Assert.assertNull;
+import static org.junit.Assert.assertTrue;
+import static org.junit.Assert.fail;
+
+import java.net.ServerSocket;
+import java.nio.charset.StandardCharsets;
+import java.util.List;
+
+import org.apache.curator.framework.CuratorFramework;
+import org.apache.curator.framework.CuratorFrameworkFactory;
+import org.apache.curator.retry.RetryOneTime;
+import org.apache.curator.test.TestingServer;
+import org.apache.zookeeper.ZooDefs;
+import org.apache.zookeeper.data.ACL;
+import org.apache.zookeeper.data.Id;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+
+@Category({category.SlowTest.class})
+public class ManagedDependencyZkTest {
+ private static final String BINDING_ID = "59646bd5-39eb-414e-a07e-f5469d369687";
+ private static final String CONTAINER = ManagedDependencyZk.PARENT + "/" + BINDING_ID;
+ private static final String ZNODE = CONTAINER + "/hbase";
+ private static final String PROBE_ID = "43d3dad3-d0b0-4ba2-a9ae-2ccf7ab758f2";
+
+ private CuratorFramework client;
+ private TestingServer server;
+
+ @Before
+ public void startZooKeeper() throws Exception {
+ server = new TestingServer(freePort());
+ server.start();
+ client = CuratorFrameworkFactory.newClient(server.getConnectString(), new RetryOneTime(200));
+ client.start();
+ }
+
+ @After
+ public void stopZooKeeper() throws Exception {
+ client.close();
+ server.stop();
+ }
+
+ @Test
+ public void prepareIsExactNonrecursiveAndRetryPreservesProviderData() throws Exception {
+ ManagedDependencyZk.main(arguments("PREPARE_INSECURE", null));
+ byte[] parent = client.getData().forPath(ManagedDependencyZk.PARENT);
+ byte[] container = client.getData().forPath(CONTAINER);
+ client.create().forPath(ZNODE + "/provider-data", "keep".getBytes(StandardCharsets.UTF_8));
+
+ ManagedDependencyZk.main(arguments("PREPARE_INSECURE", null));
+
+ assertArrayEquals("ambari-managed-hbase-parent:v1".getBytes(StandardCharsets.UTF_8), parent);
+ assertArrayEquals(container, client.getData().forPath(CONTAINER));
+ assertEquals(ZooDefs.Ids.OPEN_ACL_UNSAFE, client.getACL().forPath(ManagedDependencyZk.PARENT));
+ assertEquals(ZooDefs.Ids.OPEN_ACL_UNSAFE, client.getACL().forPath(CONTAINER));
+ assertEquals(ZooDefs.Ids.OPEN_ACL_UNSAFE, client.getACL().forPath(ZNODE));
+ assertNotNull(client.checkExists().forPath(ZNODE + "/provider-data"));
+ }
+
+ @Test
+ public void existingUnexpectedNamespaceFailsWithoutRepairOrDelete() throws Exception {
+ client.create().forPath(ManagedDependencyZk.PARENT, "foreign".getBytes(StandardCharsets.UTF_8));
+
+ try {
+ ManagedDependencyZk.main(arguments("PREPARE_INSECURE", null));
+ fail("expected unexpected parent metadata to fail closed");
+ } catch (IllegalStateException expected) {
+ // Expected.
+ }
+
+ assertArrayEquals("foreign".getBytes(StandardCharsets.UTF_8),
+ client.getData().forPath(ManagedDependencyZk.PARENT));
+ assertNull(client.checkExists().forPath(ZNODE));
+ }
+
+ @Test
+ public void consumerVerificationUsesAndCleansOnlyItsEphemeralProbe() throws Exception {
+ ManagedDependencyZk.main(arguments("PREPARE_INSECURE", null));
+ client.create().forPath(ZNODE + "/provider-data", "keep".getBytes(StandardCharsets.UTF_8));
+
+ ManagedDependencyZk.main(arguments("VERIFY_INSECURE", PROBE_ID));
+
+ assertNull(client.checkExists().forPath(ZNODE + "/.ambari-managed-probe-" + PROBE_ID));
+ assertNotNull(client.checkExists().forPath(ZNODE + "/provider-data"));
+ }
+
+ @Test
+ public void secureBaseAcceptsOnlyPrivateOrHBaseNativeCanonicalAcl() {
+ ACL consumer = new ACL(ZooDefs.Perms.ALL, new Id("sasl", "hbase_mc_c1"));
+ ACL worldRead = new ACL(ZooDefs.Perms.READ, ZooDefs.Ids.ANYONE_ID_UNSAFE);
+
+ assertTrue(ManagedDependencyZk.isSupportedHBaseBaseAcl(
+ List.of(consumer), "hbase_mc_c1", 0));
+ assertTrue(ManagedDependencyZk.isSupportedHBaseBaseAcl(
+ List.of(worldRead, consumer), "hbase_mc_c1", 1));
+ assertFalse(ManagedDependencyZk.isSupportedHBaseBaseAcl(
+ List.of(consumer), "hbase_mc_c1", 1));
+ assertFalse(ManagedDependencyZk.isSupportedHBaseBaseAcl(
+ List.of(consumer, new ACL(ZooDefs.Perms.ALL, ZooDefs.Ids.ANYONE_ID_UNSAFE)),
+ "hbase_mc_c1", 1));
+ assertFalse(ManagedDependencyZk.isSupportedHBaseBaseAcl(
+ List.of(new ACL(ZooDefs.Perms.ALL, new Id("sasl", "foreign")), worldRead),
+ "hbase_mc_c1", 1));
+ }
+
+ private String[] arguments(String operation, String probeId) {
+ java.util.List<String> values = new java.util.ArrayList<>(java.util.List.of(
+ "--operation", operation,
+ "--connection-string", server.getConnectString(),
+ "--parent", ManagedDependencyZk.PARENT,
+ "--container", CONTAINER,
+ "--znode", ZNODE,
+ "--binding-id", BINDING_ID,
+ "--provider-cluster-id", "2",
+ "--owner-user", "hbase_mc_c1"));
+ if (probeId != null) {
+ values.add("--probe-id");
+ values.add(probeId);
+ }
+ return values.toArray(String[]::new);
+ }
+
+ private static int freePort() throws Exception {
+ try (ServerSocket socket = new ServerSocket(0)) {
+ return socket.getLocalPort();
+ }
+ }
+}
diff --git a/ambari-agent/src/test/java/org/apache/ambari/tools/zk/ZkConnectionTest.java b/ambari-agent/src/test/java/org/apache/ambari/tools/zk/ZkConnectionTest.java
new file mode 100644
index 0000000000..422cf6f3dc
--- /dev/null
+++ b/ambari-agent/src/test/java/org/apache/ambari/tools/zk/ZkConnectionTest.java
@@ -0,0 +1,91 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ambari.tools.zk;
+
+import static org.junit.Assert.assertTrue;
+import static org.junit.Assert.fail;
+
+import java.util.concurrent.atomic.AtomicBoolean;
+
+import org.junit.Test;
+
+public class ZkConnectionTest {
+
+ @Test
+ public void interruptedOpenClosesThePendingClient() throws Exception {
+ AtomicBoolean closed = new AtomicBoolean();
+ ZkConnection.ConnectionFactory factory = (address, timeout, watcher) ->
+ new ZkConnection.PendingConnection() {
+ @Override
+ public org.apache.zookeeper.ZooKeeper client() {
+ return null;
+ }
+
+ @Override
+ public boolean isConnected() {
+ return false;
+ }
+
+ @Override
+ public void close() {
+ closed.set(true);
+ }
+ };
+
+ Thread.currentThread().interrupt();
+ try {
+ ZkConnection.open("unreachable.example.test:2181", 5000, 30000, factory);
+ fail("expected interrupted wait");
+ } catch (InterruptedException expected) {
+ // Expected.
+ }
+
+ assertTrue("pending ZooKeeper client was not closed", closed.get());
+ }
+
+ @Test
+ public void timedOutOpenClosesThePendingClient() throws Exception {
+ AtomicBoolean closed = new AtomicBoolean();
+ ZkConnection.ConnectionFactory factory = (address, timeout, watcher) ->
+ new ZkConnection.PendingConnection() {
+ @Override
+ public org.apache.zookeeper.ZooKeeper client() {
+ return null;
+ }
+
+ @Override
+ public boolean isConnected() {
+ return false;
+ }
+
+ @Override
+ public void close() {
+ closed.set(true);
+ }
+ };
+
+ try {
+ ZkConnection.open("unreachable.example.test:2181", 5000, 0, factory);
+ fail("expected connection timeout");
+ } catch (IllegalStateException expected) {
+ // Expected.
+ }
+
+ assertTrue("pending ZooKeeper client was not closed", closed.get());
+ }
+}
diff --git a/ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py b/ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py
index aeacb3d4f2..ff0613ad41 100644
--- a/ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py
+++ b/ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py
@@ -103,6 +103,65 @@ class TestCustomServiceOrchestrator(TestCase):
self.test_directory.cleanup()
sys.stdout = sys.__stdout__
+ def test_managed_hbase_configuration_replaces_only_command_scoped_types(self):
+ cached = {
+ "configurations": {
+ "core-site": {"fs.defaultFS": "hdfs://local", "local.only": "retained"},
+ "hdfs-site": {"dfs.nameservices": "local-ha"},
+ "hbase-site": {"unrelated": "retained"},
+ }
+ }
+ orchestrator = CustomServiceOrchestrator.__new__(CustomServiceOrchestrator)
+ orchestrator.configuration_builder = MagicMock()
+ orchestrator.configuration_builder.get_configuration.return_value = cached
+ managed_header = {
+ "clusterId": 11,
+ "serviceName": "HBASE",
+ "role": "HBASE_MASTER",
+ "configurationTypeOverrides": ["core-site", "hdfs-site"],
+ "commandParams": {"managed_dependency_commands": "persisted-bundle"},
+ "configurations": {
+ "core-site": {"fs.defaultFS": "hdfs://provider"},
+ "hdfs-site": {},
+ },
+ }
+
+ managed = orchestrator.generate_command(managed_header)
+
+ self.assertEqual(
+ {"fs.defaultFS": "hdfs://provider"}, managed["configurations"]["core-site"]
+ )
+ self.assertEqual(
+ {},
+ managed["configurations"]["hdfs-site"],
+ )
+ self.assertEqual(
+ {"unrelated": "retained"}, managed["configurations"]["hbase-site"]
+ )
+ self.assertEqual("hdfs://local", cached["configurations"]["core-site"]["fs.defaultFS"])
+
+ ordinary = orchestrator.generate_command(
+ {"clusterId": 11, "serviceName": "HIVE", "role": "HIVE_SERVER"}
+ )
+ self.assertEqual(
+ {"fs.defaultFS": "hdfs://local", "local.only": "retained"},
+ ordinary["configurations"]["core-site"],
+ )
+ self.assertEqual(
+ {"dfs.nameservices": "local-ha"}, ordinary["configurations"]["hdfs-site"]
+ )
+
+ def test_configuration_type_replacement_rejects_unmanaged_command(self):
+ with self.assertRaisesRegex(AgentException, "reserved for managed HBase"):
+ CustomServiceOrchestrator._apply_configuration_type_overrides(
+ {"configurations": {"core-site": {"fs.defaultFS": "hdfs://local"}}},
+ {
+ "serviceName": "HBASE",
+ "configurationTypeOverrides": ["core-site"],
+ "configurations": {"core-site": {"fs.defaultFS": "hdfs://provider"}},
+ },
+ )
+
@patch.object(OSCheck, "os_distribution", new=MagicMock(return_value=os_distro_value))
@patch("ambari_agent.hostname.public_hostname")
@patch.object(FileCache, "__init__")
diff --git a/ambari-agent/src/test/python/resource_management/TestManagedDependency.py b/ambari-agent/src/test/python/resource_management/TestManagedDependency.py
new file mode 100644
index 0000000000..43c6930903
--- /dev/null
+++ b/ambari-agent/src/test/python/resource_management/TestManagedDependency.py
@@ -0,0 +1,855 @@
+#!/usr/bin/env python3
+"""
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+"""
+
+import json
+import multiprocessing
+import os
+import shutil
+import stat
+import tempfile
+import unittest
+from copy import deepcopy
+from unittest.mock import patch
+
+from resource_management.libraries.functions import managed_dependency as dependency_module
+from resource_management.libraries.functions.managed_dependency import (
+ HDFS_BASE,
+ INITIALIZE_BINDING_JOURNAL,
+ INVALIDATE_BINDING_EPOCH,
+ PREPARE_BINDING_JOURNAL,
+ PROVISION_HDFS_NAMESPACE,
+ PROVISION_ZOOKEEPER_NAMESPACE,
+ PREPARE_HDFS_CONSUMER,
+ PREPARE_ZOOKEEPER_CONSUMER,
+ VERIFY_HDFS_CONSUMER,
+ ManagedDependencyCommand,
+ ManagedDependencyFailure,
+ ManagedDependencyJournal,
+ client_config_fingerprint,
+ managed_dependency_bundle_hash,
+ managed_dependency_request_hash,
+ parse_managed_dependency_bundle,
+ parse_managed_dependency_command,
+)
+
+
+BINDING_ID = "59646bd5-39eb-414e-a07e-f5469d369687"
+PREPARE_OPERATION_ID = "43d3dad3-d0b0-4ba2-a9ae-2ccf7ab758f2"
+# The production coordinator retains one operation UUID across all CREATE steps.
+INITIALIZE_OPERATION_ID = PREPARE_OPERATION_ID
+PROVISION_OPERATION_ID = PREPARE_OPERATION_ID
+AUTHORIZATION_ID = "3db076d6-ff66-4460-90b5-5b7d2f2b81af"
+INVALIDATE_OPERATION_ID = "dc023422-627b-452f-a0e3-ae536dcf42b4"
+IDENTITY_HASH = "sha256:427bca797fe223f7a612abe0292533df9a97eda718746863a056e186736846bb"
+PROVIDER_HASH = "sha256:540491df8aa26c85b4082bec6030c2ccaba50950c79f5dfc12208e1addb1d155"
+SNAPSHOT_HASH = "sha256:2bff3497f87dd9a5f2cf9049c3dba3f4caa589e49ecfead1fff799790853fc15"
+
+
+class TestManagedDependency(unittest.TestCase):
+ def setUp(self):
+ self.temporary_root = tempfile.mkdtemp(prefix="ambari-managed-dependency-")
+ self.state_root = os.path.join(self.temporary_root, "journal")
+
+ def tearDown(self):
+ shutil.rmtree(self.temporary_root)
+
+ def test_java_compatible_hash_uses_utf16_string_lengths(self):
+ payload = command_payload(
+ VERIFY_HDFS_CONSUMER,
+ {
+ "client.config.json": hdfs_client_config(),
+ "client.config.fingerprint": hdfs_client_config_fingerprint(),
+ "client.package.name": "hadoop_3_3_0_0_1-client",
+ "client.package.version": "3.3.0-\u7248\u672c-\U0001f600",
+ "client.software.kind": "HADOOP_CLIENT",
+ "client.software.semantic.version": "3.3.0",
+ "consumer.user": "hbase_mc_c1",
+ "expected.default.fs": "hdfs://nn.example.test:8020",
+ "expected.directory.mode": "0700",
+ "expected.namespace.root.uri": hdfs_uri("root"),
+ "expected.namespace.wal.uri": hdfs_uri("wal"),
+ "expected.owner.group": "hbase_mc_c1",
+ "host.id": "42",
+ "identity.fingerprint": IDENTITY_HASH,
+ "preparation.observation.fingerprint":
+ "sha256:7dc1dd44b1a75f745d6a32ca91f69159096f314ca842d43c6371d27eb467b67e",
+ "preparation.observation.id": "4be3c05a-b27c-44c7-a430-a0ff6bfddb28",
+ "preparation.request.hash":
+ "sha256:1bbb4aa3387295797ad351ea845e8864be93077934cc5a7d3e2e44c74d02ba45",
+ },
+ operation_id=PREPARE_OPERATION_ID,
+ epoch=3,
+ )
+
+ self.assertEqual(
+ "sha256:8dceee0e7453229fce1ded1bd641a6c6809995b9c87f648f120632eaafe317ca",
+ payload["envelope"]["immutableRequestHash"],
+ )
+ parse_managed_dependency_command(payload, VERIFY_HDFS_CONSUMER)
+
+ def test_bundle_hash_binds_exact_host_identity_and_prepare_command(self):
+ payload = command_payload(
+ PREPARE_HDFS_CONSUMER,
+ hdfs_prepare_parameters(),
+ operation_id=PREPARE_OPERATION_ID,
+ epoch=3,
+ )
+ command = parse_managed_dependency_command(payload, PREPARE_HDFS_CONSUMER)
+ bundle = {
+ "schemaVersion": 1,
+ "hostId": 42,
+ "consumerUser": "hbase_mc_c1",
+ "identityFingerprint": IDENTITY_HASH,
+ "commands": [payload],
+ "immutableBundleHash": managed_dependency_bundle_hash(
+ 1, 42, "hbase_mc_c1", IDENTITY_HASH, [command]
+ ),
+ }
+
+ parsed = parse_managed_dependency_bundle(bundle)
+
+ self.assertEqual(command.request_hash, parsed.commands[0].request_hash)
+ bundle["hostId"] = 43
+ with self.assertRaisesRegex(ManagedDependencyFailure, "DEPENDENCY_COMMAND_INVALID"):
+ parse_managed_dependency_bundle(bundle)
+
+ verify = command_payload(
+ VERIFY_HDFS_CONSUMER,
+ hdfs_verify_parameters(),
+ operation_id=PREPARE_OPERATION_ID,
+ epoch=3,
+ )
+ bundle["hostId"] = 42
+ bundle["commands"] = [verify]
+ verified = parse_managed_dependency_command(verify, VERIFY_HDFS_CONSUMER)
+ bundle["immutableBundleHash"] = managed_dependency_bundle_hash(
+ 1, 42, "hbase_mc_c1", IDENTITY_HASH, [verified]
+ )
+ with self.assertRaisesRegex(ManagedDependencyFailure, "DEPENDENCY_COMMAND_INVALID"):
+ parse_managed_dependency_bundle(bundle)
+
+ def test_schema_two_subset_is_required_canonical_and_hash_bound(self):
+ payload = command_payload(
+ PREPARE_HDFS_CONSUMER,
+ hdfs_prepare_parameters(),
+ operation_id=PREPARE_OPERATION_ID,
+ epoch=3,
+ )
+ command = parse_managed_dependency_command(payload, PREPARE_HDFS_CONSUMER)
+ bundle = {
+ "schemaVersion": 2,
+ "hostId": 42,
+ "consumerUser": "hbase_mc_c1",
+ "identityFingerprint": IDENTITY_HASH,
+ "commands": [payload],
+ "preparationBindingIds": [BINDING_ID],
+ "immutableBundleHash": managed_dependency_bundle_hash(
+ 2, 42, "hbase_mc_c1", IDENTITY_HASH, [command], [BINDING_ID]
+ ),
+ }
+
+ parsed = parse_managed_dependency_bundle(bundle)
+ self.assertEqual((BINDING_ID,), parsed.preparation_binding_ids)
+ self.assertEqual(
+ (BINDING_ID,), tuple(item.binding_id for item in parsed.preparation_commands)
+ )
+ self.assertIs(parsed.commands[0], parsed.preparation_commands[0])
+
+ missing = deepcopy(bundle)
+ del missing["preparationBindingIds"]
+ with self.assertRaisesRegex(ManagedDependencyFailure, "DEPENDENCY_COMMAND_INVALID"):
+ parse_managed_dependency_bundle(missing)
+
+ unknown = deepcopy(bundle)
+ unknown["unexpected"] = True
+ with self.assertRaisesRegex(ManagedDependencyFailure, "DEPENDENCY_COMMAND_INVALID"):
+ parse_managed_dependency_bundle(unknown)
+
+ duplicate = deepcopy(bundle)
+ duplicate["preparationBindingIds"] = [BINDING_ID, BINDING_ID]
+ with self.assertRaisesRegex(ManagedDependencyFailure, "DEPENDENCY_COMMAND_INVALID"):
+ parse_managed_dependency_bundle(duplicate)
+
+ non_subset = deepcopy(bundle)
+ non_subset["preparationBindingIds"] = ["f0000000-0000-4000-8000-000000000002"]
+ non_subset["immutableBundleHash"] = managed_dependency_bundle_hash(
+ 2, 42, "hbase_mc_c1", IDENTITY_HASH, [command],
+ non_subset["preparationBindingIds"],
+ )
+ with self.assertRaisesRegex(ManagedDependencyFailure, "DEPENDENCY_COMMAND_INVALID"):
+ parse_managed_dependency_bundle(non_subset)
+
+ tampered = deepcopy(bundle)
+ tampered["immutableBundleHash"] = "sha256:" + "0" * 64
+ with self.assertRaisesRegex(ManagedDependencyFailure, "DEPENDENCY_COMMAND_INVALID"):
+ parse_managed_dependency_bundle(tampered)
+
+ wrong_host = deepcopy(bundle)
+ wrong_host["commands"][0]["parameters"]["host.id"] = "43"
+ rehash(wrong_host["commands"][0])
+ with self.assertRaisesRegex(ManagedDependencyFailure, "DEPENDENCY_COMMAND_INVALID"):
+ parse_managed_dependency_bundle(wrong_host)
+
+ wrong_identity = deepcopy(bundle)
+ wrong_identity["commands"][0]["parameters"]["consumer.user"] = "other_user"
+ rehash(wrong_identity["commands"][0])
+ with self.assertRaisesRegex(ManagedDependencyFailure, "DEPENDENCY_COMMAND_INVALID"):
+ parse_managed_dependency_bundle(wrong_identity)
+
+ def test_schema_two_hash_vector_uses_canonical_string_uuid_ordering(self):
+ low_bit = ManagedDependencyCommand(
+ PREPARE_HDFS_CONSUMER,
+ {
+ "bindingId": "10000000-0000-4000-8000-000000000001",
+ "operationId": PREPARE_OPERATION_ID,
+ "epoch": 3,
+ "snapshotVersion": 7,
+ "immutableRequestHash": "sha256:1f9d4536627f840954773f5f6a5c4c446d6e092d241bc11b1dec0ae6a3c029f9",
+ },
+ {"provider.service": "HDFS"},
+ )
+ high_bit = ManagedDependencyCommand(
+ PREPARE_ZOOKEEPER_CONSUMER,
+ {
+ "bindingId": "f0000000-0000-4000-8000-000000000002",
+ "operationId": "f0000000-0000-4000-8000-000000000003",
+ "epoch": 4,
+ "snapshotVersion": 7,
+ "immutableRequestHash": "sha256:2c7af9e4ed49692ed09859ab822bfb0f5d0f0e3080ebb608dc6635920f5417b0",
+ },
+ {"provider.service": "ZOOKEEPER"},
+ )
+
+ self.assertEqual(
+ "sha256:1cdc7d2691963e519c4353d3d920b1cf2b7680f07e5239c5535388d99a4fa6a6",
+ managed_dependency_bundle_hash(
+ 2,
+ 42,
+ "hbase_mc_c1",
+ IDENTITY_HASH,
+ [high_bit, low_bit],
+ [high_bit.binding_id, low_bit.binding_id],
+ ),
+ )
+ self.assertEqual(
+ "sha256:1cdc7d2691963e519c4353d3d920b1cf2b7680f07e5239c5535388d99a4fa6a6",
+ managed_dependency_bundle_hash(
+ 2, 42, "hbase_mc_c1", IDENTITY_HASH, [high_bit, low_bit]
+ ),
+ )
+
+ def test_client_software_and_observation_lineage_are_strictly_typed(self):
+ payload = command_payload(
+ VERIFY_HDFS_CONSUMER,
+ hdfs_verify_parameters(),
+ operation_id=PREPARE_OPERATION_ID,
+ epoch=3,
+ )
+ payload["parameters"]["client.software.semantic.version"] = "3"
+ rehash(payload)
+ with self.assertRaisesRegex(ManagedDependencyFailure, "DEPENDENCY_COMMAND_INVALID"):
+ parse_managed_dependency_command(payload, VERIFY_HDFS_CONSUMER)
+
+ payload = command_payload(
+ VERIFY_HDFS_CONSUMER,
+ hdfs_verify_parameters(),
+ operation_id=PREPARE_OPERATION_ID,
+ epoch=3,
+ )
+ payload["parameters"]["preparation.observation.id"] = "not-a-uuid"
+ rehash(payload)
+ with self.assertRaisesRegex(ManagedDependencyFailure, "DEPENDENCY_COMMAND_INVALID"):
+ parse_managed_dependency_command(payload, VERIFY_HDFS_CONSUMER)
+
+ preparation = command_payload(
+ PREPARE_HDFS_CONSUMER,
+ hdfs_prepare_parameters(),
+ operation_id=PREPARE_OPERATION_ID,
+ epoch=3,
+ )
+ preparation["parameters"]["client.package.version"] = "untrusted-echo"
+ rehash(preparation)
+ with self.assertRaisesRegex(ManagedDependencyFailure, "DEPENDENCY_COMMAND_INVALID"):
+ parse_managed_dependency_command(preparation, PREPARE_HDFS_CONSUMER)
+
+ def test_traversal_and_unknown_fields_fail_even_with_recomputed_hash(self):
+ payload = command_payload(
+ PROVISION_HDFS_NAMESPACE,
+ hdfs_provision_parameters(),
+ operation_id=PROVISION_OPERATION_ID,
+ )
+ payload["parameters"]["namespace.root.uri"] = (
+ f"hdfs://nn.example.test:8020{HDFS_BASE}/{BINDING_ID}/../../foreign/root"
+ )
+ rehash(payload)
+ with self.assertRaisesRegex(ManagedDependencyFailure, "DEPENDENCY_COMMAND_INVALID"):
+ parse_managed_dependency_command(payload, PROVISION_HDFS_NAMESPACE)
+
+ payload = command_payload(
+ PROVISION_HDFS_NAMESPACE,
+ hdfs_provision_parameters(),
+ operation_id=PROVISION_OPERATION_ID,
+ )
+ payload["parameters"]["arbitrary.path"] = "/user/hbase"
+ rehash(payload)
+ with self.assertRaisesRegex(ManagedDependencyFailure, "DEPENDENCY_COMMAND_INVALID"):
+ parse_managed_dependency_command(payload, PROVISION_HDFS_NAMESPACE)
+
+ def test_secure_zookeeper_handoff_fields_are_exact_and_hash_bound(self):
+ container = f"/ambari-managed-hbase/{BINDING_ID}"
+ payload = command_payload(
+ PROVISION_ZOOKEEPER_NAMESPACE,
+ {
+ "consumer.zk.sasl.id": "hbase_mc_c1",
+ "expected.client.port": "2181",
+ "expected.quorum": "zk1.example.test,zk2.example.test",
+ "namespace.container.znode": container,
+ "namespace.ledger.znode": f"/ambari-managed-hbase/.bindings/{BINDING_ID}",
+ "namespace.parent.znode": "/ambari-managed-hbase",
+ "namespace.znode": container + "/hbase",
+ "owner.user": "hbase_mc_c1",
+ "parent.acl.policy": "PROVIDER_ADMIN_CREATE_DELETE",
+ "provider.service": "ZOOKEEPER",
+ "security.mode": "KERBEROS",
+ "subtree.acl.policy": "SASL_CONSUMER_PRIVATE",
+ },
+ operation_id=PROVISION_OPERATION_ID,
+ )
+
+ parsed = parse_managed_dependency_command(
+ payload, PROVISION_ZOOKEEPER_NAMESPACE
+ )
+ self.assertEqual("hbase_mc_c1", parsed.parameters["consumer.zk.sasl.id"])
+ payload["parameters"]["namespace.ledger.znode"] += "-foreign"
+ rehash(payload)
+ with self.assertRaisesRegex(ManagedDependencyFailure, "DEPENDENCY_COMMAND_INVALID"):
+ parse_managed_dependency_command(payload, PROVISION_ZOOKEEPER_NAMESPACE)
+
+ def test_challenge_initialize_mutate_retry_and_lost_journal(self):
+ journal = ManagedDependencyJournal(self.state_root)
+ prepare = parsed_command(
+ PREPARE_BINDING_JOURNAL,
+ {},
+ operation_id=PREPARE_OPERATION_ID,
+ )
+ challenge = journal.prepare(prepare)["initialization.challenge"]
+ self.assertEqual(challenge, journal.prepare(prepare)["initialization.challenge"])
+
+ initialize = parsed_command(
+ INITIALIZE_BINDING_JOURNAL,
+ {
+ "initialization.authorization.id": AUTHORIZATION_ID,
+ "initialization.challenge": challenge,
+ "initialization.prepare.request.hash": prepare.request_hash,
+ },
+ operation_id=INITIALIZE_OPERATION_ID,
+ )
+ initialized = journal.initialize(initialize)
+ self.assertEqual("true", initialized["journal.initialized"])
+ self.assertEqual(initialized, journal.initialize(initialize))
+
+ provision = parsed_command(
+ PROVISION_HDFS_NAMESPACE,
+ hdfs_provision_parameters(),
+ operation_id=PROVISION_OPERATION_ID,
+ )
+ namespace = os.path.join(self.temporary_root, "provider-namespace")
+ calls = []
+
+ def first_provider_mutation(progress):
+ calls.append("mutated")
+ os.mkdir(namespace, 0o700)
+ progress.checkpoint("root-created")
+ return {
+ "applied.snapshot.fingerprint": SNAPSHOT_HASH,
+ "directory.mode": "0700",
+ "namespace.root.exists": "true",
+ "namespace.wal.exists": "true",
+ "owner.group": "hbase_mc_c1",
+ "owner.user": "hbase_mc_c1",
+ "provider.action.host.id": "21",
+ }
+
+ facts = journal.execute(provision, first_provider_mutation)
+ self.assertEqual("true", facts["namespace.root.exists"])
+ self.assertEqual(0o700, stat.S_IMODE(os.stat(namespace).st_mode))
+ self.assertEqual(facts, journal.execute(provision, lambda unused: self.fail("replayed")))
+ self.assertEqual(["mutated"], calls)
+
+ os.unlink(os.path.join(self.state_root, f"hdfs-{BINDING_ID}.journal.json"))
+ replacement_challenge = journal.prepare(prepare)["initialization.challenge"]
+ self.assertNotEqual(challenge, replacement_challenge)
+ with self.assertRaisesRegex(ManagedDependencyFailure, "DEPENDENCY_FENCING_UNCERTAIN"):
+ journal.initialize(initialize)
+ with self.assertRaisesRegex(ManagedDependencyFailure, "DEPENDENCY_FENCING_UNCERTAIN"):
+ journal.execute(provision, first_provider_mutation)
+
+ def test_initialized_epoch_rejects_a_different_operation_without_mutation(self):
+ journal = self._initialized_journal()
+ command = parsed_command(PROVISION_HDFS_NAMESPACE, hdfs_provision_parameters(),
+ operation_id=INVALIDATE_OPERATION_ID)
+ with self.assertRaisesRegex(ManagedDependencyFailure, "DEPENDENCY_OPERATION_STALE"):
+ journal.execute(command, lambda unused: self.fail("foreign operation mutated"))
+
+ def test_initial_provision_cannot_change_snapshot_or_epoch_under_one_operation(self):
+ journal = self._initialized_journal()
+ for field, value in (("epoch", 2), ("snapshotVersion", 8)):
+ payload = command_payload(PROVISION_HDFS_NAMESPACE, hdfs_provision_parameters(),
+ operation_id=PROVISION_OPERATION_ID)
+ payload["envelope"][field] = value
+ rehash(payload)
+ command = parse_managed_dependency_command(payload, PROVISION_HDFS_NAMESPACE)
+ with self.assertRaisesRegex(ManagedDependencyFailure, "DEPENDENCY_OPERATION_STALE"):
+ journal.execute(command, lambda unused: self.fail("changed identity mutated"))
+
+ def test_update_uses_existing_journal_and_replays_only_its_exact_command(self):
+ journal = self._initialized_journal()
+ first = parsed_command(PROVISION_HDFS_NAMESPACE, hdfs_provision_parameters(),
+ operation_id=PROVISION_OPERATION_ID)
+ journal.execute(first, lambda unused: {"applied": "first"})
+ update = parsed_command(PROVISION_HDFS_NAMESPACE, hdfs_provision_parameters(),
+ operation_id=INVALIDATE_OPERATION_ID, epoch=2)
+ self.assertEqual({"applied": "update"}, journal.execute(update, lambda unused: {"applied": "update"}))
+ self.assertEqual({"applied": "update"}, journal.execute(update, lambda unused: self.fail("replayed")))
+ changed = command_payload(PROVISION_HDFS_NAMESPACE, hdfs_provision_parameters(),
+ operation_id=INVALIDATE_OPERATION_ID, epoch=2)
+ changed["parameters"]["snapshot.fingerprint"] = PROVIDER_HASH
+ rehash(changed)
+ with self.assertRaisesRegex(ManagedDependencyFailure, "DEPENDENCY_OPERATION_STALE"):
+ journal.execute(parse_managed_dependency_command(changed, PROVISION_HDFS_NAMESPACE),
+ lambda unused: self.fail("changed command mutated"))
+ with self.assertRaisesRegex(ManagedDependencyFailure, "DEPENDENCY_OPERATION_STALE"):
+ journal.execute(first, lambda unused: self.fail("retired epoch mutated"))
+
+ def test_symlink_lock_file_is_rejected(self):
+ os.mkdir(self.state_root, 0o700)
+ target = os.path.join(self.temporary_root, "target")
+ with open(target, "w", encoding="utf-8") as stream:
+ stream.write("do not follow")
+ os.symlink(target, os.path.join(self.state_root, f"hdfs-{BINDING_ID}.lock"))
+ prepare = parsed_command(
+ PREPARE_BINDING_JOURNAL,
+ {},
+ operation_id=PREPARE_OPERATION_ID,
+ )
+
+ with self.assertRaisesRegex(ManagedDependencyFailure, "DEPENDENCY_FENCING_UNCERTAIN"):
+ ManagedDependencyJournal(self.state_root).prepare(prepare)
+ with open(target, encoding="utf-8") as stream:
+ self.assertEqual("do not follow", stream.read())
+
+ def test_processes_serialize_on_one_binding_lock(self):
+ prepare_payload = command_payload(
+ PREPARE_BINDING_JOURNAL,
+ {},
+ operation_id=PREPARE_OPERATION_ID,
+ )
+ start = multiprocessing.Event()
+ results = multiprocessing.Queue()
+ processes = [
+ multiprocessing.Process(
+ target=prepare_in_process,
+ args=(self.state_root, prepare_payload, start, results),
+ )
+ for unused in range(3)
+ ]
+ for process in processes:
+ process.start()
+ start.set()
+ for process in processes:
+ process.join(5)
+ self.assertEqual(0, process.exitcode)
+
+ challenges = [results.get(timeout=1) for unused in processes]
+ self.assertEqual(1, len(set(challenges)))
+
+ def test_exact_invalidation_retry_reconciles_after_tombstone(self):
+ journal = self._initialized_journal()
+ invalidate = parsed_command(
+ INVALIDATE_BINDING_EPOCH,
+ {},
+ operation_id=INVALIDATE_OPERATION_ID,
+ epoch=2,
+ )
+ expected = {
+ "highest.accepted.epoch": "2",
+ "provider.action.host.id": "21",
+ }
+
+ self.assertEqual(expected, journal.execute(invalidate, lambda unused: expected))
+ self.assertEqual(
+ expected,
+ journal.execute(invalidate, lambda unused: self.fail("replayed invalidation")),
+ )
+ later_provision = parsed_command(
+ PROVISION_HDFS_NAMESPACE,
+ hdfs_provision_parameters(),
+ operation_id="b262c96a-b214-4091-a3f8-951b734c6b86",
+ epoch=3,
+ )
+ with self.assertRaisesRegex(
+ ManagedDependencyFailure, "DEPENDENCY_OPERATION_STALE"
+ ):
+ journal.execute(later_provision, lambda unused: self.fail("retired mutation"))
+
+ def test_initialize_recovers_crashes_before_and_after_pending_rewrite(self):
+ journal = ManagedDependencyJournal(self.state_root)
+ prepare = parsed_command(
+ PREPARE_BINDING_JOURNAL, {}, operation_id=PREPARE_OPERATION_ID
+ )
+ challenge = journal.prepare(prepare)["initialization.challenge"]
+ initialize = parsed_command(
+ INITIALIZE_BINDING_JOURNAL,
+ {
+ "initialization.authorization.id": AUTHORIZATION_ID,
+ "initialization.challenge": challenge,
+ "initialization.prepare.request.hash": prepare.request_hash,
+ },
+ operation_id=INITIALIZE_OPERATION_ID,
+ )
+ original_write = dependency_module._JournalStore.write
+
+ def fail_before_rewrite(store, name, value):
+ if value.get("status") == "INITIALIZED":
+ raise ManagedDependencyFailure(
+ "DEPENDENCY_FENCING_UNCERTAIN", "injected pre-rewrite interruption"
+ )
+ return original_write(store, name, value)
+
+ with patch.object(dependency_module._JournalStore, "write", fail_before_rewrite):
+ with self.assertRaisesRegex(
+ ManagedDependencyFailure, "DEPENDENCY_FENCING_UNCERTAIN"
+ ):
+ journal.initialize(initialize)
+
+ def fail_before_rename(unused_store, unused_source, unused_destination):
+ raise ManagedDependencyFailure(
+ "DEPENDENCY_FENCING_UNCERTAIN", "injected pre-rename interruption"
+ )
+
+ with patch.object(dependency_module._JournalStore, "rename", fail_before_rename):
+ with self.assertRaisesRegex(
+ ManagedDependencyFailure, "DEPENDENCY_FENCING_UNCERTAIN"
+ ):
+ journal.initialize(initialize)
+
+ facts = journal.initialize(initialize)
+ self.assertEqual("true", facts["journal.initialized"])
+ pending = os.path.join(self.state_root, f"hdfs-{BINDING_ID}.pending.json")
+ final = os.path.join(self.state_root, f"hdfs-{BINDING_ID}.journal.json")
+ self.assertFalse(os.path.exists(pending))
+ self.assertTrue(os.path.exists(final))
+
+ def test_scalar_bounds_and_corrupt_journal_identity_fail_closed(self):
+ payload = command_payload(
+ PREPARE_BINDING_JOURNAL, {}, operation_id=PREPARE_OPERATION_ID
+ )
+ payload["envelope"]["protocolVersion"] = True
+ rehash(payload)
+ with self.assertRaisesRegex(ManagedDependencyFailure, "DEPENDENCY_COMMAND_INVALID"):
+ parse_managed_dependency_command(payload, PREPARE_BINDING_JOURNAL)
+
+ payload = command_payload(
+ PREPARE_BINDING_JOURNAL, {}, operation_id=PREPARE_OPERATION_ID
+ )
+ payload["envelope"]["epoch"] = 1 << 63
+ rehash(payload)
+ with self.assertRaisesRegex(ManagedDependencyFailure, "DEPENDENCY_COMMAND_INVALID"):
+ parse_managed_dependency_command(payload, PREPARE_BINDING_JOURNAL)
+
+ journal = self._initialized_journal()
+ provision = parsed_command(
+ PROVISION_HDFS_NAMESPACE,
+ hdfs_provision_parameters(),
+ operation_id=PROVISION_OPERATION_ID,
+ )
+ journal_path = os.path.join(
+ self.state_root, f"hdfs-{BINDING_ID}.journal.json"
+ )
+ with open(journal_path, encoding="utf-8") as stream:
+ original = json.load(stream)
+ corruptions = {
+ "highestEpoch": True,
+ "initializationChallenge": "not-a-uuid",
+ "lastRequestHash": "sha256:not-a-hash",
+ "schemaVersion": True,
+ }
+ for field, value in corruptions.items():
+ damaged = dict(original)
+ damaged[field] = value
+ with open(journal_path, "w", encoding="utf-8") as stream:
+ json.dump(damaged, stream)
+ os.chmod(journal_path, 0o600)
+ with self.subTest(field=field), self.assertRaisesRegex(
+ ManagedDependencyFailure, "DEPENDENCY_FENCING_UNCERTAIN"
+ ):
+ journal.execute(provision, lambda unused: self.fail("corrupt journal"))
+
+ def test_mutation_lock_covers_remote_effect_and_durable_result(self):
+ self._initialized_journal()
+ effect_started = multiprocessing.Event()
+ result_persisted = multiprocessing.Event()
+ release_result = multiprocessing.Event()
+ invalidation_attempted = multiprocessing.Event()
+ invalidation_finished = multiprocessing.Event()
+ results = multiprocessing.Queue()
+ provision_payload = command_payload(
+ PROVISION_HDFS_NAMESPACE,
+ hdfs_provision_parameters(),
+ operation_id=PROVISION_OPERATION_ID,
+ )
+ invalidate_payload = command_payload(
+ INVALIDATE_BINDING_EPOCH,
+ {},
+ operation_id=INVALIDATE_OPERATION_ID,
+ epoch=2,
+ )
+ provision_process = multiprocessing.Process(
+ target=provision_with_delayed_result,
+ args=(
+ self.state_root,
+ provision_payload,
+ effect_started,
+ result_persisted,
+ release_result,
+ results,
+ ),
+ )
+ invalidate_process = multiprocessing.Process(
+ target=invalidate_after_effect,
+ args=(
+ self.state_root,
+ invalidate_payload,
+ effect_started,
+ invalidation_attempted,
+ invalidation_finished,
+ results,
+ ),
+ )
+ provision_process.start()
+ invalidate_process.start()
+ try:
+ self.assertTrue(result_persisted.wait(2))
+ self.assertTrue(invalidation_attempted.wait(2))
+ self.assertFalse(invalidation_finished.wait(0.2))
+ finally:
+ release_result.set()
+ for process in (provision_process, invalidate_process):
+ process.join(5)
+ for process in (provision_process, invalidate_process):
+ self.assertEqual(0, process.exitcode)
+ self.assertEqual(
+ {"provision", "invalidate"},
+ {results.get(timeout=1), results.get(timeout=1)},
+ )
+
+ def _initialized_journal(self):
+ journal = ManagedDependencyJournal(self.state_root)
+ prepare = parsed_command(
+ PREPARE_BINDING_JOURNAL, {}, operation_id=PREPARE_OPERATION_ID
+ )
+ challenge = journal.prepare(prepare)["initialization.challenge"]
+ initialize = parsed_command(
+ INITIALIZE_BINDING_JOURNAL,
+ {
+ "initialization.authorization.id": AUTHORIZATION_ID,
+ "initialization.challenge": challenge,
+ "initialization.prepare.request.hash": prepare.request_hash,
+ },
+ operation_id=INITIALIZE_OPERATION_ID,
+ )
+ journal.initialize(initialize)
+ return journal
+
+
+def prepare_in_process(state_root, payload, start, results):
+ start.wait(2)
+ command = parse_managed_dependency_command(payload, PREPARE_BINDING_JOURNAL)
+ results.put(ManagedDependencyJournal(state_root).prepare(command)["initialization.challenge"])
+
+
+def provision_with_delayed_result(
+ state_root,
+ payload,
+ effect_started,
+ result_persisted,
+ release_result,
+ results,
+):
+ command = parse_managed_dependency_command(payload, PROVISION_HDFS_NAMESPACE)
+ journal = ManagedDependencyJournal(state_root)
+ original_write = dependency_module._JournalStore.write
+
+ def write_then_wait(store, name, value):
+ original_write(store, name, value)
+ if (
+ value.get("lastCommandName") == PROVISION_HDFS_NAMESPACE
+ and value.get("status") == "SUCCEEDED"
+ ):
+ result_persisted.set()
+ release_result.wait(5)
+
+ def mutation(unused_progress):
+ effect_started.set()
+ return {
+ "applied.snapshot.fingerprint": SNAPSHOT_HASH,
+ "directory.mode": "0700",
+ "namespace.root.exists": "true",
+ "namespace.wal.exists": "true",
+ "owner.group": "hbase_mc_c1",
+ "owner.user": "hbase_mc_c1",
+ "provider.action.host.id": "21",
+ }
+
+ with patch.object(dependency_module._JournalStore, "write", write_then_wait):
+ journal.execute(command, mutation)
+ results.put("provision")
+
+
+def invalidate_after_effect(
+ state_root,
+ payload,
+ effect_started,
+ attempted,
+ finished,
+ results,
+):
+ effect_started.wait(5)
+ command = parse_managed_dependency_command(payload, INVALIDATE_BINDING_EPOCH)
+ attempted.set()
+ ManagedDependencyJournal(state_root).execute(
+ command,
+ lambda unused: {
+ "highest.accepted.epoch": str(command.epoch),
+ "provider.action.host.id": command.action_host_id,
+ },
+ )
+ finished.set()
+ results.put("invalidate")
+
+
+def parsed_command(name, extra_parameters, operation_id, epoch=1):
+ return parse_managed_dependency_command(
+ command_payload(name, extra_parameters, operation_id, epoch), name
+ )
+
+
+def command_payload(name, extra_parameters, operation_id, epoch=1):
+ parameters = {
+ "provider.cluster.id": "2",
+ "provider.fingerprint": PROVIDER_HASH,
+ "provider.service": "HDFS",
+ "security.mode": "INSECURE",
+ "snapshot.fingerprint": SNAPSHOT_HASH,
+ }
+ if name in {
+ PREPARE_BINDING_JOURNAL,
+ INITIALIZE_BINDING_JOURNAL,
+ PROVISION_HDFS_NAMESPACE,
+ PROVISION_ZOOKEEPER_NAMESPACE,
+ INVALIDATE_BINDING_EPOCH,
+ }:
+ parameters["provider.action.host.id"] = "21"
+ parameters.update(extra_parameters)
+ envelope = {
+ "protocolVersion": 1,
+ "bindingId": BINDING_ID,
+ "operationId": operation_id,
+ "epoch": epoch,
+ "snapshotVersion": 7,
+ "immutableRequestHash": "sha256:" + "0" * 64,
+ }
+ command = ManagedDependencyCommand(name, envelope, parameters)
+ envelope["immutableRequestHash"] = managed_dependency_request_hash(command)
+ return {"name": name, "envelope": envelope, "parameters": parameters}
+
+
+def rehash(payload):
+ command = ManagedDependencyCommand(
+ payload["name"], payload["envelope"], payload["parameters"]
+ )
+ payload["envelope"]["immutableRequestHash"] = managed_dependency_request_hash(command)
+
+
+def hdfs_provision_parameters():
+ return {
+ "directory.mode": "0700",
+ "namespace.root.uri": hdfs_uri("root"),
+ "namespace.wal.uri": hdfs_uri("wal"),
+ "owner.group": "hbase_mc_c1",
+ "owner.user": "hbase_mc_c1",
+ }
+
+
+def hdfs_verify_parameters():
+ return {
+ "client.config.json": hdfs_client_config(),
+ "client.config.fingerprint": hdfs_client_config_fingerprint(),
+ "client.package.name": "hadoop_3_3_0_0_1-client",
+ "client.package.version": "3.3.0-1",
+ "client.software.kind": "HADOOP_CLIENT",
+ "client.software.semantic.version": "3.3.0",
+ "consumer.user": "hbase_mc_c1",
+ "expected.default.fs": "hdfs://nn.example.test:8020",
+ "expected.directory.mode": "0700",
+ "expected.namespace.root.uri": hdfs_uri("root"),
+ "expected.namespace.wal.uri": hdfs_uri("wal"),
+ "expected.owner.group": "hbase_mc_c1",
+ "host.id": "42",
+ "identity.fingerprint": IDENTITY_HASH,
+ "preparation.observation.fingerprint": "sha256:" + "4" * 64,
+ "preparation.observation.id": "f70e1c9a-437f-4651-aed7-61cfcc6d9e2c",
+ "preparation.request.hash": "sha256:" + "5" * 64,
+ }
+
+
+def hdfs_prepare_parameters():
+ strict = {
+ "client.package.version",
+ "preparation.observation.fingerprint",
+ "preparation.observation.id",
+ "preparation.request.hash",
+ }
+ return {
+ key: value for key, value in hdfs_verify_parameters().items() if key not in strict
+ }
+
+
+def hdfs_client_config():
+ return json.dumps(
+ {
+ "coreSite": {"fs.defaultFS": "hdfs://nn.example.test:8020"},
+ "hdfsSite": {},
+ "zooKeeperClient": {},
+ },
+ sort_keys=True,
+ separators=(",", ":"),
+ )
+
+
+def hdfs_client_config_fingerprint():
+ return client_config_fingerprint(json.loads(hdfs_client_config()))
+
+
+def hdfs_uri(child):
+ return f"hdfs://nn.example.test:8020{HDFS_BASE}/{BINDING_ID}/{child}"
+
+
+if __name__ == "__main__":
+ unittest.main()
diff --git a/ambari-common/src/main/python/resource_management/libraries/functions/managed_dependency.py b/ambari-common/src/main/python/resource_management/libraries/functions/managed_dependency.py
new file mode 100644
index 0000000000..88f2c33ff4
--- /dev/null
+++ b/ambari-common/src/main/python/resource_management/libraries/functions/managed_dependency.py
@@ -0,0 +1,1633 @@
+#!/usr/bin/env python3
+"""
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+"""
+
+from contextlib import contextmanager
+import fcntl
+import hashlib
+import json
+import os
+import re
+import secrets
+import stat
+import uuid
+from urllib.parse import urlsplit
+
+from resource_management.core.exceptions import Fail
+
+
+PROTOCOL_VERSION = 1
+JOURNAL_SCHEMA_VERSION = 1
+BUNDLE_SCHEMA_VERSION = 2
+ZOOKEEPER_PARENT = "/ambari-managed-hbase"
+ZOOKEEPER_LEDGER_PARENT = ZOOKEEPER_PARENT + "/.bindings"
+ZOOKEEPER_HBASE_CHILD = "hbase"
+HDFS_BASE = "/apps/ambari-managed/hbase"
+
+PREPARE_BINDING_JOURNAL = "PREPARE_BINDING_JOURNAL"
+INITIALIZE_BINDING_JOURNAL = "INITIALIZE_BINDING_JOURNAL"
+PROVISION_HDFS_NAMESPACE = "PROVISION_HDFS_NAMESPACE"
+PROVISION_ZOOKEEPER_NAMESPACE = "PROVISION_ZOOKEEPER_NAMESPACE"
+INVALIDATE_BINDING_EPOCH = "INVALIDATE_BINDING_EPOCH"
+PREPARE_HDFS_CONSUMER = "PREPARE_HDFS_CONSUMER"
+PREPARE_ZOOKEEPER_CONSUMER = "PREPARE_ZOOKEEPER_CONSUMER"
+VERIFY_HDFS_CONSUMER = "VERIFY_HDFS_CONSUMER"
+VERIFY_ZOOKEEPER_CONSUMER = "VERIFY_ZOOKEEPER_CONSUMER"
+
+_HASH = re.compile(r"sha256:[0-9a-f]{64}\Z", re.ASCII)
+_POSITIVE_INTEGER = re.compile(r"[1-9][0-9]*\Z", re.ASCII)
+_MAX_SIGNED_LONG = (1 << 63) - 1
+_SAFE_SERVICE = re.compile(r"(?:HDFS|ZOOKEEPER)\Z", re.ASCII)
+_SAFE_PACKAGE = re.compile(r"[A-Za-z0-9][A-Za-z0-9+._~-]{0,127}\Z", re.ASCII)
+_SOFTWARE_VERSION = re.compile(
+ r"[0-9]+(?:\.[0-9]+){1,3}(?:[-+][A-Za-z0-9][A-Za-z0-9._-]*)?\Z", re.ASCII
+)
+_SAFE_IDENTITY = re.compile(r"[A-Za-z_][A-Za-z0-9._-]{0,127}\Z", re.ASCII)
+_KERBEROS_REALM = re.compile(r"[A-Za-z0-9][A-Za-z0-9._-]{0,254}\Z", re.ASCII)
+_HA_TOKEN = re.compile(r"[A-Za-z0-9][A-Za-z0-9.-]*\Z", re.ASCII)
+_STANDARD_HA_PROVIDER = (
+ "org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider"
+)
+_MAX_CLIENT_CONFIG_BYTES = 32 * 1024
+_MAX_CONSUMER_RULE_BYTES = 24 * 1024
+_MAX_CONSUMER_RULE_COUNT = 256
+_AUTH_TO_LOCAL = "hadoop.security.auth_to_local"
+_CORE_SITE_KEYS = {
+ "fs.defaultFS",
+ "hadoop.rpc.protection",
+ "hadoop.security.authentication",
+ "hadoop.security.authorization",
+ "ipc.client.connect.max.retries",
+ "ipc.client.connect.retry.interval",
+ "ipc.client.fallback-to-simple-auth-allowed",
+}
+_HDFS_SITE_KEYS = {
+ "dfs.block.access.token.enable",
+ "dfs.client.failover.connection.retries",
+ "dfs.client.failover.connection.retries.on.timeouts",
+ "dfs.client.socket-timeout",
+ "dfs.client.use.datanode.hostname",
+ "dfs.data.transfer.protection",
+ "dfs.datanode.kerberos.principal",
+ "dfs.encrypt.data.transfer",
+ "dfs.namenode.kerberos.principal",
+ "dfs.nameservices",
+}
+_ZOOKEEPER_KEYS = {
+ "hbase.zookeeper.property.clientPort",
+ "hbase.zookeeper.quorum",
+ "zookeeper.sasl.client",
+ "zookeeper.sasl.client.username",
+ "zookeeper.sasl.clientconfig",
+ "zookeeper.znode.parent",
+}
+_COMMON_PARAMETERS = {
+ "provider.cluster.id",
+ "provider.fingerprint",
+ "provider.service",
+ "security.mode",
+ "snapshot.fingerprint",
+}
+_ACTION_HOST = {"provider.action.host.id"}
+_CONSUMER_SECURITY_PROOFS = {
+ "consumer.mapping.profile.fingerprint",
+ "consumer.mapping.proof.fingerprint",
+ "consumer.mapping.rules.fingerprint",
+ "provider.security.pair.proof.fingerprint",
+ "provider.security.policy.fingerprint",
+}
+_CONSUMER_PREPARATION_PARAMETERS = {
+ "client.config.json",
+ "client.config.fingerprint",
+ "client.package.name",
+ "client.software.kind",
+ "client.software.semantic.version",
+ "consumer.user",
+ "host.id",
+ "identity.fingerprint",
+}
+_STRICT_VERIFICATION_PARAMETERS = {
+ "client.package.version",
+ "preparation.observation.fingerprint",
+ "preparation.observation.id",
+ "preparation.request.hash",
+}
+_HDFS_CONSUMER_PARAMETERS = {
+ "expected.default.fs",
+ "expected.directory.mode",
+ "expected.namespace.root.uri",
+ "expected.namespace.wal.uri",
+ "expected.owner.group",
+}
+_ZOOKEEPER_CONSUMER_PARAMETERS = {
+ "expected.namespace.container.znode",
+ "expected.namespace.parent.znode",
+ "expected.namespace.znode",
+}
+_PARAMETERS = {
+ PREPARE_BINDING_JOURNAL: _COMMON_PARAMETERS | _ACTION_HOST,
+ INITIALIZE_BINDING_JOURNAL: _COMMON_PARAMETERS
+ | _ACTION_HOST
+ | {
+ "initialization.authorization.id",
+ "initialization.challenge",
+ "initialization.prepare.request.hash",
+ },
+ PROVISION_HDFS_NAMESPACE: _COMMON_PARAMETERS
+ | _ACTION_HOST
+ | {
+ "directory.mode",
+ "namespace.root.uri",
+ "namespace.wal.uri",
+ "owner.group",
+ "owner.user",
+ },
+ PROVISION_ZOOKEEPER_NAMESPACE: _COMMON_PARAMETERS
+ | _ACTION_HOST
+ | {
+ "expected.client.port",
+ "expected.quorum",
+ "namespace.container.znode",
+ "namespace.parent.znode",
+ "namespace.znode",
+ "owner.user",
+ "parent.acl.policy",
+ "subtree.acl.policy",
+ },
+ INVALIDATE_BINDING_EPOCH: _COMMON_PARAMETERS | _ACTION_HOST,
+ PREPARE_HDFS_CONSUMER: _COMMON_PARAMETERS
+ | _CONSUMER_PREPARATION_PARAMETERS
+ | _HDFS_CONSUMER_PARAMETERS,
+ PREPARE_ZOOKEEPER_CONSUMER: _COMMON_PARAMETERS
+ | _CONSUMER_PREPARATION_PARAMETERS
+ | _ZOOKEEPER_CONSUMER_PARAMETERS,
+ VERIFY_HDFS_CONSUMER: _COMMON_PARAMETERS
+ | _CONSUMER_PREPARATION_PARAMETERS
+ | _STRICT_VERIFICATION_PARAMETERS
+ | _HDFS_CONSUMER_PARAMETERS,
+ VERIFY_ZOOKEEPER_CONSUMER: _COMMON_PARAMETERS
+ | _CONSUMER_PREPARATION_PARAMETERS
+ | _STRICT_VERIFICATION_PARAMETERS
+ | _ZOOKEEPER_CONSUMER_PARAMETERS,
+}
+_OPTIONAL_PARAMETERS = {
+ PROVISION_ZOOKEEPER_NAMESPACE: {
+ "consumer.zk.sasl.id",
+ "namespace.ledger.znode",
+ },
+ PREPARE_HDFS_CONSUMER: _CONSUMER_SECURITY_PROOFS
+ | {"consumer.kerberos.principal.pattern"},
+ PREPARE_ZOOKEEPER_CONSUMER: _CONSUMER_SECURITY_PROOFS
+ | {
+ "consumer.kerberos.principal.pattern",
+ "expected.consumer.zk.sasl.id",
+ },
+ VERIFY_HDFS_CONSUMER: _CONSUMER_SECURITY_PROOFS
+ | {"consumer.kerberos.principal.pattern"},
+ VERIFY_ZOOKEEPER_CONSUMER: _CONSUMER_SECURITY_PROOFS
+ | {
+ "consumer.kerberos.principal.pattern",
+ "expected.consumer.zk.sasl.id",
+ },
+}
+_PROVIDER_COMMANDS = {
+ PREPARE_BINDING_JOURNAL,
+ INITIALIZE_BINDING_JOURNAL,
+ PROVISION_HDFS_NAMESPACE,
+ PROVISION_ZOOKEEPER_NAMESPACE,
+ INVALIDATE_BINDING_EPOCH,
+}
+
+
+class ManagedDependencyFailure(Fail):
+ def __init__(self, code, message, facts=None):
+ self.code = code
+ self.sanitized_message = message
+ self.facts = dict(facts or {})
+ super().__init__(f"{code}: {message}")
+
+
+class ManagedDependencyCommand:
+ def __init__(self, name, envelope, parameters):
+ self.name = name
+ self.envelope = dict(envelope)
+ self.parameters = dict(parameters)
+
+ @property
+ def binding_id(self):
+ return self.envelope["bindingId"]
+
+ @property
+ def operation_id(self):
+ return self.envelope["operationId"]
+
+ @property
+ def epoch(self):
+ return self.envelope["epoch"]
+
+ @property
+ def snapshot_version(self):
+ return self.envelope["snapshotVersion"]
+
+ @property
+ def request_hash(self):
+ return self.envelope["immutableRequestHash"]
+
+ @property
+ def provider_service(self):
+ return self.parameters["provider.service"]
+
+ @property
+ def action_host_id(self):
+ return self.parameters.get("provider.action.host.id")
+
+
+class ManagedDependencyCommandBundle:
+ def __init__(
+ self,
+ schema_version,
+ host_id,
+ consumer_user,
+ identity_fingerprint,
+ commands,
+ preparation_binding_ids,
+ immutable_bundle_hash,
+ ):
+ self.schema_version = schema_version
+ self.host_id = host_id
+ self.consumer_user = consumer_user
+ self.identity_fingerprint = identity_fingerprint
+ self.commands = tuple(commands)
+ self.preparation_binding_ids = tuple(preparation_binding_ids)
+ self.immutable_bundle_hash = immutable_bundle_hash
+
+ @property
+ def preparation_commands(self):
+ selected = set(self.preparation_binding_ids)
+ return tuple(command for command in self.commands if command.binding_id in selected)
+
+ def command_for(self, provider_service):
+ for command in self.commands:
+ if command.provider_service == provider_service:
+ return command
+ return None
+
+
+def parse_managed_dependency_command(value, expected_name=None):
+ if not isinstance(value, dict) or set(value) != {"name", "envelope", "parameters"}:
+ _fail_invalid("Command must contain only name, envelope, and parameters")
+ name = value["name"]
+ if name not in _PARAMETERS or expected_name is not None and name != expected_name:
+ _fail_invalid("Command name is missing, unexpected, or unsupported")
+ envelope = value["envelope"]
+ expected_envelope = {
+ "bindingId",
+ "epoch",
+ "immutableRequestHash",
+ "operationId",
+ "protocolVersion",
+ "snapshotVersion",
+ }
+ if not isinstance(envelope, dict) or set(envelope) != expected_envelope:
+ _fail_invalid("Command envelope contains missing or unknown fields")
+ parameters = value["parameters"]
+ required_parameters = _PARAMETERS[name]
+ allowed_parameters = required_parameters | _OPTIONAL_PARAMETERS.get(name, set())
+ if (
+ not isinstance(parameters, dict)
+ or not required_parameters.issubset(parameters)
+ or not set(parameters).issubset(allowed_parameters)
+ ):
+ _fail_invalid("Command parameters contain missing or unknown fields")
+ if any(not isinstance(key, str) or not isinstance(item, str) for key, item in parameters.items()):
+ _fail_invalid("Command parameter names and values must be strings")
+
+ if (
+ type(envelope["protocolVersion"]) is not int
+ or envelope["protocolVersion"] != PROTOCOL_VERSION
+ ):
+ _fail_invalid("Unsupported protocol version")
+ envelope = dict(envelope)
+ envelope["bindingId"] = _canonical_uuid(envelope["bindingId"], "bindingId")
+ envelope["operationId"] = _canonical_uuid(envelope["operationId"], "operationId")
+ envelope["epoch"] = _positive_integer(envelope["epoch"], "epoch")
+ envelope["snapshotVersion"] = _positive_integer(
+ envelope["snapshotVersion"], "snapshotVersion"
+ )
+ _require_hash(envelope["immutableRequestHash"], "immutableRequestHash")
+ _positive_integer(parameters["provider.cluster.id"], "provider.cluster.id")
+ _require_hash(parameters["provider.fingerprint"], "provider.fingerprint")
+ _require_hash(parameters["snapshot.fingerprint"], "snapshot.fingerprint")
+ if parameters["security.mode"] not in {"INSECURE", "KERBEROS"}:
+ _fail_invalid("security.mode is unsupported")
+ if _SAFE_SERVICE.fullmatch(parameters["provider.service"]) is None:
+ _fail_invalid("provider.service is unsupported")
+
+ if name in _PROVIDER_COMMANDS:
+ _positive_integer(parameters["provider.action.host.id"], "provider.action.host.id")
+ if name in {PREPARE_BINDING_JOURNAL, INITIALIZE_BINDING_JOURNAL}:
+ pass
+ elif name in {
+ PROVISION_HDFS_NAMESPACE,
+ PREPARE_HDFS_CONSUMER,
+ VERIFY_HDFS_CONSUMER,
+ }:
+ if parameters["provider.service"] != "HDFS":
+ _fail_invalid("The HDFS command requires provider.service=HDFS")
+ root_field = (
+ "namespace.root.uri"
+ if name == PROVISION_HDFS_NAMESPACE
+ else "expected.namespace.root.uri"
+ )
+ wal_field = (
+ "namespace.wal.uri"
+ if name == PROVISION_HDFS_NAMESPACE
+ else "expected.namespace.wal.uri"
+ )
+ authority = _validate_hdfs_namespaces(
+ envelope["bindingId"], parameters[root_field], parameters[wal_field]
+ )
+ if name in {PREPARE_HDFS_CONSUMER, VERIFY_HDFS_CONSUMER}:
+ default_fs = _parse_hdfs_uri(parameters["expected.default.fs"], "expected.default.fs")
+ if default_fs.path or default_fs.netloc != authority:
+ _fail_invalid("expected.default.fs does not match the namespace authority")
+ elif name in {
+ PROVISION_ZOOKEEPER_NAMESPACE,
+ PREPARE_ZOOKEEPER_CONSUMER,
+ VERIFY_ZOOKEEPER_CONSUMER,
+ }:
+ if parameters["provider.service"] != "ZOOKEEPER":
+ _fail_invalid("The ZooKeeper command requires provider.service=ZOOKEEPER")
+ container_field = (
+ "namespace.container.znode"
+ if name == PROVISION_ZOOKEEPER_NAMESPACE
+ else "expected.namespace.container.znode"
+ )
+ znode_field = (
+ "namespace.znode" if name == PROVISION_ZOOKEEPER_NAMESPACE
+ else "expected.namespace.znode"
+ )
+ expected_container = f"{ZOOKEEPER_PARENT}/{envelope['bindingId']}"
+ if (
+ parameters[container_field] != expected_container
+ or parameters[znode_field] != f"{expected_container}/{ZOOKEEPER_HBASE_CHILD}"
+ or name in {PREPARE_ZOOKEEPER_CONSUMER, VERIFY_ZOOKEEPER_CONSUMER}
+ and parameters["expected.namespace.parent.znode"] != ZOOKEEPER_PARENT
+ ):
+ _fail_invalid("ZooKeeper container or HBase root does not match the binding UUID")
+
+ if name == INITIALIZE_BINDING_JOURNAL:
+ _canonical_uuid(parameters["initialization.authorization.id"], "initialization.authorization.id")
+ _canonical_uuid(parameters["initialization.challenge"], "initialization.challenge")
+ _require_hash(
+ parameters["initialization.prepare.request.hash"],
+ "initialization.prepare.request.hash",
+ )
+ if name == PROVISION_HDFS_NAMESPACE:
+ _validate_hdfs_ownership(parameters)
+ if name == PROVISION_ZOOKEEPER_NAMESPACE:
+ _validate_zookeeper_policies(parameters, envelope["bindingId"])
+ consumer_commands = {
+ PREPARE_HDFS_CONSUMER,
+ PREPARE_ZOOKEEPER_CONSUMER,
+ VERIFY_HDFS_CONSUMER,
+ VERIFY_ZOOKEEPER_CONSUMER,
+ }
+ if name in consumer_commands:
+ hdfs = name in {PREPARE_HDFS_CONSUMER, VERIFY_HDFS_CONSUMER}
+ verification = name in {VERIFY_HDFS_CONSUMER, VERIFY_ZOOKEEPER_CONSUMER}
+ _positive_integer(parameters["host.id"], "host.id")
+ if _SAFE_PACKAGE.fullmatch(parameters["client.package.name"]) is None:
+ _fail_invalid("client.package.name is not a bounded operating-system package name")
+ expected_kind = "HADOOP_CLIENT" if hdfs else "HBASE_CLIENT"
+ if parameters["client.software.kind"] != expected_kind:
+ _fail_invalid("client.software.kind does not match the managed dependency type")
+ _require_software_version(parameters["client.software.semantic.version"])
+ if verification:
+ _require_package_version(
+ parameters["client.package.version"], "client.package.version"
+ )
+ _canonical_uuid(
+ parameters["preparation.observation.id"], "preparation.observation.id"
+ )
+ _require_hash(
+ parameters["preparation.request.hash"], "preparation.request.hash"
+ )
+ _require_hash(
+ parameters["preparation.observation.fingerprint"],
+ "preparation.observation.fingerprint",
+ )
+ _require_nonblank(parameters["consumer.user"], "consumer.user")
+ _require_hash(parameters["identity.fingerprint"], "identity.fingerprint")
+ _require_hash(parameters["client.config.fingerprint"], "client.config.fingerprint")
+ secure = parameters["security.mode"] == "KERBEROS"
+ if secure:
+ _require_principal_pattern(
+ parameters.get("consumer.kerberos.principal.pattern"),
+ parameters["consumer.user"],
+ )
+ if not _CONSUMER_SECURITY_PROOFS.issubset(parameters):
+ _fail_invalid("Kerberos consumer command is missing typed security proofs")
+ for field in _CONSUMER_SECURITY_PROOFS:
+ _require_hash(parameters[field], field)
+ elif (
+ "consumer.kerberos.principal.pattern" in parameters
+ or not _CONSUMER_SECURITY_PROOFS.isdisjoint(parameters)
+ ):
+ _fail_invalid("Insecure consumer command carries Kerberos security proofs")
+ client_config = _parse_client_config(parameters["client.config.json"])
+ if parameters["client.config.fingerprint"] != client_config_fingerprint(client_config):
+ _fail_invalid("client.config.fingerprint does not match client.config.json")
+ if hdfs:
+ expected_group = _require_nonblank(
+ parameters["expected.owner.group"], "expected.owner.group"
+ )
+ if parameters["expected.directory.mode"] != "0700" and not (
+ parameters["expected.directory.mode"] == "0750" and expected_group != "hadoop"
+ ):
+ _fail_invalid("Expected HDFS ownership is not private per consumer")
+ if (
+ set(client_config["coreSite"])
+ - (_CORE_SITE_KEYS | ({_AUTH_TO_LOCAL} if secure else set()))
+ or set(client_config["hdfsSite"])
+ - _allowed_hdfs_client_keys(client_config["hdfsSite"])
+ or client_config["zooKeeperClient"]
+ or client_config["coreSite"].get("fs.defaultFS")
+ != parameters["expected.default.fs"]
+ ):
+ _fail_invalid("HDFS client config is outside the approved snapshot")
+ _validate_hdfs_client_values(
+ parameters["security.mode"],
+ client_config["coreSite"],
+ client_config["hdfsSite"],
+ )
+ consumer_rules = client_config["coreSite"].get(_AUTH_TO_LOCAL)
+ if secure:
+ if (
+ consumer_rules is None
+ or parameters["consumer.mapping.rules.fingerprint"]
+ != consumer_mapping_rules_fingerprint(consumer_rules)
+ ):
+ _fail_invalid(
+ "Consumer auth-to-local rules do not match their approved fingerprint"
+ )
+ elif consumer_rules is not None:
+ _fail_invalid(
+ "Insecure HDFS client config carries consumer auth-to-local rules"
+ )
+ elif (
+ client_config["coreSite"]
+ or client_config["hdfsSite"]
+ or set(client_config["zooKeeperClient"]) - _ZOOKEEPER_KEYS
+ or client_config["zooKeeperClient"].get("zookeeper.znode.parent")
+ != parameters["expected.namespace.znode"]
+ ):
+ _fail_invalid("ZooKeeper client config is outside the approved snapshot")
+ else:
+ _validate_zookeeper_client_values(
+ parameters["security.mode"], client_config["zooKeeperClient"]
+ )
+ sasl_id = parameters.get("expected.consumer.zk.sasl.id")
+ if secure:
+ _require_identity(sasl_id, "expected.consumer.zk.sasl.id")
+ if sasl_id != parameters["consumer.user"]:
+ _fail_invalid("Expected ZooKeeper SASL identity differs from consumer.user")
+ elif sasl_id is not None:
+ _fail_invalid("Insecure ZooKeeper consumer command carries a SASL identity")
+
+ command = ManagedDependencyCommand(name, envelope, parameters)
+ if command.request_hash != managed_dependency_request_hash(command):
+ _fail_invalid("immutableRequestHash does not match the command content")
+ return command
+
+
+def parse_managed_dependency_bundle(value):
+ base_fields = {
+ "schemaVersion",
+ "hostId",
+ "consumerUser",
+ "identityFingerprint",
+ "commands",
+ "immutableBundleHash",
+ }
+ if not isinstance(value, dict) or not base_fields.issubset(value):
+ _fail_invalid("Managed dependency bundle contains missing or unknown fields")
+ schema_version = value["schemaVersion"]
+ fields = base_fields | ({"preparationBindingIds"} if schema_version == BUNDLE_SCHEMA_VERSION else set())
+ if set(value) != fields:
+ _fail_invalid("Managed dependency bundle contains missing or unknown fields")
+ if type(schema_version) is not int or schema_version not in {1, BUNDLE_SCHEMA_VERSION}:
+ _fail_invalid("Managed dependency bundle schemaVersion is unsupported")
+ host_id = _positive_integer(value["hostId"], "hostId")
+ consumer_user = _require_nonblank(value["consumerUser"], "consumerUser")
+ identity_fingerprint = value["identityFingerprint"]
+ _require_hash(identity_fingerprint, "identityFingerprint")
+ raw_commands = value["commands"]
+ if not isinstance(raw_commands, list) or not 1 <= len(raw_commands) <= 2:
+ _fail_invalid("Managed dependency bundle requires one or two PREPARE commands")
+ commands = tuple(parse_managed_dependency_command(item) for item in raw_commands)
+ services = set()
+ binding_ids = set()
+ security_modes = set()
+ mapping_profiles = set()
+ for command in commands:
+ if command.name not in {PREPARE_HDFS_CONSUMER, PREPARE_ZOOKEEPER_CONSUMER}:
+ _fail_invalid("Managed dependency bundles contain only consumer PREPARE commands")
+ if command.provider_service in services or command.binding_id in binding_ids:
+ _fail_invalid("Managed dependency bundle contains a duplicate type or binding UUID")
+ services.add(command.provider_service)
+ binding_ids.add(command.binding_id)
+ security_modes.add(command.parameters["security.mode"])
+ mapping_profile = command.parameters.get("consumer.mapping.profile.fingerprint")
+ if mapping_profile is not None:
+ mapping_profiles.add(mapping_profile)
+ if (
+ command.parameters["host.id"] != str(host_id)
+ or command.parameters["consumer.user"] != consumer_user
+ or command.parameters["identity.fingerprint"] != identity_fingerprint
+ ):
+ _fail_invalid("Managed dependency bundle commands target different hosts or identities")
+ if (
+ len(security_modes) != 1
+ or len(mapping_profiles) > 1
+ or "KERBEROS" in security_modes
+ and len(mapping_profiles) != 1
+ or "INSECURE" in security_modes
+ and mapping_profiles
+ ):
+ _fail_invalid("Managed dependency bundle commands use different security profiles")
+ command_binding_ids = {command.binding_id for command in commands}
+ if schema_version == 1:
+ preparation_binding_ids = tuple(sorted(command_binding_ids))
+ else:
+ raw_preparation_ids = value["preparationBindingIds"]
+ if not isinstance(raw_preparation_ids, list):
+ _fail_invalid("preparationBindingIds must be a list")
+ try:
+ preparation_binding_ids = tuple(sorted(
+ _canonical_uuid(item, "preparationBindingIds") for item in raw_preparation_ids
+ ))
+ except (TypeError, ValueError):
+ _fail_invalid("preparationBindingIds must contain UUID strings")
+ if len(set(preparation_binding_ids)) != len(preparation_binding_ids):
+ _fail_invalid("preparationBindingIds must be duplicate-free")
+ if not set(preparation_binding_ids).issubset(command_binding_ids):
+ _fail_invalid("preparationBindingIds must be a subset of command binding IDs")
+ bundle_hash = value["immutableBundleHash"]
+ _require_hash(bundle_hash, "immutableBundleHash")
+ if bundle_hash != managed_dependency_bundle_hash(
+ schema_version,
+ host_id,
+ consumer_user,
+ identity_fingerprint,
+ commands,
+ preparation_binding_ids,
+ ):
+ _fail_invalid("immutableBundleHash does not match the selected commands")
+ return ManagedDependencyCommandBundle(
+ schema_version,
+ host_id,
+ consumer_user,
+ identity_fingerprint,
+ sorted(commands, key=lambda command: command.provider_service),
+ preparation_binding_ids,
+ bundle_hash,
+ )
+
+
+def managed_dependency_bundle_hash(
+ schema_version, host_id, consumer_user, identity_fingerprint, commands,
+ preparation_binding_ids=None
+):
+ canonical = []
+ _append_java_strings(
+ canonical,
+ "managed-dependency-command-bundle",
+ str(schema_version),
+ str(host_id),
+ consumer_user,
+ identity_fingerprint,
+ )
+ for command in sorted(commands, key=lambda item: item.provider_service):
+ _append_java_strings(
+ canonical,
+ command.provider_service,
+ command.binding_id,
+ str(command.epoch),
+ str(command.snapshot_version),
+ command.request_hash,
+ )
+ if schema_version >= BUNDLE_SCHEMA_VERSION:
+ ids = sorted(
+ command.binding_id for command in commands
+ ) if preparation_binding_ids is None else sorted(preparation_binding_ids)
+ _append_java_strings(canonical, "preparation-binding-ids", str(len(ids)))
+ for binding_id in ids:
+ _append_java_strings(canonical, binding_id)
+ return "sha256:" + hashlib.sha256("".join(canonical).encode("utf-8")).hexdigest()
+
+
+def managed_dependency_request_hash(command):
+ canonical = []
+ _append_java_strings(
+ canonical,
+ command.name,
+ str(command.envelope["protocolVersion"]),
+ command.binding_id,
+ command.operation_id,
+ str(command.epoch),
+ str(command.snapshot_version),
+ )
+ for key in sorted(command.parameters):
+ _append_java_strings(canonical, key, command.parameters[key])
+ return "sha256:" + hashlib.sha256("".join(canonical).encode("utf-8")).hexdigest()
+
+
+def client_config_fingerprint(client_config):
+ canonical = json.dumps(
+ client_config, sort_keys=True, separators=(",", ":"), ensure_ascii=False
+ )
+ return "sha256:" + hashlib.sha256(canonical.encode("utf-8")).hexdigest()
+
+
+def consumer_mapping_rules_fingerprint(rules):
+ rules = _require_consumer_mapping_rules(rules)
+ canonical = []
+ _append_java_strings(
+ canonical,
+ "managed-hdfs-auth-to-local",
+ "1",
+ "consumer-local-rules",
+ rules.encode("utf-8").hex(),
+ )
+ return "sha256:" + hashlib.sha256("".join(canonical).encode("utf-8")).hexdigest()
+
+
+class ManagedDependencyJournal:
+ def __init__(self, state_root, expected_uid=None):
+ if not isinstance(state_root, str) or not os.path.isabs(state_root):
+ raise ManagedDependencyFailure(
+ "DEPENDENCY_FENCING_UNCERTAIN", "Journal root must be an absolute path"
+ )
+ self.state_root = state_root
+ self.expected_uid = os.geteuid() if expected_uid is None else expected_uid
+
+ def prepare(self, command):
+ self._require_command(command, PREPARE_BINDING_JOURNAL)
+ with self._locked_store(command) as store:
+ if store.exists(store.journal_name):
+ raise ManagedDependencyFailure(
+ "DEPENDENCY_FENCING_UNCERTAIN",
+ "Binding journal already exists; initialization cannot be repeated",
+ )
+ pending = store.read_optional(store.pending_name)
+ if pending is not None:
+ self._validate_pending(command, pending)
+ return {"initialization.challenge": pending["challenge"]}
+ challenge = str(uuid.UUID(bytes=secrets.token_bytes(16)))
+ pending = {
+ "schemaVersion": JOURNAL_SCHEMA_VERSION,
+ "state": "CHALLENGED",
+ "bindingId": command.binding_id,
+ "providerClusterId": command.parameters["provider.cluster.id"],
+ "providerService": command.provider_service,
+ "actionHostId": command.action_host_id,
+ "epoch": command.epoch,
+ "snapshotVersion": command.snapshot_version,
+ "prepareOperationId": command.operation_id,
+ "prepareRequestHash": command.request_hash,
+ "challenge": challenge,
+ }
+ store.write(store.pending_name, pending)
+ return {"initialization.challenge": challenge}
+
+ def initialize(self, command):
+ self._require_command(command, INITIALIZE_BINDING_JOURNAL)
+ with self._locked_store(command) as store:
+ journal = store.read_optional(store.journal_name)
+ if journal is not None:
+ self._validate_journal_identity(command, journal)
+ if (
+ journal["lastOperationId"] == command.operation_id
+ and journal["lastRequestHash"] == command.request_hash
+ and journal["status"] == "INITIALIZED"
+ ):
+ return self._initialization_facts(command)
+ raise ManagedDependencyFailure(
+ "DEPENDENCY_FENCING_UNCERTAIN",
+ "Binding journal was already initialized by another operation",
+ )
+ pending = store.read_optional(store.pending_name)
+ if pending is None:
+ raise ManagedDependencyFailure(
+ "DEPENDENCY_FENCING_UNCERTAIN",
+ "Initialization requires the matching local challenge record",
+ )
+ if pending.get("status") == "INITIALIZED":
+ self._validate_journal_identity(command, pending)
+ if (
+ pending["lastOperationId"] == command.operation_id
+ and pending["lastRequestHash"] == command.request_hash
+ and pending["initializationChallenge"]
+ == command.parameters["initialization.challenge"]
+ and pending["prepareRequestHash"]
+ == command.parameters["initialization.prepare.request.hash"]
+ and pending["initializationAuthorizationId"]
+ == command.parameters["initialization.authorization.id"]
+ ):
+ store.rename(store.pending_name, store.journal_name)
+ return dict(pending["resultFacts"])
+ raise ManagedDependencyFailure(
+ "DEPENDENCY_FENCING_UNCERTAIN",
+ "Authorized journal initialization does not match this operation",
+ )
+ self._validate_pending_identity(command, pending)
+ if (
+ pending["challenge"] != command.parameters["initialization.challenge"]
+ or pending["prepareRequestHash"]
+ != command.parameters["initialization.prepare.request.hash"]
+ or pending["prepareOperationId"] != command.operation_id
+ ):
+ raise ManagedDependencyFailure(
+ "DEPENDENCY_FENCING_UNCERTAIN",
+ "Initialization authorization does not match the local challenge",
+ )
+ journal = {
+ "schemaVersion": JOURNAL_SCHEMA_VERSION,
+ "bindingId": command.binding_id,
+ "providerClusterId": command.parameters["provider.cluster.id"],
+ "providerService": command.provider_service,
+ "actionHostId": command.action_host_id,
+ "highestEpoch": command.epoch,
+ "snapshotVersion": command.snapshot_version,
+ "lastCommandName": command.name,
+ "lastOperationId": command.operation_id,
+ "lastRequestHash": command.request_hash,
+ "initializationAuthorizationId": command.parameters[
+ "initialization.authorization.id"
+ ],
+ "initializationChallenge": command.parameters["initialization.challenge"],
+ "prepareRequestHash": command.parameters["initialization.prepare.request.hash"],
+ "status": "INITIALIZED",
+ "tombstoned": False,
+ "completedSteps": [],
+ "resultFacts": self._initialization_facts(command),
+ }
+ store.write(store.pending_name, journal)
+ store.rename(store.pending_name, store.journal_name)
+ return dict(journal["resultFacts"])
+
+ def execute(self, command, mutation):
+ if command.name not in {
+ PROVISION_HDFS_NAMESPACE,
+ PROVISION_ZOOKEEPER_NAMESPACE,
+ INVALIDATE_BINDING_EPOCH,
+ }:
+ _fail_invalid("Journal execution requires a provider mutation command")
+ with self._locked_store(command) as store:
+ journal = store.read_optional(store.journal_name)
+ if journal is None:
+ raise ManagedDependencyFailure(
+ "DEPENDENCY_FENCING_UNCERTAIN",
+ "Provider mutation requires a previously initialized binding journal",
+ )
+ self._validate_journal_identity(command, journal)
+ same_operation = journal["lastOperationId"] == command.operation_id
+ same_step = same_operation and journal["lastCommandName"] == command.name
+ # One backend operation owns initialize and provision. Immutable hashes
+ # identify individual commands; only the initialized-to-provision transition
+ # may advance to another command under the same operation and epoch.
+ initial_provision = (
+ same_operation
+ and journal["status"] == "INITIALIZED"
+ and journal["lastCommandName"] == INITIALIZE_BINDING_JOURNAL
+ and command.name in {PROVISION_HDFS_NAMESPACE, PROVISION_ZOOKEEPER_NAMESPACE}
+ and command.epoch == journal["highestEpoch"]
+ and command.snapshot_version == journal["snapshotVersion"]
+ )
+ if same_operation and not initial_provision and (
+ not same_step or journal["lastRequestHash"] != command.request_hash
+ ):
+ raise ManagedDependencyFailure(
+ "DEPENDENCY_OPERATION_STALE",
+ "Operation UUID was reused with different immutable content",
+ )
+ if journal["tombstoned"]:
+ if (
+ command.name == INVALIDATE_BINDING_EPOCH
+ and same_step
+ and journal["status"] == "SUCCEEDED"
+ ):
+ return dict(journal["resultFacts"])
+ raise ManagedDependencyFailure(
+ "DEPENDENCY_OPERATION_STALE", "Provider command epoch is retired"
+ )
+ if command.epoch < journal["highestEpoch"]:
+ raise ManagedDependencyFailure(
+ "DEPENDENCY_OPERATION_STALE", "Provider command epoch is retired"
+ )
+ if same_step and journal["status"] == "SUCCEEDED":
+ return dict(journal["resultFacts"])
+ if not same_operation and command.epoch == journal["highestEpoch"]:
+ raise ManagedDependencyFailure(
+ "DEPENDENCY_OPERATION_STALE",
+ "A different operation already owns this binding epoch",
+ )
+
+ if not same_step:
+ journal["completedSteps"] = []
+ journal["resultFacts"] = {}
+ journal.update(
+ {
+ "highestEpoch": command.epoch,
+ "snapshotVersion": command.snapshot_version,
+ "lastCommandName": command.name,
+ "lastOperationId": command.operation_id,
+ "lastRequestHash": command.request_hash,
+ "status": "RUNNING",
+ }
+ )
+ store.write(store.journal_name, journal)
+ progress = _JournalProgress(store, journal)
+ try:
+ result = mutation(progress)
+ if not isinstance(result, dict):
+ raise ManagedDependencyFailure(
+ "DEPENDENCY_COMMAND_INVALID", "Provider mutation returned invalid facts"
+ )
+ journal["resultFacts"] = dict(result)
+ journal["status"] = "SUCCEEDED"
+ if command.name == INVALIDATE_BINDING_EPOCH:
+ journal["tombstoned"] = True
+ store.write(store.journal_name, journal)
+ return dict(result)
+ except ManagedDependencyFailure:
+ journal["status"] = "FAILED"
+ store.write(store.journal_name, journal)
+ raise
+ except Exception as error:
+ journal["status"] = "FAILED"
+ store.write(store.journal_name, journal)
+ raise ManagedDependencyFailure(
+ "DEPENDENCY_PROVIDER_ACTION_FAILED",
+ "Provider mutation failed; inspect the protected agent log",
+ ) from error
+
+ @contextmanager
+ def _locked_store(self, command):
+ store = _JournalStore(
+ self.state_root, command.provider_service, command.binding_id, self.expected_uid
+ )
+ with store.locked():
+ yield store
+
+ @staticmethod
+ def _require_command(command, expected_name):
+ if not isinstance(command, ManagedDependencyCommand) or command.name != expected_name:
+ _fail_invalid(f"Expected {expected_name}")
+
+ @staticmethod
+ def _validate_pending(command, pending):
+ ManagedDependencyJournal._validate_pending_identity(command, pending)
+ if (
+ pending["prepareOperationId"] != command.operation_id
+ or pending["prepareRequestHash"] != command.request_hash
+ ):
+ raise ManagedDependencyFailure(
+ "DEPENDENCY_FENCING_UNCERTAIN",
+ "Another journal challenge already exists for this binding",
+ )
+
+ @staticmethod
+ def _validate_pending_identity(command, pending):
+ required = {
+ "actionHostId",
+ "bindingId",
+ "challenge",
+ "epoch",
+ "prepareOperationId",
+ "prepareRequestHash",
+ "providerClusterId",
+ "providerService",
+ "schemaVersion",
+ "snapshotVersion",
+ "state",
+ }
+ if set(pending) != required or pending.get("state") != "CHALLENGED":
+ raise ManagedDependencyFailure(
+ "DEPENDENCY_FENCING_UNCERTAIN", "Local journal challenge is corrupt"
+ )
+ try:
+ if type(pending["schemaVersion"]) is not int:
+ raise ValueError("invalid pending schema version")
+ _canonical_uuid(pending["challenge"], "initialization.challenge")
+ _canonical_uuid(pending["prepareOperationId"], "prepareOperationId")
+ _require_hash(pending["prepareRequestHash"], "prepareRequestHash")
+ _positive_integer(pending["providerClusterId"], "providerClusterId")
+ _positive_integer(pending["actionHostId"], "actionHostId")
+ if (
+ type(pending["epoch"]) is not int
+ or type(pending["snapshotVersion"]) is not int
+ or not 0 < pending["epoch"] <= _MAX_SIGNED_LONG
+ or not 0 < pending["snapshotVersion"] <= _MAX_SIGNED_LONG
+ ):
+ raise ValueError("invalid pending numeric field")
+ except (ManagedDependencyFailure, ValueError) as error:
+ raise ManagedDependencyFailure(
+ "DEPENDENCY_FENCING_UNCERTAIN", "Local journal challenge is corrupt"
+ ) from error
+ expected = {
+ "schemaVersion": JOURNAL_SCHEMA_VERSION,
+ "bindingId": command.binding_id,
+ "providerClusterId": command.parameters["provider.cluster.id"],
+ "providerService": command.provider_service,
+ "actionHostId": command.action_host_id,
+ "epoch": command.epoch,
+ "snapshotVersion": command.snapshot_version,
+ }
+ if any(pending.get(key) != value for key, value in expected.items()):
+ raise ManagedDependencyFailure(
+ "DEPENDENCY_FENCING_UNCERTAIN",
+ "Local journal challenge does not match this binding and pinned host",
+ )
+
+ @staticmethod
+ def _validate_journal_identity(command, journal):
+ required = {
+ "actionHostId",
+ "bindingId",
+ "completedSteps",
+ "highestEpoch",
+ "initializationAuthorizationId",
+ "initializationChallenge",
+ "lastCommandName",
+ "lastOperationId",
+ "lastRequestHash",
+ "prepareRequestHash",
+ "providerClusterId",
+ "providerService",
+ "resultFacts",
+ "schemaVersion",
+ "snapshotVersion",
+ "status",
+ "tombstoned",
+ }
+ if (
+ set(journal) != required
+ or type(journal.get("schemaVersion")) is not int
+ or journal.get("status") not in {"INITIALIZED", "RUNNING", "SUCCEEDED", "FAILED"}
+ or type(journal.get("highestEpoch")) is not int
+ or not 0 < journal.get("highestEpoch", 0) <= _MAX_SIGNED_LONG
+ or type(journal.get("snapshotVersion")) is not int
+ or not 0 < journal.get("snapshotVersion", 0) <= _MAX_SIGNED_LONG
+ or type(journal.get("tombstoned")) is not bool
+ or not isinstance(journal.get("completedSteps"), list)
+ or not isinstance(journal.get("resultFacts"), dict)
+ ):
+ raise ManagedDependencyFailure(
+ "DEPENDENCY_FENCING_UNCERTAIN", "Binding journal is structurally invalid"
+ )
+ try:
+ _canonical_uuid(
+ journal["initializationAuthorizationId"], "initializationAuthorizationId"
+ )
+ _canonical_uuid(
+ journal["initializationChallenge"], "initializationChallenge"
+ )
+ _canonical_uuid(journal["lastOperationId"], "lastOperationId")
+ _require_hash(journal["prepareRequestHash"], "prepareRequestHash")
+ _require_hash(journal["lastRequestHash"], "lastRequestHash")
+ _positive_integer(journal["providerClusterId"], "providerClusterId")
+ _positive_integer(journal["actionHostId"], "actionHostId")
+ if journal["lastCommandName"] not in _PROVIDER_COMMANDS:
+ raise ValueError("invalid last command")
+ if (
+ any(
+ not isinstance(step, str) or not step or len(step) > 64
+ for step in journal["completedSteps"]
+ )
+ or len(set(journal["completedSteps"])) != len(journal["completedSteps"])
+ or any(
+ not isinstance(key, str) or not isinstance(value, str)
+ for key, value in journal["resultFacts"].items()
+ )
+ ):
+ raise ValueError("invalid journal progress or facts")
+ except (ManagedDependencyFailure, ValueError) as error:
+ raise ManagedDependencyFailure(
+ "DEPENDENCY_FENCING_UNCERTAIN", "Binding journal is structurally invalid"
+ ) from error
+ expected = {
+ "schemaVersion": JOURNAL_SCHEMA_VERSION,
+ "bindingId": command.binding_id,
+ "providerClusterId": command.parameters["provider.cluster.id"],
+ "providerService": command.provider_service,
+ "actionHostId": command.action_host_id,
+ }
+ if any(journal.get(key) != value for key, value in expected.items()):
+ raise ManagedDependencyFailure(
+ "DEPENDENCY_FENCING_UNCERTAIN",
+ "Binding journal identity or pinned action host does not match",
+ )
+
+ @staticmethod
+ def _initialization_facts(command):
+ return {
+ "initialization.authorization.id": command.parameters[
+ "initialization.authorization.id"
+ ],
+ "journal.initialized": "true",
+ "provider.action.host.id": command.action_host_id,
+ }
+
+
+class _JournalProgress:
+ def __init__(self, store, journal):
+ self.store = store
+ self.journal = journal
+
+ @property
+ def completed_steps(self):
+ return tuple(self.journal["completedSteps"])
+
+ def checkpoint(self, step):
+ if not isinstance(step, str) or not step or len(step) > 64:
+ _fail_invalid("Journal checkpoint name is invalid")
+ if step not in self.journal["completedSteps"]:
+ self.journal["completedSteps"].append(step)
+ self.store.write(self.store.journal_name, self.journal)
+
+
+class _JournalStore:
+ def __init__(self, root, service, binding_id, expected_uid):
+ self.root = root
+ self.expected_uid = expected_uid
+ prefix = f"{service.lower()}-{binding_id}"
+ self.lock_name = prefix + ".lock"
+ self.pending_name = prefix + ".pending.json"
+ self.journal_name = prefix + ".journal.json"
+ self.directory_descriptor = None
+ self.lock_descriptor = None
+
+ @contextmanager
+ def locked(self):
+ self._open_root()
+ try:
+ self.lock_descriptor = self._open_secure(
+ self.lock_name, os.O_RDWR | os.O_CREAT, 0o600
+ )
+ fcntl.flock(self.lock_descriptor, fcntl.LOCK_EX)
+ yield self
+ finally:
+ if self.lock_descriptor is not None:
+ try:
+ fcntl.flock(self.lock_descriptor, fcntl.LOCK_UN)
+ finally:
+ os.close(self.lock_descriptor)
+ self.lock_descriptor = None
+ if self.directory_descriptor is not None:
+ os.close(self.directory_descriptor)
+ self.directory_descriptor = None
+
+ def exists(self, name):
+ try:
+ file_stat = os.stat(name, dir_fd=self.directory_descriptor, follow_symlinks=False)
+ except FileNotFoundError:
+ return False
+ self._validate_file_stat(file_stat)
+ return True
+
+ def read_optional(self, name):
+ try:
+ descriptor = self._open_secure(name, os.O_RDONLY)
+ except FileNotFoundError:
+ return None
+ try:
+ chunks = []
+ remaining = 65537
+ while remaining:
+ chunk = os.read(descriptor, min(remaining, 8192))
+ if not chunk:
+ break
+ chunks.append(chunk)
+ remaining -= len(chunk)
+ if remaining == 0:
+ raise ValueError("journal record exceeds 64 KiB")
+ value = json.loads(b"".join(chunks).decode("utf-8"))
+ if not isinstance(value, dict):
+ raise ValueError("journal record is not an object")
+ return value
+ except (UnicodeDecodeError, ValueError, json.JSONDecodeError) as error:
+ raise ManagedDependencyFailure(
+ "DEPENDENCY_FENCING_UNCERTAIN",
+ "Binding journal is unreadable or corrupt",
+ ) from error
+ finally:
+ os.close(descriptor)
+
+ def write(self, name, value):
+ encoded = (json.dumps(value, sort_keys=True, separators=(",", ":")) + "\n").encode(
+ "utf-8"
+ )
+ if len(encoded) > 65536:
+ raise ManagedDependencyFailure(
+ "DEPENDENCY_FENCING_UNCERTAIN", "Binding journal exceeds 64 KiB"
+ )
+ temporary_name = f".{name}.{secrets.token_hex(12)}.tmp"
+ descriptor = None
+ try:
+ descriptor = self._open_secure(
+ temporary_name, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600
+ )
+ view = memoryview(encoded)
+ while view:
+ written = os.write(descriptor, view)
+ if written <= 0:
+ raise OSError("journal write made no progress")
+ view = view[written:]
+ os.fsync(descriptor)
+ os.close(descriptor)
+ descriptor = None
+ os.replace(
+ temporary_name,
+ name,
+ src_dir_fd=self.directory_descriptor,
+ dst_dir_fd=self.directory_descriptor,
+ )
+ os.fsync(self.directory_descriptor)
+ finally:
+ if descriptor is not None:
+ os.close(descriptor)
+ try:
+ os.unlink(temporary_name, dir_fd=self.directory_descriptor)
+ except FileNotFoundError:
+ pass
+
+ def rename(self, source, destination):
+ os.replace(
+ source,
+ destination,
+ src_dir_fd=self.directory_descriptor,
+ dst_dir_fd=self.directory_descriptor,
+ )
+ os.fsync(self.directory_descriptor)
+
+ def _open_root(self):
+ parent, name = os.path.split(self.root)
+ parent_descriptor = None
+ try:
+ flags = os.O_RDONLY | os.O_DIRECTORY | os.O_CLOEXEC
+ if not hasattr(os, "O_NOFOLLOW"):
+ raise ManagedDependencyFailure(
+ "DEPENDENCY_FENCING_UNCERTAIN", "Journal storage requires O_NOFOLLOW"
+ )
+ parent_descriptor = os.open(parent, flags | os.O_NOFOLLOW)
+ parent_stat = os.fstat(parent_descriptor)
+ if (
+ not stat.S_ISDIR(parent_stat.st_mode)
+ or parent_stat.st_uid != self.expected_uid
+ or stat.S_IMODE(parent_stat.st_mode) & 0o022
+ ):
+ raise ManagedDependencyFailure(
+ "DEPENDENCY_FENCING_UNCERTAIN",
+ "Journal parent must be owned by the agent user and not group/world writable",
+ )
+ try:
+ os.mkdir(name, 0o700, dir_fd=parent_descriptor)
+ os.fsync(parent_descriptor)
+ except FileExistsError:
+ pass
+ self.directory_descriptor = os.open(
+ name, flags | os.O_NOFOLLOW, dir_fd=parent_descriptor
+ )
+ except ManagedDependencyFailure:
+ raise
+ except OSError as error:
+ raise ManagedDependencyFailure(
+ "DEPENDENCY_FENCING_UNCERTAIN", "Journal root is missing or unsafe"
+ ) from error
+ finally:
+ if parent_descriptor is not None:
+ os.close(parent_descriptor)
+ root_stat = os.fstat(self.directory_descriptor)
+ if (
+ not stat.S_ISDIR(root_stat.st_mode)
+ or root_stat.st_uid != self.expected_uid
+ or stat.S_IMODE(root_stat.st_mode) != 0o700
+ ):
+ os.close(self.directory_descriptor)
+ self.directory_descriptor = None
+ raise ManagedDependencyFailure(
+ "DEPENDENCY_FENCING_UNCERTAIN",
+ "Journal root must be owned by the agent user with mode 0700",
+ )
+
+ def _open_secure(self, name, flags, mode=0o600):
+ try:
+ descriptor = os.open(
+ name,
+ flags | os.O_CLOEXEC | os.O_NOFOLLOW,
+ mode,
+ dir_fd=self.directory_descriptor,
+ )
+ except FileNotFoundError:
+ raise
+ except OSError as error:
+ raise ManagedDependencyFailure(
+ "DEPENDENCY_FENCING_UNCERTAIN", "Journal or lock path is unsafe"
+ ) from error
+ try:
+ self._validate_file_stat(os.fstat(descriptor))
+ return descriptor
+ except BaseException:
+ os.close(descriptor)
+ raise
+
+ def _validate_file_stat(self, file_stat):
+ if (
+ not stat.S_ISREG(file_stat.st_mode)
+ or file_stat.st_uid != self.expected_uid
+ or stat.S_IMODE(file_stat.st_mode) != 0o600
+ or file_stat.st_nlink != 1
+ ):
+ raise ManagedDependencyFailure(
+ "DEPENDENCY_FENCING_UNCERTAIN",
+ "Journal or lock file ownership, mode, or link count is unsafe",
+ )
+
+
+def _parse_client_config(value):
+ try:
+ encoded_size = len(value.encode("utf-8")) if isinstance(value, str) else -1
+ except UnicodeEncodeError as error:
+ raise ManagedDependencyFailure(
+ "DEPENDENCY_COMMAND_INVALID", "client.config.json contains invalid Unicode"
+ ) from error
+ if encoded_size < 0 or encoded_size > _MAX_CLIENT_CONFIG_BYTES:
+ _fail_invalid("client.config.json is missing or exceeds the size limit")
+ _require_json_nesting(value, 2)
+ try:
+ parsed = json.loads(value)
+ except (TypeError, ValueError, UnicodeDecodeError, RecursionError, json.JSONDecodeError) as error:
+ raise ManagedDependencyFailure(
+ "DEPENDENCY_COMMAND_INVALID", "client.config.json is not valid JSON"
+ ) from error
+ expected = {"coreSite", "hdfsSite", "zooKeeperClient"}
+ if not isinstance(parsed, dict) or set(parsed) != expected:
+ _fail_invalid("client.config.json must contain exactly the approved config maps")
+ for field in expected:
+ config = parsed[field]
+ if (
+ not isinstance(config, dict)
+ or len(config) > 64
+ or any(
+ not isinstance(key, str)
+ or not isinstance(item, str)
+ or len(key) > 256
+ or len(item.encode("utf-8"))
+ > (_MAX_CONSUMER_RULE_BYTES if field == "coreSite" and key == _AUTH_TO_LOCAL else 4096)
+ for key, item in config.items()
+ )
+ ):
+ _fail_invalid(f"{field} must contain only bounded string properties")
+ canonical = json.dumps(parsed, sort_keys=True, separators=(",", ":"), ensure_ascii=False)
+ if value != canonical:
+ _fail_invalid("client.config.json must use canonical sorted compact JSON")
+ return parsed
+
+
+def _require_consumer_mapping_rules(value):
+ try:
+ encoded = value.encode("utf-8") if isinstance(value, str) else b""
+ except UnicodeEncodeError as error:
+ raise ManagedDependencyFailure(
+ "DEPENDENCY_COMMAND_INVALID", "Consumer auth-to-local rules contain invalid Unicode"
+ ) from error
+ rules = value.split("\n") if isinstance(value, str) else []
+ if (
+ not encoded
+ or len(encoded) > _MAX_CONSUMER_RULE_BYTES
+ or "\r" in value
+ or value.startswith("\n")
+ or value.endswith("\n")
+ or len(rules) > _MAX_CONSUMER_RULE_COUNT
+ or any(not rule for rule in rules)
+ ):
+ _fail_invalid("Consumer auth-to-local rules are empty, noncanonical, or too large")
+ return value
+
+
+def _require_json_nesting(value, maximum_depth):
+ depth = 0
+ quoted = False
+ escaped = False
+ for character in value:
+ if quoted:
+ if escaped:
+ escaped = False
+ elif character == "\\":
+ escaped = True
+ elif character == '"':
+ quoted = False
+ elif character == '"':
+ quoted = True
+ elif character in "[{":
+ depth += 1
+ if depth > maximum_depth:
+ _fail_invalid("client.config.json exceeds the supported nesting depth")
+ elif character in "]}":
+ depth -= 1
+
+
+def _allowed_hdfs_client_keys(config):
+ name_service = config.get("dfs.nameservices")
+ if name_service is None:
+ return _HDFS_SITE_KEYS
+ if _HA_TOKEN.fullmatch(name_service) is None:
+ _fail_invalid("dfs.nameservices must identify one approved HA nameservice")
+ ids_key = f"dfs.ha.namenodes.{name_service}"
+ name_node_ids = config.get(ids_key, "").split(",")
+ if len(name_node_ids) < 2 or any(
+ _HA_TOKEN.fullmatch(name_node_id) is None for name_node_id in name_node_ids
+ ):
+ _fail_invalid("HA client config contains invalid NameNode IDs")
+ dynamic = {ids_key}
+ dynamic.update(
+ f"dfs.namenode.rpc-address.{name_service}.{name_node_id}"
+ for name_node_id in name_node_ids
+ )
+ provider_key = f"dfs.client.failover.proxy.provider.{name_service}"
+ dynamic.add(provider_key)
+ if (
+ not dynamic.issubset(config)
+ or config.get(provider_key) != _STANDARD_HA_PROVIDER
+ ):
+ _fail_invalid("HA client config is incomplete or uses an unsupported failover provider")
+ return _HDFS_SITE_KEYS | dynamic
+
+
+def _validate_hdfs_client_values(security_mode, core_site, hdfs_site):
+ _validate_enum(
+ core_site,
+ "hadoop.security.authentication",
+ {"simple", "kerberos"},
+ )
+ _validate_enum(
+ core_site,
+ "hadoop.rpc.protection",
+ {"authentication", "integrity", "privacy"},
+ )
+ _validate_boolean(core_site, "hadoop.security.authorization")
+ _validate_boolean(core_site, "ipc.client.fallback-to-simple-auth-allowed")
+ _validate_integer(core_site, "ipc.client.connect.max.retries", 0, 50)
+ _validate_integer(core_site, "ipc.client.connect.retry.interval", 100, 60_000)
+ _validate_boolean(hdfs_site, "dfs.block.access.token.enable")
+ _validate_integer(hdfs_site, "dfs.client.failover.connection.retries", 0, 100)
+ _validate_integer(
+ hdfs_site, "dfs.client.failover.connection.retries.on.timeouts", 0, 100
+ )
+ _validate_integer(hdfs_site, "dfs.client.socket-timeout", 1_000, 600_000)
+ _validate_boolean(hdfs_site, "dfs.client.use.datanode.hostname")
+ _validate_boolean(hdfs_site, "dfs.encrypt.data.transfer")
+ protection = hdfs_site.get("dfs.data.transfer.protection")
+ if protection is not None and any(
+ value.strip() not in {"authentication", "integrity", "privacy"}
+ for value in protection.split(",")
+ ):
+ _fail_invalid("dfs.data.transfer.protection contains an unsupported value")
+ if security_mode == "KERBEROS":
+ if (
+ core_site.get("hadoop.security.authentication") != "kerberos"
+ or core_site.get("ipc.client.fallback-to-simple-auth-allowed") != "false"
+ or not hdfs_site.get("dfs.namenode.kerberos.principal")
+ or not hdfs_site.get("dfs.datanode.kerberos.principal")
+ ):
+ _fail_invalid(
+ "Secure HDFS client config requires Kerberos principals and no simple fallback"
+ )
+ elif core_site.get("hadoop.security.authentication") == "kerberos":
+ _fail_invalid("Insecure HDFS client config cannot enable Kerberos")
+
+
+def _validate_zookeeper_client_values(security_mode, config):
+ _validate_boolean(config, "zookeeper.sasl.client")
+ context = config.get("zookeeper.sasl.clientconfig")
+ if context is not None and context != "Client":
+ _fail_invalid("zookeeper.sasl.clientconfig must use Client")
+ service = config.get("zookeeper.sasl.client.username")
+ if service is not None and _SAFE_IDENTITY.fullmatch(service) is None:
+ _fail_invalid("zookeeper.sasl.client.username is invalid")
+ if security_mode == "KERBEROS" and (
+ config.get("zookeeper.sasl.client") != "true"
+ or not service
+ or context != "Client"
+ ):
+ _fail_invalid("Secure ZooKeeper client config requires exact SASL settings")
+ if security_mode == "INSECURE" and config.get("zookeeper.sasl.client") == "true":
+ _fail_invalid("Insecure ZooKeeper client config cannot enable SASL")
+
+
+def _validate_enum(values, key, allowed):
+ value = values.get(key)
+ if value is not None and value not in allowed:
+ _fail_invalid(f"{key} has an unsupported value")
+
+
+def _validate_boolean(values, key):
+ _validate_enum(values, key, {"true", "false"})
+
+
+def _validate_integer(values, key, minimum, maximum):
+ value = values.get(key)
+ if value is None:
+ return
+ if not value.isdigit() or str(int(value)) != value or not minimum <= int(value) <= maximum:
+ _fail_invalid(f"{key} must be a canonical integer in the supported range")
+
+
+def _validate_hdfs_ownership(parameters):
+ _require_nonblank(parameters["owner.user"], "owner.user")
+ group = _require_nonblank(parameters["owner.group"], "owner.group")
+ mode = parameters["directory.mode"]
+ if mode != "0700" and not (mode == "0750" and group != "hadoop"):
+ _fail_invalid("HDFS ownership must enforce private per-consumer authority")
+
+
+def _validate_zookeeper_policies(parameters, binding_id):
+ _require_nonblank(parameters["owner.user"], "owner.user")
+ if parameters["namespace.parent.znode"] != ZOOKEEPER_PARENT:
+ _fail_invalid("ZooKeeper parent is not reserved for managed HBase")
+ client_port = _positive_integer(parameters["expected.client.port"], "expected.client.port")
+ if client_port > 65535:
+ _fail_invalid("expected.client.port must be between 1 and 65535")
+ quorum = _require_nonblank(parameters["expected.quorum"], "expected.quorum")
+ hosts = quorum.split(",")
+ if (
+ any(_HA_TOKEN.fullmatch(host) is None for host in hosts)
+ or len(hosts) != len(set(hosts))
+ or hosts != sorted(hosts)
+ ):
+ _fail_invalid("expected.quorum must contain canonical unique sorted hosts")
+ secure = parameters["security.mode"] == "KERBEROS"
+ expected_parent = "PROVIDER_ADMIN_CREATE_DELETE" if secure else "INSECURE_PROVIDER_PREPARED"
+ expected_subtree = "SASL_CONSUMER_PRIVATE" if secure else "INSECURE_BINDING_SCOPED"
+ if (
+ parameters["parent.acl.policy"] != expected_parent
+ or parameters["subtree.acl.policy"] != expected_subtree
+ ):
+ _fail_invalid("ZooKeeper ACL policies do not match security.mode")
+ sasl_id = parameters.get("consumer.zk.sasl.id")
+ ledger = parameters.get("namespace.ledger.znode")
+ if secure:
+ _require_identity(sasl_id, "consumer.zk.sasl.id")
+ if (
+ sasl_id != parameters["owner.user"]
+ or ledger != f"{ZOOKEEPER_LEDGER_PARENT}/{binding_id}"
+ ):
+ _fail_invalid("Secure ZooKeeper handoff identity or ledger is invalid")
+ elif sasl_id is not None or ledger is not None:
+ _fail_invalid("Insecure ZooKeeper commands carry secure handoff fields")
+
+
+def _validate_hdfs_namespaces(binding_id, root_value, wal_value):
+ root = _parse_hdfs_uri(root_value, "namespace.root.uri")
+ wal = _parse_hdfs_uri(wal_value, "namespace.wal.uri")
+ expected = f"{HDFS_BASE}/{binding_id}"
+ if root.path != expected + "/root" or wal.path != expected + "/wal":
+ _fail_invalid("HDFS namespaces do not match the binding UUID")
+ if root.netloc != wal.netloc:
+ _fail_invalid("HDFS root and WAL authorities differ")
+ return root.netloc
+
+
+def _parse_hdfs_uri(value, field):
+ value = _require_nonblank(value, field)
+ if "%" in value or "\\" in value or any(character.isspace() for character in value):
+ _fail_invalid(f"{field} is not a normalized HDFS URI")
+ try:
+ parsed = urlsplit(value)
+ port = parsed.port
+ except ValueError as error:
+ raise ManagedDependencyFailure(
+ "DEPENDENCY_COMMAND_INVALID", f"{field} is not a valid HDFS URI"
+ ) from error
+ if (
+ parsed.scheme != "hdfs"
+ or not parsed.netloc
+ or parsed.username is not None
+ or parsed.password is not None
+ or parsed.query
+ or parsed.fragment
+ or any(segment in {".", ".."} for segment in parsed.path.split("/"))
+ or port is not None and not 0 < port <= 65535
+ ):
+ _fail_invalid(f"{field} is not a normalized credential-free HDFS URI")
+ return parsed
+
+
+def _append_java_strings(target, *values):
+ for value in values:
+ if not isinstance(value, str):
+ _fail_invalid("Canonical request hash fields must be strings")
+ try:
+ java_length = len(value.encode("utf-16-be")) // 2
+ value.encode("utf-8")
+ except UnicodeEncodeError as error:
+ raise ManagedDependencyFailure(
+ "DEPENDENCY_COMMAND_INVALID", "Command contains an invalid Unicode value"
+ ) from error
+ target.append(f"{java_length}:{value}")
+
+
+def _canonical_uuid(value, field):
+ if not isinstance(value, str):
+ _fail_invalid(f"{field} must be a canonical UUID string")
+ try:
+ parsed = uuid.UUID(value)
+ except (ValueError, AttributeError) as error:
+ raise ManagedDependencyFailure(
+ "DEPENDENCY_COMMAND_INVALID", f"{field} must be a canonical UUID string"
+ ) from error
+ canonical = str(parsed)
+ if value != canonical:
+ _fail_invalid(f"{field} must be a canonical lower-case UUID string")
+ return canonical
+
+
+def _positive_integer(value, field):
+ if isinstance(value, bool):
+ _fail_invalid(f"{field} must be a positive integer")
+ if isinstance(value, int):
+ if value <= 0 or value > _MAX_SIGNED_LONG:
+ _fail_invalid(f"{field} must be a positive integer")
+ return value
+ if not isinstance(value, str) or _POSITIVE_INTEGER.fullmatch(value) is None:
+ _fail_invalid(f"{field} must be a positive integer")
+ result = int(value)
+ if result > _MAX_SIGNED_LONG:
+ _fail_invalid(f"{field} exceeds the supported integer range")
+ return result
+
+
+def _require_hash(value, field):
+ if not isinstance(value, str) or _HASH.fullmatch(value) is None:
+ _fail_invalid(f"{field} must use sha256:<64 lower-case hex>")
+ return value
+
+
+def _require_nonblank(value, field):
+ if not isinstance(value, str) or not value or value != value.strip():
+ _fail_invalid(f"{field} must be a nonblank normalized string")
+ return value
+
+
+def _require_package_version(value, field):
+ _require_nonblank(value, field)
+ if len(value) > 512 or any(ord(character) < 32 or ord(character) == 127 for character in value):
+ _fail_invalid(f"{field} is not a bounded single-line package-manager version")
+ return value
+
+
+def _require_software_version(value):
+ _require_nonblank(value, "client.software.semantic.version")
+ if len(value) > 128 or _SOFTWARE_VERSION.fullmatch(value) is None:
+ _fail_invalid("client.software.semantic.version is not a supported semantic version")
+ return value
+
+
+def _require_identity(value, field):
+ _require_nonblank(value, field)
+ if _SAFE_IDENTITY.fullmatch(value) is None:
+ _fail_invalid(f"{field} must be a bounded simple authorization identity")
+ return value
+
+
+def _require_principal_pattern(value, consumer_user):
+ _require_nonblank(value, "consumer.kerberos.principal.pattern")
+ _require_identity(consumer_user, "consumer.user")
+ prefix = f"{consumer_user}/_HOST@"
+ realm = value[len(prefix) :] if value.startswith(prefix) else ""
+ if len(value) > 512 or _KERBEROS_REALM.fullmatch(realm) is None:
+ _fail_invalid(
+ "consumer.kerberos.principal.pattern must match consumer.user/_HOST@REALM"
+ )
+ return value
+
+
+def _fail_invalid(message):
+ raise ManagedDependencyFailure("DEPENDENCY_COMMAND_INVALID", message)
diff --git a/ambari-common/src/main/python/resource_management/libraries/functions/managed_dependency_client.py b/ambari-common/src/main/python/resource_management/libraries/functions/managed_dependency_client.py
new file mode 100644
index 0000000000..5d2c0b2160
--- /dev/null
+++ b/ambari-common/src/main/python/resource_management/libraries/functions/managed_dependency_client.py
@@ -0,0 +1,106 @@
+#!/usr/bin/env python3
+"""
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+"""
+
+import json
+import subprocess
+
+from resource_management.core.signal_utils import TerminateStrategy
+from resource_management.libraries.functions.managed_dependency import ManagedDependencyFailure
+
+
+CLIENT_HELPER_JAR = "/var/lib/ambari-agent/tools/zkmigrator.jar"
+CLIENT_HELPER_CLASS = "org.apache.ambari.tools.hadoop.ManagedDependencyClient"
+ERROR_CODES = {
+ "DEPENDENCY_AUTHORIZATION_FAILED",
+ "DEPENDENCY_DATA_INTEGRITY_FAILED",
+ "DEPENDENCY_DATANODE_READ_WRITE_FAILED",
+ "DEPENDENCY_PROBE_CLEANUP_FAILED",
+ "DEPENDENCY_CLIENT_OBSERVATION_FAILED",
+}
+RESULT_FIELDS = {
+ "VERSION": {"kind": str, "version": str},
+ "ACL": {"entries": list},
+ "COUNT": {"empty": bool},
+ "MKDIR": {"created": bool},
+ "SET_OWNER": {"applied": bool},
+ "SET_PERMISSION": {"applied": bool},
+ "READ_MARKER": {"marker": dict},
+ "WRITE_MARKER": {"written": bool},
+ "PROBE": {"readWriteVerified": bool, "bytes": int, "cleaned": bool},
+}
+
+
+def observe_client(call, executable, config_dir, command, operation, payload,
+ user, environment, timeout=60, error_code="DEPENDENCY_CLIENT_OBSERVATION_FAILED"):
+ identity = dict(command.envelope)
+ arguments = (executable, "--config", config_dir, CLIENT_HELPER_CLASS,
+ operation, config_dir, json.dumps(identity, sort_keys=True),
+ json.dumps(payload, sort_keys=True))
+ # Use the selected distribution's launcher so its SDK precedes the Agent helper.
+ environment = {**(environment or {}), "HADOOP_CLASSPATH": CLIENT_HELPER_JAR,
+ "HBASE_CLASSPATH": CLIENT_HELPER_JAR}
+ try:
+ code, output = call(arguments, user=user, env=environment, stderr=subprocess.PIPE,
+ timeout=timeout, timeout_kill_strategy=TerminateStrategy.KILL_PROCESS_GROUP,
+ shell=False, quiet=True)[:2]
+ if code != 0 or not isinstance(output, str) or len(output.encode("utf-8")) > 32 * 1024:
+ raise ValueError("Invalid client process result")
+ result = json.loads(output)
+ if (not isinstance(result, dict) or type(result.get("schemaVersion")) is not int
+ or result["schemaVersion"] != 1 or json.dumps(result.get("identity"), sort_keys=True) != json.dumps(identity, sort_keys=True)
+ or result.get("operation") != operation):
+ raise ValueError("Invalid client response identity or schema")
+ if result.get("status") == "FAILED":
+ failure_code = result.get("errorCode")
+ if failure_code not in ERROR_CODES:
+ raise ValueError("Invalid client error code")
+ raise ManagedDependencyFailure(
+ error_code if failure_code == "DEPENDENCY_CLIENT_OBSERVATION_FAILED" else failure_code,
+ "The managed dependency client API could not complete the observation")
+ if result.get("status") != "SUCCEEDED" or "errorCode" in result:
+ raise ValueError("Invalid client response status")
+ facts = result.get("result")
+ fields = RESULT_FIELDS.get(operation)
+ if operation == "STAT":
+ if not isinstance(facts, dict) or type(facts.get("exists")) is not bool:
+ raise ValueError("Invalid client file status")
+ fields = {"exists": bool}
+ if facts["exists"]:
+ fields = {**fields, "type": str, "owner": str, "group": str, "mode": int}
+ if fields is None or not isinstance(facts, dict) or set(facts) != set(fields):
+ raise ValueError("Invalid client observation fields")
+ if any(type(facts[key]) is not expected for key, expected in fields.items()):
+ raise ValueError("Invalid client observation types")
+ if operation == "STAT" and facts["exists"]:
+ if facts["type"] not in {"DIRECTORY", "FILE", "SYMLINK"} or not 0 <= facts["mode"] <= 0o7777:
+ raise ValueError("Invalid file type or permission bits")
+ if operation == "ACL":
+ for entry in facts["entries"]:
+ if (not isinstance(entry, dict) or set(entry) != {"scope", "type", "name", "permission"}
+ or entry["scope"] not in {"ACCESS", "DEFAULT"}
+ or entry["type"] not in {"USER", "GROUP", "MASK", "OTHER"}
+ or type(entry["name"]) is not str or type(entry["permission"]) is not int
+ or not 0 <= entry["permission"] <= 7):
+ raise ValueError("Invalid typed ACL entry")
+ return facts
+ except ManagedDependencyFailure:
+ raise
+ except Exception as error:
+ raise ManagedDependencyFailure(error_code,
+ "The managed dependency client returned no valid structured observation") from error
diff --git a/ambari-server/pom.xml b/ambari-server/pom.xml
index 30a05a95c3..ddd5656b7b 100644
--- a/ambari-server/pom.xml
+++ b/ambari-server/pom.xml
@@ -68,7 +68,7 @@
<mockito.inline.version>3.5.10</mockito.inline.version>
<objenesis.tck.version>1.2</objenesis.tck.version>
<cglib.version>3.3.0</cglib.version>
- <gson.version>2.9.0</gson.version>
+ <gson.version>2.11.0</gson.version>
<jsr305.version>1.3.9</jsr305.version>
<quartz.version>2.3.2</quartz.version>
<quartz.jobs.version>2.3.2</quartz.jobs.version>
@@ -772,6 +772,7 @@
--add-opens java.base/java.util=ALL-UNNAMED
--add-opens java.base/java.util.regex=ALL-UNNAMED
--add-opens java.base/java.util.concurrent=ALL-UNNAMED
+ --add-opens java.base/java.util.concurrent.atomic=ALL-UNNAMED
--add-opens java.base/java.net=ALL-UNNAMED
--add-opens java.base/java.util.stream=ALL-UNNAMED
--add-opens java.base/java.math=ALL-UNNAMED
@@ -1001,6 +1002,21 @@
</excludes>
</resource>
</resources>
+ <testResources>
+ <testResource>
+ <directory>src/test/resources</directory>
+ </testResource>
+ <testResource>
+ <directory>src/main/resources</directory>
+ <targetPath>managed-dependency-fixtures</targetPath>
+ <includes>
+ <include>stacks/BIGTOP/3.2.0/services/HBASE/kerberos.json</include>
+ <include>stacks/BIGTOP/3.2.0/services/ZOOKEEPER/kerberos.json</include>
+ <include>stacks/BIGTOP/3.2.0/services/KERBEROS/configuration/krb5-conf.xml</include>
+ <include>stacks/BIGTOP/3.2.0/services/KERBEROS/properties/krb5_conf.j2</include>
+ </includes>
+ </testResource>
+ </testResources>
</build>
<profiles>
<profile>
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java b/ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java
index bb376f4f57..51576717e8 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java
@@ -30,6 +30,8 @@ import java.util.concurrent.TimeUnit;
import java.util.concurrent.locks.ReadWriteLock;
import java.util.concurrent.locks.ReentrantReadWriteLock;
+import jakarta.persistence.EntityManager;
+
import org.apache.ambari.annotations.TransactionalLock;
import org.apache.ambari.annotations.TransactionalLock.LockArea;
import org.apache.ambari.annotations.TransactionalLock.LockType;
@@ -42,6 +44,8 @@ import org.apache.ambari.server.audit.event.AuditEvent;
import org.apache.ambari.server.audit.event.OperationStatusAuditEvent;
import org.apache.ambari.server.audit.event.TaskStatusAuditEvent;
import org.apache.ambari.server.configuration.Configuration;
+import org.apache.ambari.server.controller.dependencies.ManagedDependencyCredentialManager;
+import org.apache.ambari.server.controller.dependencies.ManagedDependencyRuntimePlanner;
import org.apache.ambari.server.controller.internal.CalculatedStatus;
import org.apache.ambari.server.events.HostsRemovedEvent;
import org.apache.ambari.server.events.RequestFinishedEvent;
@@ -79,6 +83,7 @@ import com.google.common.cache.Cache;
import com.google.common.cache.CacheBuilder;
import com.google.common.eventbus.Subscribe;
import com.google.inject.Inject;
+import com.google.inject.Provider;
import com.google.inject.Singleton;
import com.google.inject.name.Named;
import com.google.inject.persist.Transactional;
@@ -128,6 +133,18 @@ public class ActionDBAccessorImpl implements ActionDBAccessor {
@Inject
Configuration configuration;
+ @Inject
+ Provider<ManagedDependencyRuntimePlanner> managedDependencyRuntimePlanner;
+
+ @Inject
+ ManagedDependencyCredentialManager managedDependencyCredentialManager;
+
+ @Inject
+ ActionPersistenceTransaction actionPersistenceTransaction;
+
+ @Inject
+ Provider<EntityManager> entityManagerProvider;
+
@Inject
AmbariEventPublisher ambariEventPublisher;
@@ -333,9 +350,28 @@ public class ActionDBAccessorImpl implements ActionDBAccessor {
}
@Override
- @Transactional
- @TransactionalLock(lockArea = LockArea.HRC_STATUS_CACHE, lockType = LockType.WRITE)
+ @Transactional(rollbackOn = {RuntimeException.class, AmbariException.class})
public void persistActions(Request request) throws AmbariException {
+ try {
+ managedDependencyRuntimePlanner.get().executeWithPreparationParentLocks(request,
+ () -> actionPersistenceTransaction.persist(this, request));
+ } catch (AmbariException e) {
+ markCurrentTransactionRollbackOnly();
+ throw e;
+ }
+ }
+
+ private void markCurrentTransactionRollbackOnly() {
+ if (entityManagerProvider == null) {
+ return;
+ }
+ EntityManager entityManager = entityManagerProvider.get();
+ if (entityManager.getTransaction().isActive()) {
+ entityManager.getTransaction().setRollbackOnly();
+ }
+ }
+
+ void persistActionsInTransaction(Request request) throws AmbariException {
RequestEntity requestEntity = request.constructNewPersistenceEntity();
@@ -378,6 +414,9 @@ public class ActionDBAccessorImpl implements ActionDBAccessor {
hostRoleCommandEntities.add(hostRoleCommandEntity);
hostRoleCommand.setTaskId(hostRoleCommandEntity.getTaskId());
+ managedDependencyRuntimePlanner.get().applyDeferredCustomLifecycleState(hostRoleCommand);
+ managedDependencyRuntimePlanner.get().planPreparationCommands(hostRoleCommand);
+ managedDependencyRuntimePlanner.get().associatePreparationTask(hostRoleCommand);
String prefix = "";
String output = "output-" + hostRoleCommandEntity.getTaskId() + ".txt";
@@ -437,6 +476,8 @@ public class ActionDBAccessorImpl implements ActionDBAccessor {
stageEntity = stageDAO.merge(stageEntity);
}
+ managedDependencyCredentialManager.associate(requestId, hostRoleCommands);
+
requestEntity.setStages(stageEntities);
requestDAO.merge(requestEntity);
@@ -457,6 +498,29 @@ public class ActionDBAccessorImpl implements ActionDBAccessor {
}
}
+ /** Starts action publication after the caller's managed dependency parent locks are held. */
+ @Singleton
+ public static class ActionPersistenceTransaction {
+ @Inject
+ private Provider<EntityManager> entityManagerProvider;
+
+ @Transactional(rollbackOn = {RuntimeException.class, AmbariException.class})
+ @TransactionalLock(lockArea = LockArea.HRC_STATUS_CACHE, lockType = LockType.WRITE)
+ public void persist(ActionDBAccessorImpl accessor, Request request) throws AmbariException {
+ try {
+ accessor.persistActionsInTransaction(request);
+ } catch (AmbariException e) {
+ if (entityManagerProvider != null) {
+ EntityManager entityManager = entityManagerProvider.get();
+ if (entityManager.getTransaction().isActive()) {
+ entityManager.getTransaction().setRollbackOnly();
+ }
+ }
+ throw e;
+ }
+ }
+ }
+
@Override
@Transactional
public void startRequest(long requestId) {
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionManager.java b/ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionManager.java
index 2015e34c0c..5926879c55 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionManager.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionManager.java
@@ -23,10 +23,12 @@ import java.util.Collections;
import java.util.Comparator;
import java.util.List;
import java.util.Map;
+import java.util.Objects;
import java.util.concurrent.atomic.AtomicLong;
import org.apache.ambari.server.AmbariException;
import org.apache.ambari.server.agent.CommandReport;
+import org.apache.ambari.server.agent.ExecutionCommand;
import org.apache.ambari.server.controller.ExecuteActionRequest;
import org.apache.ambari.server.security.authorization.AuthorizationHelper;
import org.apache.ambari.server.topology.TopologyManager;
@@ -138,12 +140,8 @@ public class ActionManager {
return;
}
- Collections.sort(reports, new Comparator<CommandReport>() {
- @Override
- public int compare(CommandReport o1, CommandReport o2) {
- return (int) (o1.getTaskId()-o2.getTaskId());
- }
- });
+ reports = getValidTaskReports(hostname, reports, commands);
+ reports.sort(Comparator.comparingLong(CommandReport::getTaskId));
List<CommandReport> reportsToProcess = new ArrayList<>();
//persist the action response into the db.
for (CommandReport report : reports) {
@@ -166,7 +164,79 @@ public class ActionManager {
reportsToProcess.add(report);
}
- db.updateHostRoleStates(reportsToProcess);
+ if (!reportsToProcess.isEmpty()) {
+ db.updateHostRoleStates(reportsToProcess);
+ }
+ }
+
+ /**
+ * Validates agent-supplied identities before reports reach any state machine.
+ * The hostname must come from the authenticated agent session. Task, stage and
+ * execution-command identities come from persisted server commands, not from
+ * the report's cluster envelope. The returned list does not modify the input.
+ */
+ public List<CommandReport> getValidTaskReports(String hostname, List<CommandReport> reports,
+ Map<Long, HostRoleCommand> commands) {
+ List<CommandReport> accepted = new ArrayList<>();
+ if (reports == null) {
+ return accepted;
+ }
+ for (CommandReport report : reports) {
+ if (report == null) {
+ continue;
+ }
+ HostRoleCommand command = commands.get(report.getTaskId());
+ if (isValidTaskReport(hostname, report, command)) {
+ accepted.add(report);
+ } else {
+ // Do not log untrusted output or serialized execution commands (secrets).
+ LOG.warn("Rejected agent report for task {}: identity or status does not match the persisted task",
+ report.getTaskId());
+ }
+ }
+ return accepted;
+ }
+
+ private boolean isValidTaskReport(String hostname, CommandReport report, HostRoleCommand command) {
+ if (command == null || hostname == null || !hostname.equals(command.getHostName())
+ || command.getTaskId() != report.getTaskId()
+ || command.getRole() == null || !command.getRole().name().equals(report.getRole())
+ || command.getRoleCommand() == null
+ || !command.getRoleCommand().name().equals(report.getRoleCommand())
+ || !StageUtils.getActionId(command.getRequestId(), command.getStageId()).equals(report.getActionId())) {
+ return false;
+ }
+ try {
+ HostRoleStatus status = HostRoleStatus.valueOf(report.getStatus());
+ long clusterId = Long.parseLong(report.getClusterId());
+ Stage stage = getAction(command.getRequestId(), command.getStageId());
+ if (stage == null || stage.getClusterId() != clusterId) {
+ return false;
+ }
+ // Read the stored command without expanding or refreshing configurations.
+ // This also works when a task is loaded after a server restart.
+ ExecutionCommand execution = StageUtils.getGson().fromJson(
+ command.getExecutionCommandWrapper().getJson(), ExecutionCommand.class);
+ if (execution == null || !Long.toString(clusterId).equals(execution.getClusterId())
+ || !Objects.equals(agentServiceName(execution.getServiceName()),
+ agentServiceName(report.getServiceName()))) {
+ return false;
+ }
+ String customCommand = execution.getCommandParams() == null ? null
+ : execution.getCommandParams().get("custom_command");
+ // The agent adds customCommand to execution results, not progress templates.
+ return (report.getCustomCommand() == null && status != HostRoleStatus.COMPLETED)
+ || Objects.equals(customCommand, report.getCustomCommand());
+ } catch (RuntimeException ex) {
+ // Missing/corrupt persisted identity and malformed input both fail closed.
+ // Continue validating siblings; never emit configuration or report secrets.
+ return false;
+ }
+ }
+
+ private static String agentServiceName(String serviceName) {
+ // ActionQueue uses the literal "null" for commands with no service scope.
+ return serviceName == null ? "null" : serviceName;
}
/**
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/actionmanager/HostRoleCommand.java b/ambari-server/src/main/java/org/apache/ambari/server/actionmanager/HostRoleCommand.java
index b236de4938..aac5adb76b 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/actionmanager/HostRoleCommand.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/actionmanager/HostRoleCommand.java
@@ -67,6 +67,7 @@ public class HostRoleCommand {
private RoleCommand roleCommand;
private String commandDetail;
private String customCommandName;
+ private boolean futureCommand;
private ExecutionCommandWrapper executionCommandWrapper;
private boolean isBackgroundCommand = false;
private String opsDisplayName;
@@ -312,6 +313,14 @@ public class HostRoleCommand {
this.customCommandName = customCommandName;
}
+ public boolean isFutureCommand() {
+ return futureCommand;
+ }
+
+ public void setFutureCommand(boolean futureCommand) {
+ this.futureCommand = futureCommand;
+ }
+
public HostRoleStatus getStatus() {
return status;
}
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java b/ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
index 71d719d6cd..97dff603ae 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
@@ -106,6 +106,10 @@ public class ExecutionCommand extends AgentCommand {
@SerializedName("configurations")
private Map<String, Map<String, String>> configurations;
+ @SerializedName("configurationTypeOverrides")
+ @com.fasterxml.jackson.annotation.JsonProperty("configurationTypeOverrides")
+ private Set<String> configurationTypeOverrides = new HashSet<>();
+
@SerializedName("forceRefreshConfigTagsBeforeExecution")
@JsonIgnore
private boolean overrideConfigs = false;
@@ -330,6 +334,15 @@ public class ExecutionCommand extends AgentCommand {
this.configurations = configurations;
}
+ public Set<String> getConfigurationTypeOverrides() {
+ return configurationTypeOverrides;
+ }
+
+ public void setConfigurationTypeOverrides(Set<String> configurationTypeOverrides) {
+ this.configurationTypeOverrides = configurationTypeOverrides == null
+ ? new HashSet<>() : new HashSet<>(configurationTypeOverrides);
+ }
+
/**
* Gets whether configuration tags shoudl be refreshed right before the
* command is scheduled.
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java b/ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java
index 22ed8ca410..9307e374ea 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java
@@ -235,8 +235,9 @@ public class HeartBeatHandler {
}
public void handleCommandReportStatus(List<CommandReport> reports, String hostname) throws AmbariException {
- heartbeatProcessor.processCommandReports(reports, hostname, System.currentTimeMillis());
- heartbeatProcessor.processHostStatus(null, reports, hostname);
+ List<CommandReport> acceptedReports =
+ heartbeatProcessor.processCommandReports(reports, hostname, System.currentTimeMillis());
+ heartbeatProcessor.processHostStatus(null, acceptedReports, hostname);
}
public void handleHostReportStatus(HostStatusReport hostStatusReport, String hostname) throws AmbariException {
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatMonitor.java b/ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatMonitor.java
index d7cf416b95..c11538dc76 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatMonitor.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatMonitor.java
@@ -84,9 +84,18 @@ public class HeartbeatMonitor implements Runnable {
private final AmbariEventPublisher ambariEventPublisher;
private final HostLevelParamsHolder hostLevelParamsHolder;
private final RecoveryTopologyManager recoveryTopologyManager;
+ private final java.util.function.LongSupplier clock;
+ private volatile long monitoringStartedAt;
public HeartbeatMonitor(Clusters clusters, ActionManager am,
int threadWakeupInterval, Injector injector) {
+ this(clusters, am, threadWakeupInterval, injector, System::currentTimeMillis);
+ }
+
+ HeartbeatMonitor(Clusters clusters, ActionManager am, int threadWakeupInterval,
+ Injector injector, java.util.function.LongSupplier clock) {
+ this.clock = clock;
+ monitoringStartedAt = clock.getAsLong();
this.clusters = clusters;
actionManager = am;
this.threadWakeupInterval = threadWakeupInterval;
@@ -107,6 +116,7 @@ public class HeartbeatMonitor implements Runnable {
}
public void start() {
+ monitoringStartedAt = clock.getAsLong();
monitorThread = new Thread(this, "ambari-hearbeat-monitor");
monitorThread.start();
}
@@ -144,9 +154,9 @@ public class HeartbeatMonitor implements Runnable {
//Go through all the nodes, check for last heartbeat or any waiting state
//If heartbeat is lost, update node clusters state, purge the action queue
//notify action manager for node failure.
- private void doWork() throws InvalidStateTransitionException, AmbariException {
+ void doWork() throws InvalidStateTransitionException, AmbariException {
List<Host> allHosts = clusters.getHosts();
- long now = System.currentTimeMillis();
+ long now = clock.getAsLong();
for (Host hostObj : allHosts) {
if (hostObj.getState() == HostState.HEARTBEAT_LOST) {
//do not check if host already known be lost
@@ -161,7 +171,9 @@ public class HeartbeatMonitor implements Runnable {
} catch (AmbariException e) {
LOG.warn("Exception in getting host object; Is it fatal?", e);
}
- if (lastHeartbeat + 2 * threadWakeupInterval < now) {
+ // Heartbeat timestamps are process-local. A restarted monitor must observe
+ // the existing timeout window before declaring an unreconnected host lost.
+ if (Math.max(lastHeartbeat, monitoringStartedAt) + 2L * threadWakeupInterval < now) {
handleHeartbeatLost(hostId);
}
if (hostState == HostState.WAITING_FOR_HOST_STATUS_UPDATES) {
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatProcessor.java b/ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatProcessor.java
index 9ce6a4da00..05c7c087fa 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatProcessor.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatProcessor.java
@@ -46,6 +46,7 @@ import org.apache.ambari.server.agent.stomp.dto.ComponentVersionReport;
import org.apache.ambari.server.agent.stomp.dto.ComponentVersionReports;
import org.apache.ambari.server.api.services.AmbariMetaInfo;
import org.apache.ambari.server.controller.MaintenanceStateHelper;
+import org.apache.ambari.server.controller.dependencies.ManagedDependencyTaskResultProcessor;
import org.apache.ambari.server.events.ActionFinalReportReceivedEvent;
import org.apache.ambari.server.events.AlertEvent;
import org.apache.ambari.server.events.AlertReceivedEvent;
@@ -147,6 +148,9 @@ public class HeartbeatProcessor extends AbstractService{
@Inject
Provider<MetadataHolder> metadataHolder;
+ @Inject
+ ManagedDependencyTaskResultProcessor managedDependencyTaskResultProcessor;
+
@Inject
public HeartbeatProcessor(Clusters clusterFsm, ActionManager am, HeartbeatMonitor heartbeatMonitor,
Injector injector) {
@@ -227,9 +231,9 @@ public class HeartbeatProcessor extends AbstractService{
//process status reports before command reports to prevent status override immediately after task finish
processStatusReports(heartbeat);
- processCommandReports(heartbeat, now);
+ List<CommandReport> acceptedReports = processCommandReports(heartbeat, now);
//host status calculation are based on task and status reports, should be performed last
- processHostStatus(heartbeat);
+ processHostStatus(heartbeat.getComponentStatus(), acceptedReports, heartbeat.getHostname());
}
@@ -327,19 +331,25 @@ public class HeartbeatProcessor extends AbstractService{
* @param now cached current time
* @throws AmbariException
*/
- protected void processCommandReports(HeartBeat heartbeat, long now) throws AmbariException {
- processCommandReports(heartbeat.getReports(), heartbeat.getHostname(), now);
+ protected List<CommandReport> processCommandReports(HeartBeat heartbeat, long now) throws AmbariException {
+ return processCommandReports(heartbeat.getReports(), heartbeat.getHostname(), now);
}
- protected void processCommandReports(List<CommandReport> reports, String hostName, Long now)
+ protected List<CommandReport> processCommandReports(List<CommandReport> reports, String hostName, Long now)
throws AmbariException {
+ if (reports == null) {
+ return new ArrayList<>();
+ }
// Cache HostRoleCommand entities because we will need them few times
List<Long> taskIds = new ArrayList<>();
for (CommandReport report : reports) {
- taskIds.add(report.getTaskId());
+ if (report != null) {
+ taskIds.add(report.getTaskId());
+ }
}
Map<Long, HostRoleCommand> commands = actionManager.getTasksMap(taskIds);
+ reports = actionManager.getValidTaskReports(hostName, reports, commands);
for (CommandReport report : reports) {
@@ -546,6 +556,10 @@ public class HeartbeatProcessor extends AbstractService{
//Update state machines from reports
actionManager.processTaskResponse(hostName, reports, commands);
+ for (CommandReport report : reports) {
+ managedDependencyTaskResultProcessor.process(report, hostName, commands.get(report.getTaskId()));
+ }
+ return reports;
}
/**
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/PrometheusTargetDiscovery.java b/ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/PrometheusTargetDiscovery.java
index 96932f6cf3..dac3d514b3 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/PrometheusTargetDiscovery.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/PrometheusTargetDiscovery.java
@@ -101,7 +101,7 @@ public class PrometheusTargetDiscovery {
ArrayNode groups = JsonNodeFactory.instance.arrayNode();
for (HostAssignment hostAssignment : assignments) {
String address = renderAddress(hostAssignment.host.getHostName());
- groups.add(targetGroup(address, "/metrics", cluster.getClusterName(),
+ groups.add(targetGroup(address, "/metrics", cluster.getClusterId(), cluster.getClusterName(),
hostAssignment.host.getHostName(), null, null, "host"));
JsonNode targets = hostAssignment.event.getAssignment().path("targets");
@@ -114,7 +114,7 @@ public class PrometheusTargetDiscovery {
continue;
}
String routeId = target.path("id").asText();
- groups.add(targetGroup(address, "/metrics/components/" + routeId,
+ groups.add(targetGroup(address, "/metrics/components/" + routeId, cluster.getClusterId(),
cluster.getClusterName(), hostAssignment.host.getHostName(),
target.path("service").asText(), target.path("component").asText(),
"component"));
@@ -129,12 +129,14 @@ public class PrometheusTargetDiscovery {
}
}
- private ObjectNode targetGroup(String address, String metricsPath, String clusterName,
- String hostName, String serviceName, String componentName, String targetType) {
+ private ObjectNode targetGroup(String address, String metricsPath, long clusterId,
+ String clusterName, String hostName, String serviceName, String componentName,
+ String targetType) {
ObjectNode group = JsonNodeFactory.instance.objectNode();
group.putArray("targets").add(address);
ObjectNode labels = group.putObject("labels");
labels.put("__metrics_path__", metricsPath);
+ labels.put("ambari_cluster_id", Long.toString(clusterId));
labels.put("cluster", clusterName);
labels.put("host", hostName);
labels.put("ambari_target", targetType);
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/dto/AlertGroupUpdate.java b/ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/dto/AlertGroupUpdate.java
index 89b61b0a08..f58250d1a6 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/dto/AlertGroupUpdate.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/agent/stomp/dto/AlertGroupUpdate.java
@@ -66,6 +66,11 @@ public class AlertGroupUpdate {
this.id = id;
}
+ public AlertGroupUpdate(Long id, Long clusterId) {
+ this.id = id;
+ this.clusterId = clusterId;
+ }
+
public Long getClusterId() {
return clusterId;
}
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/api/ContentTypeOverrideFilter.java b/ambari-server/src/main/java/org/apache/ambari/server/api/ContentTypeOverrideFilter.java
index 73763845f1..6d8be830ab 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/api/ContentTypeOverrideFilter.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/api/ContentTypeOverrideFilter.java
@@ -40,6 +40,7 @@ import jakarta.ws.rs.Path;
import jakarta.ws.rs.core.HttpHeaders;
import jakarta.ws.rs.core.MediaType;
+import org.glassfish.jersey.uri.UriTemplate;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -151,7 +152,6 @@ public class ContentTypeOverrideFilter implements Filter {
ClassPath classPath = ClassPath.from(ClassLoader.getSystemClassLoader());
ImmutableSet<ClassPath.ClassInfo> classes = classPath.getTopLevelClassesRecursive("org.apache.ambari.server.api");
- restart:
for (ClassPath.ClassInfo classInfo: classes) {
Class<?> clazz = classInfo.load();
if (clazz.isAnnotationPresent(Path.class)) {
@@ -161,8 +161,14 @@ public class ContentTypeOverrideFilter implements Filter {
Consumes consumesAnnotation = method.getAnnotation(Consumes.class);
for (String consume : consumesAnnotation.value()) {
if (MediaType.APPLICATION_JSON.equals(consume)) {
- excludedUrls.add(Pattern.compile(path.value()));
- continue restart;
+ Path methodPath = method.getAnnotation(Path.class);
+ String resourcePath = path.value();
+ if (methodPath != null) {
+ resourcePath = resourcePath.replaceAll("/+$", "") + "/"
+ + methodPath.value().replaceAll("^/+", "");
+ }
+ // Use the same path-template syntax as Jersey, including parameters.
+ excludedUrls.add(Pattern.compile(new UriTemplate(resourcePath).getPattern().getRegex()));
}
}
}
@@ -180,7 +186,7 @@ public class ContentTypeOverrideFilter implements Filter {
}
private boolean isUrlExcluded(String pathInfo) {
- return excludedUrls.stream().anyMatch(p -> p.matcher(pathInfo).matches());
+ return pathInfo != null && excludedUrls.stream().anyMatch(p -> p.matcher(pathInfo).matches());
}
@Override
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/api/handlers/DeleteHandler.java b/ambari-server/src/main/java/org/apache/ambari/server/api/handlers/DeleteHandler.java
index f8b0a29ef8..6177b6adce 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/api/handlers/DeleteHandler.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/api/handlers/DeleteHandler.java
@@ -55,6 +55,7 @@ public class DeleteHandler extends BaseManagementHandler implements RequestHandl
}
} catch (AuthorizationException e) {
result = new ResultImpl(new ResultStatus(ResultStatus.STATUS.FORBIDDEN, e.getMessage()));
+
} catch (SystemException e) {
result = new ResultImpl(new ResultStatus(ResultStatus.STATUS.SERVER_ERROR, e));
} catch (NoSuchParentResourceException e) {
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/api/services/BaseService.java b/ambari-server/src/main/java/org/apache/ambari/server/api/services/BaseService.java
index 5495cc21a2..d57c9d194d 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/api/services/BaseService.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/api/services/BaseService.java
@@ -37,6 +37,7 @@ import org.apache.ambari.server.api.services.serializers.CsvSerializer;
import org.apache.ambari.server.api.services.serializers.JsonSerializer;
import org.apache.ambari.server.api.services.serializers.ResultSerializer;
import org.apache.ambari.server.audit.request.RequestAuditLogger;
+import org.apache.ambari.server.controller.dependencies.ManagedDependencyIntegrationException;
import org.apache.ambari.server.controller.spi.Resource;
import org.apache.ambari.server.utils.RetryHelper;
import org.eclipse.jetty.util.ajax.JSON;
@@ -175,6 +176,13 @@ public abstract class BaseService {
result = new ResultImpl(new ResultStatus(ResultStatus.STATUS.BAD_REQUEST, e.getMessage()));
LOG.error("Bad request received: " + e.getMessage());
requestAuditLogger.log(request, result);
+ } catch (ManagedDependencyIntegrationException e) {
+ ResultStatus.STATUS status = java.util.Arrays.stream(ResultStatus.STATUS.values())
+ .filter(candidate -> candidate.getStatus() == e.getStatus()).findFirst()
+ .orElse(ResultStatus.STATUS.SERVER_ERROR);
+ requestAuditLogger.log(request, new ResultImpl(new ResultStatus(status, e.getMessage())));
+ RetryHelper.clearAffectedClusters();
+ return ManagedDependencyApiSupport.error(e.getStatus(), e.getCode(), e.getMessage());
} catch (Throwable t) {
requestAuditLogger.log(request, new ResultImpl(new ResultStatus(ResultStatus.STATUS.SERVER_ERROR, t.getMessage())));
throw t;
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/api/services/ManagedDependencyApiSupport.java b/ambari-server/src/main/java/org/apache/ambari/server/api/services/ManagedDependencyApiSupport.java
new file mode 100644
index 0000000000..53b712c414
--- /dev/null
+++ b/ambari-server/src/main/java/org/apache/ambari/server/api/services/ManagedDependencyApiSupport.java
@@ -0,0 +1,317 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ambari.server.api.services;
+
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Iterator;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.function.Supplier;
+
+import jakarta.persistence.PersistenceException;
+import jakarta.ws.rs.core.MediaType;
+import jakarta.ws.rs.core.Response;
+
+import org.apache.ambari.server.controller.dependencies.ManagedDependencyIntegrationException;
+import org.apache.ambari.server.controller.dependencies.ManagedDependencyType;
+import org.apache.ambari.server.controller.dependencies.ManagedServiceDependencyCoordinator.CreateRequest;
+import org.apache.ambari.server.controller.dependencies.ManagedServiceDependencyCoordinator.DraftReference;
+import org.apache.ambari.server.controller.dependencies.ManagedServiceDependencyCoordinator.LifecycleRequest;
+import org.apache.ambari.server.controller.dependencies.ManagedServiceDependencyCoordinator.PreviewSelection;
+import org.apache.ambari.server.controller.dependencies.ManagedServiceDependencyCoordinator.ProviderReference;
+import org.apache.ambari.server.controller.dependencies.ManagedServiceDependencyCoordinator.UpdateRequest;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+
+final class ManagedDependencyApiSupport {
+ private static final Logger LOG = LoggerFactory.getLogger(ManagedDependencyApiSupport.class);
+ private static final int MAX_REQUEST_BYTES = 65_536;
+ private static final ObjectMapper MAPPER = new ObjectMapper();
+
+ private ManagedDependencyApiSupport() {
+ }
+
+ static Response invoke(Supplier<Object> action) {
+ try {
+ return Response.ok(action.get()).type(MediaType.APPLICATION_JSON_TYPE).build();
+ } catch (ManagedDependencyIntegrationException e) {
+ return error(e.getStatus(), e.getCode(), e.getMessage());
+ } catch (IllegalArgumentException e) {
+ return error(Response.Status.BAD_REQUEST.getStatusCode(), "INVALID_DEPENDENCY_REQUEST",
+ safeMessage(e, "The dependency request is invalid."));
+ } catch (PersistenceException e) {
+ LOG.error("Managed dependency persistence failed: {}", e.getClass().getName());
+ return error(Response.Status.INTERNAL_SERVER_ERROR.getStatusCode(), "DEPENDENCY_STORAGE_FAILED",
+ "The dependency operation could not be persisted.");
+ } catch (RuntimeException e) {
+ LOG.error("Managed dependency request failed: {}", e.getClass().getName());
+ return error(Response.Status.INTERNAL_SERVER_ERROR.getStatusCode(), "DEPENDENCY_SERVER_ERROR",
+ "The dependency operation could not be completed.");
+ }
+ }
+
+ static Response accepted(Supplier<Object> action) {
+ try {
+ return Response.accepted(action.get()).type(MediaType.APPLICATION_JSON_TYPE).build();
+ } catch (ManagedDependencyIntegrationException e) {
+ return error(e.getStatus(), e.getCode(), e.getMessage());
+ } catch (IllegalArgumentException e) {
+ return error(Response.Status.BAD_REQUEST.getStatusCode(), "INVALID_DEPENDENCY_REQUEST",
+ safeMessage(e, "The dependency request is invalid."));
+ } catch (PersistenceException e) {
+ LOG.error("Managed dependency persistence failed: {}", e.getClass().getName());
+ return error(Response.Status.INTERNAL_SERVER_ERROR.getStatusCode(), "DEPENDENCY_STORAGE_FAILED",
+ "The dependency operation could not be persisted.");
+ } catch (RuntimeException e) {
+ LOG.error("Managed dependency request failed: {}", e.getClass().getName());
+ return error(Response.Status.INTERNAL_SERVER_ERROR.getStatusCode(), "DEPENDENCY_SERVER_ERROR",
+ "The dependency operation could not be completed.");
+ }
+ }
+
+ static JsonNode body(String raw, Set<String> allowedFields) {
+ if (raw == null || raw.getBytes(StandardCharsets.UTF_8).length > MAX_REQUEST_BYTES) {
+ throw new IllegalArgumentException("Dependency request body is missing or too large");
+ }
+ try {
+ JsonNode node = MAPPER.readTree(raw);
+ if (node == null || !node.isObject()) {
+ throw new IllegalArgumentException("Dependency request body must be a JSON object");
+ }
+ requireOnly(node, allowedFields);
+ return node;
+ } catch (JsonProcessingException e) {
+ throw new IllegalArgumentException("Dependency request body is not valid JSON", e);
+ }
+ }
+
+ static CreateRequest createRequest(String raw) {
+ JsonNode root = body(raw, createFields());
+ if (root.has("items")) {
+ throw new IllegalArgumentException("A single CreateRequest is required");
+ }
+ return createRequest(root);
+ }
+
+ static List<CreateRequest> createRequests(String raw) {
+ JsonNode root = body(raw, createFields());
+ if (!root.has("items")) {
+ return List.of(createRequest(root));
+ }
+ if (root.size() != 1 || !root.get("items").isArray()
+ || root.get("items").size() < 1 || root.get("items").size() > 2) {
+ throw new IllegalArgumentException("items must contain one or two CreateRequest objects");
+ }
+ List<CreateRequest> requests = new ArrayList<>();
+ for (JsonNode item : root.get("items")) {
+ if (!item.isObject()) {
+ throw new IllegalArgumentException("Each dependency item must be an object");
+ }
+ requireOnly(item, createFieldsWithoutItems());
+ requests.add(createRequest(item));
+ }
+ return List.copyOf(requests);
+ }
+
+ static boolean isCreateCollection(String raw) {
+ return body(raw, createFields()).has("items");
+ }
+
+ private static CreateRequest createRequest(JsonNode root) {
+ requireOnly(root, createFieldsWithoutItems());
+ JsonNode provider = requiredObject(root, "provider", Set.of("cluster_id", "service_name"));
+ DraftReference draft = null;
+ if (root.hasNonNull("draft")) {
+ JsonNode draftNode = requiredObject(root, "draft", Set.of("id", "revision"));
+ draft = new DraftReference(uuid(text(draftNode, "id")), positiveLong(draftNode, "revision"));
+ }
+ return new CreateRequest(uuid(text(root, "binding_id")), type(text(root, "dependency_type")),
+ provider(provider), fingerprint(root, "expected_provider_fingerprint"),
+ fingerprint(root, "expected_consumer_descriptor_fingerprint"),
+ fingerprint(root, "expected_snapshot_fingerprint"),
+ positiveInt(root, "preview_schema_version"), uuid(text(root, "operation_id")), draft);
+ }
+
+ static PreviewSelection previewSelection(JsonNode node) {
+ if (!node.isObject()) {
+ throw new IllegalArgumentException("Each dependency selection must be an object");
+ }
+ requireOnly(node, Set.of("binding_id", "dependency_type", "provider"));
+ JsonNode provider = requiredObject(node, "provider", Set.of("cluster_id", "service_name"));
+ return new PreviewSelection(type(text(node, "dependency_type")), provider(provider),
+ optionalUuid(node, "binding_id"));
+ }
+
+ private static Set<String> createFields() {
+ return Set.of("items", "binding_id", "dependency_type", "provider",
+ "expected_provider_fingerprint", "expected_consumer_descriptor_fingerprint",
+ "expected_snapshot_fingerprint", "preview_schema_version", "operation_id", "draft");
+ }
+
+ private static Set<String> createFieldsWithoutItems() {
+ return Set.of("binding_id", "dependency_type", "provider",
+ "expected_provider_fingerprint", "expected_consumer_descriptor_fingerprint",
+ "expected_snapshot_fingerprint", "preview_schema_version", "operation_id", "draft");
+ }
+
+ static LifecycleRequest lifecycleRequest(String raw) {
+ JsonNode root = body(raw, Set.of("operation_id", "expected_row_version"));
+ return new LifecycleRequest(uuid(text(root, "operation_id")),
+ nonNegativeLong(root, "expected_row_version"));
+ }
+
+ static UpdateRequest updateRequest(String raw) {
+ JsonNode root = body(raw, Set.of("operation_id", "expected_row_version",
+ "expected_snapshot_version", "expected_provider_fingerprint",
+ "expected_consumer_descriptor_fingerprint", "expected_snapshot_fingerprint",
+ "expected_snapshot_approval", "preview_schema_version"));
+ return new UpdateRequest(uuid(text(root, "operation_id")),
+ nonNegativeLong(root, "expected_row_version"),
+ positiveLong(root, "expected_snapshot_version"),
+ fingerprint(root, "expected_provider_fingerprint"),
+ fingerprint(root, "expected_consumer_descriptor_fingerprint"),
+ fingerprint(root, "expected_snapshot_fingerprint"),
+ text(root, "expected_snapshot_approval"),
+ positiveInt(root, "preview_schema_version"));
+ }
+
+ static ProviderReference provider(JsonNode node) {
+ return new ProviderReference(positiveLong(node, "cluster_id"), text(node, "service_name"));
+ }
+
+ static ManagedDependencyType type(String value) {
+ try {
+ return ManagedDependencyType.valueOf(value);
+ } catch (RuntimeException e) {
+ throw new IllegalArgumentException("dependency type must be HDFS or ZOOKEEPER", e);
+ }
+ }
+
+ static UUID uuid(String value) {
+ try {
+ UUID uuid = UUID.fromString(value);
+ if (!uuid.toString().equals(value)) {
+ throw new IllegalArgumentException("UUIDs must use canonical lower-case form");
+ }
+ return uuid;
+ } catch (RuntimeException e) {
+ throw new IllegalArgumentException("A canonical UUID is required", e);
+ }
+ }
+
+ static UUID optionalUuid(JsonNode node, String field) {
+ if (!node.has(field) || node.get(field).isNull()) {
+ return null;
+ }
+ return uuid(text(node, field));
+ }
+
+ static long positiveLong(String value, String field) {
+ try {
+ long result = Long.parseLong(value);
+ if (result <= 0) {
+ throw new NumberFormatException();
+ }
+ return result;
+ } catch (RuntimeException e) {
+ throw new IllegalArgumentException(field + " must be a positive integer", e);
+ }
+ }
+
+ static JsonNode requiredObject(JsonNode parent, String field, Set<String> fields) {
+ JsonNode value = parent.get(field);
+ if (value == null || !value.isObject()) {
+ throw new IllegalArgumentException(field + " must be an object");
+ }
+ requireOnly(value, fields);
+ return value;
+ }
+
+ static String text(JsonNode node, String field) {
+ JsonNode value = node.get(field);
+ if (value == null || !value.isTextual() || value.textValue().isBlank()) {
+ throw new IllegalArgumentException(field + " must be a non-empty string");
+ }
+ return value.textValue().trim();
+ }
+
+ static String fingerprint(JsonNode node, String field) {
+ String value = text(node, field);
+ if (!value.matches("sha256:[0-9a-f]{64}")) {
+ throw new IllegalArgumentException(field + " must be a SHA-256 fingerprint");
+ }
+ return value;
+ }
+
+ static long positiveLong(JsonNode node, String field) {
+ JsonNode value = node.get(field);
+ if (value == null || !value.isIntegralNumber() || !value.canConvertToLong()
+ || value.longValue() <= 0) {
+ throw new IllegalArgumentException(field + " must be a positive integer");
+ }
+ return value.longValue();
+ }
+
+ static int positiveInt(JsonNode node, String field) {
+ JsonNode value = node.get(field);
+ if (value == null || !value.isIntegralNumber() || !value.canConvertToInt()
+ || value.intValue() <= 0) {
+ throw new IllegalArgumentException(field + " must be a positive integer");
+ }
+ return value.intValue();
+ }
+
+ static long nonNegativeLong(JsonNode node, String field) {
+ JsonNode value = node.get(field);
+ if (value == null || !value.isIntegralNumber() || !value.canConvertToLong()
+ || value.longValue() < 0) {
+ throw new IllegalArgumentException(field + " must be a non-negative integer");
+ }
+ return value.longValue();
+ }
+
+ static void requireOnly(JsonNode object, Set<String> allowed) {
+ Iterator<String> names = object.fieldNames();
+ while (names.hasNext()) {
+ String name = names.next();
+ if (!allowed.contains(name)) {
+ throw new IllegalArgumentException("Unknown dependency request field: " + name);
+ }
+ }
+ }
+
+ static Response error(int status, String code, String message) {
+ Map<String, Object> response = new LinkedHashMap<>();
+ response.put("code", code);
+ response.put("message", message);
+ return Response.status(status).type(MediaType.APPLICATION_JSON_TYPE).entity(response).build();
+ }
+
+ private static String safeMessage(IllegalArgumentException exception, String fallback) {
+ String message = exception.getMessage();
+ return message == null || message.length() > 256 ? fallback : message;
+ }
+}
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/api/services/ManagedServiceDependencyService.java b/ambari-server/src/main/java/org/apache/ambari/server/api/services/ManagedServiceDependencyService.java
new file mode 100644
index 0000000000..d7211bc14d
--- /dev/null
+++ b/ambari-server/src/main/java/org/apache/ambari/server/api/services/ManagedServiceDependencyService.java
@@ -0,0 +1,234 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ambari.server.api.services;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+import jakarta.ws.rs.DELETE;
+import jakarta.ws.rs.GET;
+import jakarta.ws.rs.POST;
+import jakarta.ws.rs.Path;
+import jakarta.ws.rs.PathParam;
+import jakarta.ws.rs.QueryParam;
+import jakarta.ws.rs.core.Response;
+
+import org.apache.ambari.server.StaticallyInject;
+import org.apache.ambari.server.controller.dependencies.ManagedDependencyDeploymentCoordinator;
+import org.apache.ambari.server.controller.dependencies.ManagedDependencyType;
+import org.apache.ambari.server.controller.dependencies.ManagedServiceDependencyCoordinator;
+import org.apache.ambari.server.controller.dependencies.ManagedServiceDependencyCoordinator.ConsumerReference;
+import org.apache.ambari.server.controller.dependencies.ManagedServiceDependencyCoordinator.PreviewSelection;
+import org.apache.ambari.server.utils.StageUtils;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.google.inject.Inject;
+import com.google.inject.Provider;
+
+import io.swagger.annotations.ApiOperation;
+
+@StaticallyInject
+public class ManagedServiceDependencyService {
+ @Inject
+ private static Provider<ManagedServiceDependencyCoordinator> coordinator;
+
+ @Inject
+ private static Provider<ManagedDependencyDeploymentCoordinator> deployments;
+
+ private final String clusterName;
+ private final String serviceName;
+
+ public ManagedServiceDependencyService(String clusterName, String serviceName) {
+ this.clusterName = clusterName;
+ this.serviceName = serviceName;
+ }
+
+ @GET
+ @ApiOperation(value = "Lists managed dependencies for a service")
+ public Response list() {
+ return ManagedDependencyApiSupport.invoke(
+ () -> Map.of("items", coordinator.get().list(clusterName, serviceName)));
+ }
+
+ @GET
+ @Path("/candidates")
+ @ApiOperation(value = "Lists candidate providers for a managed dependency")
+ public Response candidates(@QueryParam("type") String dependencyType) {
+ return ManagedDependencyApiSupport.invoke(() -> {
+ long clusterId = coordinator.get().consumerClusterId(clusterName, serviceName);
+ return Map.of("items", coordinator.get().candidates(ConsumerReference.service(clusterId),
+ ManagedDependencyApiSupport.type(dependencyType)));
+ });
+ }
+
+ @POST
+ @Path("/preview")
+ @ApiOperation(value = "Previews a managed dependency")
+ public Response preview(String body) {
+ return ManagedDependencyApiSupport.invoke(() -> {
+ JsonNode root = ManagedDependencyApiSupport.body(body,
+ Set.of("binding_id", "dependency_type", "provider", "selections"));
+ if (root.has("selections")) {
+ if (root.size() != 1 || !root.get("selections").isArray()
+ || root.get("selections").size() < 1 || root.get("selections").size() > 2) {
+ throw new IllegalArgumentException(
+ "A complete preview requires one or two selections");
+ }
+ List<PreviewSelection> selections = new ArrayList<>();
+ for (JsonNode selection : root.get("selections")) {
+ selections.add(ManagedDependencyApiSupport.previewSelection(selection));
+ }
+ long clusterId = coordinator.get().consumerClusterId(clusterName, serviceName);
+ return coordinator.get().preview(ConsumerReference.service(clusterId), selections);
+ }
+ ManagedDependencyType type = ManagedDependencyApiSupport.type(
+ ManagedDependencyApiSupport.text(root, "dependency_type"));
+ JsonNode provider = ManagedDependencyApiSupport.requiredObject(root, "provider",
+ Set.of("cluster_id", "service_name"));
+ long clusterId = coordinator.get().consumerClusterId(clusterName, serviceName);
+ return coordinator.get().preview(ConsumerReference.service(clusterId), type,
+ ManagedDependencyApiSupport.provider(provider),
+ ManagedDependencyApiSupport.optionalUuid(root, "binding_id"));
+ });
+ }
+
+ @POST
+ @ApiOperation(value = "Creates a managed dependency")
+ public Response create(String body) {
+ return ManagedDependencyApiSupport.accepted(() -> {
+ List<org.apache.ambari.server.controller.dependencies.ManagedServiceDependencyCoordinator.CreateRequest>
+ requests = ManagedDependencyApiSupport.createRequests(body);
+ List<Map<String, Object>> responses = coordinator.get().create(clusterName, serviceName, requests);
+ return ManagedDependencyApiSupport.isCreateCollection(body)
+ ? Map.of("items", responses) : responses.get(0);
+ });
+ }
+
+ @POST
+ @Path("/deployments/{deploymentId}")
+ @ApiOperation(value = "Launches a managed dependency deployment")
+ public Response launchDeployment(@PathParam("deploymentId") String deploymentId, String body) {
+ return ManagedDependencyApiSupport.accepted(() -> {
+ coordinator.get().consumerClusterId(clusterName, serviceName);
+ JsonNode root = ManagedDependencyApiSupport.body(body, Set.of("targets", "install_only"));
+ if (!root.path("targets").isArray() || !root.path("install_only").isBoolean()) {
+ throw new IllegalArgumentException("Exact targets and an explicit installation mode are required");
+ }
+ List<ManagedDependencyDeploymentCoordinator.Target> targets = new ArrayList<>();
+ for (JsonNode target : root.get("targets")) {
+ if (!target.isObject()) throw new IllegalArgumentException("A deployment target must be an object");
+ target.fieldNames().forEachRemaining(field -> {
+ if (!Set.of("serviceName", "componentName", "hostName", "hostId").contains(field)) {
+ throw new IllegalArgumentException("Unknown deployment target field");
+ }
+ });
+ for (String field : Set.of("serviceName", "componentName", "hostName")) {
+ ManagedDependencyApiSupport.text(target, field);
+ }
+ if (target.has("hostId") && (!target.get("hostId").canConvertToLong()
+ || !target.get("hostId").isIntegralNumber() || target.get("hostId").longValue() <= 0)) {
+ throw new IllegalArgumentException("A host identity must be a positive integer");
+ }
+ targets.add(StageUtils.getGson().fromJson(target.toString(), ManagedDependencyDeploymentCoordinator.Target.class));
+ }
+ return deployments.get().launch(clusterName, ManagedDependencyApiSupport.uuid(deploymentId),
+ targets, root.get("install_only").booleanValue());
+ });
+ }
+
+ @GET
+ @Path("/deployments/{deploymentId}")
+ @ApiOperation(value = "Returns a managed dependency deployment")
+ public Response getDeployment(@PathParam("deploymentId") String deploymentId) {
+ return ManagedDependencyApiSupport.invoke(() -> {
+ coordinator.get().consumerClusterId(clusterName, serviceName);
+ return deployments.get().get(clusterName, ManagedDependencyApiSupport.uuid(deploymentId));
+ });
+ }
+
+ @POST
+ @Path("/deployments/{deploymentId}/actions/retry")
+ @ApiOperation(value = "Retries a managed dependency deployment")
+ public Response retryDeployment(@PathParam("deploymentId") String deploymentId, String body) {
+ return ManagedDependencyApiSupport.accepted(() -> {
+ coordinator.get().consumerClusterId(clusterName, serviceName);
+ JsonNode root = ManagedDependencyApiSupport.body(body, Set.of("operation_id"));
+ return deployments.get().retry(clusterName, ManagedDependencyApiSupport.uuid(deploymentId),
+ ManagedDependencyApiSupport.uuid(ManagedDependencyApiSupport.text(root, "operation_id")));
+ });
+ }
+
+ @POST
+ @Path("/{bindingId}/actions/verify-credentials")
+ @ApiOperation(value = "Verifies managed dependency credentials")
+ public Response verifyCredentials(@PathParam("bindingId") String bindingId, String body) {
+ return ManagedDependencyApiSupport.accepted(() -> {
+ JsonNode root = ManagedDependencyApiSupport.body(body, Set.of("expected_epoch"));
+ if (!root.path("expected_epoch").isIntegralNumber() || root.get("expected_epoch").longValue() <= 0) {
+ throw new IllegalArgumentException("An exact operation epoch is required");
+ }
+ return coordinator.get().verifyManualCredentials(clusterName, serviceName,
+ ManagedDependencyApiSupport.uuid(bindingId), root.get("expected_epoch").longValue());
+ });
+ }
+
+ @GET
+ @Path("/{bindingId}")
+ @ApiOperation(value = "Returns a managed dependency")
+ public Response get(@PathParam("bindingId") String bindingId) {
+ return ManagedDependencyApiSupport.invoke(() -> coordinator.get().get(clusterName, serviceName,
+ ManagedDependencyApiSupport.uuid(bindingId)));
+ }
+
+ @GET
+ @Path("/{bindingId}/preview-update")
+ @ApiOperation(value = "Previews a managed dependency update")
+ public Response previewUpdate(@PathParam("bindingId") String bindingId) {
+ return ManagedDependencyApiSupport.invoke(() -> coordinator.get().previewUpdate(
+ clusterName, serviceName, ManagedDependencyApiSupport.uuid(bindingId)));
+ }
+
+ @POST
+ @Path("/{bindingId}/actions/update")
+ @ApiOperation(value = "Updates a managed dependency")
+ public Response update(@PathParam("bindingId") String bindingId, String body) {
+ return ManagedDependencyApiSupport.accepted(() -> coordinator.get().update(
+ clusterName, serviceName, ManagedDependencyApiSupport.uuid(bindingId),
+ ManagedDependencyApiSupport.updateRequest(body)));
+ }
+
+ @POST
+ @Path("/{bindingId}/actions/retry")
+ @ApiOperation(value = "Retries a managed dependency operation")
+ public Response retry(@PathParam("bindingId") String bindingId, String body) {
+ return ManagedDependencyApiSupport.accepted(() -> coordinator.get().retry(
+ clusterName, serviceName, ManagedDependencyApiSupport.uuid(bindingId),
+ ManagedDependencyApiSupport.lifecycleRequest(body)));
+ }
+
+ @DELETE
+ @Path("/{bindingId}")
+ @ApiOperation(value = "Detaches a managed dependency")
+ public Response detach(@PathParam("bindingId") String bindingId, String body) {
+ return ManagedDependencyApiSupport.accepted(() -> coordinator.get().detach(
+ clusterName, serviceName, ManagedDependencyApiSupport.uuid(bindingId),
+ ManagedDependencyApiSupport.lifecycleRequest(body)));
+ }
+}
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/api/services/ManagedServiceDependentsService.java b/ambari-server/src/main/java/org/apache/ambari/server/api/services/ManagedServiceDependentsService.java
new file mode 100644
index 0000000000..4fb33ba278
--- /dev/null
+++ b/ambari-server/src/main/java/org/apache/ambari/server/api/services/ManagedServiceDependentsService.java
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ambari.server.api.services;
+
+import jakarta.ws.rs.GET;
+import jakarta.ws.rs.QueryParam;
+import jakarta.ws.rs.core.Response;
+
+import org.apache.ambari.server.StaticallyInject;
+import org.apache.ambari.server.controller.dependencies.ManagedServiceDependencyCoordinator;
+
+import com.google.inject.Inject;
+import com.google.inject.Provider;
+
+import io.swagger.annotations.ApiOperation;
+
+@StaticallyInject
+public class ManagedServiceDependentsService {
+ @Inject
+ private static Provider<ManagedServiceDependencyCoordinator> coordinator;
+
+ private final String clusterName;
+ private final String serviceName;
+ private final boolean impact;
+
+ public ManagedServiceDependentsService(String clusterName, String serviceName, boolean impact) {
+ this.clusterName = clusterName;
+ this.serviceName = serviceName;
+ this.impact = impact;
+ }
+
+ @GET
+ @ApiOperation(value = "Returns dependents or dependency impact for a service")
+ public Response get(@QueryParam("action") String action) {
+ return ManagedDependencyApiSupport.invoke(() -> impact
+ ? coordinator.get().impact(clusterName, serviceName, action)
+ : coordinator.get().dependents(clusterName, serviceName));
+ }
+}
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/api/services/PersistKeyValueImpl.java b/ambari-server/src/main/java/org/apache/ambari/server/api/services/PersistKeyValueImpl.java
index b1288584fc..15106ce55e 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/api/services/PersistKeyValueImpl.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/api/services/PersistKeyValueImpl.java
@@ -18,28 +18,147 @@
package org.apache.ambari.server.api.services;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.Collection;
+import java.util.Collections;
+import java.util.EnumSet;
import java.util.HashMap;
+import java.util.LinkedHashMap;
import java.util.List;
+import java.util.Locale;
import java.util.Map;
+import java.util.Set;
import java.util.UUID;
+import java.util.regex.Pattern;
+import jakarta.persistence.PersistenceException;
import jakarta.ws.rs.WebApplicationException;
+import jakarta.ws.rs.core.MediaType;
import jakarta.ws.rs.core.Response;
+import org.apache.ambari.server.AmbariException;
+import org.apache.ambari.server.ClusterNotFoundException;
+import org.apache.ambari.server.orm.dao.ClusterDAO;
import org.apache.ambari.server.orm.dao.KeyValueDAO;
+import org.apache.ambari.server.orm.dao.ScopedWorkflowStateDAO;
+import org.apache.ambari.server.orm.dao.UserDAO;
+import org.apache.ambari.server.orm.entities.ClusterEntity;
import org.apache.ambari.server.orm.entities.KeyValueEntity;
+import org.apache.ambari.server.orm.entities.ScopedWorkflowStateEntity;
+import org.apache.ambari.server.orm.entities.UserEntity;
+import org.apache.ambari.server.security.authorization.AuthorizationException;
+import org.apache.ambari.server.security.authorization.AuthorizationHelper;
+import org.apache.ambari.server.security.authorization.ResourceType;
+import org.apache.ambari.server.security.authorization.RoleAuthorization;
+import org.apache.ambari.server.state.Cluster;
+import org.apache.ambari.server.state.ClusterCreationContext;
+import org.apache.ambari.server.state.Clusters;
+import org.apache.ambari.server.utils.StageUtils;
+import com.google.gson.JsonParseException;
import com.google.inject.Inject;
import com.google.inject.Singleton;
@Singleton
public class PersistKeyValueImpl {
+ static final String SCOPED_KEY_PREFIX = "ambari.internal.workflow:";
+
+ public static String creationDraftStorageKey(int ownerUserId, UUID draftId) {
+ if (ownerUserId <= 0) {
+ throw new IllegalArgumentException("A positive draft owner user ID is required");
+ }
+ return SCOPED_KEY_PREFIX + "drafts:" + ownerUserId + ":" +
+ java.util.Objects.requireNonNull(draftId, "draftId");
+ }
+ private static final int MAX_DRAFT_SUMMARIES = 100;
+ private static final int MAX_PHASE_LENGTH = 128;
+ static final int MAX_SCOPED_REQUEST_BYTES = 2 * 1024 * 1024;
+ private static final int MAX_VALUE_DEPTH = 20;
+ private static final int MAX_VALUE_NODES = 100000;
+ private static final Pattern SENSITIVE_KEY = Pattern.compile(
+ ".*(password|secret|private.?key|ssh.?key|credential|token|cookie|keytab).*", Pattern.CASE_INSENSITIVE);
+ private static final Pattern OPAQUE_SENSITIVE_VALUE = Pattern.compile(
+ ".*(password|secret|private.?key|ssh.?key|credential|token|cookie|keytab)[\\s\\\"'\\\\]*[:=].*",
+ Pattern.CASE_INSENSITIVE | Pattern.DOTALL);
+ private static final Pattern PRIVATE_KEY_VALUE = Pattern.compile(
+ ".*-----BEGIN(?: [A-Z0-9]+)* PRIVATE KEY-----.*", Pattern.CASE_INSENSITIVE | Pattern.DOTALL);
+ private static final Set<String> VALUE_FIELDS = Set.of(
+ "value", "property_value", "propertyvalue", "savedvalue", "recommendedvalue",
+ "initialvalue", "defaultvalue", "currentvalue", "previousvalue", "priorvalue",
+ "originalvalue", "uservalue", "changedvalue", "newvalue", "oldvalue",
+ "default_value", "initial_value", "previous_value", "recommended_value",
+ "new_value", "old_value", "confirmpassword", "confirm_password");
+ private static final Set<String> REPOSITORY_URL_FIELDS = Set.of(
+ "baseurl", "defaulturl", "version_url", "base_url", "default_base_url");
+ private static final Set<String> LEGACY_USER_PREFERENCE_SUFFIXES = Set.of("supports", "dashboard");
+ private static final Map<String, RoleAuthorization> CLUSTER_WORKFLOW_AUTHORIZATIONS;
+ private static final Map<String, RoleAuthorization> LEGACY_CLUSTER_KEYS;
+ private static final Set<String> AMBIGUOUS_LEGACY_KEYS = Set.of(
+ "USER_REDIRECTION_URL", "CLUSTER_CURRENT", "CLUSTER_CURRENT_STATUS");
+
+ static {
+ Map<String, RoleAuthorization> authorizations = new LinkedHashMap<>();
+ authorizations.put("ADD_SERVICE", RoleAuthorization.SERVICE_ADD_DELETE_SERVICES);
+ authorizations.put("ADD_HOST", RoleAuthorization.HOST_ADD_DELETE_HOSTS);
+ authorizations.put("REASSIGN_COMPONENT", RoleAuthorization.SERVICE_MOVE);
+ authorizations.put("ENABLING_KERBEROS", RoleAuthorization.CLUSTER_TOGGLE_KERBEROS);
+ authorizations.put("HIGH_AVAILIBILITY_NAMENODE", RoleAuthorization.SERVICE_ENABLE_HA);
+ authorizations.put("ROLLBACK_HIGH_AVAILABILITY", RoleAuthorization.SERVICE_ENABLE_HA);
+ authorizations.put("HIGH_AVAILIBILITY_RM_HA", RoleAuthorization.SERVICE_ENABLE_HA);
+ authorizations.put("HIGH_AVAILIBILITY_RANGER_HA", RoleAuthorization.SERVICE_ENABLE_HA);
+ authorizations.put("MANAGE_JOURNALNODES", RoleAuthorization.SERVICE_ENABLE_HA);
+ authorizations.put("OBSERVER_NAMENODE", RoleAuthorization.SERVICE_ENABLE_HA);
+ authorizations.put("NAMENODE_FEDERATION", RoleAuthorization.SERVICE_ENABLE_HA);
+ authorizations.put("ROUTER_FEDERATION", RoleAuthorization.SERVICE_ENABLE_HA);
+ authorizations.put("ADD_HAWQ_STANDBY", RoleAuthorization.SERVICE_ENABLE_HA);
+ authorizations.put("REMOVE_HAWQ_STANDBY", RoleAuthorization.SERVICE_ENABLE_HA);
+ authorizations.put("ACTIVATE_HAWQ_STANDBY", RoleAuthorization.SERVICE_ENABLE_HA);
+ authorizations.put("UPGRADE", RoleAuthorization.CLUSTER_UPGRADE_DOWNGRADE_STACK);
+ CLUSTER_WORKFLOW_AUTHORIZATIONS = Collections.unmodifiableMap(authorizations);
+
+ Map<String, RoleAuthorization> legacyKeys = new HashMap<>();
+ legacyKeys.put("ADD_SERVICE", RoleAuthorization.SERVICE_ADD_DELETE_SERVICES);
+ legacyKeys.put("ADD_HOST", RoleAuthorization.HOST_ADD_DELETE_HOSTS);
+ legacyKeys.put("REASSIGN_COMPONENT", RoleAuthorization.SERVICE_MOVE);
+ legacyKeys.put("ENABLING_KERBEROS", RoleAuthorization.CLUSTER_TOGGLE_KERBEROS);
+ legacyKeys.put("HIGH_AVAILIBILITY_NAMENODE", RoleAuthorization.SERVICE_ENABLE_HA);
+ legacyKeys.put("HIGH_AVAILIBILITY_RM_HA", RoleAuthorization.SERVICE_ENABLE_HA);
+ legacyKeys.put("HIGH_AVAILIBILITY_RANGER_HA", RoleAuthorization.SERVICE_ENABLE_HA);
+ legacyKeys.put("MANAGE_JOURNALNODES", RoleAuthorization.SERVICE_ENABLE_HA);
+ legacyKeys.put("OBSERVER_NAMENODE", RoleAuthorization.SERVICE_ENABLE_HA);
+ legacyKeys.put("NAMENODE_FEDERATION", RoleAuthorization.SERVICE_ENABLE_HA);
+ legacyKeys.put("ROUTER_FEDERATION", RoleAuthorization.SERVICE_ENABLE_HA);
+ legacyKeys.put("ADD_HAWQ_STANDBY", RoleAuthorization.SERVICE_ENABLE_HA);
+ legacyKeys.put("REMOVE_HAWQ_STANDBY", RoleAuthorization.SERVICE_ENABLE_HA);
+ legacyKeys.put("ACTIVATE_HAWQ_STANDBY", RoleAuthorization.SERVICE_ENABLE_HA);
+ legacyKeys.put("isPatchUpgrade", RoleAuthorization.CLUSTER_UPGRADE_DOWNGRADE_STACK);
+ legacyKeys.put("upgradeIsFinalizeItem", RoleAuthorization.CLUSTER_UPGRADE_DOWNGRADE_STACK);
+ legacyKeys.put("upgradeVersionDisplayName", RoleAuthorization.CLUSTER_UPGRADE_DOWNGRADE_STACK);
+ legacyKeys.put("versionOperations", RoleAuthorization.CLUSTER_UPGRADE_DOWNGRADE_STACK);
+ legacyKeys.put("CLUSTER_STATE", null);
+ legacyKeys.put("wizard-data", null);
+ LEGACY_CLUSTER_KEYS = Collections.unmodifiableMap(legacyKeys);
+ }
+
@Inject
KeyValueDAO keyValueDAO;
+ @Inject
+ ScopedWorkflowStateDAO scopedWorkflowStateDAO;
+
+ @Inject
+ ClusterDAO clusterDAO;
+
+ @Inject
+ UserDAO userDAO;
+
+ @Inject
+ Clusters clusters;
+
public String generateKey() {
return UUID.randomUUID().toString();
}
@@ -53,6 +172,7 @@ public class PersistKeyValueImpl {
}
public synchronized String getValue(String key) {
+ rejectReservedKey(key);
KeyValueEntity keyValueEntity = keyValueDAO.findByKey(key);
if (keyValueEntity != null) {
return keyValueEntity.getValue();
@@ -67,6 +187,7 @@ public class PersistKeyValueImpl {
}
public synchronized void put(String key, String value) {
+ rejectReservedKey(key);
KeyValueEntity keyValueEntity = keyValueDAO.findByKey(key);
if (keyValueEntity != null) {
keyValueEntity.setValue(value);
@@ -82,8 +203,673 @@ public class PersistKeyValueImpl {
public synchronized Map<String, String> getAllKeyValues() {
Map<String, String> map = new HashMap<>();
for (KeyValueEntity keyValueEntity : keyValueDAO.findAll()) {
- map.put(keyValueEntity.getKey(), keyValueEntity.getValue());
+ if (!isReservedKey(keyValueEntity.getKey())) {
+ map.put(keyValueEntity.getKey(), keyValueEntity.getValue());
+ }
}
return map;
}
+
+ public String getLegacyValue(String key) throws AuthorizationException {
+ authorizeLegacyKey(key, false);
+ return getValue(key);
+ }
+
+ public void putLegacyValues(Map<String, String> values) throws AuthorizationException {
+ for (String key : values.keySet()) {
+ authorizeLegacyKey(key, true);
+ }
+ for (Map.Entry<String, String> entry : values.entrySet()) {
+ put(entry.getKey(), entry.getValue());
+ }
+ }
+
+ public Collection<String> putLegacyGeneratedValues(Collection<String> values) throws AuthorizationException {
+ AuthorizationHelper.verifyAuthorization(ResourceType.AMBARI, null,
+ EnumSet.of(RoleAuthorization.AMBARI_MANAGE_CONFIGURATION));
+ Collection<String> keys = new ArrayList<>(values.size());
+ for (String value : values) {
+ keys.add(put(value));
+ }
+ return keys;
+ }
+
+ public Map<String, String> getAllLegacyKeyValues() {
+ Map<String, String> visible = new HashMap<>();
+ for (Map.Entry<String, String> entry : getAllKeyValues().entrySet()) {
+ if (isLegacyKeyAuthorized(entry.getKey(), false)) {
+ visible.put(entry.getKey(), entry.getValue());
+ }
+ }
+ return visible;
+ }
+
+ public ScopedWorkflowState getScopedState(String scopeType, String scopeId, boolean summary)
+ throws AuthorizationException {
+ AuthenticatedUser user = authenticatedUser();
+ ScopeKey scope = resolveScope(scopeType, scopeId, user);
+ authorizeRead(scope, summary);
+ return toState(scopedWorkflowStateDAO.findByKey(scope.storageKey));
+ }
+
+ /**
+ * Returns the authenticated user's exact active cluster workflow checkpoint.
+ * This is used for side-effect-free planning that must not adopt another
+ * administrator's in-progress workflow.
+ */
+ public ScopedWorkflowState getActiveOwnedClusterWorkflowState(long clusterId,
+ String workflow, long expectedRevision) throws AuthorizationException {
+ AuthenticatedUser user = authenticatedUser();
+ ScopeKey scope = resolveScope("clusters", Long.toString(clusterId), user);
+ authorizeRead(scope, false);
+ ScopedWorkflowStateEntity entity = scopedWorkflowStateDAO.findByKey(scope.storageKey);
+ if (entity == null || !user.id.equals(entity.getOwnerUserId())
+ || !workflow.equals(entity.getWorkflow())
+ || entity.getRevision() == null || entity.getRevision() != expectedRevision) {
+ throw conflict("WORKFLOW_VERSION_CONFLICT",
+ "Reload the active workflow checkpoint before previewing dependencies");
+ }
+ return toState(entity);
+ }
+
+ public ScopedWorkflowState putScopedState(String scopeType, String scopeId, ScopedWorkflowUpdate update)
+ throws AuthorizationException {
+ AuthenticatedUser user = authenticatedUser();
+ ScopeKey scope = resolveScope(scopeType, scopeId, user);
+ authorizeUpdateScope(scope);
+ validateUpdate(scope, update);
+ authorizeWorkflowUpdate(scope, update.getWorkflow());
+ update.setValues(sanitizeValues(update.getValues(), 0, new int[] {0}, false));
+
+ try {
+ return updateScopedState(scope.storageKey, user, update);
+ } catch (PersistenceException e) {
+ // A missing-row race is complete and rolled back when the DAO proxy returns.
+ // Retry against the winning row so optimistic concurrency reports a stable 409.
+ if (update.getExpectedRevision() == 0 && scopedWorkflowStateDAO.findByKey(scope.storageKey) != null) {
+ return updateScopedState(scope.storageKey, user, update);
+ }
+ throw e;
+ }
+ }
+
+ public ClusterCreationContext validateClusterCreationDraft(String scopeId)
+ throws AuthorizationException {
+ AuthenticatedUser user = authenticatedUser();
+ ScopeKey scope = resolveScope("drafts", scopeId, user);
+ authorizeRead(scope, true);
+ ScopedWorkflowStateEntity draft = scopedWorkflowStateDAO.findByKey(scope.storageKey);
+ if (draft == null) {
+ throw conflict("INVALID_CREATION_DRAFT",
+ "The creation draft is not active for the authenticated user");
+ }
+ String draftId = canonicalDraftId(scopeId);
+ if (draft.getCreatedClusterId() != null
+ && clusterForDraftAssociation(draft, user.id, draftId) == null) {
+ throw conflict("CREATION_DRAFT_CONSUMED",
+ "The creation draft was already consumed and cannot create another cluster");
+ }
+ if (!user.id.equals(draft.getOwnerUserId())
+ || !"CLUSTER_CREATE".equals(draft.getWorkflow())) {
+ throw conflict("INVALID_CREATION_DRAFT",
+ "The creation draft is not active for the authenticated user");
+ }
+ return new ClusterCreationContext(user.id, draftId, scope.storageKey);
+ }
+
+ public Map<String, Object> getCreationDraftCluster(String scopeId)
+ throws AuthorizationException {
+ AuthenticatedUser user = authenticatedUser();
+ ScopeKey scope = resolveScope("drafts", scopeId, user);
+ authorizeRead(scope, true);
+ ScopedWorkflowStateEntity draft = scopedWorkflowStateDAO.findByKey(scope.storageKey);
+ if (draft == null) {
+ throw new WebApplicationException(Response.Status.NOT_FOUND);
+ }
+ ClusterEntity cluster = clusterForDraftAssociation(draft, user.id, canonicalDraftId(scopeId));
+ if (cluster == null) {
+ throw new WebApplicationException(Response.Status.NOT_FOUND);
+ }
+ Map<String, Object> response = new LinkedHashMap<>();
+ response.put("cluster_id", cluster.getClusterId());
+ response.put("cluster_name", cluster.getClusterName());
+ return response;
+ }
+
+ public Map<String, Object> getCreationDraftDirectory() throws AuthorizationException {
+ AuthenticatedUser user = authenticatedUser();
+ AuthorizationHelper.verifyAuthorization(ResourceType.AMBARI, null,
+ EnumSet.of(RoleAuthorization.AMBARI_ADD_DELETE_CLUSTERS));
+ String prefix = SCOPED_KEY_PREFIX + "drafts:" + user.id + ":";
+ Map<Long, ClusterEntity> clustersById = new HashMap<>();
+ for (ClusterEntity cluster : clusterDAO.findByCreatorUserId(user.id)) {
+ clustersById.put(cluster.getClusterId(), cluster);
+ }
+ List<Map<String, Object>> items = new ArrayList<>();
+ for (ScopedWorkflowStateEntity draft : scopedWorkflowStateDAO.findActiveCreationDrafts(
+ user.id, prefix, MAX_DRAFT_SUMMARIES)) {
+ String draftId = draft.getScopeKey().substring(prefix.length());
+ ClusterEntity cluster = draft.getCreatedClusterId() == null
+ ? null : clustersById.get(draft.getCreatedClusterId());
+ if (cluster != null && !draftId.equals(cluster.getCreationDraftId())) {
+ cluster = null;
+ }
+ items.add(new ScopedWorkflowDraftSummary(draftId, draft.getRevision(), draft.getWorkflow(),
+ draft.getPhase(), cluster == null ? null : cluster.getClusterId(),
+ cluster == null ? null : cluster.getClusterName()).toResponse());
+ }
+ return Map.of("items", items);
+ }
+
+ static boolean isReservedKey(String key) {
+ return key != null && key.startsWith(SCOPED_KEY_PREFIX);
+ }
+
+ private void authorizeLegacyKey(String key, boolean write) throws AuthorizationException {
+ rejectReservedKey(key);
+ if (AMBIGUOUS_LEGACY_KEYS.contains(key)
+ || (LEGACY_CLUSTER_KEYS.containsKey(key) && clusters.getClusters().size() != 1)) {
+ throw conflict("LEGACY_PERSIST_SCOPE_REQUIRED",
+ "This legacy key has no unambiguous user and cluster scope");
+ }
+ if (!isLegacyKeyAuthorized(key, write)) {
+ throw new AuthorizationException();
+ }
+ }
+
+ private boolean isLegacyKeyAuthorized(String key, boolean write) {
+ AuthenticatedUser user = authenticatedUserOrNull();
+ if (user == null) {
+ return false;
+ }
+ if (AMBIGUOUS_LEGACY_KEYS.contains(key)) {
+ return false;
+ }
+
+ if (isUserPreferenceKey(key, user.username)) {
+ return AuthorizationHelper.isAuthorized(ResourceType.CLUSTER, null,
+ RoleAuthorization.CLUSTER_MANAGE_USER_PERSISTED_DATA);
+ }
+
+ if (LEGACY_CLUSTER_KEYS.containsKey(key)) {
+ Map<String, Cluster> existingClusters = clusters.getClusters();
+ if (existingClusters.size() != 1) {
+ return false;
+ }
+ Cluster cluster = existingClusters.values().iterator().next();
+ Long resourceId = cluster.getResourceId();
+ if (!AuthorizationHelper.isAuthorized(ResourceType.CLUSTER, resourceId,
+ RoleAuthorization.CLUSTER_MANAGE_USER_PERSISTED_DATA)) {
+ return false;
+ }
+ if (!write && !AuthorizationHelper.isAuthorized(ResourceType.CLUSTER, resourceId,
+ RoleAuthorization.CLUSTER_VIEW_CONFIGS)) {
+ return false;
+ }
+ RoleAuthorization workflowAuthorization = LEGACY_CLUSTER_KEYS.get(key);
+ return workflowAuthorization == null || AuthorizationHelper.isAuthorized(
+ ResourceType.CLUSTER, resourceId, workflowAuthorization);
+ }
+
+ return AuthorizationHelper.isAuthorized(ResourceType.AMBARI, null,
+ RoleAuthorization.AMBARI_MANAGE_CONFIGURATION);
+ }
+
+ private boolean isUserPreferenceKey(String key, String username) {
+ String normalizedUser = username.toLowerCase(Locale.ROOT);
+ String normalizedKey = key == null ? "" : key.toLowerCase(Locale.ROOT);
+ if (normalizedKey.equals("admin-settings-show-bg-" + normalizedUser)
+ || normalizedKey.equals("admin-settings-timezone-" + normalizedUser)) {
+ return true;
+ }
+ for (String suffix : LEGACY_USER_PREFERENCE_SUFFIXES) {
+ if (normalizedKey.equals("user-pref-" + normalizedUser + "-" + suffix)) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+ private ScopedWorkflowState updateScopedState(String storageKey, AuthenticatedUser user,
+ ScopedWorkflowUpdate update) {
+ ScopedWorkflowStateEntity value = scopedWorkflowStateDAO.updateWithLock(storageKey, entity -> {
+ ScopedWorkflowState current = toState(entity);
+ if (current.getRevision() != update.getExpectedRevision()) {
+ throw conflict("WORKFLOW_VERSION_CONFLICT", String.format(
+ "Expected workflow revision %d but current revision is %d",
+ update.getExpectedRevision(), current.getRevision()));
+ }
+ if (entity.getOwnerUserId() != null && !entity.getOwnerUserId().equals(user.id)) {
+ throw conflict("WORKFLOW_OWNED", "The workflow is active for another user");
+ }
+ if (!"IDLE".equals(current.getWorkflow()) && !"IDLE".equals(update.getWorkflow())
+ && !current.getWorkflow().equals(update.getWorkflow())) {
+ throw conflict("WORKFLOW_ACTIVE",
+ "Release the active workflow before starting a different workflow type");
+ }
+ if (storageKey.startsWith(SCOPED_KEY_PREFIX + "drafts:")
+ && "IDLE".equals(current.getWorkflow()) && current.getRevision() > 0
+ && "CLUSTER_CREATE".equals(update.getWorkflow())) {
+ throw conflict("CREATION_DRAFT_RETIRED",
+ "A released creation draft cannot be reused; start with a new draft ID");
+ }
+
+ boolean release = "IDLE".equals(update.getWorkflow());
+ Map<String, Object> values = release ? Collections.emptyMap() : update.getValues();
+ String payload = StageUtils.getGson().toJson(values);
+ if (payload.getBytes(StandardCharsets.UTF_8).length > MAX_SCOPED_REQUEST_BYTES) {
+ throw badRequest("WORKFLOW_STATE_TOO_LARGE",
+ "The workflow state exceeds the maximum persisted size");
+ }
+ entity.setRevision(current.getRevision() + 1);
+ entity.setOwnerUserId(release ? null : user.id);
+ entity.setOwnerName(release ? null : user.username);
+ entity.setWorkflow(release ? "IDLE" : update.getWorkflow());
+ entity.setPhase(release ? "IDLE" : update.getPhase());
+ entity.setPayload(payload);
+ return entity;
+ });
+ return toState(value);
+ }
+
+ private ScopeKey resolveScope(String scopeType, String scopeId, AuthenticatedUser user)
+ throws AuthorizationException {
+ if ("clusters".equals(scopeType)) {
+ long clusterId;
+ try {
+ clusterId = Long.parseLong(scopeId);
+ } catch (NumberFormatException e) {
+ throw badRequest("INVALID_WORKFLOW_SCOPE", "Cluster scope ID must be numeric");
+ }
+ if (clusterId <= 0) {
+ throw badRequest("INVALID_WORKFLOW_SCOPE", "Cluster scope ID must be positive");
+ }
+
+ Cluster cluster;
+ try {
+ cluster = clusters.getClusterById(clusterId);
+ } catch (ClusterNotFoundException e) {
+ if (AuthorizationHelper.isAuthorized(ResourceType.AMBARI, null,
+ RoleAuthorization.AMBARI_ADD_DELETE_CLUSTERS)) {
+ throw badRequest("INVALID_WORKFLOW_SCOPE", "Cluster scope does not exist");
+ }
+ throw new AuthorizationException();
+ } catch (AmbariException e) {
+ throw new WebApplicationException(e, Response.Status.INTERNAL_SERVER_ERROR);
+ }
+ return ScopeKey.cluster(cluster, SCOPED_KEY_PREFIX + "clusters:" + clusterId);
+ }
+
+ if ("drafts".equals(scopeType)) {
+ UUID draftId;
+ try {
+ draftId = UUID.fromString(scopeId);
+ } catch (IllegalArgumentException e) {
+ throw badRequest("INVALID_WORKFLOW_SCOPE", "Draft scope ID must be a UUID");
+ }
+ String storageKey = creationDraftStorageKey(user.id, draftId);
+ return ScopeKey.draft(storageKey);
+ }
+
+ throw badRequest("INVALID_WORKFLOW_SCOPE", "Scope type must be clusters or drafts");
+ }
+
+ private String canonicalDraftId(String scopeId) {
+ try {
+ return UUID.fromString(scopeId).toString();
+ } catch (IllegalArgumentException e) {
+ throw badRequest("INVALID_WORKFLOW_SCOPE", "Draft scope ID must be a UUID");
+ }
+ }
+
+ private void authorizeRead(ScopeKey scope, boolean summary) throws AuthorizationException {
+ if (scope.cluster != null) {
+ if (summary) {
+ AuthorizationHelper.verifyAuthorization(ResourceType.CLUSTER, scope.cluster.getResourceId(),
+ RoleAuthorization.AUTHORIZATIONS_VIEW_CLUSTER);
+ } else {
+ AuthorizationHelper.verifyAuthorization(ResourceType.CLUSTER, scope.cluster.getResourceId(),
+ EnumSet.of(RoleAuthorization.CLUSTER_MANAGE_USER_PERSISTED_DATA));
+ AuthorizationHelper.verifyAuthorization(ResourceType.CLUSTER, scope.cluster.getResourceId(),
+ EnumSet.of(RoleAuthorization.CLUSTER_VIEW_CONFIGS));
+ }
+ } else {
+ AuthorizationHelper.verifyAuthorization(ResourceType.AMBARI, null,
+ EnumSet.of(RoleAuthorization.AMBARI_ADD_DELETE_CLUSTERS));
+ }
+ }
+
+ private void authorizeUpdateScope(ScopeKey scope) throws AuthorizationException {
+ if (scope.cluster == null) {
+ AuthorizationHelper.verifyAuthorization(ResourceType.AMBARI, null,
+ EnumSet.of(RoleAuthorization.AMBARI_ADD_DELETE_CLUSTERS));
+ return;
+ }
+
+ AuthorizationHelper.verifyAuthorization(ResourceType.CLUSTER, scope.cluster.getResourceId(),
+ EnumSet.of(RoleAuthorization.CLUSTER_MANAGE_USER_PERSISTED_DATA));
+ }
+
+ private void authorizeWorkflowUpdate(ScopeKey scope, String workflow) throws AuthorizationException {
+ if (scope.cluster != null && !"IDLE".equals(workflow)) {
+ AuthorizationHelper.verifyAuthorization(ResourceType.CLUSTER, scope.cluster.getResourceId(),
+ EnumSet.of(CLUSTER_WORKFLOW_AUTHORIZATIONS.get(workflow)));
+ }
+ }
+
+ private void validateUpdate(ScopeKey scope, ScopedWorkflowUpdate update) {
+ if (update == null || update.getExpectedRevision() == null || update.getExpectedRevision() < 0) {
+ throw badRequest("INVALID_WORKFLOW_STATE", "expected_revision must be a non-negative number");
+ }
+ String workflow = update.getWorkflow();
+ boolean allowedWorkflow = "IDLE".equals(workflow)
+ || (scope.cluster == null && "CLUSTER_CREATE".equals(workflow))
+ || (scope.cluster != null && CLUSTER_WORKFLOW_AUTHORIZATIONS.containsKey(workflow));
+ if (!allowedWorkflow) {
+ throw badRequest("INVALID_WORKFLOW_STATE", "Unsupported workflow type");
+ }
+ if (!"IDLE".equals(workflow)
+ && (update.getPhase() == null || update.getPhase().isEmpty()
+ || update.getPhase().length() > MAX_PHASE_LENGTH)) {
+ throw badRequest("INVALID_WORKFLOW_STATE", "phase must contain 1 to 128 characters");
+ }
+ if (update.getValues() == null) {
+ update.setValues(Collections.emptyMap());
+ }
+ }
+
+ private ClusterEntity clusterForDraftAssociation(ScopedWorkflowStateEntity draft,
+ int creatorUserId, String draftId) {
+ if (draft == null || draft.getCreatedClusterId() == null) {
+ return null;
+ }
+ ClusterEntity cluster = clusterDAO.findById(draft.getCreatedClusterId());
+ if (cluster == null || !Integer.valueOf(creatorUserId).equals(cluster.getCreatorUserId())
+ || !draftId.equals(cluster.getCreationDraftId())) {
+ return null;
+ }
+ return cluster;
+ }
+
+ @SuppressWarnings("unchecked")
+ private Map<String, Object> sanitizeValues(Map<String, Object> values, int depth, int[] nodes,
+ boolean sensitiveContext) {
+ Object sanitized = sanitizeValue(values, depth, nodes, sensitiveContext);
+ return sanitized == null ? new LinkedHashMap<>() : (Map<String, Object>) sanitized;
+ }
+
+ private Object sanitizeValue(Object value, int depth, int[] nodes, boolean sensitiveContext) {
+ if (depth > MAX_VALUE_DEPTH || ++nodes[0] > MAX_VALUE_NODES) {
+ throw badRequest("INVALID_WORKFLOW_STATE", "Workflow values exceed structural limits");
+ }
+ if (value instanceof Map) {
+ Map<?, ?> source = (Map<?, ?>) value;
+ boolean mapSensitive = sensitiveContext || isSensitiveProperty(source);
+ Map<String, Object> sanitized = new LinkedHashMap<>();
+ boolean redacted = false;
+ for (Map.Entry<?, ?> entry : source.entrySet()) {
+ String key = String.valueOf(entry.getKey());
+ String normalizedKey = key.toLowerCase(Locale.ROOT);
+ boolean sensitiveKey = SENSITIVE_KEY.matcher(key).matches();
+ if ("versiondefinitionsource".equals(normalizedKey)) {
+ sanitized.put(key, sanitizeVersionDefinitionSource(entry.getValue(), depth + 1, nodes));
+ continue;
+ }
+ if ((mapSensitive && VALUE_FIELDS.contains(normalizedKey)
+ && !isSafePathMetadata(source, entry.getValue()))
+ || (sensitiveKey && !(entry.getValue() instanceof Map))
+ || (REPOSITORY_URL_FIELDS.contains(normalizedKey)
+ && containsUrlUserInfo(entry.getValue()))) {
+ redacted = true;
+ continue;
+ }
+ Object sanitizedValue = sanitizeValue(entry.getValue(), depth + 1, nodes,
+ sensitiveKey);
+ if (sanitizedValue != null) {
+ sanitized.put(key, sanitizedValue);
+ }
+ }
+ if (redacted) {
+ sanitized.put("requires_reentry", true);
+ }
+ return sanitized;
+ } else if (value instanceof Collection) {
+ if (sensitiveContext) {
+ return Collections.emptyList();
+ }
+ List<Object> sanitized = new ArrayList<>();
+ for (Object item : (Collection<?>) value) {
+ Object sanitizedItem = sanitizeValue(item, depth + 1, nodes, false);
+ if (sanitizedItem != null) {
+ sanitized.add(sanitizedItem);
+ }
+ }
+ return sanitized;
+ } else if (value instanceof String && isOpaqueCredentialValue((String) value)) {
+ throw badRequest("SENSITIVE_WORKFLOW_STATE",
+ "Opaque credential-bearing values cannot be persisted in workflow state");
+ } else if (sensitiveContext) {
+ return null;
+ }
+ return value;
+ }
+
+ private Map<String, Object> sanitizeVersionDefinitionSource(Object value, int depth, int[] nodes) {
+ if (depth > MAX_VALUE_DEPTH || ++nodes[0] > MAX_VALUE_NODES) {
+ throw badRequest("INVALID_WORKFLOW_STATE", "Workflow values exceed structural limits");
+ }
+ Map<String, Object> sanitized = new LinkedHashMap<>();
+ if (!(value instanceof Map)) {
+ sanitized.put("requires_reentry", true);
+ return sanitized;
+ }
+
+ Map<?, ?> source = (Map<?, ?>) value;
+ String type = source.get("type") instanceof String
+ ? ((String) source.get("type")).toLowerCase(Locale.ROOT) : null;
+ if (type != null) {
+ sanitized.put("type", type);
+ }
+ if (!"url".equals(type)) {
+ sanitized.put("requires_reentry", true);
+ return sanitized;
+ }
+
+ Object payload = source.get("payload");
+ if (payload instanceof Map) {
+ Object sanitizedPayload = sanitizeValue(payload, depth + 1, nodes, false);
+ sanitized.put("payload", sanitizedPayload);
+ if (containsReentryMarker(sanitizedPayload)) {
+ sanitized.put("requires_reentry", true);
+ }
+ } else {
+ sanitized.put("requires_reentry", true);
+ }
+ if (source.containsKey("headers")) {
+ sanitized.put("requires_reentry", true);
+ }
+ return sanitized;
+ }
+
+ private boolean containsReentryMarker(Object value) {
+ if (value instanceof Map) {
+ Map<?, ?> map = (Map<?, ?>) value;
+ if (Boolean.TRUE.equals(map.get("requires_reentry"))) {
+ return true;
+ }
+ for (Object child : map.values()) {
+ if (containsReentryMarker(child)) {
+ return true;
+ }
+ }
+ } else if (value instanceof Collection) {
+ for (Object child : (Collection<?>) value) {
+ if (containsReentryMarker(child)) {
+ return true;
+ }
+ }
+ }
+ return false;
+ }
+
+ private boolean isSensitiveProperty(Map<?, ?> value) {
+ for (String nameKey : List.of("name", "property_name", "propertyName", "key")) {
+ Object name = value.get(nameKey);
+ if (name != null && SENSITIVE_KEY.matcher(String.valueOf(name)).matches()) {
+ return true;
+ }
+ }
+ for (String attributesKey : List.of("propertyAttributes", "property_attributes",
+ "property_value_attributes")) {
+ Object attributes = value.get(attributesKey);
+ if (attributes instanceof Map) {
+ Object type = ((Map<?, ?>) attributes).get("type");
+ if (type != null && "password".equalsIgnoreCase(String.valueOf(type))) {
+ return true;
+ }
+ }
+ }
+ return false;
+ }
+
+ private boolean isSafePathMetadata(Map<?, ?> property, Object value) {
+ if (!(value instanceof String) || !((String) value).startsWith("/")) {
+ return false;
+ }
+ String propertyName = null;
+ for (String nameKey : List.of("name", "property_name", "propertyName", "key")) {
+ if (property.get(nameKey) != null) {
+ propertyName = String.valueOf(property.get(nameKey)).toLowerCase(Locale.ROOT);
+ break;
+ }
+ }
+ return propertyName != null
+ && (propertyName.contains("keytab") || propertyName.contains("principal"))
+ && !((String) value).contains("\n") && !((String) value).contains("\u0000");
+ }
+
+ private boolean containsUrlUserInfo(Object value) {
+ if (!(value instanceof String)) {
+ return false;
+ }
+ try {
+ return new URI((String) value).getRawUserInfo() != null;
+ } catch (URISyntaxException e) {
+ return false;
+ }
+ }
+
+ private boolean isOpaqueCredentialValue(String value) {
+ return OPAQUE_SENSITIVE_VALUE.matcher(value).matches()
+ || PRIVATE_KEY_VALUE.matcher(value).matches();
+ }
+
+ @SuppressWarnings("unchecked")
+ private ScopedWorkflowState toState(ScopedWorkflowStateEntity entity) {
+ if (entity == null || entity.getRevision() == null || entity.getRevision() == 0) {
+ return ScopedWorkflowState.empty();
+ }
+ try {
+ if (entity.getRevision() < 0 || entity.getWorkflow() == null || entity.getPhase() == null
+ || entity.getPayload() == null
+ || (!"IDLE".equals(entity.getWorkflow()) && entity.getOwnerUserId() == null)) {
+ throw new JsonParseException("Required workflow state fields are missing");
+ }
+ Map<String, Object> values = StageUtils.getGson().fromJson(entity.getPayload(), Map.class);
+ if (values == null) {
+ values = Collections.emptyMap();
+ }
+ return new ScopedWorkflowState(entity.getRevision(), entity.getOwnerName(),
+ entity.getWorkflow(), entity.getPhase(), values);
+ } catch (JsonParseException | ClassCastException e) {
+ throw serverError("WORKFLOW_STATE_CORRUPT", "Persisted workflow state is invalid", e);
+ }
+ }
+
+ private AuthenticatedUser authenticatedUser() throws AuthorizationException {
+ AuthenticatedUser user = authenticatedUserOrNull();
+ if (user == null) {
+ throw new AuthorizationException();
+ }
+ return user;
+ }
+
+ private AuthenticatedUser authenticatedUserOrNull() {
+ String username = AuthorizationHelper.getAuthenticatedName();
+ int userId = AuthorizationHelper.getAuthenticatedId();
+ if (username == null || username.isEmpty() || userId < 0) {
+ return null;
+ }
+ UserEntity user = userDAO.findByPK(userId);
+ if (user == null || !Boolean.TRUE.equals(user.getActive())
+ || !username.equalsIgnoreCase(user.getUserName())) {
+ return null;
+ }
+ return new AuthenticatedUser(user.getUserId(), user.getUserName());
+ }
+
+ private void rejectReservedKey(String key) {
+ if (isReservedKey(key)) {
+ throw new WebApplicationException(Response.status(Response.Status.FORBIDDEN)
+ .type(MediaType.APPLICATION_JSON_TYPE)
+ .entity(error("RESERVED_PERSIST_KEY", "The requested key is reserved for scoped workflow state"))
+ .build());
+ }
+ }
+
+ private WebApplicationException badRequest(String code, String message) {
+ return errorResponse(Response.Status.BAD_REQUEST, code, message, null);
+ }
+
+ private WebApplicationException conflict(String code, String message) {
+ return errorResponse(Response.Status.CONFLICT, code, message, null);
+ }
+
+ private WebApplicationException serverError(String code, String message, Throwable cause) {
+ return errorResponse(Response.Status.INTERNAL_SERVER_ERROR, code, message, cause);
+ }
+
+ private WebApplicationException errorResponse(Response.Status status, String code, String message,
+ Throwable cause) {
+ Response response = Response.status(status)
+ .type(MediaType.APPLICATION_JSON_TYPE)
+ .entity(error(code, message))
+ .build();
+ return cause == null ? new WebApplicationException(response) : new WebApplicationException(cause, response);
+ }
+
+ private Map<String, String> error(String code, String message) {
+ Map<String, String> error = new LinkedHashMap<>();
+ error.put("code", code);
+ error.put("message", message);
+ return error;
+ }
+
+ private static final class ScopeKey {
+ private final Cluster cluster;
+ private final String storageKey;
+
+ private ScopeKey(Cluster cluster, String storageKey) {
+ this.cluster = cluster;
+ this.storageKey = storageKey;
+ }
+
+ private static ScopeKey cluster(Cluster cluster, String storageKey) {
+ return new ScopeKey(cluster, storageKey);
+ }
+
+ private static ScopeKey draft(String storageKey) {
+ return new ScopeKey(null, storageKey);
+ }
+ }
+
+ private static final class AuthenticatedUser {
+ private final Integer id;
+ private final String username;
+
+ private AuthenticatedUser(Integer id, String username) {
+ this.id = id;
+ this.username = username;
+ }
+ }
}
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/api/services/PersistKeyValueService.java b/ambari-server/src/main/java/org/apache/ambari/server/api/services/PersistKeyValueService.java
index 33bd3c80b9..15f3a9647a 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/api/services/PersistKeyValueService.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/api/services/PersistKeyValueService.java
@@ -19,21 +19,26 @@
package org.apache.ambari.server.api.services;
import java.io.IOException;
-import java.util.ArrayList;
+import java.nio.charset.StandardCharsets;
import java.util.Collection;
import java.util.Map;
+import jakarta.ws.rs.Consumes;
+import jakarta.ws.rs.DefaultValue;
import jakarta.ws.rs.GET;
import jakarta.ws.rs.POST;
import jakarta.ws.rs.PUT;
import jakarta.ws.rs.Path;
import jakarta.ws.rs.PathParam;
import jakarta.ws.rs.Produces;
+import jakarta.ws.rs.QueryParam;
import jakarta.ws.rs.WebApplicationException;
+import jakarta.ws.rs.core.MediaType;
import jakarta.ws.rs.core.Response;
import jakarta.xml.bind.JAXBException;
import org.apache.ambari.annotations.ApiIgnore;
+import org.apache.ambari.server.security.authorization.AuthorizationException;
import org.apache.ambari.server.state.fsm.InvalidStateTransitionException;
import org.apache.ambari.server.utils.StageUtils;
import org.slf4j.Logger;
@@ -56,46 +61,94 @@ public class PersistKeyValueService {
@Produces("text/plain")
public Response update(String keyValues)
throws WebApplicationException, InvalidStateTransitionException,
- JAXBException, IOException {
- LOG.debug("Received message from UI {}", keyValues);
+ JAXBException, IOException, AuthorizationException {
+ LOG.debug("Received persisted key-value update");
Map<String, String> keyValuesMap = StageUtils.fromJson(keyValues, Map.class);
- /* Call into the heartbeat handler */
-
- for (Map.Entry<String, String> keyValue: keyValuesMap.entrySet()) {
- persistKeyVal.put(keyValue.getKey(), keyValue.getValue());
- }
+ persistKeyVal.putLegacyValues(keyValuesMap);
return Response.status(Response.Status.ACCEPTED).build();
}
@SuppressWarnings("unchecked")
@PUT @ApiIgnore // until documented
@Produces("text/plain")
- public String store(String values) throws IOException, JAXBException {
- LOG.debug("Received message from UI {}", values);
+ public String store(String values) throws IOException, JAXBException, AuthorizationException {
+ LOG.debug("Received generated-key persisted value update");
Collection<String> valueCollection = StageUtils.fromJson(values, Collection.class);
- Collection<String> keys = new ArrayList<>(valueCollection.size());
- for (String s : valueCollection) {
- keys.add(persistKeyVal.put(s));
- }
+ Collection<String> keys = persistKeyVal.putLegacyGeneratedValues(valueCollection);
String stringRet = StageUtils.jaxbToString(keys);
- LOG.debug("Returning {}", stringRet);
return stringRet;
}
+ @GET
+ @ApiIgnore
+ @Path("scopes/{scopeType}/{scopeId}")
+ @Produces(MediaType.APPLICATION_JSON)
+ public Response getScopedState(@PathParam("scopeType") String scopeType,
+ @PathParam("scopeId") String scopeId,
+ @QueryParam("summary") @DefaultValue("false") boolean summary) throws AuthorizationException {
+ ScopedWorkflowState state = persistKeyVal.getScopedState(scopeType, scopeId, summary);
+ return Response.ok(state.toResponse(summary), MediaType.APPLICATION_JSON_TYPE).build();
+ }
+
+ @GET
+ @ApiIgnore
+ @Path("scopes/drafts")
+ @Produces(MediaType.APPLICATION_JSON)
+ public Response getCreationDraftDirectory() throws AuthorizationException {
+ return Response.ok(persistKeyVal.getCreationDraftDirectory(), MediaType.APPLICATION_JSON_TYPE).build();
+ }
+
+ @GET
+ @ApiIgnore
+ @Path("scopes/drafts/{scopeId}/cluster")
+ @Produces(MediaType.APPLICATION_JSON)
+ public Response getCreationDraftCluster(@PathParam("scopeId") String scopeId)
+ throws AuthorizationException {
+ return Response.ok(persistKeyVal.getCreationDraftCluster(scopeId),
+ MediaType.APPLICATION_JSON_TYPE).build();
+ }
+
+ @PUT
+ @ApiIgnore
+ @Path("scopes/{scopeType}/{scopeId}")
+ @Consumes(MediaType.APPLICATION_JSON)
+ @Produces(MediaType.APPLICATION_JSON)
+ public Response putScopedState(@PathParam("scopeType") String scopeType,
+ @PathParam("scopeId") String scopeId, String requestBody) throws AuthorizationException {
+ if (requestBody == null || requestBody.length() > PersistKeyValueImpl.MAX_SCOPED_REQUEST_BYTES
+ || requestBody.getBytes(StandardCharsets.UTF_8).length > PersistKeyValueImpl.MAX_SCOPED_REQUEST_BYTES) {
+ throw new WebApplicationException(Response.status(Response.Status.REQUEST_ENTITY_TOO_LARGE)
+ .type(MediaType.APPLICATION_JSON_TYPE)
+ .entity(Map.of("code", "WORKFLOW_STATE_TOO_LARGE",
+ "message", "The workflow request exceeds the maximum persisted size"))
+ .build());
+ }
+ ScopedWorkflowUpdate update;
+ try {
+ update = StageUtils.fromJson(requestBody, ScopedWorkflowUpdate.class);
+ } catch (IOException e) {
+ throw new WebApplicationException(Response.status(Response.Status.BAD_REQUEST)
+ .type(MediaType.APPLICATION_JSON_TYPE)
+ .entity(Map.of("code", "INVALID_WORKFLOW_STATE", "message", "Request body is not valid JSON"))
+ .build());
+ }
+ ScopedWorkflowState state = persistKeyVal.putScopedState(scopeType, scopeId, update);
+ return Response.ok(state.toResponse(false), MediaType.APPLICATION_JSON_TYPE).build();
+ }
+
@GET @ApiIgnore // until documented
@Produces("text/plain")
@Path("{keyName}")
- public String getKey( @PathParam("keyName") String keyName) {
+ public String getKey( @PathParam("keyName") String keyName) throws AuthorizationException {
LOG.debug("Looking for keyName {}", keyName);
- return persistKeyVal.getValue(keyName);
+ return persistKeyVal.getLegacyValue(keyName);
}
@GET @ApiIgnore // until documented
@Produces("text/plain")
public String getAllKeyValues() throws JAXBException, IOException {
- Map<String, String> ret = persistKeyVal.getAllKeyValues();
+ Map<String, String> ret = persistKeyVal.getAllLegacyKeyValues();
String stringRet = StageUtils.jaxbToString(ret);
- LOG.debug("Returning {}", stringRet);
return stringRet;
}
}
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/api/services/ScopedWorkflowDraftSummary.java b/ambari-server/src/main/java/org/apache/ambari/server/api/services/ScopedWorkflowDraftSummary.java
new file mode 100644
index 0000000000..6c015a430d
--- /dev/null
+++ b/ambari-server/src/main/java/org/apache/ambari/server/api/services/ScopedWorkflowDraftSummary.java
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ambari.server.api.services;
+
+import java.util.LinkedHashMap;
+import java.util.Map;
+
+/** Credential-free summary used by the owned creation draft directory. */
+public class ScopedWorkflowDraftSummary {
+ private final String draftId;
+ private final long revision;
+ private final String workflow;
+ private final String phase;
+ private final Long clusterId;
+ private final String clusterName;
+
+ public ScopedWorkflowDraftSummary(String draftId, long revision, String workflow, String phase,
+ Long clusterId, String clusterName) {
+ this.draftId = draftId;
+ this.revision = revision;
+ this.workflow = workflow;
+ this.phase = phase;
+ this.clusterId = clusterId;
+ this.clusterName = clusterName;
+ }
+
+ public Map<String, Object> toResponse() {
+ Map<String, Object> response = new LinkedHashMap<>();
+ response.put("draft_id", draftId);
+ response.put("revision", revision);
+ response.put("workflow", workflow);
+ response.put("phase", phase);
+ if (clusterId != null) {
+ response.put("cluster_id", clusterId);
+ response.put("cluster_name", clusterName);
+ }
+ return response;
+ }
+}
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/api/services/ScopedWorkflowState.java b/ambari-server/src/main/java/org/apache/ambari/server/api/services/ScopedWorkflowState.java
new file mode 100644
index 0000000000..366f6b00d6
--- /dev/null
+++ b/ambari-server/src/main/java/org/apache/ambari/server/api/services/ScopedWorkflowState.java
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ambari.server.api.services;
+
+import java.util.Collections;
+import java.util.LinkedHashMap;
+import java.util.Map;
+
+public class ScopedWorkflowState {
+ private long revision;
+ private String owner;
+ private String workflow;
+ private String phase;
+ private Map<String, Object> values;
+
+ public ScopedWorkflowState() {
+ values = new LinkedHashMap<>();
+ }
+
+ public ScopedWorkflowState(long revision, String owner, String workflow, String phase,
+ Map<String, Object> values) {
+ this.revision = revision;
+ this.owner = owner;
+ this.workflow = workflow;
+ this.phase = phase;
+ this.values = values == null ? new LinkedHashMap<>() : new LinkedHashMap<>(values);
+ }
+
+ public static ScopedWorkflowState empty() {
+ return new ScopedWorkflowState(0, null, "IDLE", "IDLE", Collections.emptyMap());
+ }
+
+ public long getRevision() {
+ return revision;
+ }
+
+ public void setRevision(long revision) {
+ this.revision = revision;
+ }
+
+ public String getOwner() {
+ return owner;
+ }
+
+ public void setOwner(String owner) {
+ this.owner = owner;
+ }
+
+ public String getWorkflow() {
+ return workflow;
+ }
+
+ public void setWorkflow(String workflow) {
+ this.workflow = workflow;
+ }
+
+ public String getPhase() {
+ return phase;
+ }
+
+ public void setPhase(String phase) {
+ this.phase = phase;
+ }
+
+ public Map<String, Object> getValues() {
+ return values;
+ }
+
+ public void setValues(Map<String, Object> values) {
+ this.values = values == null ? new LinkedHashMap<>() : new LinkedHashMap<>(values);
+ }
+
+ public Map<String, Object> toResponse(boolean summary) {
+ Map<String, Object> response = new LinkedHashMap<>();
+ response.put("revision", revision);
+ response.put("owner", owner);
+ response.put("workflow", workflow);
+ response.put("phase", phase);
+ if (!summary) {
+ response.put("values", values == null ? Collections.emptyMap() : values);
+ }
+ return response;
+ }
+}
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/api/services/ScopedWorkflowUpdate.java b/ambari-server/src/main/java/org/apache/ambari/server/api/services/ScopedWorkflowUpdate.java
new file mode 100644
index 0000000000..2cbb4d164c
--- /dev/null
+++ b/ambari-server/src/main/java/org/apache/ambari/server/api/services/ScopedWorkflowUpdate.java
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ambari.server.api.services;
+
+import java.util.LinkedHashMap;
+import java.util.Map;
+
+import com.fasterxml.jackson.annotation.JsonProperty;
+
+public class ScopedWorkflowUpdate {
+ @JsonProperty("expected_revision")
+ private Long expectedRevision;
+ private String workflow;
+ private String phase;
+ private Map<String, Object> values;
+
+ public Long getExpectedRevision() {
+ return expectedRevision;
+ }
+
+ public void setExpectedRevision(Long expectedRevision) {
+ this.expectedRevision = expectedRevision;
+ }
+
+ public String getWorkflow() {
+ return workflow;
+ }
+
+ public void setWorkflow(String workflow) {
+ this.workflow = workflow;
+ }
+
+ public String getPhase() {
+ return phase;
+ }
+
+ public void setPhase(String phase) {
+ this.phase = phase;
+ }
+
+ public Map<String, Object> getValues() {
+ return values == null ? new LinkedHashMap<>() : values;
+ }
+
+ public void setValues(Map<String, Object> values) {
+ this.values = values;
+ }
+}
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/api/services/ServiceDependenciesApiService.java b/ambari-server/src/main/java/org/apache/ambari/server/api/services/ServiceDependenciesApiService.java
new file mode 100644
index 0000000000..642e5763bb
--- /dev/null
+++ b/ambari-server/src/main/java/org/apache/ambari/server/api/services/ServiceDependenciesApiService.java
@@ -0,0 +1,146 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ambari.server.api.services;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+import jakarta.ws.rs.GET;
+import jakarta.ws.rs.POST;
+import jakarta.ws.rs.Path;
+import jakarta.ws.rs.QueryParam;
+import jakarta.ws.rs.core.Response;
+
+import org.apache.ambari.server.StaticallyInject;
+import org.apache.ambari.server.controller.dependencies.ManagedDependencyType;
+import org.apache.ambari.server.controller.dependencies.ManagedServiceDependencyCoordinator;
+import org.apache.ambari.server.controller.dependencies.ManagedServiceDependencyCoordinator.ConsumerReference;
+import org.apache.ambari.server.controller.dependencies.ManagedServiceDependencyCoordinator.PreviewSelection;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.google.inject.Inject;
+import com.google.inject.Provider;
+
+import io.swagger.annotations.ApiOperation;
+
+@StaticallyInject
+@Path("/service-dependencies")
+public class ServiceDependenciesApiService {
+ @Inject
+ private static Provider<ManagedServiceDependencyCoordinator> coordinator;
+
+ @GET
+ @Path("/candidates")
+ @ApiOperation(value = "Lists candidate providers for a managed dependency")
+ public Response candidates(@QueryParam("consumer_scope") String consumerScope,
+ @QueryParam("draft_id") String draftId,
+ @QueryParam("expected_draft_revision") String revision,
+ @QueryParam("cluster_id") String clusterId,
+ @QueryParam("expected_workflow_revision") String workflowRevision,
+ @QueryParam("dependency_type") String dependencyType) {
+ return ManagedDependencyApiSupport.invoke(() -> {
+ ConsumerReference consumer;
+ if ("draft".equalsIgnoreCase(consumerScope)) {
+ if (clusterId != null || workflowRevision != null) {
+ throw new IllegalArgumentException("Draft and service plan fields cannot be mixed");
+ }
+ consumer = ConsumerReference.draft(
+ ManagedDependencyApiSupport.uuid(draftId),
+ ManagedDependencyApiSupport.positiveLong(revision, "expected_draft_revision"));
+ } else if ("service_plan".equalsIgnoreCase(consumerScope)) {
+ if (draftId != null || revision != null) {
+ throw new IllegalArgumentException("Draft and service plan fields cannot be mixed");
+ }
+ consumer = ConsumerReference.servicePlan(
+ ManagedDependencyApiSupport.positiveLong(clusterId, "cluster_id"),
+ ManagedDependencyApiSupport.positiveLong(
+ workflowRevision, "expected_workflow_revision"));
+ } else {
+ throw new IllegalArgumentException("consumer_scope must be draft or service_plan");
+ }
+ return Map.of("items", coordinator.get().candidates(consumer,
+ ManagedDependencyApiSupport.type(dependencyType)));
+ });
+ }
+
+ @POST
+ @Path("/preview")
+ @ApiOperation(value = "Previews a managed dependency")
+ public Response preview(String body) {
+ return ManagedDependencyApiSupport.invoke(() -> {
+ JsonNode root = ManagedDependencyApiSupport.body(body,
+ Set.of("binding_id", "consumer", "dependency_type", "provider", "selections"));
+ if (root.has("selections")) {
+ if (root.size() != 2 || !root.has("consumer")
+ || !root.get("selections").isArray()
+ || root.get("selections").size() < 1 || root.get("selections").size() > 2) {
+ throw new IllegalArgumentException(
+ "A complete preview requires consumer and one or two selections");
+ }
+ JsonNode consumer = ManagedDependencyApiSupport.requiredObject(root, "consumer",
+ Set.of("scope", "draft_id", "cluster_id", "expected_revision"));
+ List<PreviewSelection> selections = new ArrayList<>();
+ for (JsonNode selection : root.get("selections")) {
+ selections.add(ManagedDependencyApiSupport.previewSelection(selection));
+ }
+ return coordinator.get().preview(consumerReference(consumer), selections);
+ }
+ JsonNode consumer = ManagedDependencyApiSupport.requiredObject(root, "consumer",
+ Set.of("scope", "draft_id", "cluster_id", "expected_revision"));
+ ConsumerReference consumerReference = consumerReference(consumer);
+ JsonNode provider = ManagedDependencyApiSupport.requiredObject(root, "provider",
+ Set.of("cluster_id", "service_name"));
+ ManagedDependencyType type = ManagedDependencyApiSupport.type(
+ ManagedDependencyApiSupport.text(root, "dependency_type"));
+ return coordinator.get().preview(consumerReference, type,
+ ManagedDependencyApiSupport.provider(provider),
+ ManagedDependencyApiSupport.optionalUuid(root, "binding_id"));
+ });
+ }
+
+ private static ConsumerReference consumerReference(JsonNode consumer) {
+ String scope = ManagedDependencyApiSupport.text(consumer, "scope");
+ if ("DRAFT".equals(scope)) {
+ if (consumer.has("cluster_id")) {
+ throw new IllegalArgumentException("Draft and service plan fields cannot be mixed");
+ }
+ return ConsumerReference.draft(
+ ManagedDependencyApiSupport.uuid(
+ ManagedDependencyApiSupport.text(consumer, "draft_id")),
+ ManagedDependencyApiSupport.positiveLong(consumer, "expected_revision"));
+ }
... 98993 lines suppressed ...
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@ambari.apache.org
For additional commands, e-mail: commits-help@ambari.apache.org