You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@ambari.apache.org by ji...@apache.org on 2026/09/07 06:47:41 UTC
(ambari) branch trunk updated: AMBARI-26638: Replace legacy Ambari Metrics with Prometheus and VictoriaMetrics (#4182)
This is an automated email from the ASF dual-hosted git repository.
JiaLiangC pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/ambari.git
The following commit(s) were added to refs/heads/trunk by this push:
new 40392df355 AMBARI-26638: Replace legacy Ambari Metrics with Prometheus and VictoriaMetrics (#4182)
40392df355 is described below
commit 40392df35500954ed1d633fd358771b152758c81
Author: jialiang <ji...@gmail.com>
AuthorDate: Mon Sep 7 06:47:33 2026 +0000
AMBARI-26638: Replace legacy Ambari Metrics with Prometheus and VictoriaMetrics (#4182)
* AMBARI-26638: Add Ambari Agent Prometheus telemetry
* AMBARI-26638: Replace legacy monitoring with Prometheus
* AMBARI-26638: Expose VictoriaMetrics in React service navigation
* AMBARI-26638: Provision managed Metrics discovery identity
* AMBARI-26638: Prepare Metrics runtime directories
* AMBARI-26638: Preserve PromQL query parameter encoding
* AMBARI-26638: Fix monitoring route contract types
* AMBARI-26638: Guard missing service update messages
* AMBARI-26638: Allow Metrics RPM version override
* AMBARI-26638: Add native Linux telemetry dashboards
* AMBARI-26638: Harden React monitoring data handling
* AMBARI-26638: Enforce VictoriaMetrics deployment topology
* AMBARI-26638: Exclude Metrics RPM from stack select
* AMBARI-26638: Provision Metrics identity before startup
* AMBARI-26638: Show Hive monitoring dashboards
* AMBARI-26638: Add generic service summary and live health state
* AMBARI-26638: Add runtime monitoring evidence
* AMBARI-26638: Replace legacy Ambari Metrics with Prometheus and VictoriaMetrics
* AMBARI-26638: Document monitoring architecture in English and Chinese
* AMBARI-26638: Preserve monitoring architecture links
* AMBARI-26638: Migrate monitoring dashboard to native React
* AMBARI-26638: Document native monitoring dashboard migration
* AMBARI-26638: Fix dashboard type imports
* AMBARI-26638: Tighten dashboard schema parsing types
* AMBARI-26638: Improve dashboard chart primitives
* AMBARI-26638: Add rich dashboard visualizations
* AMBARI-26638: Add dashboard workspace editors
* AMBARI-26638: Add editable dashboard workspace
* AMBARI-26638: Polish dashboard workspace styling
* AMBARI-26638: Refresh built-in monitoring dashboards
* AMBARI-26638: Document dashboard migration design
* AMBARI-26638: Expose chart editing from embedded dashboards
* AMBARI-26638: Expand native Linux host metrics
* AMBARI-26638: Harden built-in dashboard lifecycle
* AMBARI-26638: Improve dashboard data rendering
* AMBARI-26638: Refine dashboard editing workflow
* AMBARI-26638: Refresh native monitoring dashboards
* AMBARI-26638: Update dashboard migration evidence
* AMBARI-26638: Repair managed metrics credential drift
* AMBARI-26638: Complete trunk conflict resolution
* AMBARI-26638: Fix VictoriaMetrics advisor module loading
* AMBARI-26638: Add license to metrics contract fixture
* AMBARI-26638: Use standard JSON for telemetry cache
* AMBARI-26638: Implement cached telemetry lookup contract
* AMBARI-26638: Align VictoriaMetrics with stack contracts
* AMBARI-26638: Remove obsolete AMS test contracts
* AMBARI-26638: Repair metrics server test contracts
* AMBARI-26638: Align topology and stack test expectations
* AMBARI-26638: Remove stale monitoring imports
* AMBARI-26638: Stabilize monitoring UI tests
* AMBARI-26638: Add Simplified Chinese support to React UI
Add a complete Chinese locale for the PR branch, browser language detection, persistent language selection, and localized login and navigation controls. Preserve existing authentication and monitoring behavior.
Tested with 35 focused tests, TypeScript, a production build, targeted ESLint, and Chromium checks at desktop and mobile sizes. The full suite passed 1070 tests; two existing ServiceComponents tests also fail at the original PR head because HostsListStateProvider is missing.
* AMBARI-26638: Stabilize AlertDispatch DAO test isolation
* AMBARI-26638: Provide Hosts state in service summary tests
---
Jenkinsfile | 16 -
LICENSE.txt | 3 +
README.md | 8 +-
.../app/scripts/services/AddVersionModal.js | 2 +-
.../ui/admin-web/app/scripts/services/Stack.js | 6 +-
.../ui/ambari-admin/src/__mocks__/mockRbacData.ts | 26 +-
.../src/__mocks__/mockVersionDefinitions.ts | 17 -
ambari-agent/conf/unix/ambari-agent.ini | 6 +
.../python/ambari_agent/AlertSchedulerHandler.py | 4 -
.../src/main/python/ambari_agent/AmbariConfig.py | 37 +
.../src/main/python/ambari_agent/Constants.py | 2 +
.../main/python/ambari_agent/HeartbeatThread.py | 44 +-
.../src/main/python/ambari_agent/HostInfo.py | 4 -
.../main/python/ambari_agent/InitializerModule.py | 6 +
.../python/ambari_agent/TelemetryConfigCache.py | 166 +
.../main/python/ambari_agent/alerts/ams_alert.py | 274 -
.../listeners/TelemetryEventListener.py | 57 +
ambari-agent/src/main/python/ambari_agent/main.py | 3 +
.../main/python/ambari_agent/metrics}/__init__.py | 6 +-
.../src/main/python/ambari_agent/metrics/core.py | 162 +
.../src/main/python/ambari_agent/metrics/linux.py | 694 +
.../src/main/python/ambari_agent/metrics/server.py | 168 +
.../main/python/ambari_agent/metrics/telemetry.py | 656 +
.../src/test/python/ambari_agent/TestASTChecker.py | 251 -
.../test/python/ambari_agent/TestActionQueue.py | 8 +-
.../python/ambari_agent/TestActualConfigHandler.py | 3 -
.../ambari_agent/TestAlertSchedulerHandler.py | 19 -
.../src/test/python/ambari_agent/TestAlerts.py | 119 +-
.../src/test/python/ambari_agent/TestAmsAlert.py | 265 -
.../test/python/ambari_agent/TestHostCleanup.py | 4 +-
.../python/ambari_agent/TestPrometheusMetrics.py | 530 +
.../python/ambari_agent/TestTelemetryHeartbeat.py | 93 +
.../python/ambari_agent/TestTelemetryMetrics.py | 560 +
.../ambari_agent/dummy_files/dummy_current_stack | 1 -
.../ambari_agent/examples/ControllerTester.py | 6 -
.../fixtures/telemetry/hbase-master-jmx.json | 42 +
.../fixtures/telemetry/hbase-regionserver-jmx.json | 59 +
.../fixtures/telemetry/hiveserver2-jmx.json | 48 +
.../fixtures/telemetry/nodemanager-jmx.json | 24 +
.../python/ambari_commons/ambari_metrics_helper.py | 288 -
.../repo_manager/repo_manager_helper.py | 72 -
.../pluggable_stack_definition/configs/ODP.json | 12 -
.../pluggable_stack_definition/configs/PHD.json | 24 -
.../pluggable_stack_definition/configs/SAPHD.json | 12 -
.../libraries/functions/constants.py | 2 -
.../libraries/functions/curl_krb_request.py | 16 +-
.../libraries/functions/package_conditions.py | 12 -
.../src/test/resources/metrics.properties | 24 -
.../stacks/HDP/1.2.0/services/GANGLIA/metainfo.xml | 97 -
.../services/GANGLIA/configuration/global.xml | 55 -
.../stacks/HDP/1.3.0/services/GANGLIA/metainfo.xml | 97 -
.../stacks/HDP/1.3.1/services/GANGLIA/metainfo.xml | 97 -
.../stacks/HDP/1.3.4/services/GANGLIA/metainfo.xml | 96 -
.../stacks/HDP/2.0.1/services/GANGLIA/metainfo.xml | 129 -
.../stacks/HDP/2.0.5/services/GANGLIA/metainfo.xml | 130 -
.../stacks/HDP/2.0.5/services/HBASE/metrics.json | 3206 -
.../stacks/HDP/2.0.5/services/HDFS/metrics.json | 2961 -
.../stacks/HDP/2.0.5/services/YARN/metrics.json | 1661 -
.../stacks/HDP/2.0.6.1/role_command_order.json | 5 +-
.../stacks/HDP/2.0.6/role_command_order.json | 5 +-
.../stacks/HDP/2.0.7/role_command_order.json | 5 +-
.../HIVE/package/scripts/hive_metastore.py | 1 -
.../services/HIVE/package/scripts/hive_server.py | 1 -
.../stacks/HDP/2.0.8/role_command_order.json | 5 +-
.../stacks/HDP/2.0.8/services/HDFS/metrics.json | 7860 -
.../stacks/HDP/2.1.1/role_command_order.json | 5 +-
.../HDP/2.1.1/services/AMBARI_METRICS/metainfo.xml | 120 -
.../stacks/HDP/2.1.1/services/STORM/metrics.json | 118 -
.../stacks/HDP/2.2.0/role_command_order.json | 5 +-
.../stacks/HDP/2.2.0/services/GANGLIA/metainfo.xml | 39 -
.../HDP/2.2.0/upgrades/upgrade_test_checks.xml | 6 -
.../stacks/OTHER/1.0/role_command_order.json | 5 +-
.../stacks/OTHER/1.0/services/HDFS/metrics.json | 7861 -
.../stacks/OTHER/1.0/services/STORM/metrics.json | 103 -
.../stacks/OTHER/2.0/services/HBASE/widgets.json | 190 -
ambari-metrics/README.md | 42 +
ambari-metrics/pom.xml | 152 +
.../src/main/package/rpm/preinstall.sh | 35 +-
.../resources/victoriametrics-checksums.sha256 | 34 +
.../src/main/scripts/prepare-victoriametrics.sh | 196 +
ambari-server/checkstyle.xml | 2 +-
ambari-server/conf/unix/ambari.properties | 3 +
ambari-server/conf/unix/metrics.properties | 66 -
ambari-server/docs/api/generated/index.html | 956 -
ambari-server/docs/api/generated/swagger.json | 191 +-
ambari-server/docs/api/v1/clusters-cluster.md | 7 -
ambari-server/docs/api/v1/index.md | 151 +-
ambari-server/docs/api/v1/schemas.md | 60 -
ambari-server/docs/api/v1/service-resources.md | 4 +-
ambari-server/docs/api/v1/user-get.md | 4 +-
ambari-server/docs/configuration/index.md | 11 -
ambari-server/pom.xml | 17 -
ambari-server/src/main/assemblies/server.xml | 10 -
.../main/java/org/apache/ambari/server/Role.java | 5 -
.../apache/ambari/server/actionmanager/Stage.java | 3 +-
.../ambari/server/agent/HeartBeatHandler.java | 2 +
.../ambari/server/agent/RegistrationResponse.java | 12 +
.../agent/stomp/AgentCurrentDataController.java | 8 +
.../agent/stomp/PrometheusTargetDiscovery.java | 223 +
.../agent/stomp/TelemetryAssignmentCompiler.java | 486 +
.../ambari/server/agent/stomp/TelemetryHolder.java | 142 +
.../api/query/render/ClusterBlueprintRenderer.java | 2 +-
.../api/query/render/MetricsPaddingRenderer.java | 46 -
.../ActiveWidgetLayoutResourceDefinition.java | 89 -
.../api/resources/BaseResourceDefinition.java | 5 -
.../api/resources/ResourceInstanceFactoryImpl.java | 12 -
.../api/resources/UserResourceDefinition.java | 1 -
.../resources/WidgetLayoutResourceDefinition.java | 83 -
.../ambari/server/api/services/AmbariMetaInfo.java | 47 -
.../ambari/server/api/services/ClusterService.java | 94 +-
.../server/api/services/WidgetLayoutService.java | 124 -
.../ambari/server/api/services/WidgetService.java | 126 -
.../api/services/metrics/BoardApiService.java | 235 +
.../server/api/services/metrics/BoardRequest.java | 56 +
.../server/api/services/metrics/BoardResponse.java | 50 +
.../services/metrics/ChartShareRequest.java} | 25 +-
.../api/services/metrics/ChartShareResource.java | 120 +
.../services/metrics/ChartShareResponse.java} | 26 +-
.../api/services/metrics/DatasourceApiService.java | 286 +
.../metrics/DatasourceProxyApiService.java | 108 +
.../api/services/metrics/DatasourceRequest.java | 66 +
.../api/services/metrics/DatasourceResponse.java | 83 +
.../services/metrics/DatasourceResponseMapper.java | 181 +
.../api/services/metrics/PrometheusApiService.java | 307 +
.../services/users/ActiveWidgetLayoutService.java | 112 -
...MetricsHadoopSinkVersionCompatibilityCheck.java | 260 -
.../ambari/server/checks/ServicesUpCheck.java | 2 +-
.../configuration/ComponentSSLConfiguration.java | 11 +-
.../ambari/server/configuration/Configuration.java | 194 +-
.../controller/ActiveWidgetLayoutRequest.java | 66 -
.../controller/ActiveWidgetLayoutResponse.java | 142 -
.../controller/AmbariManagementController.java | 28 +-
.../controller/AmbariManagementControllerImpl.java | 199 +-
.../ambari/server/controller/AmbariServer.java | 19 -
.../ambari/server/controller/ControllerModule.java | 8 -
.../ambari/server/controller/WidgetResponse.java | 192 -
.../AbstractControllerResourceProvider.java | 6 -
.../internal/AbstractPropertyProvider.java | 31 -
.../internal/AbstractProviderModule.java | 309 +-
.../ActiveWidgetLayoutResourceProvider.java | 276 -
.../internal/BlueprintConfigurationProcessor.java | 65 +-
.../internal/OverriddenMetricsHostProvider.java | 106 -
.../server/controller/internal/PropertyInfo.java | 18 -
.../internal/ServiceResourceProvider.java | 3 -
.../internal/StackArtifactResourceProvider.java | 105 +-
.../internal/StackDefinedPropertyProvider.java | 45 +-
.../internal/WidgetLayoutResourceProvider.java | 376 -
.../internal/WidgetResourceProvider.java | 410 -
.../metrics/CollectorHostDownRefreshCounter.java | 41 -
.../controller/metrics/MetricHostProvider.java | 56 -
.../metrics/MetricsCollectorHAClusterState.java | 210 -
.../metrics/MetricsCollectorHAManager.java | 153 -
.../metrics/MetricsDataTransferMethod.java | 22 -
.../metrics/MetricsDataTransferMethodFactory.java | 80 -
.../metrics/MetricsDownsamplingMethod.java | 49 -
.../metrics/MetricsDownsamplingMethodFactory.java | 174 -
.../controller/metrics/MetricsPaddingMethod.java | 129 -
.../metrics/MetricsPropertyProvider.java | 193 -
.../metrics/MetricsPropertyProviderProxy.java | 203 -
.../metrics/MetricsReportPropertyProvider.java | 82 -
.../MetricsReportPropertyProviderProxy.java | 124 -
.../ganglia/GangliaComponentPropertyProvider.java | 72 -
.../GangliaHostComponentPropertyProvider.java | 71 -
.../ganglia/GangliaHostPropertyProvider.java | 84 -
.../controller/metrics/ganglia/GangliaMetric.java | 257 -
.../metrics/ganglia/GangliaPropertyProvider.java | 628 -
.../ganglia/GangliaReportPropertyProvider.java | 225 -
.../timeline/AMSComponentPropertyProvider.java | 54 -
.../timeline/AMSHostComponentPropertyProvider.java | 56 -
.../metrics/timeline/AMSHostPropertyProvider.java | 52 -
.../metrics/timeline/AMSPropertyProvider.java | 740 -
.../timeline/AMSReportPropertyProvider.java | 318 -
.../metrics/timeline/MetricsRequestHelper.java | 182 -
.../timeline/cache/TimelineAppMetricCacheKey.java | 128 -
.../cache/TimelineAppMetricCacheKeySerializer.java | 78 -
.../timeline/cache/TimelineMetricCache.java | 124 -
.../cache/TimelineMetricCacheCustomExpiry.java | 55 -
.../cache/TimelineMetricCacheEntryFactory.java | 345 -
.../cache/TimelineMetricCacheProvider.java | 122 -
.../timeline/cache/TimelineMetricsCacheValue.java | 109 -
.../cache/TimelineMetricsCacheValueSerializer.java | 79 -
.../ambari/server/controller/spi/Resource.java | 6 -
.../controller/utilities/PropertyHelper.java | 113 +-
.../apache/ambari/server/events/AmbariEvent.java | 5 -
.../server/events/DefaultMessageEmitter.java | 2 +
.../events/MetricsCollectorHostDownEvent.java | 47 -
.../apache/ambari/server/events/STOMPEvent.java | 1 +
.../ambari/server/events/TelemetryUpdateEvent.java | 70 +
.../ambari/server/metrics/system/MetricsSink.java | 43 -
.../server/metrics/system/MetricsSource.java | 34 -
.../metrics/system/impl/AbstractMetricsSource.java | 34 -
.../metrics/system/impl/AmbariMetricSinkImpl.java | 350 -
.../system/impl/AmbariPerformanceMonitor.java | 141 -
.../metrics/system/impl/DatabaseMetricsSource.java | 199 -
.../metrics/system/impl/JvmMetricsSource.java | 118 -
.../metrics/system/impl/MetricsConfiguration.java | 153 -
.../metrics/system/impl/MetricsServiceImpl.java | 154 -
.../system/impl/StompEventsMetricsSource.java | 105 -
.../system/impl/StompStatsMetricsSource.java | 128 -
.../org/apache/ambari/server/orm/dao/BoardDAO.java | 92 +
.../dao/BoardPayloadDAO.java} | 28 +-
.../ambari/server/orm/dao/ChartShareDAO.java | 48 +
.../ambari/server/orm/dao/DatasourceDAO.java | 82 +
.../apache/ambari/server/orm/dao/WidgetDAO.java | 121 -
.../ambari/server/orm/dao/WidgetLayoutDAO.java | 127 -
.../ambari/server/orm/entities/BoardEntity.java | 160 +
.../entities/BoardPayloadEntity.java} | 41 +-
.../server/orm/entities/ChartShareEntity.java | 74 +
.../ambari/server/orm/entities/ClusterEntity.java | 6 -
.../server/orm/entities/DatasourceEntity.java | 283 +
.../ambari/server/orm/entities/UserEntity.java | 11 -
.../ambari/server/orm/entities/WidgetEntity.java | 268 -
.../server/orm/entities/WidgetLayoutEntity.java | 175 -
.../orm/entities/WidgetLayoutUserWidgetEntity.java | 111 -
.../entities/WidgetLayoutUserWidgetEntityPK.java | 71 -
.../authorization/AmbariAuthorizationFilter.java | 4 -
.../security/authorization/RoleAuthorization.java | 1 -
.../metrics/BuiltinDashboardProvisioner.java | 158 +
.../metrics/BuiltinDatasourceProvisioner.java | 175 +
.../server/service/metrics/ChartShareService.java | 161 +
.../server/service/metrics/DashboardService.java | 527 +
.../metrics/DatasourceConfigurationSecrets.java | 175 +
.../server/service/metrics/DatasourceService.java | 553 +
.../metrics/ManagedMetricsIdentityListener.java | 99 +
.../metrics/ManagedMetricsIdentityService.java | 220 +
.../metrics/PrometheusClientException.java} | 34 +-
.../service/metrics/PrometheusQueryClient.java | 303 +
.../ambari/server/stack/ExtensionModule.java | 4 +-
.../ambari/server/stack/ServiceDirectory.java | 32 +-
.../apache/ambari/server/stack/ServiceModule.java | 8 +-
.../apache/ambari/server/stack/StackDirectory.java | 1 +
.../apache/ambari/server/state/ComponentInfo.java | 2 +-
.../org/apache/ambari/server/state/Service.java | 2 -
.../apache/ambari/server/state/ServiceInfo.java | 42 +-
.../server/state/alert/AlertDefinitionFactory.java | 5 -
.../ambari/server/state/alert/AmsSource.java | 164 -
.../ambari/server/state/alert/MetricSource.java | 16 +-
.../ambari/server/state/alert/PercentSource.java | 16 +-
.../ambari/server/state/alert/SourceType.java | 5 -
.../state/services/MetricsRetrievalService.java | 2 +-
.../apache/ambari/server/state/stack/Metric.java | 13 +-
.../ambari/server/state/stack/WidgetLayout.java | 75 -
.../server/state/stack/WidgetLayoutInfo.java | 115 -
.../ProvisionActionPredicateBuilder.java | 6 +-
.../server/upgrade/AbstractUpgradeCatalog.java | 107 -
.../ambari/server/upgrade/SchemaUpgradeHelper.java | 1 +
.../ambari/server/upgrade/UpgradeCatalog260.java | 43 -
.../ambari/server/upgrade/UpgradeCatalog270.java | 75 -
.../ambari/server/upgrade/UpgradeCatalog272.java | 4 -
.../ambari/server/upgrade/UpgradeCatalog310.java | 189 +
.../org/apache/ambari/server/utils/StageUtils.java | 1 -
.../src/main/python/ambari_server/serverUpgrade.py | 2 +-
.../src/main/python/ambari_server/setupActions.py | 1 -
.../src/main/python/ambari_server/setupHttps.py | 3 -
.../src/main/resources/Ambari-DDL-Derby-CREATE.sql | 100 +-
.../src/main/resources/Ambari-DDL-H2-CREATE.sql | 99 +-
.../src/main/resources/Ambari-DDL-MySQL-CREATE.sql | 99 +-
.../main/resources/Ambari-DDL-Oracle-CREATE.sql | 99 +-
.../main/resources/Ambari-DDL-Postgres-CREATE.sql | 99 +-
.../resources/Ambari-DDL-SQLAnywhere-CREATE.sql | 99 +-
.../main/resources/Ambari-DDL-SQLServer-CREATE.sql | 99 +-
.../src/main/resources/META-INF/persistence.xml | 7 +-
.../3.0.0/package/scripts/params.py | 32 -
.../AMBARI_INFRA_SOLR/3.0.0/properties/solr.xml.j2 | 80 -
.../AMBARI_METRICS/3.0.0/alerts.json | 192 -
.../AMBARI_METRICS/3.0.0/configuration/ams-env.xml | 218 -
.../3.0.0/configuration/ams-grafana-env.xml | 135 -
.../3.0.0/configuration/ams-grafana-ini.xml | 402 -
.../3.0.0/configuration/ams-hbase-env.xml | 277 -
.../3.0.0/configuration/ams-hbase-log4j.xml | 188 -
.../3.0.0/configuration/ams-hbase-policy.xml | 53 -
.../configuration/ams-hbase-security-site.xml | 163 -
.../3.0.0/configuration/ams-hbase-site.xml | 673 -
.../3.0.0/configuration/ams-log4j.xml | 86 -
.../3.0.0/configuration/ams-site.xml | 826 -
.../3.0.0/configuration/ams-ssl-client.xml | 42 -
.../3.0.0/configuration/ams-ssl-server.xml | 86 -
.../AMBARI_METRICS/3.0.0/kerberos.json | 157 -
.../AMBARI_METRICS/3.0.0/metainfo.xml | 229 -
.../AMBARI_METRICS/3.0.0/metrics.json | 2472 -
.../package/alerts/alert_ambari_metrics_monitor.py | 118 -
.../BIGTOP/grafana-hbase-home.json | 2559 -
.../BIGTOP/grafana-hbase-misc.json | 1812 -
.../BIGTOP/grafana-hbase-regionservers.json | 10635 -
.../BIGTOP/grafana-hbase-tables.json | 2285 -
.../BIGTOP/grafana-hbase-users.json | 565 -
.../BIGTOP/grafana-hdfs-datanodes.json | 1771 -
.../BIGTOP/grafana-hdfs-home.json | 757 -
.../BIGTOP/grafana-hdfs-namenodes.json | 1496 -
.../BIGTOP/grafana-hdfs-topn.json | 862 -
.../BIGTOP/grafana-hdfs-users.json | 270 -
.../BIGTOP/grafana-hive-hivemetastore.json | 587 -
.../BIGTOP/grafana-hive-hiverserver2.json | 619 -
.../BIGTOP/grafana-hive-home.json | 1121 -
.../BIGTOP/grafana-infra-solr-collections.json | 2088 -
.../BIGTOP/grafana-infra-solr-cores.json | 2113 -
.../BIGTOP/grafana-infra-solr-hosts.json | 1184 -
.../BIGTOP/grafana-kafka-home.json | 1091 -
.../BIGTOP/grafana-kafka-hosts.json | 2009 -
.../BIGTOP/grafana-kafka-topics.json | 521 -
.../BIGTOP/grafana-llapdaemon-daemons.json | 2174 -
.../BIGTOP/grafana-llapdaemon-heatmaps.json | 364 -
.../BIGTOP/grafana-llapdaemon-overview.json | 1815 -
.../BIGTOP/grafana-logsearch-home.json | 600 -
.../BIGTOP/grafana-solr-cores-dashboard.json | 1597 -
.../BIGTOP/grafana-solr-hosts-dashboard.json | 285 -
.../BIGTOP/grafana-storm-components.json | 579 -
.../BIGTOP/grafana-storm-home.json | 501 -
.../BIGTOP/grafana-storm-kafka-offset.json | 258 -
.../BIGTOP/grafana-storm-topology.json | 894 -
.../BIGTOP/grafana-yarn-applications.json | 519 -
.../BIGTOP/grafana-yarn-home.json | 636 -
.../BIGTOP/grafana-yarn-jobhistoryserver.json | 402 -
.../BIGTOP/grafana-yarn-nodemanagers.json | 1827 -
.../BIGTOP/grafana-yarn-queues.json | 1522 -
.../BIGTOP/grafana-yarn-resourcemanagers.json | 958 -
.../BIGTOP/grafana-yarn-timelineserver.json | 700 -
.../default/grafana-ambari-server-database.json | 918 -
.../default/grafana-ambari-server-stats.json | 1619 -
.../default/grafana-ambari-server-topn.json | 453 -
.../default/grafana-ambari-server.json | 744 -
.../default/grafana-ams-hbase-home.json | 2558 -
.../default/grafana-ams-hbase-misc.json | 1696 -
.../default/grafana-ams-hbase-regionservers.json | 9078 -
.../default/grafana-system-home.json | 1768 -
.../default/grafana-system-servers.json | 2613 -
.../files/service-metrics/AMBARI_METRICS.txt | 245 -
.../3.0.0/package/files/service-metrics/HBASE.txt | 588 -
.../3.0.0/package/files/service-metrics/HDFS.txt | 342 -
.../3.0.0/package/files/service-metrics/HIVE.txt | 184 -
.../3.0.0/package/files/service-metrics/HOST.txt | 55 -
.../3.0.0/package/files/service-metrics/KAFKA.txt | 1104 -
.../3.0.0/package/files/service-metrics/YARN.txt | 178 -
.../AMBARI_METRICS/3.0.0/package/scripts/ams.py | 477 -
.../3.0.0/package/scripts/ams_service.py | 240 -
.../3.0.0/package/scripts/functions.py | 54 -
.../AMBARI_METRICS/3.0.0/package/scripts/hbase.py | 241 -
.../3.0.0/package/scripts/hbase_service.py | 95 -
.../3.0.0/package/scripts/metrics_collector.py | 90 -
.../3.0.0/package/scripts/metrics_grafana.py | 132 -
.../3.0.0/package/scripts/metrics_grafana_util.py | 514 -
.../3.0.0/package/scripts/metrics_monitor.py | 86 -
.../3.0.0/package/scripts/metrics_process.py | 258 -
.../3.0.0/package/scripts/metrics_utils.py | 243 -
.../AMBARI_METRICS/3.0.0/package/scripts/params.py | 806 -
.../3.0.0/package/scripts/params_linux.py | 58 -
.../3.0.0/package/scripts/service_check.py | 411 -
.../AMBARI_METRICS/3.0.0/package/scripts/status.py | 106 -
.../3.0.0/package/scripts/status_params.py | 64 -
.../3.0.0/package/templates/ams.conf.j2 | 35 -
.../package/templates/ams_collector_jaas.conf.j2 | 26 -
.../package/templates/ams_zookeeper_jaas.conf.j2 | 26 -
.../templates/hadoop-metrics2-hbase.properties.j2 | 62 -
.../package/templates/hbase_master_jaas.conf.j2 | 36 -
.../templates/hbase_regionserver_jaas.conf.j2 | 36 -
.../3.0.0/package/templates/metric_groups.conf.j2 | 37 -
.../3.0.0/package/templates/metric_monitor.ini.j2 | 52 -
.../templates/metrics_grafana_datasource.json.j2 | 33 -
.../3.0.0/package/templates/regionservers.j2 | 20 -
.../package/templates/smoketest_metrics.json.j2 | 14 -
.../3.0.0/quicklinks/quicklinks.json | 34 -
.../AMBARI_METRICS/3.0.0/role_command_order.json | 9 -
.../AMBARI_METRICS/3.0.0/service_advisor.py | 1124 -
.../AMBARI_METRICS/3.0.0/themes/credentials.json | 42 -
.../AMBARI_METRICS/3.0.0/themes/directories.json | 247 -
.../AMBARI_METRICS/3.0.0/themes/theme.json | 288 -
.../resources/custom_actions/scripts/check_host.py | 9 -
.../custom_actions/scripts/validate_configs.py | 8 -
.../src/main/resources/ganglia_properties.json | 18906 -
.../resources/host_scripts/alert_version_select.py | 3 +-
.../integrations/Linux/dashboards/HDFS.json | 973 +
.../Linux/dashboards/HDFS_DataNode.json | 2834 +
.../Linux/dashboards/HDFS_NameNode.json | 1383 +
.../Linux/dashboards/HIVE_Server2.json | 339 +
.../integrations/Linux/dashboards/HbaseMaster.json | 1961 +
.../Linux/dashboards/HbaseRegionServer.json | 1955 +
.../Linux/dashboards/Linux_Fleet_Overview.json | 404 +
.../Linux/dashboards/Linux_Host_Detail.json | 3330 +
.../Linux/dashboards/YARN_NodeManager.json | 1792 +
.../Linux/dashboards/YARN_ResourceManager.json | 1413 +
.../Linux/dashboards/YARN_ResourceManager_Sys.json | 1779 +
.../resources/scripts/takeover_files_mapping.json | 5 -
.../stack-hooks/before-ANY/scripts/params.py | 4 -
.../stack-hooks/before-INSTALL/scripts/params.py | 7 -
.../stack-hooks/before-START/scripts/params.py | 173 -
.../before-START/scripts/shared_initialization.py | 8 -
.../templates/hadoop-metrics2.properties.j2 | 114 -
.../stacks/BIGTOP/3.2.0/role_command_order.json | 10 +-
.../3.2.0/services/AMBARI-METRICS/metainfo.xml | 26 -
.../BIGTOP/3.2.0/services/HBASE/metrics.json | 9398 +-
.../3.2.0/services/HBASE/package/scripts/hbase.py | 5 +-
.../services/HBASE/package/scripts/params_linux.py | 82 -
...oop-metrics2-hbase.properties-GANGLIA-MASTER.j2 | 133 -
.../hadoop-metrics2-hbase.properties-GANGLIA-RS.j2 | 131 -
.../templates/hadoop-metrics2-hbase.properties.j2} | 10 +-
.../HBASE/telemetry-profiles/hbase-master-2.4.json | 221 +
.../telemetry-profiles/hbase-regionserver-2.4.json | 327 +
.../BIGTOP/3.2.0/services/HBASE/telemetry.json | 49 +
.../BIGTOP/3.2.0/services/HBASE/widgets.json | 486 -
.../stacks/BIGTOP/3.2.0/services/HDFS/alerts.json | 1046 +-
.../services/HDFS/configuration/core-site.xml | 9 +
.../stacks/BIGTOP/3.2.0/services/HDFS/metrics.json | 7908 +-
.../HDFS/package/alerts/alert_metrics_deviation.py | 621 -
.../3.2.0/services/HDFS/role_command_order.json | 5 +-
.../BIGTOP/3.2.0/services/HDFS/telemetry.json | 69 +
.../stacks/BIGTOP/3.2.0/services/HDFS/widgets.json | 456 -
.../3.2.0/services/HIVE/package/scripts/hive.py | 15 -
.../3.2.0/services/HIVE/package/scripts/params.py | 79 -
.../hadoop-metrics2-hivemetastore.properties.j2 | 62 -
.../hadoop-metrics2-hiveserver2.properties.j2 | 62 -
.../3.2.0/services/HIVE/quicklinks/quicklinks.json | 27 +-
.../HIVE/telemetry-profiles/hiveserver2-3.1.json | 231 +
.../BIGTOP/3.2.0/services/HIVE/telemetry.json | 37 +
.../services/KAFKA/configuration/kafka-broker.xml | 107 -
.../services/KAFKA/configuration/kafka-env.xml | 4 -
.../BIGTOP/3.2.0/services/KAFKA/metrics.json | 238 -
.../3.2.0/services/KAFKA/package/scripts/kafka.py | 16 -
.../3.2.0/services/KAFKA/package/scripts/params.py | 50 -
.../BIGTOP/3.2.0/services/KAFKA/widgets.json | 181 -
.../3.2.0/services/SOLR/package/scripts/params.py | 25 -
.../configuration/spark-metrics-properties.xml | 9 -
.../configuration/victoriametrics-auth.xml | 56 +
.../configuration/victoriametrics-env.xml | 104 +
.../configuration/victoriametrics-scrape.xml | 155 +
.../configuration/victoriametrics.xml | 136 +
.../3.2.0/services/VICTORIAMETRICS/metainfo.xml | 163 +
.../VICTORIAMETRICS/package/scripts/params.py | 318 +
.../package/scripts/service_check.py | 87 +
.../package/scripts/status_params.py | 51 +-
.../package/scripts/victoriametrics.py | 302 +
.../package/templates/promscrape.yml.j2 | 49 +
.../package/templates/vmauth.yml.j2 | 124 +
.../VICTORIAMETRICS/quicklinks/quicklinks.json | 56 +
.../services/VICTORIAMETRICS/service_advisor.py | 300 +
.../3.2.0/services/YARN/MAPREDUCE2_metrics.json | 2596 -
.../BIGTOP/3.2.0/services/YARN/YARN_metrics.json | 3486 -
.../BIGTOP/3.2.0/services/YARN/YARN_widgets.json | 670 -
.../stacks/BIGTOP/3.2.0/services/YARN/metainfo.xml | 6 -
.../services/YARN/package/scripts/params_linux.py | 105 -
.../3.2.0/services/YARN/package/scripts/yarn.py | 22 -
.../templates/hadoop-metrics2-hbase.properties.j2 | 62 -
.../templates/yarn_hbase_secure.yarnfile.j2 | 7 +-
.../templates/yarn_hbase_unsecure.yarnfile.j2 | 7 +-
.../YARN/telemetry-profiles/nodemanager-3.3.json | 125 +
.../BIGTOP/3.2.0/services/YARN/telemetry.json | 70 +
.../services/ZOOKEEPER/role_command_order.json | 3 +-
.../stacks/BIGTOP/3.3.0/role_command_order.json | 10 +-
.../3.3.0/services/AMBARI-METRICS/metainfo.xml | 26 -
.../stacks/BIGTOP/3.4.0/role_command_order.json | 20 +-
.../3.4.0/services/AMBARI-METRICS/metainfo.xml | 26 -
.../src/main/resources/stacks/stack_advisor.py | 54 -
ambari-server/src/main/resources/widgets.json | 95 -
.../apache/ambari/server/H2DatabaseCleaner.java | 2 -
.../server/actionmanager/TestActionScheduler.java | 36 +-
.../agent/stomp/PrometheusTargetDiscoveryTest.java | 93 +
.../stomp/TelemetryAssignmentCompilerTest.java | 172 +
.../query/render/MetricsPaddingRendererTest.java | 71 -
.../api/resources/UserResourceDefinitionTest.java | 1 -
.../server/api/services/AmbariMetaInfoTest.java | 74 -
.../metrics/DatasourceResponseMapperTest.java | 91 +
.../AmbariMetricsHadoopSinkVersionCheckTest.java | 288 -
.../checks/ComponentsInstallationCheckTest.java | 60 +-
.../ambari/server/checks/ServicesUpCheckTest.java | 62 +-
.../ComponentSSLConfigurationTest.java | 4 +-
.../AmbariCustomCommandExecutionHelperTest.java | 82 +-
.../AmbariManagementControllerImplTest.java | 12 +-
.../controller/AmbariManagementControllerTest.java | 91 +-
.../server/controller/AuthToLocalBuilderTest.java | 6 +-
.../internal/AbstractPropertyProviderTest.java | 11 +-
.../ActiveWidgetLayoutResourceProviderTest.java | 430 -
.../BlueprintConfigurationProcessorTest.java | 385 +-
.../internal/BlueprintResourceProviderTest.java | 7 +-
.../ClusterStackVersionResourceProviderTest.java | 48 +-
.../internal/HttpPropertyProviderTest.java | 6 +-
.../internal/MetricsServiceProviderTest.java | 41 -
.../OverriddenMetricsHostProviderTest.java | 95 -
.../StackArtifactResourceProviderTest.java | 52 +-
.../internal/StackDefinedPropertyProviderTest.java | 1244 +-
.../internal/WidgetLayoutResourceProviderTest.java | 398 -
.../internal/WidgetResourceProviderTest.java | 538 -
.../controller/logging/LogQueryResponseTest.java | 8 +-
.../logging/LoggingRequestHelperImplTest.java | 8 +-
.../metrics/JMXPropertyProviderTest.java | 28 -
.../metrics/RestMetricsPropertyProviderTest.java | 46 +-
.../metrics/ganglia/GangliaMetricTest.java | 119 -
.../ganglia/GangliaPropertyProviderTest.java | 1028 -
.../ganglia/GangliaReportPropertyProviderTest.java | 140 -
.../metrics/ganglia/TestStreamProvider.java | 73 -
.../metrics/timeline/AMSPropertyProviderTest.java | 1230 -
.../timeline/AMSReportPropertyProviderTest.java | 155 -
.../metrics/timeline/MetricsPaddingMethodTest.java | 279 -
.../metrics/timeline/MetricsRequestHelperTest.java | 117 -
.../timeline/cache/TimelineMetricCacheTest.java | 528 -
.../controller/utilities/PropertyHelperTest.java | 372 +-
.../upgrade/HostVersionOutOfSyncListenerTest.java | 22 +-
.../ambari/server/metadata/RoleGraphTest.java | 5 -
.../metric/system/impl/MetricsServiceTest.java | 44 -
.../metric/system/impl/MetricsSourceTest.java | 188 -
.../system/impl/TestAmbariMetricsSinkImpl.java | 93 -
.../metric/system/impl/TestMetricsSource.java | 28 -
.../system/impl/StompStatsMetricsSourceTest.java | 112 -
.../server/orm/dao/AlertDispatchDAOTest.java | 4 +-
.../ambari/server/orm/dao/WidgetDAOTest.java | 130 -
.../ambari/server/orm/dao/WidgetLayoutDAOTest.java | 139 -
.../server/orm/entities/MonitoringEntityTest.java | 83 +
.../AmbariAuthorizationFilterTest.java | 3 -
.../metrics/BuiltinDashboardContractTest.java | 326 +
.../metrics/BuiltinDashboardProvisionerTest.java | 114 +
.../metrics/BuiltinDatasourceProvisionerTest.java | 150 +
.../service/metrics/DashboardServiceTest.java | 149 +
.../DatasourceConfigurationSecretsTest.java | 60 +
.../service/metrics/DatasourceServiceTest.java | 130 +
.../metrics/ManagedMetricsIdentityServiceTest.java | 226 +
.../service/metrics/PrometheusQueryClientTest.java | 105 +
.../metrics/PrometheusScrapeContractTest.java | 272 +
.../ambari/server/stack/ServiceModuleTest.java | 1 -
.../ambari/server/stack/StackManagerTest.java | 97 +-
.../upgrade/orchestrate/UpgradeHelperTest.java | 6 +-
.../server/stageplanner/TestStagePlanner.java | 32 +-
.../ambari/server/state/ConfigHelperTest.java | 36 +-
.../ambari/server/state/ServiceInfoTest.java | 11 +-
.../ambari/server/state/cluster/ClusterTest.java | 62 +-
.../state/repository/VersionDefinitionTest.java | 8 +-
.../ambari/server/state/stack/UpgradePackTest.java | 20 -
.../ProvisionActionPredicateBuilderTest.java | 40 +-
.../server/upgrade/UpgradeCatalog260Test.java | 158 -
.../server/upgrade/UpgradeCatalog270Test.java | 153 -
.../server/upgrade/UpgradeCatalog310Test.java | 133 +
.../python/TestAmbariMetricsBigtopConfiguration.py | 192 -
.../python/TestAmbariMetricsBigtopIntegration.py | 140 -
.../python/TestAmbariMetricsBigtopLifecycle.py | 471 -
.../test/python/TestAmbariMetricsBigtopMetadata.py | 113 -
.../test/python/TestAmbariMetricsBigtopRuntime.py | 110 -
.../python/TestBigtopServiceCredentialContracts.py | 4 +-
.../python/TestBigtopServiceProcessContracts.py | 1 -
.../src/test/python/TestBigtopStackCatalog.py | 2 -
.../src/test/python/TestExecutionCommand.py | 11 -
.../test/python/TestExecutionCommand_command.json | 8 +-
.../src/test/python/TestResourceFilesKeeper.py | 2 +-
.../src/test/python/TestServerClassPath.py | 3 -
.../src/test/python/TestServiceAdvisor.py | 113 +
ambari-server/src/test/python/TestStackAdvisor.py | 36 +-
.../src/test/python/TestYarnBigtopConfig.py | 9 -
ambari-server/src/test/python/hdfs.json | 13 +-
.../src/test/python/stacks/configs/default.json | 200 -
.../python/stacks/configs/repository_file.json | 197 -
.../src/test/python/stacks/configs/secured.json | 20 -
.../python/stacks/configs/secured_no_jce_name.json | 11 -
.../stack-hooks/before-START/test_before_start.py | 36 -
.../package/scripts/hive_metastore.py | 1 -
.../0.11.0.2.0.5.0/package/scripts/hive_server.py | 1 -
.../HIVE/package/scripts/hive_metastore.py | 1 -
.../HIVE/package/scripts/hive_server.py | 1 -
.../multinode-default.json | 2 +-
.../resources/ams/aggregate_cluster_metrics.json | 100 -
.../resources/ams/aggregate_component_metric.json | 42 -
.../test/resources/ams/embedded_host_metric.json | 198 -
.../ams/multiple_component_regexp_metrics.json | 496 -
.../test/resources/ams/multiple_host_metrics.json | 218 -
.../resources/ams/single_component_metrics.json | 249 -
.../ams/single_host_component_metrics.json | 21 -
.../ams/single_host_component_metrics_h1.json | 21 -
.../ams/single_host_component_metrics_h2.json | 22 -
.../src/test/resources/ams/single_host_metric.json | 124 -
.../curl-setup-multiple-hbase-master.sh | 98 -
ambari-server/src/test/resources/deploy_HDP2.sh | 85 -
.../src/test/resources/flume_ganglia_data.txt | 478707 ------------------
.../src/test/resources/gsInstaller-hosts.txt | 34 -
.../test/resources/hbase_hbasemaster_jmx_2.json | 6 +-
.../src/test/resources/hdfs_datanode_jmx.json | 6 +-
.../src/test/resources/hdfs_namenode_jmx.json | 6 +-
.../test/resources/host_temporal_ganglia_data.txt | 511 -
.../test/resources/journalnode_ganglia_data.txt | 949 -
.../kerberos/test_kerberos_descriptor_2_1_3.json | 108 -
.../test_kerberos_descriptor_2_5_infra_solr.json | 4 +-
.../test/resources/mapreduce_jobtracker_jmx.json | 6 +-
.../test/resources/mapreduce_tasktracker_jmx.json | 6 +-
.../src/test/resources/metrics.properties | 28 -
.../contracts/prometheus-exporter-types.txt | 221 +
.../mpacks-v2/upgrade-packs/upgrade-basic.xml | 3 -
.../ambari/server/upgrade/kerberos_descriptor.json | 132 +-
.../src/test/resources/resourcemanager_jmx.json | 4 -
.../stacks/HDP/0.1/role_command_order.json | 5 +-
.../stacks/HDP/1.2.0/services/GANGLIA/metainfo.xml | 97 -
.../services/GANGLIA/configuration/global.xml | 59 -
.../stacks/HDP/1.3.0/services/GANGLIA/metainfo.xml | 97 -
.../stacks/HDP/1.3.1/services/GANGLIA/metainfo.xml | 97 -
.../stacks/HDP/1.3.4/services/GANGLIA/metainfo.xml | 96 -
.../stacks/HDP/2.0.1/services/GANGLIA/metainfo.xml | 129 -
.../stacks/HDP/2.0.5/role_command_order.json | 5 +-
.../stacks/HDP/2.0.5/services/GANGLIA/metainfo.xml | 130 -
.../stacks/HDP/2.0.5/services/HBASE/metrics.json | 3206 -
.../stacks/HDP/2.0.5/services/HDFS/metrics.json | 2961 -
.../stacks/HDP/2.0.5/services/YARN/metrics.json | 1661 -
.../stacks/HDP/2.0.6/role_command_order.json | 5 +-
.../stacks/HDP/2.0.7/role_command_order.json | 5 +-
.../HIVE/package/scripts/hive_metastore.py | 1 -
.../services/HIVE/package/scripts/hive_server.py | 1 -
.../stacks/HDP/2.0.8/role_command_order.json | 5 +-
.../stacks/HDP/2.0.8/services/HDFS/metrics.json | 7860 -
.../stacks/HDP/2.1.1/role_command_order.json | 5 +-
.../HDP/2.1.1/services/AMBARI_METRICS/metainfo.xml | 119 -
.../stacks/HDP/2.1.1/services/STORM/metrics.json | 118 -
.../stacks/HDP/2.2.0/role_command_order.json | 5 +-
.../stacks/HDP/2.2.0/services/GANGLIA/metainfo.xml | 39 -
.../GANGLIA/upgrades/HDP/rolling-upgrade.xml | 41 -
.../HDP/2.2.0/upgrades/upgrade_test_15388.xml | 1 -
.../HDP/2.2.0/upgrades/upgrade_test_checks.xml | 6 +-
.../stacks/OTHER/1.0/role_command_order.json | 5 +-
.../stacks/OTHER/1.0/services/STORM/metrics.json | 103 -
.../stacks/OTHER/2.0/services/HBASE/widgets.json | 190 -
.../OTHER/1.0/role_command_order.json | 5 +-
.../OTHER/1.0/services/HDFS/metrics.json | 7860 -
.../HDP/2.2.0/role_command_order.json | 4 +-
.../HDP/2.2.0/upgrades/upgrade_test_15388.xml | 1 -
.../src/test/resources/temporal_ganglia_data.txt | 359 -
.../src/test/resources/temporal_ganglia_data_1.txt | 359 -
.../temporal_ganglia_data_yarn_queues.txt | 6635 -
.../resources/temporal_ganglia_report_data.json | 970 -
ambari-server/src/test/resources/test_api.sh | 62 -
.../src/test/resources/test_multnode_api.sh | 78 -
.../test/resources/upgrade-packs/upgrade-basic.xml | 3 -
ambari-server/src/test/resources/widgets.json | 95 -
.../src/test/resources/yarn_ganglia_data.txt | 1433 -
ambari-web/latest/package-lock.json | 78 +
ambari-web/latest/package.json | 2 +
.../src/Initializers/WizardConfigInitializer.ts | 4 -
ambari-web/latest/src/Utils/Utility.ts | 549 -
.../src/Utils/centralizedServiceStateUtils.ts | 21 +-
ambari-web/latest/src/Utils/configs.ts | 25 +-
ambari-web/latest/src/Utils/metricProcessing.ts | 504 -
.../latest/src/Utils/serviceMetricsCalcUtils.ts | 32 -
.../latest/src/Utils/serviceNavigation.test.ts | 10 +
ambari-web/latest/src/Utils/serviceNavigation.ts | 5 +-
ambari-web/latest/src/Utils/sslProtocolUtils.ts | 7 -
.../src/api/{hostsApi.test.ts => adminApi.test.ts} | 41 +-
ambari-web/latest/src/api/adminApi.ts | 8 +-
ambari-web/latest/src/api/cachedServiceApi.ts | 19 +-
ambari-web/latest/src/api/clusterApi.ts | 10 -
ambari-web/latest/src/api/hostsApi.test.ts | 9 +
ambari-web/latest/src/api/hostsApi.ts | 10 +-
ambari-web/latest/src/api/metricsApi.test.ts | 174 +
ambari-web/latest/src/api/metricsApi.ts | 614 +-
ambari-web/latest/src/api/serviceApi.ts | 2 +-
.../latest/src/api/servicesConfigsApi.test.ts | 2 +-
.../latest/src/components/Heatmap/HeatmapGrid.tsx | 71 -
.../latest/src/components/Heatmap/HeatmapHost.tsx | 149 -
.../src/components/Heatmap/HeatmapLegend.tsx | 84 -
.../latest/src/components/Heatmap/HeatmapRack.tsx | 81 -
.../LanguageSelector.tsx} | 28 +-
ambari-web/latest/src/components/Navbar.tsx | 24 +-
.../src/components/Sidebar/SideItemList.test.tsx | 54 +
.../latest/src/components/Sidebar/SideItemList.tsx | 69 +-
.../latest/src/components/Sidebar/Sidebar.tsx | 47 +-
ambari-web/latest/src/constants.ts | 10 +-
.../latest/src/data/BIGTOP/site_properties.ts | 70 -
.../latest/src/data/configs/propertyNameMapping.ts | 5 -
.../configs/services/ambari_metrics_properties.ts | 190 -
.../src/data/configs/wizards/ha_properties.ts | 17 +-
...ntMetricsEnums.ts => ServiceComponentFields.ts} | 97 +-
.../src/hooks/useAmbariMetricsConfigUpdater.ts | 308 -
ambari-web/latest/src/hooks/useConfigSaver.tsx | 1 -
ambari-web/latest/src/hooks/useConfigsTags.ts | 9 -
.../latest/src/hooks/useHDFSConfigUpdater.ts | 616 +-
.../latest/src/hooks/useHbaseConfigUpdater.ts | 254 +-
.../latest/src/hooks/useHiveConfigUpdater.ts | 14 +-
.../latest/src/hooks/useHostConfigUpdater.ts | 81 +-
.../latest/src/hooks/useKerberosConfigUpdater.ts | 10 +-
.../latest/src/hooks/useKyuubiConfigUpdater.ts | 10 +-
ambari-web/latest/src/hooks/useLazyQuicklinks.ts | 110 +-
.../latest/src/hooks/useMapReduce2ConfigUpdater.ts | 10 +-
.../latest/src/hooks/usePinotConfigUpdater.ts | 34 +-
.../latest/src/hooks/useRangerConfigUpdater.ts | 89 +-
.../latest/src/hooks/useRangerKMSConfigUpdater.ts | 8 +-
ambari-web/latest/src/hooks/useSSMConfigUpdater.ts | 16 +-
ambari-web/latest/src/hooks/useServiceDeletion.tsx | 3 +-
.../latest/src/hooks/useSpark3ConfigUpdater.ts | 30 +-
.../latest/src/hooks/useSqoopConfigUpdater.ts | 14 +-
ambari-web/latest/src/hooks/useTezConfigUpdater.ts | 10 +-
.../latest/src/hooks/useTrinoConfigUpdater.tsx | 22 +-
.../src/hooks/useTrinoGatewayConfigUpdater.ts | 10 +-
.../latest/src/hooks/useYarnConfigUpdater.ts | 475 +-
ambari-web/latest/src/hooks/useZkConfigUpdater.ts | 12 +-
ambari-web/latest/src/i18n.test.ts | 145 +
ambari-web/latest/src/i18n.ts | 44 +-
ambari-web/latest/src/locales/en/translation.json | 578 +-
ambari-web/latest/src/locales/zh/translation.json | 2709 +
ambari-web/latest/src/mappers/hostsMapper.ts | 10 -
ambari-web/latest/src/models/ambari_metrics.ts | 81 -
ambari-web/latest/src/models/hbase.ts | 145 +-
ambari-web/latest/src/models/hdfs.ts | 352 +-
ambari-web/latest/src/models/host.ts | 92 -
ambari-web/latest/src/models/hostComponent.ts | 16 -
ambari-web/latest/src/models/mapreduce2.ts | 3 +-
ambari-web/latest/src/models/ranger.ts | 44 +-
ambari-web/latest/src/models/service.ts | 1 -
ambari-web/latest/src/models/spark3.ts | 1 -
ambari-web/latest/src/models/trino.ts | 1 -
ambari-web/latest/src/models/yarn.ts | 278 +-
ambari-web/latest/src/router/RoutesList.test.tsx | 44 +-
ambari-web/latest/src/router/RoutesList.tsx | 68 +-
.../src/screens/Authentication/Login.i18n.test.tsx | 116 +
.../src/screens/Authentication/Login.test.tsx | 4 +-
.../latest/src/screens/Authentication/Login.tsx | 26 +-
.../latest/src/screens/ClusterWizard/Step10.tsx | 1 -
.../latest/src/screens/ClusterWizard/Step4.tsx | 1 -
.../latest/src/screens/ClusterWizard/Step6.tsx | 12 +-
.../src/screens/ClusterWizard/Step7/index.tsx | 2 -
.../latest/src/screens/ClusterWizard/constants.ts | 6 +-
.../ClusterWizard/hooks/useHostComponents.test.tsx | 11 +
.../ClusterWizard/hooks/useHostComponents.ts | 2 +-
.../latest/src/screens/ClusterWizard/utils.ts | 3 +-
.../src/screens/CommonConfigs/ConfigUtils.ts | 6 -
.../latest/src/screens/CommonConfigs/index.tsx | 1 -
.../src/screens/CommonConfigs/themeEngine.test.ts | 3 +-
.../screens/ConfigVersions/ConfigsComparator.tsx | 5 -
.../src/screens/Dashboard/ChartContainer.tsx | 42 -
.../src/screens/Dashboard/DashboardHeatmaps.tsx | 519 -
.../latest/src/screens/Dashboard/Index.test.tsx | 75 +
ambari-web/latest/src/screens/Dashboard/Index.tsx | 42 +-
.../latest/src/screens/Dashboard/Metrics.tsx | 247 -
.../src/screens/Dashboard/WidgetContainer.tsx | 118 -
.../src/screens/Dashboard/dashboard_widgets.tsx | 248 -
.../Dashboard/widgets/CpuClusterMetrics.tsx | 302 -
.../src/screens/Dashboard/widgets/DataNodeLive.tsx | 54 -
.../screens/Dashboard/widgets/HbaseMasterHeap.tsx | 79 -
.../Dashboard/widgets/HbaseMasterUptime.tsx | 35 -
.../Dashboard/widgets/HbaseRegionsInTransition.tsx | 36 -
.../Dashboard/widgets/LoadClusterMetrics.tsx | 300 -
.../Dashboard/widgets/MemoryClusterMetrics.tsx | 324 -
.../screens/Dashboard/widgets/NameNodeCapacity.tsx | 83 -
.../Dashboard/widgets/NameNodeCpuPieChartView.tsx | 128 -
.../src/screens/Dashboard/widgets/NameNodeHeap.tsx | 110 -
.../src/screens/Dashboard/widgets/NameNodeRpc.tsx | 46 -
.../screens/Dashboard/widgets/NameNodeUptime.tsx | 87 -
.../Dashboard/widgets/NetworkClusterMetrics.tsx | 288 -
.../screens/Dashboard/widgets/NodeManagersLive.tsx | 78 -
.../widgets/ResourceManagerHeapPieChart.tsx | 141 -
.../Dashboard/widgets/ResourceManagerUptime.tsx | 42 -
.../Dashboard/widgets/YARNMemoryPieChart.tsx | 133 -
.../screens/Dashboard/widgets/YarnContainers.tsx | 78 -
ambari-web/latest/src/screens/Heatmaps/index.tsx | 427 -
.../latest/src/screens/Hosts/HostConfigs.tsx | 2 -
.../latest/src/screens/Hosts/HostMetrics.tsx | 126 -
.../latest/src/screens/Hosts/HostMetricsGraph.tsx | 596 -
.../latest/src/screens/Hosts/HostSummary.tsx | 74 +-
ambari-web/latest/src/screens/Hosts/HostsList.tsx | 16 +-
ambari-web/latest/src/screens/Hosts/constants.ts | 19 -
.../src/screens/Metrics/BrowseWidgetsModal.tsx | 51 -
.../latest/src/screens/Metrics/GaugeWidgetView.tsx | 98 -
.../latest/src/screens/Metrics/GraphWidgetView.tsx | 192 -
.../latest/src/screens/Metrics/ServiceMetrics.tsx | 815 -
ambari-web/latest/src/screens/Metrics/type.ts | 34 -
.../Monitoring/Dashboard/DashboardLayout.tsx | 118 +
.../Monitoring/Dashboard/DashboardPage.test.tsx | 133 +
.../screens/Monitoring/Dashboard/DashboardPage.tsx | 450 +
.../Monitoring/Dashboard/DashboardPanel.test.tsx | 186 +
.../Monitoring/Dashboard/DashboardPanel.tsx | 275 +
.../Monitoring/Dashboard/DashboardPanelEditor.tsx | 391 +
.../Dashboard/DashboardQueryVariable.test.tsx | 54 +
.../Dashboard/DashboardQueryVariable.tsx | 73 +
.../Monitoring/Dashboard/DashboardRow.test.tsx | 52 +
.../screens/Monitoring/Dashboard/DashboardRow.tsx | 20 +-
.../Dashboard/DashboardVisualizationOptions.tsx | 135 +
.../Dashboard/DashboardWorkspaceDialogs.tsx | 186 +
.../screens/Monitoring/Dashboard/PanelRenderer.tsx | 68 +
.../Dashboard/dashboardWorkspace.test.ts | 68 +
.../Monitoring/Dashboard/dashboardWorkspace.ts | 163 +
.../Monitoring/Dashboard/data/panelData.test.ts | 92 +
.../screens/Monitoring/Dashboard/data/panelData.ts | 181 +
.../Dashboard/layout/dashboardLayout.test.ts | 86 +
.../Monitoring/Dashboard/layout/dashboardLayout.ts | 96 +
.../Dashboard/renderers/BarChartRenderer.tsx | 86 +
.../Dashboard/renderers/BarGaugeRenderer.tsx | 77 +
.../Dashboard/renderers/GaugeRenderer.tsx | 71 +
.../Dashboard/renderers/HeatmapRenderer.tsx | 75 +
.../Dashboard/renderers/HexbinRenderer.tsx | 99 +
.../Dashboard/renderers/IframeRenderer.tsx | 42 +
.../Monitoring/Dashboard/renderers/PieRenderer.tsx | 70 +
.../Dashboard/renderers/StatRenderer.tsx | 67 +
.../Dashboard/renderers/TableRenderer.tsx | 224 +
.../Dashboard/renderers/TextRenderer.tsx | 37 +
.../Dashboard/renderers/TimeSeriesRenderer.tsx | 67 +
.../latest/src/screens/Monitoring/Dashboards.tsx | 153 +
.../latest/src/screens/Monitoring/Datasources.tsx | 340 +
.../src/screens/Monitoring/EmbeddedDashboards.tsx | 70 +
.../latest/src/screens/Monitoring/Explorer.tsx | 142 +
.../screens/Monitoring/MonitoringLayout.test.tsx | 73 +
.../src/screens/Monitoring/MonitoringLayout.tsx | 65 +
.../src/screens/Monitoring/PrometheusChart.tsx | 291 +
.../src/screens/Monitoring/SharedCharts.test.tsx | 103 +
.../latest/src/screens/Monitoring/SharedCharts.tsx | 79 +
.../latest/src/screens/Monitoring/Targets.tsx | 82 +
.../latest/src/screens/Monitoring/monitoring.scss | 1271 +
ambari-web/latest/src/screens/Monitoring/types.ts | 212 +
.../latest/src/screens/Monitoring/utils.test.ts | 214 +
ambari-web/latest/src/screens/Monitoring/utils.ts | 333 +
.../src/screens/Monitoring/valueFormatter.test.ts | 62 +
.../src/screens/Monitoring/valueFormatter.ts | 127 +
.../latest/src/screens/ServiceConfigs/index.tsx | 2 -
ambari-web/latest/src/screens/Services/Actions.tsx | 1 -
.../src/screens/Services/HDFSFederationSummary.tsx | 869 -
.../Services/HDFSFederationTopology.test.tsx | 69 +
.../screens/Services/HDFSFederationTopology.tsx | 137 +
.../src/screens/Services/SampleMetricsConsumer.tsx | 28 -
.../screens/Services/ServiceComponents.test.tsx | 125 +
.../src/screens/Services/ServiceComponents.tsx | 901 +-
.../src/screens/Services/ServiceDashboard.tsx | 34 +-
.../latest/src/screens/Services/ServiceLoader.tsx | 1 -
.../latest/src/screens/Services/ServiceMetrics.tsx | 218 -
.../src/screens/Services/ServiceQuicklinks.tsx | 1 -
.../latest/src/screens/Services/ServiceSummary.tsx | 30 +-
.../latest/src/screens/Services/constants.ts | 382 +-
.../Services/highAvailibility/nameNode/Step9.tsx | 9 -
.../src/screens/Services/reassign/constants.ts | 1 -
.../src/screens/Services/serviceRestartUtils.ts | 2 -
.../src/screens/Services/styles/services.scss | 3 -
ambari-web/latest/src/screens/messages.ts | 524 +-
ambari-web/latest/src/store/OptimizedUpdater.tsx | 2 -
ambari-web/latest/src/store/ServiceContext.tsx | 6 +-
ambari-web/latest/src/store/Updater.tsx | 4 -
ambari-web/latest/src/store/UserContext.test.tsx | 29 +
ambari-web/latest/src/store/UserContext.tsx | 6 +-
ambari-web/latest/src/styles/app.scss | 4 -
ambari-web/latest/src/types/heatmap.ts | 90 -
.../addons/package/rpm/create_ganglia_addon_rpm.sh | 87 -
.../addons/package/rpm/hdp_mon_ganglia_addons.spec | 80 -
.../addOns/ganglia/conf/cluster_HDPJobTracker.json | 4 -
.../addOns/ganglia/conf/cluster_HDPNameNode.json | 3 -
.../src/addOns/ganglia/conf/cluster_HDPSlaves.json | 4 -
.../ganglia/graph.d/hdp_mon_disk_report.json | 10 -
.../ganglia/graph.d/hdp_mon_hbase_hlog_report.json | 10 -
.../hdp_mon_hbase_hlog_split_size_report.json | 10 -
...p_mon_hbase_master_cluster_requests_report.json | 10 -
..._regionserver_compaction_queue_size_report.json | 10 -
...hbase_regionserver_flush_queue_size_report.json | 10 -
..._hbase_regionserver_fs_read_latency_report.json | 10 -
...hbase_regionserver_fs_write_latency_report.json | 10 -
...on_hbase_regionserver_read_requests_report.json | 10 -
.../hdp_mon_hbase_regionserver_regions_report.json | 10 -
...n_hbase_regionserver_write_requests_report.json | 10 -
.../hdp_mon_hdfs_capacity_remaining_report.json | 10 -
.../ganglia/graph.d/hdp_mon_hdfs_io_report.json | 10 -
.../ganglia/graph.d/hdp_mon_hdfs_ops_report.json | 14 -
...mon_hdfs_pending_replication_blocks_report.json | 10 -
...dp_mon_hdfs_under_replicated_blocks_report.json | 10 -
.../hdp_mon_jobtracker_heartbeats_report.json | 10 -
.../hdp_mon_jobtracker_jobs_completed_report.json | 10 -
.../hdp_mon_jobtracker_jobs_failed_report.json | 10 -
.../hdp_mon_jobtracker_jobs_running_report.json | 10 -
.../hdp_mon_jobtracker_jobs_submitted_report.json | 10 -
.../hdp_mon_jobtracker_map_slot_report.json | 14 -
.../hdp_mon_jobtracker_mapreduce_report.json | 12 -
.../hdp_mon_jobtracker_reduce_slot_report.json | 14 -
.../ganglia/graph.d/hdp_mon_jvm_gc_report.json | 10 -
.../graph.d/hdp_mon_jvm_mem_heap_used_report.json | 10 -
.../graph.d/hdp_mon_jvm_threads_report.json | 16 -
.../graph.d/hdp_mon_rpc_latency_report.json | 12 -
.../hdp_mon_rpc_queue_time_avg_time_report.json | 10 -
.../hdp_mon_rpc_queue_time_num_ops_report.json | 10 -
...p_mon_rpcdetailed_heartbeat_num_ops_report.json | 10 -
.../hdp_mon_tasktracker_mapreduce_report.json | 12 -
.../graph.d/hdp_mon_tasktracker_task_report.json | 14 -
.../jmx/data/cluster_configuration.json | 93 -
.../jmx/data/cluster_configuration.json.nohbase | 93 -
.../jmx/data/sample_hbasemaster_jmx.json | 1652 -
.../jmx/data/sample_jobtracker_jmx.json | 894 -
.../dataServices/jmx/data/sample_namenode_jmx.json | 843 -
.../test/dataServices/jmx/test_config_load.php | 120 -
.../test/dataServices/jmx/test_jmx_parsing.php | 255 -
contrib/addons/test/ui/json/clusterSummary.json | 3 +-
.../addons/test/ui/json/get_graph_info_all.json | 24 -
.../utils/dataServices/ganglia/generateAll.sh | 36 -
.../ganglia/generate_dashboard_hdp_json.php | 66 -
.../ganglia/generate_dashboard_json.php | 61 -
.../ganglia/generate_hdfs_hdp_json.php | 66 -
.../dataServices/ganglia/generate_hdfs_json.php | 1360 -
.../ganglia/generate_mapreduce_hdp_json.php | 74 -
.../ganglia/generate_mapreduce_json.php | 1441 -
.../1.0.0/configuration/nifi-ambari-config.xml | 17 -
.../1.0.0/configuration/nifi-bootstrap-env.xml | 7 -
.../NIFI/1.0.0/configuration/nifi-flow-env.xml | 68 -
.../common-services/NIFI/1.0.0/metainfo.xml | 3 -
.../common-services/NIFI/1.0.0/metrics.json | 131 -
.../NIFI/1.0.0/package/scripts/nifi.py | 69 -
.../NIFI/1.0.0/package/scripts/params.py | 14 -
.../common-services/NIFI/1.0.0/widgets.json | 463 -
.../HDF/2.0/hooks/before-ANY/scripts/params.py | 7 -
.../HDF/2.0/hooks/before-INSTALL/scripts/params.py | 6 -
.../HDF/2.0/hooks/before-START/scripts/params.py | 44 -
.../before-START/scripts/shared_initialization.py | 7 -
.../templates/hadoop-metrics2.properties.j2 | 112 -
.../stacks/HDF/2.0/properties/stack_features.json | 5 -
.../stacks/HDF/2.0/role_command_order.json | 8 -
.../HDF/2.0/services/AMBARI_METRICS/metainfo.xml | 26 -
.../stacks/HDF/2.0/services/stack_advisor.py | 705 +-
.../src/main/resources/stacks/HDF/2.0/widgets.json | 95 -
.../addon-services/ONEFS/1.0.0/metrics.json | 230 +-
.../addon-services/ONEFS/1.0.0/widgets.json | 141 -
.../non_root/setup_and_start_non_root_agent.sh | 2 +-
...bari-agent-prometheus-telemetry-architecture.md | 416 +
...gent-prometheus-telemetry-architecture.zh-CN.md | 348 +
docs/ambari-agent-prometheus-telemetry-plan.md | 269 +
.../ambari-26638-n9e-dashboard-migration-plan.md | 806 +
.../frontend-refactor/prometheus-migration-plan.md | 591 +
.../prometheus-source-baseline.md | 210 +
.../prometheus-source-inventory.md | 230 +
.../react-current/01-auth-shell-gap.md | 60 +
.../09-namenode-journalnode-ha-gap.md | 18 +-
.../react-current/13-permissions-flags-gap.md | 1 -
.../runtime-evidence/AMBARI-26638/README.md | 33 +
.../AMBARI-26638/hbase-service-metrics.png | Bin 0 -> 218234 bytes
.../AMBARI-26638/hdfs-service-metrics.png | Bin 0 -> 283077 bytes
.../AMBARI-26638/linux-fleet-dashboard.png | Bin 0 -> 238538 bytes
.../AMBARI-26638/monitoring-dashboard-catalog.png | Bin 0 -> 248780 bytes
.../AMBARI-26638/prometheus-scrape-targets.png | Bin 0 -> 261426 bytes
.../victoriametrics-service-summary.png | Bin 0 -> 98727 bytes
docs/src/site/apt/index.apt | 3 +-
docs/src/site/apt/install-0.9.apt | 17 -
docs/src/site/apt/install.apt | 16 -
docs/src/site/apt/whats-new.apt | 5 -
pom.xml | 6 +
922 files changed, 49607 insertions(+), 726702 deletions(-)
diff --git a/Jenkinsfile b/Jenkinsfile
index 1c4aec26bc..d307887619 100644
--- a/Jenkinsfile
+++ b/Jenkinsfile
@@ -23,7 +23,6 @@
Ambari WebUI -> ambari-web, ambari-admin
Ambari Agent -> ambari-agent, ambari-utility
Ambari Server -> ambari-server, ambari-views
- Ambari Metrics -> (cd) -> ambari-metrics (only build, no checks)
ServiceAdvisor -> ambari-serviceadvisor
Ambari LogSearch -> (cd) -> ambari-logsearch
Ambari Infra -> (cd) -> ambari-infra
@@ -83,21 +82,6 @@ pipeline {
publishPRLink env.CHANGE_ID, env.CHANGE_URL, env.CHANGE_TITLE
}
}
- stage('Ambari Metrics Build (deps)') {
- steps {
- script{
- def dirExists = fileExists 'ambari-metrics'
- if (dirExists) {
- echo "Ambari-Metrics is here!"
- dir('ambari-metrics') {
- sh 'mvn -T 3C install -DskipSurefireTests -DskipPythonTests -Dmaven.test.failure.ignore -DskipTests -Dfindbugs.skip -Drat.skip -Dmaven.artifact.threads=10 -X'
- }
- } else {
- echo "Ignoring ambari-metrics, as no such directory found"
- }
- }
- }
- }
stage('Ambari Service Advisor') {
steps {
sh 'mvn -T 3C -am install -pl ambari-serviceadvisor -DskipSurefireTests -DskipPythonTests -Dmaven.test.failure.ignore -DskipTests -DskipUiBuild=true -Dfindbugs.skip -Drat.skip -Dmaven.artifact.threads=10'
diff --git a/LICENSE.txt b/LICENSE.txt
index 4625c63a0e..c43caf4c7d 100644
--- a/LICENSE.txt
+++ b/LICENSE.txt
@@ -208,6 +208,9 @@ notices and license terms. Your use of the source code for the these
subcomponents is subject to the terms and conditions of the following
licenses.
+This product bundles VictoriaMetrics (https://victoriametrics.com/),
+which is available under the Apache License, Version 2.0.
+
and some parts:
Copyright (C) 2011 Krzysztof Wilczynski
diff --git a/README.md b/README.md
index c31e69f40d..446cd655e8 100644
--- a/README.md
+++ b/README.md
@@ -18,11 +18,15 @@
[](https://builds.apache.org/view/A/view/Ambari/job/Ambari-trunk-Commit/)
-Apache Ambari is a tool for provisioning, managing, and monitoring Apache Hadoop clusters. Ambari consists of a set of RESTful APIs and a browser-based management interface.
+Apache Ambari is a tool for provisioning, managing, and monitoring Apache Hadoop clusters. Ambari consists of a set of RESTful APIs and a browser-based management interface. Its monitoring stack uses Prometheus-compatible APIs, Ambari Agent exporters, and a bundled VictoriaMetrics storage provider.
+
+The optional [Ambari Metrics RPM](ambari-metrics/README.md) packages the pinned
+VictoriaMetrics release used by the default deployment. It replaces the legacy
+Ambari Metrics System and Ganglia integrations; it is not the former external
+`apache/ambari-metrics` project.
## Sub-projects
-- Ambari Metrics ([GitHub](https://github.com/apache/ambari-metrics), [GitBox](https://gitbox.apache.org/repos/asf?p=ambari-metrics.git))
- Ambari Log Search ([GitHub](https://github.com/apache/ambari-logsearch), [GitBox](https://gitbox.apache.org/repos/asf?p=ambari-logsearch.git))
- Ambari Infra ([GitHub](https://github.com/apache/ambari-infra), [GitBox](https://gitbox.apache.org/repos/asf?p=ambari-infra.git))
diff --git a/ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/AddVersionModal.js b/ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/AddVersionModal.js
index cd86a78e2f..88a73ad780 100644
--- a/ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/AddVersionModal.js
+++ b/ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/AddVersionModal.js
@@ -122,7 +122,7 @@ angular.module('ambariAdminConsole')
};
var services = [];
angular.forEach(repo.VersionDefinition.stack_services, function (service) {
- var servicesToExclude = ['GANGLIA', 'KERBEROS', 'MAPREDUCE2'];
+ var servicesToExclude = ['KERBEROS', 'MAPREDUCE2'];
if (servicesToExclude.indexOf(service.name) === -1) {
services.push({
name: service.name,
diff --git a/ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/Stack.js b/ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/Stack.js
index d0d298b908..8c568adef9 100644
--- a/ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/Stack.js
+++ b/ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/Stack.js
@@ -135,7 +135,7 @@ angular.module('ambariAdminConsole')
var services = [];
angular.forEach(version.VersionDefinition.stack_services, function (service) {
// services that should not be shown on UI
- var servicesToExclude = ['GANGLIA', 'KERBEROS', 'MAPREDUCE2'];
+ var servicesToExclude = ['KERBEROS', 'MAPREDUCE2'];
if (servicesToExclude.indexOf(service.name) === -1) {
services.push({
name: service.name,
@@ -288,7 +288,7 @@ angular.module('ambariAdminConsole')
};
var services = [];
angular.forEach(data.repository_versions[0].RepositoryVersions.stack_services, function (service) {
- var servicesToExclude = ['GANGLIA', 'KERBEROS', 'MAPREDUCE2'];
+ var servicesToExclude = ['KERBEROS', 'MAPREDUCE2'];
if (servicesToExclude.indexOf(service.name) === -1) {
services.push({
name: service.name,
@@ -442,7 +442,7 @@ angular.module('ambariAdminConsole')
return s.name;
});
return response.services.filter(function (service) {
- var skipServices = ['MAPREDUCE2', 'GANGLIA', 'KERBEROS'];
+ var skipServices = ['MAPREDUCE2', 'KERBEROS'];
return skipServices.indexOf(service.name) === -1 && availableServices.indexOf(service.name) !== -1;
}) || [];
}
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/__mocks__/mockRbacData.ts b/ambari-admin/src/main/resources/ui/ambari-admin/src/__mocks__/mockRbacData.ts
index e9a85b49c6..3c4f298172 100644
--- a/ambari-admin/src/main/resources/ui/ambari-admin/src/__mocks__/mockRbacData.ts
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/__mocks__/mockRbacData.ts
@@ -172,14 +172,6 @@ export const rbacData = {
"permission_id": 1
}
},
- {
- "href": "http://host.example.com:8080/api/v1/permissions/1/authorizations/CLUSTER.MANAGE_WIDGETS",
- "AuthorizationInfo": {
- "authorization_id": "CLUSTER.MANAGE_WIDGETS",
- "authorization_name": "Manage widgets",
- "permission_id": 1
- }
- },
{
"href": "http://host.example.com:8080/api/v1/permissions/1/authorizations/CLUSTER.MODIFY_CONFIGS",
"AuthorizationInfo": {
@@ -660,14 +652,6 @@ export const rbacData = {
"permission_id": 3
}
},
- {
- "href": "http://host.example.com:8080/api/v1/permissions/3/authorizations/CLUSTER.MANAGE_WIDGETS",
- "AuthorizationInfo": {
- "authorization_id": "CLUSTER.MANAGE_WIDGETS",
- "authorization_name": "Manage widgets",
- "permission_id": 3
- }
- },
{
"href": "http://host.example.com:8080/api/v1/permissions/3/authorizations/CLUSTER.MODIFY_CONFIGS",
"AuthorizationInfo": {
@@ -1000,14 +984,6 @@ export const rbacData = {
"permission_id": 5
}
},
- {
- "href": "http://host.example.com:8080/api/v1/permissions/5/authorizations/CLUSTER.MANAGE_WIDGETS",
- "AuthorizationInfo": {
- "authorization_id": "CLUSTER.MANAGE_WIDGETS",
- "authorization_name": "Manage widgets",
- "permission_id": 5
- }
- },
{
"href": "http://host.example.com:8080/api/v1/permissions/5/authorizations/CLUSTER.VIEW_ALERTS",
"AuthorizationInfo": {
@@ -1595,4 +1571,4 @@ export const rbacData = {
]
}
]
-}
\ No newline at end of file
+}
diff --git a/ambari-admin/src/main/resources/ui/ambari-admin/src/__mocks__/mockVersionDefinitions.ts b/ambari-admin/src/main/resources/ui/ambari-admin/src/__mocks__/mockVersionDefinitions.ts
index 252af05977..8542f07401 100644
--- a/ambari-admin/src/main/resources/ui/ambari-admin/src/__mocks__/mockVersionDefinitions.ts
+++ b/ambari-admin/src/main/resources/ui/ambari-admin/src/__mocks__/mockVersionDefinitions.ts
@@ -108,14 +108,6 @@ export default {
"3.2.2"
]
},
- {
- "name": "AMBARI_METRICS",
- "display_name": "Ambari Metrics",
- "comment": "A system for metrics collection that provides storage and retrieval capability for metrics collected from the cluster\n ",
- "versions": [
- "0.2.0"
- ]
- },
{
"name": "ZOOKEEPER",
"display_name": "ZooKeeper",
@@ -811,14 +803,6 @@ export default {
"0.10.2"
]
},
- {
- "name": "AMBARI_METRICS",
- "display_name": "Ambari Metrics",
- "comment": "A system for metrics collection that provides storage and retrieval capability for metrics collected from the cluster\n ",
- "versions": [
- "0.2.1"
- ]
- },
{
"name": "RANGER_KMS",
"display_name": "Ranger KMS",
@@ -1555,4 +1539,3 @@ export default {
]
}
-
diff --git a/ambari-agent/conf/unix/ambari-agent.ini b/ambari-agent/conf/unix/ambari-agent.ini
index 61b69d99e0..e4bd54d320 100644
--- a/ambari-agent/conf/unix/ambari-agent.ini
+++ b/ambari-agent/conf/unix/ambari-agent.ini
@@ -77,6 +77,12 @@ log_max_symbols_size=900000
idle_interval_min=1
idle_interval_max=10
+[prometheus]
+; Exposes host and component metrics for Prometheus-compatible scrapers.
+enabled=true
+bind_address=0.0.0.0
+port=9101
+
[logging]
syslog_enabled=0
diff --git a/ambari-agent/src/main/python/ambari_agent/AlertSchedulerHandler.py b/ambari-agent/src/main/python/ambari_agent/AlertSchedulerHandler.py
index 96761b3a64..43ab225fbf 100644
--- a/ambari-agent/src/main/python/ambari_agent/AlertSchedulerHandler.py
+++ b/ambari-agent/src/main/python/ambari_agent/AlertSchedulerHandler.py
@@ -27,7 +27,6 @@ from apscheduler.executors.pool import ThreadPoolExecutor
from apscheduler.schedulers.background import BackgroundScheduler
from ambari_agent.alerts.collector import AlertCollector
from ambari_agent.alerts.metric_alert import MetricAlert
-from ambari_agent.alerts.ams_alert import AmsAlert
from ambari_agent.alerts.port_alert import PortAlert
from ambari_agent.alerts.script_alert import ScriptAlert
from ambari_agent.alerts.web_alert import WebAlert
@@ -42,7 +41,6 @@ logger = logging.getLogger(__name__)
class AlertSchedulerHandler:
TYPE_PORT = "PORT"
TYPE_METRIC = "METRIC"
- TYPE_AMS = "AMS"
TYPE_SCRIPT = "SCRIPT"
TYPE_WEB = "WEB"
TYPE_RECOVERY = "RECOVERY"
@@ -296,8 +294,6 @@ class AlertSchedulerHandler:
if source_type == AlertSchedulerHandler.TYPE_METRIC:
alert = MetricAlert(json_definition, source, self.config)
- elif source_type == AlertSchedulerHandler.TYPE_AMS:
- alert = AmsAlert(json_definition, source, self.config)
elif source_type == AlertSchedulerHandler.TYPE_PORT:
alert = PortAlert(json_definition, source, self.config)
elif source_type == AlertSchedulerHandler.TYPE_SCRIPT:
diff --git a/ambari-agent/src/main/python/ambari_agent/AmbariConfig.py b/ambari-agent/src/main/python/ambari_agent/AmbariConfig.py
index 835fd43741..c1ced56d42 100644
--- a/ambari-agent/src/main/python/ambari_agent/AmbariConfig.py
+++ b/ambari-agent/src/main/python/ambari_agent/AmbariConfig.py
@@ -81,6 +81,11 @@ log_max_symbols_size=900000
iddle_interval_min=1
iddle_interval_max=10
+[prometheus]
+enabled=true
+bind_address=0.0.0.0
+port=9101
+
[logging]
log_command_executes = 0
@@ -217,6 +222,38 @@ class AmbariConfig:
def host_status_report_interval(self):
return int(self.get("heartbeat", "state_interval_seconds", "60"))
+ @property
+ def prometheus_metrics_enabled(self):
+ return self.get("prometheus", "enabled", "true").lower() in (
+ "1",
+ "true",
+ "yes",
+ "on",
+ )
+
+ @property
+ def prometheus_metrics_bind_address(self):
+ return self.get("prometheus", "bind_address", "0.0.0.0")
+
+ @property
+ def prometheus_metrics_port(self):
+ configured = self.get("prometheus", "port", "9101")
+ try:
+ port = int(configured)
+ except (TypeError, ValueError):
+ logger.warning(
+ "Invalid Prometheus metrics port %r; using the default port 9101",
+ configured,
+ )
+ return 9101
+ if not 1 <= port <= 65535:
+ logger.warning(
+ "Prometheus metrics port %r is out of range; using the default port 9101",
+ configured,
+ )
+ return 9101
+ return port
+
@property
def log_max_symbols_size(self):
return int(self.get("heartbeat", "log_max_symbols_size", "900000"))
diff --git a/ambari-agent/src/main/python/ambari_agent/Constants.py b/ambari-agent/src/main/python/ambari_agent/Constants.py
index 0dc1e3fa22..70d9b29762 100644
--- a/ambari-agent/src/main/python/ambari_agent/Constants.py
+++ b/ambari-agent/src/main/python/ambari_agent/Constants.py
@@ -22,6 +22,7 @@ COMMANDS_TOPIC = "/user/commands"
CONFIGURATIONS_TOPIC = "/user/configs"
HOST_LEVEL_PARAMS_TOPIC = "/user/host_level_params"
ALERTS_DEFINITIONS_TOPIC = "/user/alert_definitions"
+TELEMETRY_TOPIC = "/user/telemetry"
METADATA_TOPIC = "/events/metadata"
TOPOLOGIES_TOPIC = "/events/topologies"
SERVER_RESPONSES_TOPIC = "/user/"
@@ -41,6 +42,7 @@ METADATA_REQUEST_ENDPOINT = "/agents/metadata"
CONFIGURATIONS_REQUEST_ENDPOINT = "/agents/configs"
HOST_LEVEL_PARAMS_TOPIC_ENPOINT = "/agents/host_level_params"
ALERTS_DEFINITIONS_REQUEST_ENDPOINT = "/agents/alert_definitions"
+TELEMETRY_REQUEST_ENDPOINT = "/agents/telemetry"
COMPONENT_STATUS_REPORTS_ENDPOINT = "/reports/component_status"
COMPONENT_VERSION_REPORTS_ENDPOINT = "/reports/component_version"
COMMANDS_STATUS_REPORTS_ENDPOINT = "/reports/commands_status"
diff --git a/ambari-agent/src/main/python/ambari_agent/HeartbeatThread.py b/ambari-agent/src/main/python/ambari_agent/HeartbeatThread.py
index f8cb4610da..5d104f8a8e 100644
--- a/ambari-agent/src/main/python/ambari_agent/HeartbeatThread.py
+++ b/ambari-agent/src/main/python/ambari_agent/HeartbeatThread.py
@@ -21,6 +21,7 @@ limitations under the License.
import logging
import os
import threading
+import time
from socket import error as socket_error
from ambari_agent import Constants
@@ -40,11 +41,14 @@ from ambari_agent.listeners.AlertDefinitionsEventListener import (
AlertDefinitionsEventListener,
)
from ambari_agent.listeners.EncryptionKeyListener import EncryptionKeyListener
+from ambari_agent.listeners.TelemetryEventListener import TelemetryEventListener
from ambari_agent import security
from ambari_agent.AmbariStompConnection import ConnectionIsAlreadyClosed
HEARTBEAT_INTERVAL = 10
REQUEST_RESPONSE_TIMEOUT = 10
+TELEMETRY_RECONCILIATION_INTERVAL = 300
+TELEMETRY_CAPABILITY = "telemetry-v1"
logger = logging.getLogger(__name__)
@@ -81,6 +85,7 @@ class HeartbeatThread(threading.Thread):
self.alert_definitions_events_listener = AlertDefinitionsEventListener(
initializer_module
)
+ self.telemetry_events_listener = TelemetryEventListener(initializer_module)
self.agent_actions_events_listener = AgentActionsListener(initializer_module)
self.component_status_executor = initializer_module.component_status_executor
self.listeners = [
@@ -91,6 +96,7 @@ class HeartbeatThread(threading.Thread):
self.configuration_events_listener,
self.host_level_params_events_listener,
self.alert_definitions_events_listener,
+ self.telemetry_events_listener,
self.agent_actions_events_listener,
self.encryption_key_events_listener,
]
@@ -128,6 +134,8 @@ class HeartbeatThread(threading.Thread):
),
]
self.responseId = 0
+ self.telemetry_supported = False
+ self.last_telemetry_reconciliation = 0
self.file_cache = initializer_module.file_cache
self.stale_alerts_monitor = initializer_module.stale_alerts_monitor
self.post_registration_actions = [
@@ -155,6 +163,7 @@ class HeartbeatThread(threading.Thread):
response = self.blocking_request(heartbeat_body, Constants.HEARTBEAT_ENDPOINT)
logger.debug("Heartbeat response id=%s", response.get("id"))
self.handle_heartbeat_reponse(response)
+ self.reconcile_telemetry_if_due()
except Exception as ex:
if isinstance(ex, (ConnectionIsAlreadyClosed)):
logger.info("Connection was closed. Re-running the registration")
@@ -202,7 +211,18 @@ class HeartbeatThread(threading.Thread):
self.handle_registration_response(response)
- for endpoint, cache, listener, subscribe_to in self.post_registration_requests:
+ post_registration_requests = list(self.post_registration_requests)
+ if self.telemetry_supported:
+ post_registration_requests.append(
+ (
+ Constants.TELEMETRY_REQUEST_ENDPOINT,
+ self.initializer_module.telemetry_cache,
+ self.telemetry_events_listener,
+ Constants.TELEMETRY_TOPIC,
+ )
+ )
+
+ for endpoint, cache, listener, subscribe_to in post_registration_requests:
try:
listener.enabled = False
self.subscribe_to_topics([subscribe_to])
@@ -284,6 +304,28 @@ class HeartbeatThread(threading.Thread):
raise Exception(error_message)
self.responseId = int(response["id"])
+ capabilities = response.get("serverCapabilities", [])
+ self.telemetry_supported = TELEMETRY_CAPABILITY in capabilities
+ self.last_telemetry_reconciliation = time.monotonic()
+
+ def reconcile_telemetry_if_due(self):
+ if not self.telemetry_supported:
+ return
+
+ now = time.monotonic()
+ if now - self.last_telemetry_reconciliation < TELEMETRY_RECONCILIATION_INTERVAL:
+ return
+
+ self.last_telemetry_reconciliation = now
+ try:
+ response = self.blocking_request(
+ {"hash": self.initializer_module.telemetry_cache.hash},
+ Constants.TELEMETRY_REQUEST_ENDPOINT,
+ log_handler=self.telemetry_events_listener.get_log_message,
+ )
+ self.telemetry_events_listener.on_event({}, response)
+ except Exception:
+ logger.exception("Unable to reconcile the telemetry assignment")
def handle_heartbeat_reponse(self, response):
serverId = int(response["id"])
diff --git a/ambari-agent/src/main/python/ambari_agent/HostInfo.py b/ambari-agent/src/main/python/ambari_agent/HostInfo.py
index 2c3fc8a3f0..14dca53725 100644
--- a/ambari-agent/src/main/python/ambari_agent/HostInfo.py
+++ b/ambari-agent/src/main/python/ambari_agent/HostInfo.py
@@ -182,7 +182,6 @@ class HostInfoLinux(HostInfo):
"falcon",
"ambari_qa",
"hadoop_deploy",
- "rrdcached",
"hcat",
"ambari-qa",
"sqoop-ambari-qa",
@@ -201,8 +200,6 @@ class HostInfoLinux(HostInfo):
"phoenix",
"ranger",
"accumulo",
- "ambari-metrics-collector",
- "ambari-metrics-monitor",
"atlas",
"zeppelin",
]
@@ -229,7 +226,6 @@ class HostInfoLinux(HostInfo):
"falcon",
"kafka",
"knox",
- "ams",
"hadoop",
"spark",
"accumulo",
diff --git a/ambari-agent/src/main/python/ambari_agent/InitializerModule.py b/ambari-agent/src/main/python/ambari_agent/InitializerModule.py
index 91f7313a2f..c6ea2b4053 100644
--- a/ambari-agent/src/main/python/ambari_agent/InitializerModule.py
+++ b/ambari-agent/src/main/python/ambari_agent/InitializerModule.py
@@ -29,6 +29,7 @@ from ambari_agent.ClusterTopologyCache import ClusterTopologyCache
from ambari_agent.ClusterMetadataCache import ClusterMetadataCache
from ambari_agent.ClusterHostLevelParamsCache import ClusterHostLevelParamsCache
from ambari_agent.ClusterAlertDefinitionsCache import ClusterAlertDefinitionsCache
+from ambari_agent.TelemetryConfigCache import TelemetryConfigCache
from ambari_agent.ActionQueue import ActionQueue
from ambari_agent.CommandStatusDict import CommandStatusDict
from ambari_agent.CustomServiceOrchestrator import CustomServiceOrchestrator
@@ -44,6 +45,7 @@ from ambari_agent.ComponentStatusExecutor import ComponentStatusExecutor
from ambari_agent.CommandStatusReporter import CommandStatusReporter
from ambari_agent.HostStatusReporter import HostStatusReporter
from ambari_agent.AlertStatusReporter import AlertStatusReporter
+from ambari_agent.metrics.server import PrometheusMetricsServer
logger = logging.getLogger(__name__)
@@ -66,6 +68,7 @@ class InitializerModule:
self.host_level_params_cache = None
self.configurations_cache = None
self.alert_definitions_cache = None
+ self.telemetry_cache = None
self.configuration_builder = None
self.stale_alerts_monitor = None
self.server_responses_listener = None
@@ -76,6 +79,7 @@ class InitializerModule:
self.commandStatuses = None
self.action_queue = None
self.alert_scheduler_handler = None
+ self.prometheus_metrics_server = None
def init(self):
"""
@@ -96,6 +100,7 @@ class InitializerModule:
self.alert_definitions_cache = ClusterAlertDefinitionsCache(
self.config.cluster_cache_dir
)
+ self.telemetry_cache = TelemetryConfigCache(self.config.cluster_cache_dir)
self.configuration_builder = ConfigurationBuilder(self)
self.stale_alerts_monitor = StaleAlertsMonitor(self)
self.server_responses_listener = ServerResponsesListener(self)
@@ -118,6 +123,7 @@ class InitializerModule:
self.host_status_reporter = HostStatusReporter(self)
self.alert_status_reporter = AlertStatusReporter(self)
self.heartbeat_thread = HeartbeatThread.HeartbeatThread(self)
+ self.prometheus_metrics_server = PrometheusMetricsServer(self)
@property
def connection(self):
diff --git a/ambari-agent/src/main/python/ambari_agent/TelemetryConfigCache.py b/ambari-agent/src/main/python/ambari_agent/TelemetryConfigCache.py
new file mode 100644
index 0000000000..0acd190565
--- /dev/null
+++ b/ambari-agent/src/main/python/ambari_agent/TelemetryConfigCache.py
@@ -0,0 +1,166 @@
+#!/usr/bin/env python3
+
+"""
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+"""
+
+import copy
+import json
+import logging
+import os
+import tempfile
+import threading
+
+from ambari_agent.metrics.telemetry import (
+ profile_digest,
+ validate_telemetry_bundle,
+)
+
+
+logger = logging.getLogger(__name__)
+
+
+class TelemetryConfigCache:
+ ASSIGNMENT_FILE = "telemetry_assignment.json"
+ PROFILES_DIRECTORY = "telemetry_profiles"
+
+ def __init__(self, cache_dir):
+ self.cache_dir = cache_dir
+ self.assignment_file = os.path.join(cache_dir, self.ASSIGNMENT_FILE)
+ self.profiles_dir = os.path.join(cache_dir, self.PROFILES_DIRECTORY)
+ self._lock = threading.RLock()
+ self._assignment = {"schemaVersion": 1, "targets": []}
+ self._profiles = {}
+ self.hash = None
+ self.last_reload_successful = True
+ self._load()
+
+ def _load(self):
+ if not os.path.isfile(self.assignment_file):
+ return
+
+ try:
+ with open(self.assignment_file, "r", encoding="utf-8") as stream:
+ envelope = json.load(stream)
+
+ assignment = envelope["assignment"]
+ cache_hash = envelope["hash"]
+ if not isinstance(cache_hash, str) or not cache_hash:
+ raise ValueError("Persisted telemetry assignment hash is invalid")
+ profiles = self._load_referenced_profiles(assignment)
+ validate_telemetry_bundle(assignment, profiles)
+ except Exception:
+ self.last_reload_successful = False
+ logger.exception("Unable to load the persisted telemetry assignment")
+ return
+
+ with self._lock:
+ self._assignment = assignment
+ self._profiles = profiles
+ self.hash = cache_hash
+
+ def _load_referenced_profiles(self, assignment):
+ profiles = {}
+ for target in assignment.get("targets", []):
+ digest = target.get("profileHash")
+ if not digest or digest in profiles:
+ continue
+ path = self._profile_path(digest)
+ with open(path, "r", encoding="utf-8") as stream:
+ profile = json.load(stream)
+ if profile_digest(profile) != digest:
+ raise ValueError('Telemetry profile digest does not match "{}"'.format(digest))
+ profiles[digest] = profile
+ return profiles
+
+ def update(self, assignment, profiles, cache_hash):
+ if not isinstance(cache_hash, str) or not cache_hash:
+ raise ValueError("Telemetry assignment hash must be a non-empty string")
+ if not isinstance(profiles, dict):
+ raise ValueError("Telemetry profiles must be keyed by their SHA-256 digest")
+
+ with self._lock:
+ candidate_profiles = copy.deepcopy(self._profiles)
+ try:
+ for digest, profile in profiles.items():
+ if profile_digest(profile) != digest:
+ raise ValueError(
+ 'Telemetry profile digest does not match "{}"'.format(digest)
+ )
+ candidate_profiles[digest] = profile
+
+ validate_telemetry_bundle(assignment, candidate_profiles)
+ referenced_profiles = {
+ target["profileHash"]: candidate_profiles[target["profileHash"]]
+ for target in assignment["targets"]
+ if target["format"] == "jmx_json"
+ }
+ self._persist_profiles(profiles)
+ self._atomic_json_write(
+ self.assignment_file,
+ {"hash": cache_hash, "assignment": assignment},
+ )
+ except Exception:
+ self.last_reload_successful = False
+ raise
+
+ self._assignment = copy.deepcopy(assignment)
+ self._profiles = copy.deepcopy(referenced_profiles)
+ self.hash = cache_hash
+ self.last_reload_successful = True
+
+ def snapshot(self):
+ with self._lock:
+ return {
+ "assignment": copy.deepcopy(self._assignment),
+ "profiles": copy.deepcopy(self._profiles),
+ }
+
+ def _persist_profiles(self, profiles):
+ for digest, profile in profiles.items():
+ path = self._profile_path(digest)
+ if os.path.isfile(path):
+ continue
+ self._atomic_json_write(path, profile)
+
+ def _profile_path(self, digest):
+ prefix = "sha256:"
+ if not isinstance(digest, str) or not digest.startswith(prefix):
+ raise ValueError('Invalid telemetry profile digest "{}"'.format(digest))
+ hex_digest = digest[len(prefix) :]
+ if len(hex_digest) != 64 or any(char not in "0123456789abcdef" for char in hex_digest):
+ raise ValueError('Invalid telemetry profile digest "{}"'.format(digest))
+ return os.path.join(self.profiles_dir, hex_digest + ".json")
+
+ def _atomic_json_write(self, path, value):
+ directory = os.path.dirname(path)
+ if not os.path.isdir(directory):
+ os.makedirs(directory)
+
+ descriptor, temporary_path = tempfile.mkstemp(prefix=".telemetry-", dir=directory)
+ try:
+ with os.fdopen(descriptor, "w", encoding="utf-8") as stream:
+ json.dump(value, stream, indent=2, sort_keys=True)
+ stream.flush()
+ os.fsync(stream.fileno())
+ os.replace(temporary_path, path)
+ except Exception:
+ try:
+ os.unlink(temporary_path)
+ except OSError:
+ pass
+ raise
diff --git a/ambari-agent/src/main/python/ambari_agent/alerts/ams_alert.py b/ambari-agent/src/main/python/ambari_agent/alerts/ams_alert.py
deleted file mode 100644
index 59ff8f0795..0000000000
--- a/ambari-agent/src/main/python/ambari_agent/alerts/ams_alert.py
+++ /dev/null
@@ -1,274 +0,0 @@
-#!/usr/bin/env python3
-
-"""
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-"""
-
-from ambari_commons import import_utils
-import time
-from alerts.metric_alert import MetricAlert, REALCODE_REGEXP
-import urllib.request, urllib.parse, urllib.error
-import json
-import logging
-import re
-import uuid
-
-from resource_management.libraries.functions.get_port_from_url import get_port_from_url
-from ambari_commons import inet_utils
-from ambari_commons.ast_checker import ASTChecker, BlacklistRule
-
-logger = logging.getLogger(__name__)
-
-AMS_METRICS_GET_URL = "/ws/v1/timeline/metrics?%s"
-
-
-class AmsAlert(MetricAlert):
- """
- Allow alerts to fire based on an AMS metrics.
- Alert is triggered if the aggregated function of the specified metric has
- grown beyond the specified threshold within a given time interval.
- """
-
- def __init__(self, alert_meta, alert_source_meta, config):
- super(AmsAlert, self).__init__(alert_meta, alert_source_meta, config)
-
- self.metric_info = None
- if "ams" in alert_source_meta:
- self.metric_info = AmsMetric(alert_source_meta["ams"])
-
- def _collect(self):
- """
- Low level function to collect alert data. The result is a tuple as:
- res[0] = the result code
- res[1] = the list of arguments supplied to the reporting text for the result code
- """
-
- if self.metric_info is None:
- raise Exception(
- "Could not determine result. Specific metric collector is not defined."
- )
-
- if self.uri_property_keys is None:
- raise Exception("Could not determine result. URL(s) were not defined.")
-
- # use the URI lookup keys to get a final URI value to query
- alert_uri = self._get_uri_from_structure(self.uri_property_keys)
-
- if logger.isEnabledFor(logging.DEBUG):
- logger.debug(
- "[Alert][{0}] Calculated metric URI to be {1} (ssl={2})".format(
- self.get_name(), alert_uri.uri, str(alert_uri.is_ssl_enabled)
- )
- )
-
- host = inet_utils.get_host_from_url(alert_uri.uri)
- if host is None:
- host = self.host_name
-
- try:
- port = int(get_port_from_url(alert_uri.uri))
- except:
- port = 6188
-
- collect_result = None
- value_list = []
-
- if isinstance(self.metric_info, AmsMetric):
- raw_data_points, http_code = self._load_metric(
- alert_uri.is_ssl_enabled, host, port, self.metric_info
- )
- if not raw_data_points and http_code not in [200, 307]:
- collect_result = self.RESULT_UNKNOWN
- value_list.append(f"HTTP {str(http_code)} response (metrics unavailable)")
- elif not raw_data_points and http_code in [200, 307]:
- raise Exception(
- f"[Alert][{self.get_name()}] Unable to extract JSON from HTTP response"
- )
- else:
- data_points = self.metric_info.calculate_value(raw_data_points)
- compute_result = self.metric_info.calculate_compute(data_points)
- value_list.append(compute_result)
-
- collect_result = self._get_result(
- value_list[0] if compute_result is None else compute_result
- )
-
- if logger.isEnabledFor(logging.DEBUG):
- logger.debug(
- f"[Alert][{self.get_name()}] Computed result = {str(value_list)}"
- )
-
- return (collect_result, value_list)
-
- def _load_metric(self, ssl, host, port, ams_metric):
- """creates a AmsMetric object that holds info about ams-based metrics"""
-
- if "0.0.0.0" in str(host):
- host = self.host_name
-
- current_time = int(time.time()) * 1000
- interval = ams_metric.interval
- get_metrics_parameters = {
- "metricNames": ",".join(ams_metric.metric_list),
- "appId": ams_metric.app_id,
- "hostname": self.host_name,
- "startTime": current_time - 60 * 1000 * interval,
- "endTime": current_time,
- "precision": "seconds",
- "grouped": "true",
- }
- encoded_get_metrics_parameters = urllib.parse.urlencode(get_metrics_parameters)
-
- path = AMS_METRICS_GET_URL % encoded_get_metrics_parameters
- url = "{0}://{1}:{2}{3}".format(
- "https" if ssl else "http", host, str(port), path
- )
-
- response = None
- data = b""
- status = None
- try:
- response = self.url_opener.open(url, timeout=self.connection_timeout)
- status = response.getcode()
- data = response.read()
- except urllib.error.HTTPError as exception:
- response = exception
- status = exception.code
- return (None, status)
- except Exception as exception:
- if logger.isEnabledFor(logging.DEBUG):
- logger.exception(
- f"[Alert][{self.get_name()}] Unable to retrieve metrics from AMS: {str(exception)}"
- )
- return (None, status)
- finally:
- if logger.isEnabledFor(logging.DEBUG):
- logger.debug(
- "[Alert][%s] AMS response status=%s bytes=%d",
- self.get_name(),
- status,
- len(data),
- )
- if response is not None:
- try:
- response.close()
- except Exception:
- logger.debug(
- f"[Alert][{self.get_name()}] Unable to close URL connection to {url}"
- )
-
- try:
- data_json = json.loads(data)
- metrics_data = data_json["metrics"]
- except Exception as exception:
- if logger.isEnabledFor(logging.DEBUG):
- logger.exception(
- "[Alert][{0}] Convert response to json failed or json doesn't contain needed data: {1}".format(
- self.get_name(), str(exception)
- )
- )
- return ([], status)
-
- metrics = []
- metric_dict = {}
- for metric_data in metrics_data:
- metric_dict[metric_data["metricname"]] = metric_data["metrics"]
-
- for metric_name in ams_metric.metric_list:
- if metric_name in metric_dict:
- metrics.append(metric_dict[metric_name])
-
- return (metrics, status)
-
-
-class AmsMetric:
- DYNAMIC_CODE_VALUE_TEMPLATE = """
-# ensure that division yields a float, use // for integer division
-from __future__ import division
-
-def f(args):
- l = []
- for k in args[0]:
- try:
- data_point = {0}
- l.append(data_point)
- except:
- continue
-
- return l
-"""
-
- DYNAMIC_CODE_COMPUTE_TEMPLATE = """
-# ensure that division yields a float, use // for integer division
-from __future__ import division
-from ambari_commons.aggregate_functions import sample_standard_deviation_percentage
-from ambari_commons.aggregate_functions import sample_standard_deviation
-from ambari_commons.aggregate_functions import mean
-from ambari_commons.aggregate_functions import count
-
-def f(args):
- func = {0}
- return func(args)
-"""
-
- def __init__(self, metric_info):
- self.custom_value_module = None
- self.custom_compute_module = None
- self.metric_list = metric_info["metric_list"]
- self.interval = metric_info["interval"] # in minutes
- self.app_id = metric_info["app_id"]
- self.minimum_value = metric_info["minimum_value"]
- self.safeChecker = ASTChecker([BlacklistRule()], use_blacklist=True)
-
- if "value" in metric_info:
- realcode = REALCODE_REGEXP.sub(r"args[\g<2>][k]", metric_info["value"])
- if not self.safeChecker.is_safe_expression(realcode):
- logger.exception(
- f"AmsMetric: Value expression {realcode} is not safe,blocked by checker"
- )
- raise Exception(f"AmsMetric: Value expression {realcode} is not safe")
-
- self.custom_value_module = import_utils.new_module(str(uuid.uuid4()))
- code = self.DYNAMIC_CODE_VALUE_TEMPLATE.format(realcode)
- exec(code, self.custom_value_module.__dict__)
-
- if "compute" in metric_info:
- realcode = metric_info["compute"]
- if not self.safeChecker.is_safe_expression(realcode):
- logger.exception(
- f"AmsMetric: compute expression {realcode} is not safe,blocked by checker"
- )
- raise Exception(f"AmsMetric: compute expression {realcode} is not safe")
- self.custom_compute_module = import_utils.new_module(str(uuid.uuid4()))
- code = self.DYNAMIC_CODE_COMPUTE_TEMPLATE.format(realcode)
- exec(code, self.custom_compute_module.__dict__)
-
- def calculate_value(self, args):
- data_points = None
- if self.custom_value_module is not None:
- data_points = self.custom_value_module.f(args)
- if self.minimum_value:
- data_points = [
- data_point for data_point in data_points if data_point > self.minimum_value
- ]
- return data_points
-
- def calculate_compute(self, args):
- compute_result = None
- if self.custom_compute_module is not None:
- compute_result = self.custom_compute_module.f(args)
- return compute_result
diff --git a/ambari-agent/src/main/python/ambari_agent/listeners/TelemetryEventListener.py b/ambari-agent/src/main/python/ambari_agent/listeners/TelemetryEventListener.py
new file mode 100644
index 0000000000..fb77bb330b
--- /dev/null
+++ b/ambari-agent/src/main/python/ambari_agent/listeners/TelemetryEventListener.py
@@ -0,0 +1,57 @@
+#!/usr/bin/env python3
+
+"""
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+"""
+
+import logging
+
+from ambari_agent import Constants
+from ambari_agent.listeners import EventListener
+
+
+logger = logging.getLogger(__name__)
+
+
+class TelemetryEventListener(EventListener):
+ """Applies complete, hash-addressed component telemetry assignments."""
+
+ def __init__(self, initializer_module):
+ super(TelemetryEventListener, self).__init__(initializer_module)
+ self.telemetry_cache = initializer_module.telemetry_cache
+
+ def on_event(self, headers, message):
+ if message == {}:
+ return
+
+ self.telemetry_cache.update(
+ message["assignment"],
+ message.get("profiles", {}),
+ message["hash"],
+ )
+
+ def get_handled_path(self):
+ return Constants.TELEMETRY_TOPIC
+
+ def get_log_message(self, headers, message_json):
+ if message_json:
+ assignment = message_json.get("assignment", {})
+ message_json["assignment"] = {
+ "targetCount": len(assignment.get("targets", []))
+ }
+ message_json["profiles"] = list(message_json.get("profiles", {}))
+ return super(TelemetryEventListener, self).get_log_message(headers, message_json)
diff --git a/ambari-agent/src/main/python/ambari_agent/main.py b/ambari-agent/src/main/python/ambari_agent/main.py
index 4bf33af230..5fd26b3409 100644
--- a/ambari-agent/src/main/python/ambari_agent/main.py
+++ b/ambari-agent/src/main/python/ambari_agent/main.py
@@ -343,6 +343,7 @@ MAX_RETRIES = 10
def run_threads(initializer_module):
initializer_module.alert_scheduler_handler.start()
+ initializer_module.prometheus_metrics_server.start()
initializer_module.heartbeat_thread.start()
initializer_module.component_status_executor.start()
initializer_module.command_status_reporter.start()
@@ -355,6 +356,7 @@ def run_threads(initializer_module):
initializer_module.action_queue.interrupt()
initializer_module.alert_scheduler_handler.stop()
+ initializer_module.prometheus_metrics_server.stop()
initializer_module.action_queue.join()
initializer_module.command_status_reporter.join()
@@ -362,6 +364,7 @@ def run_threads(initializer_module):
initializer_module.host_status_reporter.join()
initializer_module.alert_status_reporter.join()
initializer_module.heartbeat_thread.join()
+ initializer_module.prometheus_metrics_server.join()
# parse the options from command line
diff --git a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/3.0.0/package/scripts/__init__.py b/ambari-agent/src/main/python/ambari_agent/metrics/__init__.py
similarity index 85%
rename from ambari-server/src/main/resources/common-services/AMBARI_METRICS/3.0.0/package/scripts/__init__.py
rename to ambari-agent/src/main/python/ambari_agent/metrics/__init__.py
index 152033a3be..0a30822a4d 100644
--- a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/3.0.0/package/scripts/__init__.py
+++ b/ambari-agent/src/main/python/ambari_agent/metrics/__init__.py
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
+
"""
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
@@ -15,5 +16,8 @@ distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-
"""
+
+from ambari_agent.metrics.core import CollectorRegistry, MetricFamily, Sample
+
+__all__ = ["CollectorRegistry", "MetricFamily", "Sample"]
diff --git a/ambari-agent/src/main/python/ambari_agent/metrics/core.py b/ambari-agent/src/main/python/ambari_agent/metrics/core.py
new file mode 100644
index 0000000000..5ee459d22a
--- /dev/null
+++ b/ambari-agent/src/main/python/ambari_agent/metrics/core.py
@@ -0,0 +1,162 @@
+#!/usr/bin/env python3
+
+"""
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+"""
+
+import logging
+import math
+import re
+import time
+from collections import OrderedDict
+
+
+logger = logging.getLogger(__name__)
+
+METRIC_NAME_PATTERN = re.compile(r"^[a-zA-Z_:][a-zA-Z0-9_:]*$")
+LABEL_NAME_PATTERN = re.compile(r"^[a-zA-Z_][a-zA-Z0-9_]*$")
+METRIC_TYPES = frozenset(("counter", "gauge", "histogram", "summary", "untyped"))
+
+
+class Sample:
+ def __init__(self, value, labels=None):
+ self.value = value
+ self.labels = dict(labels or {})
+
+
+class MetricFamily:
+ def __init__(self, name, help_text, metric_type, samples=None):
+ if not METRIC_NAME_PATTERN.match(name):
+ raise ValueError(f'Invalid Prometheus metric name "{name}"')
+ if metric_type not in METRIC_TYPES:
+ raise ValueError(f'Invalid Prometheus metric type "{metric_type}"')
+
+ self.name = name
+ self.help_text = help_text
+ self.metric_type = metric_type
+ self.samples = samples or []
+
+ def add_sample(self, value, labels=None):
+ labels = labels or {}
+ for label_name in labels:
+ if not LABEL_NAME_PATTERN.match(label_name):
+ raise ValueError(f'Invalid Prometheus label name "{label_name}"')
+ self.samples.append(Sample(value, labels))
+
+
+class CollectorRegistry:
+ def __init__(self, collectors=None, clock=None):
+ self.collectors = list(collectors or [])
+ self.clock = clock or time.monotonic
+
+ def register(self, collector):
+ self.collectors.append(collector)
+
+ def collect(self):
+ families = []
+ collector_health = MetricFamily(
+ "ambari_agent_metrics_collector_up",
+ "Whether the Ambari Agent metric collector completed successfully.",
+ "gauge",
+ )
+ collector_duration = MetricFamily(
+ "ambari_agent_metrics_collector_duration_seconds",
+ "Time spent collecting one Ambari Agent metric collector.",
+ "gauge",
+ )
+
+ for collector in self.collectors:
+ collector_name = getattr(collector, "name", collector.__class__.__name__)
+ start = self.clock()
+ up = 1
+ try:
+ families.extend(collector.collect())
+ except Exception:
+ up = 0
+ logger.exception('Metric collector "%s" failed', collector_name)
+ finally:
+ collector_health.add_sample(up, {"collector": collector_name})
+ collector_duration.add_sample(
+ max(0, self.clock() - start), {"collector": collector_name}
+ )
+
+ families.extend((collector_health, collector_duration))
+ return families
+
+ def render(self):
+ return render_metrics(self.collect())
+
+
+def render_metrics(families):
+ merged_families = OrderedDict()
+ for family in families:
+ existing = merged_families.get(family.name)
+ if existing is None:
+ merged_families[family.name] = MetricFamily(
+ family.name,
+ family.help_text,
+ family.metric_type,
+ list(family.samples),
+ )
+ continue
+ if (
+ existing.help_text != family.help_text
+ or existing.metric_type != family.metric_type
+ ):
+ raise ValueError(f'Conflicting Prometheus metric family "{family.name}"')
+ existing.samples.extend(family.samples)
+
+ output = []
+ for family in merged_families.values():
+ output.append(f"# HELP {family.name} {_escape_help(family.help_text)}")
+ output.append(f"# TYPE {family.name} {family.metric_type}")
+ for sample in family.samples:
+ output.append(
+ f"{family.name}{_format_labels(sample.labels)} {_format_value(sample.value)}"
+ )
+
+ return ("\n".join(output) + "\n").encode("utf-8")
+
+
+def _escape_help(value):
+ return str(value).replace("\\", "\\\\").replace("\n", "\\n")
+
+
+def _escape_label_value(value):
+ return str(value).replace("\\", "\\\\").replace("\n", "\\n").replace('"', '\\"')
+
+
+def _format_labels(labels):
+ if not labels:
+ return ""
+
+ values = [f'{name}="{_escape_label_value(labels[name])}"' for name in sorted(labels)]
+ return "{" + ",".join(values) + "}"
+
+
+def _format_value(value):
+ if isinstance(value, bool):
+ return "1" if value else "0"
+ if isinstance(value, int):
+ return str(value)
+
+ number = float(value)
+ if math.isnan(number):
+ return "NaN"
+ if math.isinf(number):
+ return "+Inf" if number > 0 else "-Inf"
+ return repr(number)
diff --git a/ambari-agent/src/main/python/ambari_agent/metrics/linux.py b/ambari-agent/src/main/python/ambari_agent/metrics/linux.py
new file mode 100644
index 0000000000..0dec2ee2e7
--- /dev/null
+++ b/ambari-agent/src/main/python/ambari_agent/metrics/linux.py
@@ -0,0 +1,694 @@
+#!/usr/bin/env python3
+
+"""
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+"""
+
+import ctypes
+import logging
+import os
+import socket
+
+from ambari_agent.metrics.core import MetricFamily
+
+
+logger = logging.getLogger(__name__)
+NAMESPACE = "ambari_agent"
+CPU_MODES = (
+ "user",
+ "nice",
+ "system",
+ "idle",
+ "iowait",
+ "irq",
+ "softirq",
+ "steal",
+)
+MEMORY_FIELDS = {
+ "MemTotal": ("memory_total_bytes", "bytes"),
+ "MemFree": ("memory_free_bytes", "bytes"),
+ "MemAvailable": ("memory_available_bytes", "bytes"),
+ "Buffers": ("memory_buffers_bytes", "bytes"),
+ "Cached": ("memory_cached_bytes", "bytes"),
+ "SwapCached": ("memory_swap_cached_bytes", "bytes"),
+ "Active": ("memory_active_bytes", "bytes"),
+ "Active(anon)": ("memory_active_anon_bytes", "bytes"),
+ "Active(file)": ("memory_active_file_bytes", "bytes"),
+ "Inactive": ("memory_inactive_bytes", "bytes"),
+ "Inactive(anon)": ("memory_inactive_anon_bytes", "bytes"),
+ "Inactive(file)": ("memory_inactive_file_bytes", "bytes"),
+ "Unevictable": ("memory_unevictable_bytes", "bytes"),
+ "Mlocked": ("memory_mlocked_bytes", "bytes"),
+ "SwapTotal": ("memory_swap_total_bytes", "bytes"),
+ "SwapFree": ("memory_swap_free_bytes", "bytes"),
+ "Dirty": ("memory_dirty_bytes", "bytes"),
+ "Writeback": ("memory_writeback_bytes", "bytes"),
+ "AnonPages": ("memory_anon_pages_bytes", "bytes"),
+ "Mapped": ("memory_mapped_bytes", "bytes"),
+ "Shmem": ("memory_shmem_bytes", "bytes"),
+ "Slab": ("memory_slab_bytes", "bytes"),
+ "SReclaimable": ("memory_sreclaimable_bytes", "bytes"),
+ "SUnreclaim": ("memory_sunreclaim_bytes", "bytes"),
+ "KernelStack": ("memory_kernel_stack_bytes", "bytes"),
+ "NFS_Unstable": ("memory_nfs_unstable_bytes", "bytes"),
+ "Bounce": ("memory_bounce_bytes", "bytes"),
+ "VmallocUsed": ("memory_vmalloc_used_bytes", "bytes"),
+ "VmallocChunk": ("memory_vmalloc_chunk_bytes", "bytes"),
+ "AnonHugePages": ("memory_anon_huge_pages_bytes", "bytes"),
+ "HugePages_Total": ("memory_hugepages_total", "pages"),
+ "HugePages_Free": ("memory_hugepages_free", "pages"),
+ "HugePages_Rsvd": ("memory_hugepages_reserved", "pages"),
+ "HugePages_Surp": ("memory_hugepages_surplus", "pages"),
+ "Hugepagesize": ("memory_hugepages_size_bytes", "bytes"),
+ "DirectMap4k": ("memory_direct_map_4k_bytes", "bytes"),
+ "DirectMap2M": ("memory_direct_map_2m_bytes", "bytes"),
+ "DirectMap1G": ("memory_direct_map_1g_bytes", "bytes"),
+}
+TCP_STATES = {
+ "01": "established",
+ "02": "syn_sent",
+ "03": "syn_received",
+ "04": "fin_wait1",
+ "05": "fin_wait2",
+ "06": "time_wait",
+ "07": "close",
+ "08": "close_wait",
+ "09": "last_ack",
+ "0A": "listen",
+ "0B": "closing",
+ "0C": "new_syn_received",
+}
+PSEUDO_FILESYSTEMS = frozenset(
+ (
+ "autofs",
+ "bpf",
+ "cgroup",
+ "cgroup2",
+ "configfs",
+ "debugfs",
+ "devpts",
+ "devtmpfs",
+ "fusectl",
+ "hugetlbfs",
+ "mqueue",
+ "proc",
+ "pstore",
+ "securityfs",
+ "sysfs",
+ "tracefs",
+ )
+)
+
+
+class LinuxCollector:
+ def __init__(self, proc_root="/proc"):
+ self.proc_root = proc_root
+
+ def _read(self, name):
+ with open(os.path.join(self.proc_root, name), encoding="utf-8") as stream:
+ return stream.read()
+
+ def _read_optional(self, name):
+ try:
+ return self._read(name)
+ except OSError:
+ return None
+
+
+class CpuCollector(LinuxCollector):
+ name = "cpu"
+
+ def collect(self):
+ family = MetricFamily(
+ f"{NAMESPACE}_cpu_seconds_total",
+ "Seconds the CPUs spent in each mode.",
+ "counter",
+ )
+ ticks_per_second = float(os.sysconf("SC_CLK_TCK"))
+
+ for line in self._read("stat").splitlines():
+ fields = line.split()
+ if not fields or not fields[0].startswith("cpu"):
+ continue
+ if fields[0] != "cpu" and not fields[0][3:].isdigit():
+ continue
+
+ values = [int(value) for value in fields[1:]]
+ values.extend([0] * (10 - len(values)))
+ values[0] -= values[8]
+ values[1] -= values[9]
+ cpu = "total" if fields[0] == "cpu" else fields[0][3:]
+ for mode, ticks in zip(CPU_MODES, values):
+ family.add_sample(ticks / ticks_per_second, {"cpu": cpu, "mode": mode})
+
+ return [family]
+
+
+class MemoryCollector(LinuxCollector):
+ name = "memory"
+
+ def collect(self):
+ values = {}
+ for line in self._read("meminfo").splitlines():
+ key, raw_value = line.split(":", 1)
+ if key not in MEMORY_FIELDS:
+ continue
+ fields = raw_value.split()
+ multiplier = 1024 if len(fields) > 1 and fields[1] == "kB" else 1
+ values[key] = int(fields[0]) * multiplier
+
+ families = []
+ for source_name, (metric_name, unit) in MEMORY_FIELDS.items():
+ if source_name not in values:
+ continue
+ family = MetricFamily(
+ f"{NAMESPACE}_{metric_name}",
+ f"Linux {source_name} memory value in {unit}.",
+ "gauge",
+ )
+ family.add_sample(values[source_name])
+ families.append(family)
+ return families
+
+
+class SystemCollector(LinuxCollector):
+ name = "system"
+
+ def collect(self):
+ load_values = [float(value) for value in self._read("loadavg").split()[:3]]
+ uptime = float(self._read("uptime").split()[0])
+ stat_values = {}
+ for line in self._read("stat").splitlines():
+ fields = line.split()
+ if len(fields) >= 2 and fields[0] in (
+ "btime",
+ "ctxt",
+ "intr",
+ "processes",
+ "procs_running",
+ "procs_blocked",
+ ):
+ stat_values[fields[0]] = int(fields[1])
+
+ metrics = (
+ ("system_load1", "One minute system load average.", "gauge", load_values[0]),
+ ("system_load5", "Five minute system load average.", "gauge", load_values[1]),
+ ("system_load15", "Fifteen minute system load average.", "gauge", load_values[2]),
+ ("system_uptime_seconds", "System uptime in seconds.", "gauge", uptime),
+ (
+ "system_cpu_count",
+ "Number of logical CPUs visible to the Agent.",
+ "gauge",
+ os.cpu_count() or 0,
+ ),
+ (
+ "system_boot_time_seconds",
+ "System boot time as Unix epoch seconds.",
+ "gauge",
+ stat_values.get("btime", 0),
+ ),
+ (
+ "system_processes_forked_total",
+ "Total number of processes forked since boot.",
+ "counter",
+ stat_values.get("processes", 0),
+ ),
+ (
+ "system_processes_running",
+ "Number of processes currently runnable.",
+ "gauge",
+ stat_values.get("procs_running", 0),
+ ),
+ (
+ "system_processes_blocked",
+ "Number of processes currently blocked on I/O.",
+ "gauge",
+ stat_values.get("procs_blocked", 0),
+ ),
+ (
+ "system_context_switches_total",
+ "Total number of context switches since boot.",
+ "counter",
+ stat_values.get("ctxt", 0),
+ ),
+ (
+ "system_interrupts_total",
+ "Total number of interrupts serviced since boot.",
+ "counter",
+ stat_values.get("intr", 0),
+ ),
+ )
+
+ families = []
+ for name, help_text, metric_type, value in metrics:
+ family = MetricFamily(f"{NAMESPACE}_{name}", help_text, metric_type)
+ family.add_sample(value)
+ families.append(family)
+
+ optional_metrics = (
+ (
+ "file_descriptors_allocated",
+ "Number of allocated file descriptors.",
+ "gauge",
+ self._first_int("sys/fs/file-nr"),
+ ),
+ (
+ "file_descriptors_maximum",
+ "System-wide maximum number of file descriptors.",
+ "gauge",
+ self._first_int("sys/fs/file-max"),
+ ),
+ (
+ "entropy_available_bits",
+ "Available kernel entropy in bits.",
+ "gauge",
+ self._first_int("sys/kernel/random/entropy_avail"),
+ ),
+ (
+ "oom_kills_total",
+ "Total number of processes killed by the out-of-memory killer.",
+ "counter",
+ self._vmstat_value("oom_kill"),
+ ),
+ (
+ "conntrack_entries",
+ "Number of currently tracked network connections.",
+ "gauge",
+ self._first_int("sys/net/netfilter/nf_conntrack_count"),
+ ),
+ (
+ "conntrack_entries_limit",
+ "Maximum number of tracked network connections.",
+ "gauge",
+ self._first_int("sys/net/netfilter/nf_conntrack_max"),
+ ),
+ )
+ for name, help_text, metric_type, value in optional_metrics:
+ if value is None:
+ continue
+ family = MetricFamily(f"{NAMESPACE}_{name}", help_text, metric_type)
+ family.add_sample(value)
+ families.append(family)
+
+ tcp_connections = self._tcp_connections()
+ if tcp_connections is not None:
+ family = MetricFamily(
+ f"{NAMESPACE}_tcp_connections",
+ "Number of IPv4 and IPv6 TCP connections by state.",
+ "gauge",
+ )
+ for state in TCP_STATES.values():
+ family.add_sample(tcp_connections[state], {"state": state})
+ families.append(family)
+
+ info = MetricFamily(
+ f"{NAMESPACE}_host_info",
+ "Static information about the host running Ambari Agent.",
+ "gauge",
+ )
+ info.add_sample(1, {"hostname": socket.getfqdn()})
+ families.append(info)
+ return families
+
+ def _first_int(self, name):
+ content = self._read_optional(name)
+ if content is None:
+ return None
+ try:
+ return int(content.split()[0])
+ except (IndexError, ValueError):
+ return None
+
+ def _vmstat_value(self, key):
+ content = self._read_optional("vmstat")
+ if content is None:
+ return None
+ for line in content.splitlines():
+ fields = line.split()
+ if len(fields) == 2 and fields[0] == key:
+ try:
+ return int(fields[1])
+ except ValueError:
+ return None
+ return None
+
+ def _tcp_connections(self):
+ counts = {state: 0 for state in TCP_STATES.values()}
+ source_available = False
+ for name in ("net/tcp", "net/tcp6"):
+ content = self._read_optional(name)
+ if content is None:
+ continue
+ source_available = True
+ for line in content.splitlines()[1:]:
+ fields = line.split()
+ if len(fields) < 4:
+ continue
+ state = TCP_STATES.get(fields[3].upper())
+ if state is not None:
+ counts[state] += 1
+ return counts if source_available else None
+
+
+class FilesystemCollector(LinuxCollector):
+ name = "filesystem"
+
+ def __init__(self, proc_root="/proc", statvfs=None):
+ super().__init__(proc_root)
+ self.statvfs = statvfs or os.statvfs
+
+ def collect(self):
+ size = _filesystem_family("size_bytes", "Filesystem size in bytes.")
+ free = _filesystem_family("free_bytes", "Filesystem free space in bytes.")
+ available = _filesystem_family(
+ "available_bytes",
+ "Filesystem space available to unprivileged users in bytes.",
+ )
+ files = _filesystem_family("files", "Filesystem total file nodes.")
+ files_free = _filesystem_family("files_free", "Filesystem free file nodes.")
+ readonly = _filesystem_family(
+ "readonly", "Whether the filesystem is mounted read-only."
+ )
+ device_error = _filesystem_family(
+ "device_error", "Whether an error occurred while reading filesystem statistics."
+ )
+
+ for line in self._read("self/mounts").splitlines():
+ fields = line.split()
+ if len(fields) < 3 or fields[2] in PSEUDO_FILESYSTEMS:
+ continue
+
+ device = _decode_mount_field(fields[0])
+ mountpoint = _decode_mount_field(fields[1])
+ filesystem_type = fields[2]
+ labels = {
+ "device": device,
+ "fstype": filesystem_type,
+ "mountpoint": mountpoint,
+ }
+ try:
+ stats = self.statvfs(mountpoint)
+ except OSError:
+ logger.warning('Unable to read filesystem metrics for "%s"', mountpoint)
+ device_error.add_sample(1, labels)
+ continue
+ device_error.add_sample(0, labels)
+ size.add_sample(stats.f_blocks * stats.f_frsize, labels)
+ free.add_sample(stats.f_bfree * stats.f_frsize, labels)
+ available.add_sample(stats.f_bavail * stats.f_frsize, labels)
+ files.add_sample(stats.f_files, labels)
+ files_free.add_sample(stats.f_ffree, labels)
+ readonly.add_sample(1 if getattr(stats, "f_flag", 0) & os.ST_RDONLY else 0, labels)
+
+ return [size, free, available, files, files_free, readonly, device_error]
+
+
+class DiskIoCollector(LinuxCollector):
+ name = "disk_io"
+
+ def collect(self):
+ definitions = (
+ ("disk_reads_completed_total", "Completed disk read operations.", "counter"),
+ ("disk_reads_merged_total", "Merged disk read operations.", "counter"),
+ ("disk_read_bytes_total", "Bytes read from disk.", "counter"),
+ ("disk_read_time_seconds_total", "Seconds spent reading from disk.", "counter"),
+ ("disk_writes_completed_total", "Completed disk write operations.", "counter"),
+ ("disk_writes_merged_total", "Merged disk write operations.", "counter"),
+ ("disk_written_bytes_total", "Bytes written to disk.", "counter"),
+ ("disk_write_time_seconds_total", "Seconds spent writing to disk.", "counter"),
+ ("disk_io_in_progress", "Disk I/O operations currently in progress.", "gauge"),
+ ("disk_io_time_seconds_total", "Seconds spent doing disk I/O.", "counter"),
+ (
+ "disk_io_weighted_time_seconds_total",
+ "Weighted seconds spent doing disk I/O.",
+ "counter",
+ ),
+ )
+ families = [
+ MetricFamily(f"{NAMESPACE}_{name}", help_text, metric_type)
+ for name, help_text, metric_type in definitions
+ ]
+
+ for line in self._read("diskstats").splitlines():
+ fields = line.split()
+ if len(fields) < 14:
+ continue
+ labels = {"device": fields[2]}
+ values = (
+ int(fields[3]),
+ int(fields[4]),
+ int(fields[5]) * 512,
+ int(fields[6]) / 1000.0,
+ int(fields[7]),
+ int(fields[8]),
+ int(fields[9]) * 512,
+ int(fields[10]) / 1000.0,
+ int(fields[11]),
+ int(fields[12]) / 1000.0,
+ int(fields[13]) / 1000.0,
+ )
+ for family, value in zip(families, values):
+ family.add_sample(value, labels)
+
+ return families
+
+
+class NetworkCollector(LinuxCollector):
+ name = "network"
+
+ def collect(self):
+ definitions = (
+ ("network_receive_bytes_total", "Network bytes received."),
+ ("network_receive_packets_total", "Network packets received."),
+ ("network_receive_errors_total", "Network receive errors."),
+ ("network_receive_dropped_total", "Network receive packets dropped."),
+ ("network_transmit_bytes_total", "Network bytes transmitted."),
+ ("network_transmit_packets_total", "Network packets transmitted."),
+ ("network_transmit_errors_total", "Network transmit errors."),
+ ("network_transmit_dropped_total", "Network transmit packets dropped."),
+ )
+ families = [
+ MetricFamily(f"{NAMESPACE}_{name}", help_text, "counter")
+ for name, help_text in definitions
+ ]
+
+ for line in self._read("net/dev").splitlines()[2:]:
+ if ":" not in line:
+ continue
+ device, raw_values = line.split(":", 1)
+ device = device.strip()
+ if device == "lo":
+ continue
+ values = [int(value) for value in raw_values.split()]
+ if len(values) < 16:
+ continue
+ selected = values[0:4] + values[8:12]
+ for family, value in zip(families, selected):
+ family.add_sample(value, {"device": device})
+
+ return families
+
+
+class SocketCollector(LinuxCollector):
+ name = "sockets"
+
+ def collect(self):
+ values = {}
+ for source in ("net/sockstat", "net/sockstat6"):
+ content = self._read_optional(source)
+ if content is None:
+ continue
+ for line in content.splitlines():
+ if ":" not in line:
+ continue
+ protocol, raw_values = line.split(":", 1)
+ fields = raw_values.split()
+ for index in range(0, len(fields) - 1, 2):
+ try:
+ value = int(fields[index + 1])
+ except ValueError:
+ continue
+ key = (protocol, fields[index])
+ values[key] = values.get(key, 0) + value
+
+ definitions = (
+ ("socket_tcp_allocated", "TCP", "alloc", "Allocated TCP sockets."),
+ ("socket_tcp_in_use", "TCP", "inuse", "TCP sockets currently in use."),
+ ("socket_tcp_orphaned", "TCP", "orphan", "Orphaned TCP sockets."),
+ ("socket_tcp_time_wait", "TCP", "tw", "TCP sockets in the TIME_WAIT state."),
+ ("sockets_used", "sockets", "used", "Sockets currently in use."),
+ )
+ families = []
+ for metric_name, protocol, field, help_text in definitions:
+ value = values.get((protocol, field))
+ if value is None:
+ continue
+ family = MetricFamily(f"{NAMESPACE}_{metric_name}", help_text, "gauge")
+ family.add_sample(value)
+ families.append(family)
+ return families
+
+
+class _Timeval(ctypes.Structure):
+ _fields_ = [("tv_sec", ctypes.c_long), ("tv_usec", ctypes.c_long)]
+
+
+class _Timex(ctypes.Structure):
+ _fields_ = [
+ ("modes", ctypes.c_uint),
+ ("offset", ctypes.c_long),
+ ("freq", ctypes.c_long),
+ ("maxerror", ctypes.c_long),
+ ("esterror", ctypes.c_long),
+ ("status", ctypes.c_int),
+ ("constant", ctypes.c_long),
+ ("precision", ctypes.c_long),
+ ("tolerance", ctypes.c_long),
+ ("time", _Timeval),
+ ("tick", ctypes.c_long),
+ ("ppsfreq", ctypes.c_long),
+ ("jitter", ctypes.c_long),
+ ("shift", ctypes.c_int),
+ ("stabil", ctypes.c_long),
+ ("jitcnt", ctypes.c_long),
+ ("calcnt", ctypes.c_long),
+ ("errcnt", ctypes.c_long),
+ ("stbcnt", ctypes.c_long),
+ ("tai", ctypes.c_int),
+ ("reserved", ctypes.c_int * 11),
+ ]
+
+
+class TimexCollector:
+ name = "timex"
+ STA_NANO = 0x2000
+
+ def __init__(self, read_timex=None):
+ self.read_timex = read_timex or self._read_timex
+
+ def collect(self):
+ family = MetricFamily(
+ f"{NAMESPACE}_timex_offset_seconds",
+ "Time offset between the system clock and the kernel clock discipline source.",
+ "gauge",
+ )
+ family.add_sample(self.read_timex())
+ return [family]
+
+ def _read_timex(self):
+ libc = ctypes.CDLL(None, use_errno=True)
+ adjtimex = libc.adjtimex
+ adjtimex.argtypes = [ctypes.POINTER(_Timex)]
+ adjtimex.restype = ctypes.c_int
+ value = _Timex()
+ if adjtimex(ctypes.byref(value)) < 0:
+ error = ctypes.get_errno()
+ raise OSError(error, os.strerror(error))
+ scale = 1e-9 if value.status & self.STA_NANO else 1e-6
+ return value.offset * scale
+
+
+class ProcessCollector(LinuxCollector):
+ name = "processes"
+
+ PROCESS_STATES = {
+ "R": "running",
+ "S": "sleeping",
+ "D": "blocked",
+ "Z": "zombie",
+ "T": "stopped",
+ "t": "stopped",
+ "X": "dead",
+ "x": "dead",
+ "I": "idle",
+ "W": "paging",
+ }
+
+ def collect(self):
+ counts = {state: 0 for state in set(self.PROCESS_STATES.values())}
+ counts["unknown"] = 0
+ counts["total"] = 0
+ threads = 0
+
+ for entry in os.listdir(self.proc_root):
+ if not entry.isdigit():
+ continue
+ try:
+ stat = self._read(os.path.join(entry, "stat"))
+ except OSError:
+ continue
+
+ command_end = stat.rfind(")")
+ if command_end < 0 or len(stat) <= command_end + 2:
+ counts["unknown"] += 1
+ else:
+ stat_fields = stat[command_end + 2 :].split()
+ if not stat_fields:
+ counts["unknown"] += 1
+ else:
+ state = stat_fields[0]
+ counts[self.PROCESS_STATES.get(state, "unknown")] += 1
+ if len(stat_fields) > 17:
+ try:
+ threads += int(stat_fields[17])
+ except ValueError:
+ pass
+ counts["total"] += 1
+
+ family = MetricFamily(
+ f"{NAMESPACE}_processes",
+ "Number of processes in each Linux process state.",
+ "gauge",
+ )
+ for state in sorted(counts):
+ family.add_sample(counts[state], {"state": state})
+ thread_family = MetricFamily(
+ f"{NAMESPACE}_process_threads",
+ "Total number of threads across processes visible to Ambari Agent.",
+ "gauge",
+ )
+ thread_family.add_sample(threads)
+ return [family, thread_family]
+
+
+def default_collectors(proc_root="/proc"):
+ if not os.path.isdir(proc_root):
+ return []
+ return [
+ CpuCollector(proc_root),
+ MemoryCollector(proc_root),
+ SystemCollector(proc_root),
+ FilesystemCollector(proc_root),
+ DiskIoCollector(proc_root),
+ NetworkCollector(proc_root),
+ SocketCollector(proc_root),
+ ProcessCollector(proc_root),
+ TimexCollector(),
+ ]
+
+
+def _filesystem_family(name, help_text):
+ return MetricFamily(f"{NAMESPACE}_filesystem_{name}", help_text, "gauge")
+
+
+def _decode_mount_field(value):
+ return (
+ value.replace("\\040", " ")
+ .replace("\\011", "\t")
+ .replace("\\012", "\n")
+ .replace("\\134", "\\")
+ )
diff --git a/ambari-agent/src/main/python/ambari_agent/metrics/server.py b/ambari-agent/src/main/python/ambari_agent/metrics/server.py
new file mode 100644
index 0000000000..749eb27f86
--- /dev/null
+++ b/ambari-agent/src/main/python/ambari_agent/metrics/server.py
@@ -0,0 +1,168 @@
+#!/usr/bin/env python3
+
+"""
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+"""
+
+from http.server import BaseHTTPRequestHandler, HTTPServer
+import logging
+from socketserver import ThreadingMixIn
+import threading
+from urllib.parse import urlsplit
+
+from ambari_agent.metrics.core import CollectorRegistry
+from ambari_agent.metrics.linux import default_collectors
+from ambari_agent.metrics.telemetry import (
+ PROMETHEUS_CONTENT_TYPE,
+ KerberosTelemetryFetcher,
+ TelemetryHttpClient,
+ TelemetryRouteRegistry,
+ TelemetryScrapeError,
+)
+
+
+logger = logging.getLogger(__name__)
+
+
+class MetricsHTTPServer(ThreadingMixIn, HTTPServer):
+ allow_reuse_address = True
+ daemon_threads = True
+
+
+def create_request_handler(registry, telemetry_routes=None):
+ class MetricsRequestHandler(BaseHTTPRequestHandler):
+ def do_GET(self):
+ self._respond(include_body=True)
+
+ def do_HEAD(self):
+ self._respond(include_body=False)
+
+ def _respond(self, include_body):
+ path = urlsplit(self.path).path
+ if path == "/metrics":
+ body = registry.render()
+ content_type = PROMETHEUS_CONTENT_TYPE
+ status = 200
+ elif path.startswith("/metrics/components/"):
+ route_id = path[len("/metrics/components/") :]
+ if telemetry_routes is None or not route_id or "/" in route_id:
+ body = b"Not Found\n"
+ content_type = "text/plain; charset=utf-8"
+ status = 404
+ else:
+ try:
+ response = telemetry_routes.scrape(route_id)
+ body = response.body
+ content_type = response.content_type
+ status = 200
+ except KeyError:
+ body = b"Not Found\n"
+ content_type = "text/plain; charset=utf-8"
+ status = 404
+ except TelemetryScrapeError as err:
+ body = (str(err) + "\n").encode("utf-8")
+ content_type = "text/plain; charset=utf-8"
+ status = err.status
+ elif path == "/-/healthy":
+ body = b"Ambari Agent Prometheus exporter is healthy.\n"
+ content_type = "text/plain; charset=utf-8"
+ status = 200
+ else:
+ body = b"Not Found\n"
+ content_type = "text/plain; charset=utf-8"
+ status = 404
+
+ self.send_response(status)
+ self.send_header("Content-Type", content_type)
+ self.send_header("Content-Length", str(len(body)))
+ self.end_headers()
+ if include_body:
+ self.wfile.write(body)
+
+ def log_message(self, message_format, *args):
+ logger.debug("Prometheus endpoint: " + message_format, *args)
+
+ return MetricsRequestHandler
+
+
+class PrometheusMetricsServer(threading.Thread):
+ def __init__(
+ self,
+ initializer_module,
+ registry=None,
+ server_class=None,
+ telemetry_routes=None,
+ ):
+ super().__init__(name="prometheus-metrics-server")
+ self.agent_stop_event = initializer_module.stop_event
+ self.config = initializer_module.config
+ self.enabled = self.config.prometheus_metrics_enabled
+ self.bind_address = self.config.prometheus_metrics_bind_address
+ self.port = self.config.prometheus_metrics_port
+ self.registry = registry or CollectorRegistry(default_collectors())
+ telemetry_cache = getattr(initializer_module, "telemetry_cache", None)
+ self.telemetry_routes = telemetry_routes
+ if self.telemetry_routes is None and telemetry_cache is not None:
+ http_client = TelemetryHttpClient(
+ kerberos_fetcher=KerberosTelemetryFetcher(self.config)
+ )
+ self.telemetry_routes = TelemetryRouteRegistry(
+ telemetry_cache, http_client=http_client
+ )
+ if self.telemetry_routes is not None:
+ self.registry.register(self.telemetry_routes)
+ self.server_class = server_class or MetricsHTTPServer
+ self.http_server = None
+ self.startup_error = None
+ self.ready_event = threading.Event()
+ self.stop_event = threading.Event()
+
+ def run(self):
+ if not self.enabled:
+ self.ready_event.set()
+ return
+
+ try:
+ handler = create_request_handler(self.registry, self.telemetry_routes)
+ self.http_server = self.server_class((self.bind_address, self.port), handler)
+ self.http_server.timeout = 0.5
+ logger.info(
+ "Ambari Agent Prometheus endpoint listening on %s:%s",
+ self.bind_address,
+ self.http_server.server_address[1],
+ )
+ self.ready_event.set()
+
+ while not self.agent_stop_event.is_set() and not self.stop_event.is_set():
+ self.http_server.handle_request()
+ except Exception as err:
+ self.startup_error = err
+ self.ready_event.set()
+ logger.exception("Ambari Agent Prometheus endpoint failed")
+ finally:
+ if self.http_server is not None:
+ self.http_server.server_close()
+ self.ready_event.set()
+
+ def stop(self):
+ self.stop_event.set()
+
+ @property
+ def server_address(self):
+ if self.http_server is None:
+ return None
+ return self.http_server.server_address
diff --git a/ambari-agent/src/main/python/ambari_agent/metrics/telemetry.py b/ambari-agent/src/main/python/ambari_agent/metrics/telemetry.py
new file mode 100644
index 0000000000..d15c4cfefe
--- /dev/null
+++ b/ambari-agent/src/main/python/ambari_agent/metrics/telemetry.py
@@ -0,0 +1,656 @@
+#!/usr/bin/env python3
+
+"""
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+"""
+
+from collections import OrderedDict, defaultdict
+import hashlib
+import json
+import math
+import re
+import threading
+import time
+from urllib.error import HTTPError, URLError
+from urllib.parse import urlsplit
+from urllib.request import HTTPRedirectHandler, Request, build_opener
+
+from ambari_agent.metrics.core import (
+ LABEL_NAME_PATTERN,
+ METRIC_NAME_PATTERN,
+ MetricFamily,
+ render_metrics,
+)
+
+
+PROMETHEUS_CONTENT_TYPE = "text/plain; version=0.0.4; charset=utf-8"
+ROUTE_ID_PATTERN = re.compile(r"^[a-zA-Z0-9][a-zA-Z0-9._-]{0,127}$")
+SUPPORTED_FORMATS = frozenset(("prometheus_text", "jmx_json"))
+SUPPORTED_TYPES = frozenset(("counter", "gauge"))
+PROMETHEUS_SAMPLE_PATTERN = re.compile(
+ r"^[a-zA-Z_:][a-zA-Z0-9_:]*(?:\{.*\})?[ \t]+"
+ r"(?:[-+]?(?:(?:[0-9]+(?:\.[0-9]*)?|\.[0-9]+)(?:[eE][-+]?[0-9]+)?)"
+ r"|NaN|[+-]Inf)(?:[ \t]+[-+]?[0-9]+)?$"
+)
+MAX_PROMETHEUS_LINE_BYTES = 1024 * 1024
+
+
+class TelemetryConfigError(ValueError):
+ pass
+
+
+class TelemetryScrapeError(Exception):
+ def __init__(self, message, status=502):
+ super().__init__(message)
+ self.status = status
+
+
+class TelemetryResponse:
+ def __init__(self, body, content_type=PROMETHEUS_CONTENT_TYPE):
+ self.body = body
+ self.content_type = content_type
+
+
+class _RejectRedirects(HTTPRedirectHandler):
+ def redirect_request(self, req, fp, code, msg, headers, newurl):
+ return None
+
+
+def profile_digest(profile):
+ encoded = json.dumps(
+ profile, sort_keys=True, separators=(",", ":"), ensure_ascii=True
+ ).encode("utf-8")
+ return "sha256:" + hashlib.sha256(encoded).hexdigest()
+
+
+def validate_prometheus_text(payload):
+ if not isinstance(payload, bytes) or not payload:
+ raise TelemetryScrapeError(
+ "Telemetry upstream returned an empty Prometheus response"
+ )
+ try:
+ document = payload.decode("utf-8")
+ except UnicodeDecodeError as err:
+ raise TelemetryScrapeError(
+ "Telemetry upstream returned non-UTF-8 Prometheus text"
+ ) from err
+ if "\x00" in document:
+ raise TelemetryScrapeError(
+ "Telemetry upstream returned invalid Prometheus text"
+ )
+
+ sample_count = 0
+ for line in document.splitlines():
+ if len(line.encode("utf-8")) > MAX_PROMETHEUS_LINE_BYTES:
+ raise TelemetryScrapeError(
+ "Telemetry upstream returned an oversized Prometheus line"
+ )
+ line = line.strip()
+ if not line or line.startswith("#"):
+ continue
+ if not PROMETHEUS_SAMPLE_PATTERN.match(line):
+ raise TelemetryScrapeError(
+ "Telemetry upstream returned malformed Prometheus text"
+ )
+ sample_count += 1
+
+ if sample_count == 0:
+ raise TelemetryScrapeError(
+ "Telemetry upstream returned no Prometheus samples"
+ )
+
+
+def validate_telemetry_bundle(assignment, profiles):
+ if not isinstance(assignment, dict) or assignment.get("schemaVersion") != 1:
+ raise TelemetryConfigError("Telemetry assignment schemaVersion must be 1")
+ targets = assignment.get("targets")
+ if not isinstance(targets, list):
+ raise TelemetryConfigError("Telemetry assignment targets must be a list")
+
+ route_ids = set()
+ for target in targets:
+ _validate_target(target, profiles)
+ route_id = target["id"]
+ if route_id in route_ids:
+ raise TelemetryConfigError('Duplicate telemetry route "{}"'.format(route_id))
+ route_ids.add(route_id)
+
+
+def validate_profile(profile):
+ if not isinstance(profile, dict) or profile.get("schemaVersion") != 1:
+ raise TelemetryConfigError("Telemetry profile schemaVersion must be 1")
+ if not isinstance(profile.get("id"), str) or not profile["id"]:
+ raise TelemetryConfigError("Telemetry profile id must be a non-empty string")
+ rules = profile.get("rules")
+ if not isinstance(rules, list) or not rules:
+ raise TelemetryConfigError("Telemetry profile rules must be a non-empty list")
+ max_series = profile.get("maxSeries", 10000)
+ if (
+ not isinstance(max_series, int)
+ or isinstance(max_series, bool)
+ or not 1 <= max_series <= 100000
+ ):
+ raise TelemetryConfigError("Telemetry profile maxSeries must be between 1 and 100000")
+
+ families = {}
+ for rule in rules:
+ _validate_rule(rule, families)
+
+
+def _validate_target(target, profiles):
+ if not isinstance(target, dict):
+ raise TelemetryConfigError("Telemetry target must be an object")
+ route_id = target.get("id")
+ if not isinstance(route_id, str) or not ROUTE_ID_PATTERN.match(route_id):
+ raise TelemetryConfigError('Invalid telemetry route id "{}"'.format(route_id))
+ source_format = target.get("format")
+ if source_format not in SUPPORTED_FORMATS:
+ raise TelemetryConfigError('Unsupported telemetry format "{}"'.format(source_format))
+
+ source_url = target.get("url")
+ parsed_url = urlsplit(source_url if isinstance(source_url, str) else "")
+ if (
+ parsed_url.scheme not in ("http", "https")
+ or not parsed_url.hostname
+ or parsed_url.username
+ or parsed_url.password
+ or parsed_url.query
+ or parsed_url.fragment
+ ):
+ raise TelemetryConfigError('Invalid telemetry target URL "{}"'.format(source_url))
+
+ timeout = target.get("timeoutSeconds", 5)
+ if (
+ isinstance(timeout, bool)
+ or not isinstance(timeout, (int, float))
+ or not 0 < timeout <= 60
+ ):
+ raise TelemetryConfigError("Telemetry timeoutSeconds must be greater than 0 and at most 60")
+ max_bytes = target.get("maxResponseBytes", 32 * 1024 * 1024)
+ if (
+ isinstance(max_bytes, bool)
+ or not isinstance(max_bytes, int)
+ or not 1024 <= max_bytes <= 64 * 1024 * 1024
+ ):
+ raise TelemetryConfigError("Telemetry maxResponseBytes must be between 1024 and 67108864")
+ max_concurrent = target.get("maxConcurrentRequests", 2)
+ if (
+ isinstance(max_concurrent, bool)
+ or not isinstance(max_concurrent, int)
+ or not 1 <= max_concurrent <= 16
+ ):
+ raise TelemetryConfigError(
+ "Telemetry maxConcurrentRequests must be between 1 and 16"
+ )
+
+ auth = target.get("auth", {})
+ if not isinstance(auth, dict):
+ raise TelemetryConfigError("Telemetry authentication must be an object")
+ auth_type = auth.get("type", "none")
+ if auth_type not in ("none", "kerberos"):
+ raise TelemetryConfigError('Unsupported telemetry authentication "{}"'.format(auth_type))
+ if auth_type == "kerberos":
+ for field in ("principal", "keytab"):
+ if not isinstance(auth.get(field), str) or not auth[field]:
+ raise TelemetryConfigError(
+ 'Kerberos telemetry authentication requires "{}"'.format(field)
+ )
+
+ profile_hash = target.get("profileHash")
+ if source_format == "jmx_json":
+ if profile_hash not in profiles:
+ raise TelemetryConfigError(
+ 'Telemetry route "{}" references a missing profile'.format(route_id)
+ )
+ validate_profile(profiles[profile_hash])
+ elif profile_hash is not None:
+ raise TelemetryConfigError(
+ 'Prometheus pass-through route "{}" must not reference a profile'.format(route_id)
+ )
+
+
+def _validate_rule(rule, families):
+ if not isinstance(rule, dict):
+ raise TelemetryConfigError("Telemetry profile rule must be an object")
+ bean = rule.get("bean")
+ if (
+ not isinstance(bean, dict)
+ or not isinstance(bean.get("domain"), str)
+ or not bean["domain"]
+ ):
+ raise TelemetryConfigError("Telemetry rule bean domain must be a non-empty string")
+ properties = bean.get("properties")
+ if not isinstance(properties, dict) or not properties:
+ raise TelemetryConfigError("Telemetry rule bean properties must be a non-empty object")
+ if any(
+ not isinstance(key, str) or not isinstance(value, str)
+ for key, value in properties.items()
+ ):
+ raise TelemetryConfigError("Telemetry bean property names and values must be strings")
+
+ labels = rule.get("labels", {})
+ if not isinstance(labels, dict):
+ raise TelemetryConfigError("Telemetry rule labels must be an object")
+ for label_name, source in labels.items():
+ if not LABEL_NAME_PATTERN.match(label_name) or label_name.startswith("__"):
+ raise TelemetryConfigError('Invalid telemetry label "{}"'.format(label_name))
+ if not isinstance(source, dict) or set(source) not in ({"property"}, {"value"}):
+ raise TelemetryConfigError(
+ 'Telemetry label "{}" must define property or value'.format(label_name)
+ )
+ if "property" in source and (
+ not isinstance(source["property"], str) or not source["property"]
+ ):
+ raise TelemetryConfigError(
+ 'Telemetry label "{}" property must be a non-empty string'.format(
+ label_name
+ )
+ )
+ if "value" in source and not isinstance(source["value"], str):
+ raise TelemetryConfigError(
+ 'Telemetry label "{}" value must be a string'.format(label_name)
+ )
+
+ attributes = rule.get("attributes")
+ if not isinstance(attributes, dict) or not attributes:
+ raise TelemetryConfigError("Telemetry rule attributes must be a non-empty object")
+ for source_name, definition in attributes.items():
+ if not isinstance(source_name, str) or not source_name:
+ raise TelemetryConfigError("Telemetry source attribute must be a non-empty string")
+ _validate_metric_definition(definition, families)
+
+
+def _validate_metric_definition(definition, families):
+ if not isinstance(definition, dict):
+ raise TelemetryConfigError("Telemetry metric definition must be an object")
+ name = definition.get("name")
+ metric_type = definition.get("type")
+ help_text = definition.get("help")
+ if not isinstance(name, str) or not METRIC_NAME_PATTERN.match(name):
+ raise TelemetryConfigError('Invalid Prometheus metric name "{}"'.format(name))
+ if metric_type not in SUPPORTED_TYPES:
+ raise TelemetryConfigError('Unsupported Prometheus metric type "{}"'.format(metric_type))
+ if metric_type == "counter" and not name.endswith("_total"):
+ raise TelemetryConfigError('Counter metric "{}" must end with _total'.format(name))
+ if not isinstance(help_text, str) or not help_text.strip():
+ raise TelemetryConfigError('Metric "{}" must define help text'.format(name))
+ if not isinstance(definition.get("unit", "unitless"), str):
+ raise TelemetryConfigError('Metric "{}" unit must be a string'.format(name))
+ scale = definition.get("scale", 1.0)
+ if isinstance(scale, bool) or not isinstance(scale, (int, float)) or not math.isfinite(scale):
+ raise TelemetryConfigError('Metric "{}" scale must be finite'.format(name))
+ value_map = definition.get("valueMap")
+ if value_map is not None:
+ if not isinstance(value_map, dict) or not value_map:
+ raise TelemetryConfigError('Metric "{}" valueMap must be a non-empty object'.format(name))
+ for source_value, mapped_value in value_map.items():
+ if (
+ not isinstance(source_value, str)
+ or isinstance(mapped_value, bool)
+ or not isinstance(mapped_value, (int, float))
+ or not math.isfinite(mapped_value)
+ ):
+ raise TelemetryConfigError('Metric "{}" valueMap must map strings to finite numbers'.format(name))
+
+ metadata = (metric_type, help_text)
+ existing = families.get(name)
+ if existing is not None and existing != metadata:
+ raise TelemetryConfigError('Conflicting metric family "{}"'.format(name))
+ families[name] = metadata
+
+
+class TelemetryHttpClient:
+ def __init__(self, opener=None, kerberos_fetcher=None):
+ self.opener = opener or build_opener(_RejectRedirects())
+ self.kerberos_fetcher = kerberos_fetcher
+
+ def fetch(self, target):
+ if target.get("auth", {}).get("type", "none") == "kerberos":
+ if self.kerberos_fetcher is None:
+ raise TelemetryScrapeError("Kerberos telemetry fetcher is not configured")
+ return self.kerberos_fetcher(target)
+
+ request = Request(
+ target["url"],
+ headers={"Accept": "text/plain, application/json", "User-Agent": "ambari-agent"},
+ )
+ timeout = target.get("timeoutSeconds", 5)
+ max_bytes = target.get("maxResponseBytes", 32 * 1024 * 1024)
+ try:
+ with self.opener.open(request, timeout=timeout) as response:
+ body = response.read(max_bytes + 1)
+ if len(body) > max_bytes:
+ raise TelemetryScrapeError("Telemetry response exceeded the configured size limit")
+ content_type = response.headers.get("Content-Type", PROMETHEUS_CONTENT_TYPE)
+ return TelemetryResponse(body, content_type)
+ except TelemetryScrapeError:
+ raise
+ except HTTPError as err:
+ raise TelemetryScrapeError("Telemetry upstream returned HTTP {}".format(err.code))
+ except (URLError, OSError) as err:
+ reason = getattr(err, "reason", err)
+ if isinstance(reason, TimeoutError):
+ raise TelemetryScrapeError("Telemetry upstream timed out", status=504)
+ raise TelemetryScrapeError("Telemetry upstream request failed")
+
+
+class KerberosTelemetryFetcher:
+ def __init__(self, config):
+ self.tmp_dir = config.get("agent", "tmp_dir")
+ self.user = config.get("agent", "run_as_user", "root")
+
+ def __call__(self, target):
+ from resource_management.libraries.functions.curl_krb_request import (
+ curl_krb_request,
+ )
+
+ auth = target["auth"]
+ max_bytes = target.get("maxResponseBytes", 32 * 1024 * 1024)
+ try:
+ body, _, _ = curl_krb_request(
+ self.tmp_dir,
+ auth["keytab"],
+ auth["principal"],
+ target["url"],
+ "telemetry",
+ auth.get("kerberosExecutableSearchPaths"),
+ False,
+ target["id"],
+ self.user,
+ connection_timeout=target.get("timeoutSeconds", 5),
+ ca_certs=auth.get("caFile"),
+ follow_redirects=False,
+ fail_on_http_error=True,
+ max_response_bytes=max_bytes,
+ verify_ssl=True,
+ )
+ except Exception as err:
+ raise TelemetryScrapeError(
+ "Kerberos telemetry upstream request failed"
+ ) from err
+
+ if isinstance(body, str):
+ body = body.encode("utf-8")
+ if not isinstance(body, bytes) or not body:
+ raise TelemetryScrapeError(
+ "Kerberos telemetry upstream returned an empty response"
+ )
+ if len(body) > max_bytes:
+ raise TelemetryScrapeError(
+ "Telemetry response exceeded the configured size limit"
+ )
+ return TelemetryResponse(body)
+
+
+class TelemetryRouteRegistry:
+ def __init__(self, config_cache, http_client=None, clock=None, wall_clock=None):
+ self.name = "component_telemetry"
+ self.config_cache = config_cache
+ self.http_client = http_client or TelemetryHttpClient()
+ self.clock = clock or time.monotonic
+ self.wall_clock = wall_clock or time.time
+ self._lock = threading.RLock()
+ self._requests = defaultdict(int)
+ self._last_duration = {}
+ self._last_success = {}
+ self._conversion_failures = defaultdict(int)
+ self._route_semaphores = {}
+
+ def scrape(self, route_id):
+ snapshot = self.config_cache.snapshot()
+ targets = {target["id"]: target for target in snapshot["assignment"]["targets"]}
+ target = targets.get(route_id)
+ if target is None:
+ raise KeyError(route_id)
+
+ started = self.clock()
+ result = "success"
+ semaphore = self._route_semaphore(
+ route_id, target.get("maxConcurrentRequests", 2)
+ )
+ acquired = semaphore.acquire(blocking=False)
+ if not acquired:
+ with self._lock:
+ self._requests[(route_id, "error")] += 1
+ self._last_duration[route_id] = 0
+ raise TelemetryScrapeError(
+ "Telemetry route has reached its concurrent scrape limit", status=503
+ )
+ try:
+ upstream = self.http_client.fetch(target)
+ if target["format"] == "prometheus_text":
+ validate_prometheus_text(upstream.body)
+ response = TelemetryResponse(upstream.body, PROMETHEUS_CONTENT_TYPE)
+ else:
+ profile = snapshot["profiles"][target["profileHash"]]
+ try:
+ response = TelemetryResponse(convert_jmx_json(upstream.body, profile))
+ except Exception:
+ with self._lock:
+ self._conversion_failures[route_id] += 1
+ raise
+ with self._lock:
+ self._last_success[route_id] = self.wall_clock()
+ return response
+ except Exception:
+ result = "error"
+ raise
+ finally:
+ with self._lock:
+ self._requests[(route_id, result)] += 1
+ self._last_duration[route_id] = max(0, self.clock() - started)
+ semaphore.release()
+
+ def _route_semaphore(self, route_id, limit):
+ with self._lock:
+ configured_limit, semaphore = self._route_semaphores.get(
+ route_id, (None, None)
+ )
+ if semaphore is None or configured_limit != limit:
+ semaphore = threading.BoundedSemaphore(limit)
+ self._route_semaphores[route_id] = (limit, semaphore)
+ return semaphore
+
+ def collect(self):
+ configured = MetricFamily(
+ "ambari_agent_telemetry_target_configured",
+ "Whether a component telemetry target is configured.",
+ "gauge",
+ )
+ requests = MetricFamily(
+ "ambari_agent_telemetry_scrape_requests_total",
+ "Component telemetry scrape requests by result.",
+ "counter",
+ )
+ duration = MetricFamily(
+ "ambari_agent_telemetry_last_scrape_duration_seconds",
+ "Duration of the latest component telemetry scrape.",
+ "gauge",
+ )
+ last_success = MetricFamily(
+ "ambari_agent_telemetry_last_success_timestamp_seconds",
+ "Unix timestamp of the latest successful component telemetry scrape.",
+ "gauge",
+ )
+ conversion_failures = MetricFamily(
+ "ambari_agent_telemetry_conversion_failures_total",
+ "JMX profile conversion failures by component telemetry route.",
+ "counter",
+ )
+ reload_success = MetricFamily(
+ "ambari_agent_telemetry_config_last_reload_successful",
+ "Whether the latest telemetry configuration reload succeeded.",
+ "gauge",
+ )
+
+ snapshot = self.config_cache.snapshot()
+ for target in snapshot["assignment"]["targets"]:
+ configured.add_sample(1, {"route": target["id"]})
+ reload_success.add_sample(1 if self.config_cache.last_reload_successful else 0)
+
+ with self._lock:
+ for (route_id, result), count in sorted(self._requests.items()):
+ requests.add_sample(count, {"route": route_id, "result": result})
+ for route_id, value in sorted(self._last_duration.items()):
+ duration.add_sample(value, {"route": route_id})
+ for route_id, value in sorted(self._last_success.items()):
+ last_success.add_sample(value, {"route": route_id})
+ for route_id, value in sorted(self._conversion_failures.items()):
+ conversion_failures.add_sample(value, {"route": route_id})
+
+ return [
+ configured,
+ requests,
+ duration,
+ last_success,
+ conversion_failures,
+ reload_success,
+ ]
+
+
+def convert_jmx_json(payload, profile):
+ validate_profile(profile)
+ try:
+ document = json.loads(payload.decode("utf-8") if isinstance(payload, bytes) else payload)
+ except (TypeError, ValueError, UnicodeDecodeError):
+ raise TelemetryScrapeError("Telemetry upstream returned invalid JMX JSON")
+ beans = document.get("beans") if isinstance(document, dict) else None
+ if not isinstance(beans, list):
+ raise TelemetryScrapeError("Telemetry JMX JSON does not contain a beans list")
+
+ families = OrderedDict()
+ seen_samples = defaultdict(set)
+ series_count = 0
+ for bean in beans:
+ if not isinstance(bean, dict) or not isinstance(bean.get("name"), str):
+ continue
+ try:
+ domain, properties = parse_object_name(bean["name"])
+ except ValueError:
+ continue
+ for rule in profile["rules"]:
+ if not _bean_matches(rule["bean"], domain, properties):
+ continue
+ labels = _rule_labels(rule.get("labels", {}), properties)
+ for source_name, definition in rule["attributes"].items():
+ value = _attribute_value(bean, source_name)
+ value_map = definition.get("valueMap")
+ if value_map is not None:
+ lookup_value = str(value).lower() if isinstance(value, bool) else str(value)
+ value = value_map.get(lookup_value)
+ if isinstance(value, bool) or not isinstance(value, (int, float)):
+ continue
+ scale = definition.get("scale", 1)
+ if scale != 1:
+ value *= scale
+ metric_name = definition["name"]
+ family = families.get(metric_name)
+ if family is None:
+ family = MetricFamily(metric_name, definition["help"], definition["type"])
+ families[metric_name] = family
+ sample_key = tuple(sorted(labels.items()))
+ if sample_key in seen_samples[metric_name]:
+ raise TelemetryScrapeError('Duplicate JMX sample for metric "{}"'.format(metric_name))
+ seen_samples[metric_name].add(sample_key)
+ family.add_sample(value, labels)
+ series_count += 1
+ if series_count > profile.get("maxSeries", 10000):
+ raise TelemetryScrapeError("JMX telemetry exceeded the profile series limit")
+
+ if not families:
+ raise TelemetryScrapeError("No JMX metrics matched the telemetry profile")
+ return render_metrics(families.values())
+
+
+def parse_object_name(value):
+ if not isinstance(value, str) or ":" not in value:
+ raise ValueError("Invalid JMX ObjectName")
+ domain, properties_text = value.split(":", 1)
+ if not domain or not properties_text:
+ raise ValueError("Invalid JMX ObjectName")
+ properties = {}
+ for field in _split_unquoted(properties_text, ","):
+ key_value = _split_unquoted(field, "=", max_parts=2)
+ if len(key_value) != 2 or not key_value[0]:
+ raise ValueError("Invalid JMX ObjectName property")
+ properties[key_value[0]] = _unquote_object_name_value(key_value[1])
+ return domain, properties
+
+
+def _split_unquoted(value, delimiter, max_parts=None):
+ parts = []
+ start = 0
+ quoted = False
+ escaped = False
+ for index, char in enumerate(value):
+ if escaped:
+ escaped = False
+ continue
+ if char == "\\":
+ escaped = True
+ continue
+ if char == '"':
+ quoted = not quoted
+ continue
+ if char == delimiter and not quoted and (max_parts is None or len(parts) + 1 < max_parts):
+ parts.append(value[start:index])
+ start = index + 1
+ if quoted or escaped:
+ raise ValueError("Invalid quoted JMX ObjectName")
+ parts.append(value[start:])
+ return parts
+
+
+def _unquote_object_name_value(value):
+ if len(value) >= 2 and value[0] == '"' and value[-1] == '"':
+ value = value[1:-1]
+ return re.sub(r"\\(.)", r"\1", value)
+
+
+def _bean_matches(expected, domain, properties):
+ if domain != expected["domain"]:
+ return False
+ for name, value in expected["properties"].items():
+ if name not in properties or (value != "*" and properties[name] != value):
+ return False
+ return True
+
+
+def _rule_labels(label_definitions, properties):
+ labels = {}
+ for name, source in label_definitions.items():
+ if "property" in source:
+ property_name = source["property"]
+ if property_name not in properties:
+ raise TelemetryScrapeError(
+ 'JMX bean is missing label property "{}"'.format(property_name)
+ )
+ labels[name] = properties[property_name]
+ else:
+ labels[name] = source["value"]
+ return labels
+
+
+def _attribute_value(bean, source_name):
+ if source_name in bean:
+ return bean[source_name]
+ value = bean
+ for part in source_name.split("."):
+ if not isinstance(value, dict) or part not in value:
+ return None
+ value = value[part]
+ return value
diff --git a/ambari-agent/src/test/python/ambari_agent/TestASTChecker.py b/ambari-agent/src/test/python/ambari_agent/TestASTChecker.py
index 97358f2cd6..38469856c5 100644
--- a/ambari-agent/src/test/python/ambari_agent/TestASTChecker.py
+++ b/ambari-agent/src/test/python/ambari_agent/TestASTChecker.py
@@ -21,7 +21,6 @@ limitations under the License.
import unittest
from ambari_commons.ast_checker import ASTChecker, BlacklistRule
from ambari_agent.alerts.metric_alert import JmxMetric
-from ambari_agent.alerts.ams_alert import AmsMetric
class TestJmxMetric(unittest.TestCase):
@@ -59,239 +58,6 @@ class TestJmxMetric(unittest.TestCase):
JmxMetric(jmx_info)
-class TestAmsMetric(unittest.TestCase):
- # Safe test cases
- def test_safe_simple_calculation(self):
- metric_info = {
- "metric_list": ["a", "b"],
- "interval": 5,
- "app_id": "test_app",
- "minimum_value": 0,
- "value": "args[0][k] + args[1][k]",
- }
- metric = AmsMetric(metric_info)
- result = metric.calculate_value([{"k": 10}, {"k": 20}])
- self.assertEqual(result, [30])
-
- def test_safe_complex_calculation(self):
- metric_info = {
- "metric_list": ["x", "y", "z"],
- "interval": 5,
- "app_id": "test_app",
- "minimum_value": 0,
- "value": "max(args[0][k], args[1][k]) / min(args[1][k], args[2][k]) * 100.0",
- }
- metric = AmsMetric(metric_info)
- result = metric.calculate_value([{"k": 100}, {"k": 50}, {"k": 25}])
- self.assertEqual(result, [400.0])
-
- def test_safe_string_manipulation(self):
- metric_info = {
- "metric_list": ["str1", "str2"],
- "interval": 5,
- "app_id": "test_app",
- "minimum_value": 0,
- "value": "len(str(args[0][k])) + len(str(args[1][k]))",
- }
- metric = AmsMetric(metric_info)
- result = metric.calculate_value([{"k": "hello"}, {"k": "world"}])
- self.assertEqual(result, [10])
-
- def test_safe_list_comprehension(self):
- metric_info = {
- "metric_list": ["numbers"],
- "interval": 5,
- "app_id": "test_app",
- "minimum_value": 0,
- "value": "sum([x for x in args[0][k] if x > 5])",
- }
- metric = AmsMetric(metric_info)
- result = metric.calculate_value([{"k": [1, 6, 2, 7, 3, 8]}])
- self.assertEqual(result, [21])
-
- def test_safe_dict_manipulation(self):
- metric_info = {
- "metric_list": ["data"],
- "interval": 5,
- "app_id": "test_app",
- "minimum_value": 0,
- "value": "sum(args[0][k].values())",
- }
- metric = AmsMetric(metric_info)
- result = metric.calculate_value([{"k": {"a": 1, "b": 2, "c": 3}}])
- self.assertEqual(result, [6])
-
- def test_safe_conditional_expression(self):
- metric_info = {
- "metric_list": ["x", "y"],
- "interval": 5,
- "app_id": "test_app",
- "minimum_value": 0,
- "value": "args[0][k] if args[0][k] > args[1][k] else args[1][k]",
- }
- metric = AmsMetric(metric_info)
- result = metric.calculate_value([{"k": 10}, {"k": 20}])
- self.assertEqual(result, [20])
-
- def test_safe_boolean_operations(self):
- metric_info = {
- "metric_list": ["a", "b", "c"],
- "interval": 5,
- "app_id": "test_app",
- "minimum_value": 0,
- "value": "int(args[0][k] > 0 and args[1][k] < 10 or args[2][k] == 5)",
- }
- metric = AmsMetric(metric_info)
- result = metric.calculate_value([{"k": 1}, {"k": 5}, {"k": 5}])
- self.assertEqual(result, [1])
-
- def test_safe_compute_mean(self):
- metric_info = {
- "metric_list": ["numbers"],
- "interval": 5,
- "app_id": "test_app",
- "minimum_value": 0,
- "compute": "mean",
- }
- metric = AmsMetric(metric_info)
- result = metric.calculate_compute([1, 2, 3, 4, 5]) # Pass a flat list
- self.assertEqual(result, 3)
-
- def test_safe_compute_standard_deviation(self):
- metric_info = {
- "metric_list": ["numbers"],
- "interval": 5,
- "app_id": "test_app",
- "minimum_value": 0,
- "compute": "sample_standard_deviation",
- }
- metric = AmsMetric(metric_info)
- result = metric.calculate_compute([1, 2, 3, 4, 5]) # Pass a flat list
- self.assertAlmostEqual(result, 1.5811, places=4)
-
- def test_safe_compute_count(self):
- metric_info = {
- "metric_list": ["numbers"],
- "interval": 5,
- "app_id": "test_app",
- "minimum_value": 0,
- "compute": "count",
- }
- metric = AmsMetric(metric_info)
- result = metric.calculate_compute([1, 2, 3, 4, 5]) # Pass a flat list
- self.assertEqual(result, 5)
-
- # Unsafe test cases
- def test_unsafe_import(self):
- metric_info = {
- "metric_list": ["a"],
- "interval": 5,
- "app_id": "test_app",
- "minimum_value": 0,
- "value": "__import__('os').system('echo hacked')",
- }
- with self.assertRaises(Exception):
- AmsMetric(metric_info)
-
- def test_unsafe_eval(self):
- metric_info = {
- "metric_list": ["a"],
- "interval": 5,
- "app_id": "test_app",
- "minimum_value": 0,
- "value": "eval('2 + 2')",
- }
- with self.assertRaises(Exception):
- AmsMetric(metric_info)
-
- def test_unsafe_exec(self):
- metric_info = {
- "metric_list": ["a"],
- "interval": 5,
- "app_id": "test_app",
- "minimum_value": 0,
- "value": "exec('x = 5')",
- }
- with self.assertRaises(Exception):
- AmsMetric(metric_info)
-
- def test_unsafe_open_file(self):
- metric_info = {
- "metric_list": ["a"],
- "interval": 5,
- "app_id": "test_app",
- "minimum_value": 0,
- "value": "open('/etc/passwd', 'r').read()",
- }
- with self.assertRaises(Exception):
- AmsMetric(metric_info)
-
- def test_unsafe_subprocess(self):
- metric_info = {
- "metric_list": ["a"],
- "interval": 5,
- "app_id": "test_app",
- "minimum_value": 0,
- "value": "__import__('subprocess').call(['ls', '-l'])",
- }
- with self.assertRaises(Exception):
- AmsMetric(metric_info)
-
- def test_unsafe_globals(self):
- metric_info = {
- "metric_list": ["a"],
- "interval": 5,
- "app_id": "test_app",
- "minimum_value": 0,
- "value": "globals()['__builtins__']",
- }
- with self.assertRaises(Exception):
- AmsMetric(metric_info)
-
- def test_unsafe_attribute_access(self):
- metric_info = {
- "metric_list": ["a"],
- "interval": 5,
- "app_id": "test_app",
- "minimum_value": 0,
- "value": "args.__class__.__bases__[0].__subclasses__()",
- }
- with self.assertRaises(Exception):
- AmsMetric(metric_info)
-
- def test_unsafe_pickle(self):
- metric_info = {
- "metric_list": ["a"],
- "interval": 5,
- "app_id": "test_app",
- "minimum_value": 0,
- "value": "__import__('pickle').loads(b'cos\\nsystem\\n(S\"echo hacked\"\\ntR.')",
- }
- with self.assertRaises(Exception):
- AmsMetric(metric_info)
-
- def test_unsafe_custom_function(self):
- metric_info = {
- "metric_list": ["a"],
- "interval": 5,
- "app_id": "test_app",
- "minimum_value": 0,
- "value": "lambda: __import__('os').system('echo hacked')",
- }
- with self.assertRaises(Exception):
- AmsMetric(metric_info)
-
- def test_unsafe_builtin_override(self):
- metric_info = {
- "metric_list": ["a"],
- "interval": 5,
- "app_id": "test_app",
- "minimum_value": 0,
- "value": "__builtins__.__dict__['print'] = lambda x: exec(x)",
- }
- with self.assertRaises(Exception):
- AmsMetric(metric_info)
-
class TestBlacklistASTChecker(unittest.TestCase):
def setUp(self):
@@ -337,23 +103,6 @@ class TestBlacklistASTChecker(unittest.TestCase):
"class CustomClass: pass",
"try: 1/0\nexcept ZeroDivisionError: pass",
"safe_function([1, 2, 3])",
- # Safe expressions from ams
- "args[0][k]",
- "args[1][k]",
- "len(args[0])",
- "max(args[0][k], args[1][k])",
- "min(args[0][k], 10)",
- "args[0][k] + args[1][k]",
- "(args[1][k] - args[0][k]) / args[0][k] * 100 if args[0][k] != 0 else 0",
- "args[0][k] if args[0][k] > args[1][k] else args[1][k]",
- "(args[0][k] + args[1][k] + args[2][k]) / 3",
- "len(str(args[0][k]))",
- "int(args[0][k]) if args[0][k] is not None else 0",
- "args[0][k] * 2 if args[1][k] > 100 else args[0][k] / 2 if args[1][k] < 50 else args[0][k]",
- "sample_standard_deviation_percentage(args)",
- "sample_standard_deviation(args)",
- "mean(args)",
- "count(args)",
]
for expr in safe_expressions:
diff --git a/ambari-agent/src/test/python/ambari_agent/TestActionQueue.py b/ambari-agent/src/test/python/ambari_agent/TestActionQueue.py
index 95d804e750..d97331151c 100644
--- a/ambari-agent/src/test/python/ambari_agent/TestActionQueue.py
+++ b/ambari-agent/src/test/python/ambari_agent/TestActionQueue.py
@@ -1670,15 +1670,15 @@ class TestActionQueue(TestCase):
"'some.password': '[PROTECTED]', 'other.password': '[PROTECTED]',",
)
self.assertEqual(
- hide_passwords("u'metrics_grafana_password': u'mypassword123!'"),
- "u'metrics_grafana_password': u'[PROTECTED]'",
+ hide_passwords("u'service_admin_password': u'mypassword123!'"),
+ "u'service_admin_password': u'[PROTECTED]'",
)
self.assertEqual(
hide_passwords(
- "u'metrics_grafana_username': u'admin', u'metrics_grafana_password': u'mypassword123!', some text, u'clientssl.keystore.password': u'myKeyFilePassword', another text, "
+ "u'service_admin_username': u'admin', u'service_admin_password': u'mypassword123!', some text, u'clientssl.keystore.password': u'myKeyFilePassword', another text, "
),
- "u'metrics_grafana_username': u'admin', u'metrics_grafana_password': u'[PROTECTED]', some text, u'clientssl.keystore.password': u'[PROTECTED]', another text, ",
+ "u'service_admin_username': u'admin', u'service_admin_password': u'[PROTECTED]', some text, u'clientssl.keystore.password': u'[PROTECTED]', another text, ",
)
self.assertEqual(
hide_passwords(
diff --git a/ambari-agent/src/test/python/ambari_agent/TestActualConfigHandler.py b/ambari-agent/src/test/python/ambari_agent/TestActualConfigHandler.py
index ac3adb7d05..2e465297d2 100644
--- a/ambari-agent/src/test/python/ambari_agent/TestActualConfigHandler.py
+++ b/ambari-agent/src/test/python/ambari_agent/TestActualConfigHandler.py
@@ -38,7 +38,6 @@ class TestActualConfigHandler(TestCase):
LiveStatus.SERVICES = [
"HDFS",
"MAPREDUCE",
- "GANGLIA",
"HBASE",
"ZOOKEEPER",
"OOZIE",
@@ -75,8 +74,6 @@ class TestActualConfigHandler(TestCase):
{"serviceName": "HDFS", "componentName": "ZKFC"},
{"serviceName": "MAPREDUCE", "componentName": "JOBTRACKER"},
{"serviceName": "MAPREDUCE", "componentName": "TASKTRACKER"},
- {"serviceName": "GANGLIA", "componentName": "GANGLIA_SERVER"},
- {"serviceName": "GANGLIA", "componentName": "GANGLIA_MONITOR"},
{"serviceName": "HBASE", "componentName": "HBASE_MASTER"},
{"serviceName": "HBASE", "componentName": "HBASE_REGIONSERVER"},
{"serviceName": "FLUME", "componentName": "FLUME_SERVER"},
diff --git a/ambari-agent/src/test/python/ambari_agent/TestAlertSchedulerHandler.py b/ambari-agent/src/test/python/ambari_agent/TestAlertSchedulerHandler.py
index 44023b4088..2638591dbd 100644
--- a/ambari-agent/src/test/python/ambari_agent/TestAlertSchedulerHandler.py
+++ b/ambari-agent/src/test/python/ambari_agent/TestAlertSchedulerHandler.py
@@ -24,7 +24,6 @@ import tempfile
from ambari_agent.AlertSchedulerHandler import AlertSchedulerHandler
from ambari_agent.alerts.metric_alert import MetricAlert
-from ambari_agent.alerts.ams_alert import AmsAlert
from ambari_agent.alerts.port_alert import PortAlert
from ambari_agent.alerts.web_alert import WebAlert
@@ -240,24 +239,6 @@ class TestAlertSchedulerHandler(TestCase):
self.assertEqual(callable_result.alert_meta, json_definition)
self.assertEqual(callable_result.alert_source_meta, json_definition["source"])
- def test_json_to_callable_ams(self):
- initializer_module = InitializerModule()
- initializer_module.config.get_server_ssl_context = MagicMock()
- initializer_module.init()
-
- scheduler = AlertSchedulerHandler(initializer_module)
- # (TEST_PATH, TEST_PATH, TEST_PATH, TEST_PATH, TEST_PATH, None, self.config, None)
- json_definition = {"clusterId": "0", "source": {"type": "AMS"}}
-
- callable_result = scheduler._AlertSchedulerHandler__json_to_callable(
- "cluster", "host", "host", copy.deepcopy(json_definition)
- )
-
- self.assertTrue(callable_result is not None)
- self.assertTrue(isinstance(callable_result, AmsAlert))
- self.assertEqual(callable_result.alert_meta, json_definition)
- self.assertEqual(callable_result.alert_source_meta, json_definition["source"])
-
def test_json_to_callable_port(self):
json_definition = {"clusterId": "0", "source": {"type": "PORT"}}
diff --git a/ambari-agent/src/test/python/ambari_agent/TestAlerts.py b/ambari-agent/src/test/python/ambari_agent/TestAlerts.py
index 7fe3f23eb4..2e646f00de 100644
--- a/ambari-agent/src/test/python/ambari_agent/TestAlerts.py
+++ b/ambari-agent/src/test/python/ambari_agent/TestAlerts.py
@@ -24,7 +24,6 @@ import sys
import urllib.request, urllib.error, urllib.parse
import tempfile
import random
-from alerts.ams_alert import AmsAlert
from ambari_agent.AlertSchedulerHandler import AlertSchedulerHandler
from ambari_agent.RecoveryManager import RecoveryManager
@@ -97,7 +96,7 @@ class TestAlerts(TestCase):
definition_json = self._get_recovery_alert_definition()
is_stale_mock.return_value = False
rm_get_actions_mock.return_value = {
- "METRICS_COLLECTOR": {
+ "DATANODE": {
"count": 0,
"lastAttempt": 1447860184,
"warnedLastReset": False,
@@ -126,7 +125,7 @@ class TestAlerts(TestCase):
# WARN - "count": 1
rm_get_actions_mock.return_value = {
- "METRICS_COLLECTOR": {
+ "DATANODE": {
"count": 1,
"lastAttempt": 1447860184,
"warnedLastReset": False,
@@ -143,7 +142,7 @@ class TestAlerts(TestCase):
# CRIT - "count": 5
rm_get_actions_mock.return_value = {
- "METRICS_COLLECTOR": {
+ "DATANODE": {
"count": 5,
"lastAttempt": 1447860184,
"warnedLastReset": False,
@@ -169,7 +168,7 @@ class TestAlerts(TestCase):
# CRIT, after recovery manager window expired,
# but max_lifetime_count reached, warnedThresholdReached == True
rm_get_actions_mock.return_value = {
- "METRICS_COLLECTOR": {
+ "DATANODE": {
"count": 5,
"lastAttempt": 1447860184,
"warnedLastReset": False,
@@ -430,60 +429,6 @@ class TestAlerts(TestCase):
self.assertEqual("OK", alerts[0]["state"])
self.assertEqual("(Unit Tests) OK: 1 25 None", alerts[0]["text"])
- @patch.object(ConfigurationBuilder, "get_configuration")
- @patch.object(AmsAlert, "_load_metric")
- def test_ams_alert(self, ma_load_metric_mock, get_configuration_mock):
- definition_json = self._get_ams_alert_definition()
- configuration = {
- "ams-site": {"timeline.metrics.service.webapp.address": "0.0.0.0:6188"}
- }
-
- collector = AlertCollector()
- cluster_configuration = self.__get_cluster_configuration()
- self.__update_cluster_configuration(cluster_configuration, configuration)
-
- initializer_module = self.create_initializer_module()
- get_configuration_mock.return_value = {"configurations": configuration}
-
- alert = AmsAlert(definition_json, definition_json["source"], self.config)
- alert.set_helpers(
- collector, cluster_configuration, initializer_module.configuration_builder
- )
- alert.set_cluster("c1", "0", "c6401.ambari.apache.org")
-
- # trip an OK
- ma_load_metric_mock.return_value = ([{1: 100, 2: 100, 3: 200, 4: 200}], None)
-
- alert.collect()
- alerts = collector.alerts()
- self.assertEqual(0, len(collector.alerts()))
- self.assertEqual("OK", alerts[0]["state"])
- self.assertEqual(
- "(Unit Tests) OK: the mean used heap size is 150.0 MB.", alerts[0]["text"]
- )
-
- # trip a warning
- ma_load_metric_mock.return_value = ([{1: 800, 2: 800, 3: 900, 4: 900}], None)
-
- alert.collect()
- alerts = collector.alerts()
- self.assertEqual(0, len(collector.alerts()))
- self.assertEqual("WARNING", alerts[0]["state"])
- self.assertEqual(
- "(Unit Tests) Warning: the mean used heap size is 850.0 MB.", alerts[0]["text"]
- )
-
- # trip a critical now
- ma_load_metric_mock.return_value = ([{1: 1000, 2: 1000, 3: 2000, 4: 2000}], None)
-
- alert.collect()
- alerts = collector.alerts()
- self.assertEqual(0, len(collector.alerts()))
- self.assertEqual("CRITICAL", alerts[0]["state"])
- self.assertEqual(
- "(Unit Tests) Critical: the mean used heap size is 1500.0 MB.", alerts[0]["text"]
- )
-
@patch.object(ConfigurationBuilder, "get_configuration")
@patch.object(MetricAlert, "_load_jmx")
def test_alert_uri_structure(self, ma_load_jmx_mock, get_configuration_mock):
@@ -1685,10 +1630,10 @@ class TestAlerts(TestCase):
def _get_recovery_alert_definition(self):
return {
"definitionId": 1,
- "componentName": "METRICS_COLLECTOR",
- "name": "ams_metrics_collector_autostart",
- "label": "Metrics Collector Recovery",
- "description": "This alert is triggered if the Metrics Collector has been auto-started for number of times equal to threshold.",
+ "componentName": "DATANODE",
+ "name": "datanode_autostart",
+ "label": "DataNode Recovery",
+ "description": "This alert is triggered if the DataNode has been auto-started for number of times equal to threshold.",
"interval": 1,
"scope": "HOST",
"enabled": True,
@@ -1696,14 +1641,14 @@ class TestAlerts(TestCase):
"type": "RECOVERY",
"reporting": {
"ok": {
- "text": "Metrics Collector has not been auto-started and is running normally{0}."
+ "text": "DataNode has not been auto-started and is running normally{0}."
},
"warning": {
- "text": "Metrics Collector has been auto-started {1} times{0}.",
+ "text": "DataNode has been auto-started {1} times{0}.",
"count": 1,
},
"critical": {
- "text": "Metrics Collector has been auto-started {1} times{0}.",
+ "text": "DataNode has been auto-started {1} times{0}.",
"count": 5,
},
},
@@ -1743,48 +1688,6 @@ class TestAlerts(TestCase):
},
}
- def _get_ams_alert_definition(self):
- return {
- "ignore_host": False,
- "name": "namenode_mean_heapsize_used",
- "componentName": "NAMENODE",
- "interval": 1,
- "clusterId": 2,
- "uuid": "8a857295-ad11-4985-896e-d866dc27b963",
- "label": "NameNode Mean Used Heap Size (Hourly)",
- "definitionId": 28,
- "source": {
- "ams": {
- "compute": "mean",
- "interval": 30,
- "app_id": "NAMENODE",
- "value": "{0}",
- "metric_list": ["jvm.JvmMetrics.MemHeapUsedM"],
- "minimum_value": -1,
- },
- "reporting": {
- "units": "#",
- "warning": {
- "text": "(Unit Tests) Warning: the mean used heap size is {0} MB.",
- "value": 768,
- },
- "ok": {"text": "(Unit Tests) OK: the mean used heap size is {0} MB."},
- "critical": {
- "text": "(Unit Tests) Critical: the mean used heap size is {0} MB.",
- "value": 1024,
- },
- },
- "type": "AMS",
- "uri": {
- "http": "{{ams-site/timeline.metrics.service.webapp.address}}",
- "https_property_value": "HTTPS_ONLY",
- "https_property": "{{ams-site/timeline.metrics.service.http.policy}}",
- "https": "{{ams-site/timeline.metrics.service.webapp.address}}",
- "connection_timeout": 5.0,
- },
- },
- }
-
def _get_metric_alert_definition_with_float_division(self):
return {
"definitionId": 1,
diff --git a/ambari-agent/src/test/python/ambari_agent/TestAmsAlert.py b/ambari-agent/src/test/python/ambari_agent/TestAmsAlert.py
deleted file mode 100644
index 6fa2f9438a..0000000000
--- a/ambari-agent/src/test/python/ambari_agent/TestAmsAlert.py
+++ /dev/null
@@ -1,265 +0,0 @@
-#!/usr/bin/env python3
-
-"""
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-"""
-
-from unittest import TestCase
-from alerts.ams_alert import AmsAlert
-from unittest.mock import Mock, MagicMock, patch
-from ambari_agent.AmbariConfig import AmbariConfig
-
-
-class TestAmsAlert(TestCase):
- def setUp(self):
- self.config = AmbariConfig()
- self.ssl_context = MagicMock()
- self.config.get_server_ssl_context = MagicMock(return_value=self.ssl_context)
- self.configuration_builder = MagicMock()
- self.configuration_builder.get_configuration.return_value = {
- "configurations": {"cluster-env": {"security_enabled": "false"}}
- }
-
- @patch("alerts.metric_alert.build_url_opener")
- def test_collect_ok_low_values(self, build_url_opener):
- alert_meta = {
- "definitionId": 1,
- "name": "alert1",
- "label": "label1",
- "serviceName": "service1",
- "componentName": "component1",
- "uuid": "123",
- "enabled": "true",
- }
- alert_source_meta = {
- "ams": {
- "metric_list": ["metric1"],
- "app_id": "APP_ID",
- "interval": 60,
- "minimum_value": -1,
- "compute": "mean",
- "value": "{0}",
- },
- "uri": {
- "http": "192.168.0.10:8080",
- "https_property": "{{ams-site/timeline.metrics.service.http.policy}}",
- "https_property_value": "HTTPS_ONLY",
- },
- "reporting": {
- "ok": {"text": "OK: {0}"},
- "warning": {"text": "Warn: {0}", "value": 3},
- "critical": {"text": "Crit: {0}", "value": 5},
- },
- }
- cluster = "c1"
- cluster_id = "0"
- host = "host1"
- expected_text = "OK: 2.0"
-
- def collector_side_effect(clus, data):
- self.assertEqual(data["name"], alert_meta["name"])
- self.assertEqual(data["text"], expected_text)
- self.assertEqual(data["clusterId"], cluster_id)
- self.assertEqual(data["definitionId"], alert_meta["definitionId"])
- self.assertEqual(clus, cluster)
-
- response = MagicMock()
- response.getcode.return_value = 200
- build_url_opener.return_value.open.return_value = response
- response.read.return_value = '{"metrics":[{"metricname":"metric1","metrics":{"1459966360838":1,"1459966370838":3}}]}'
-
- mock_collector = MagicMock()
- mock_collector.put = Mock(side_effect=collector_side_effect)
-
- alert = AmsAlert(alert_meta, alert_source_meta, self.config)
- alert.set_helpers(mock_collector, MagicMock(), self.configuration_builder)
- alert.set_cluster(cluster, cluster_id, host)
-
- alert.collect()
- response.close.assert_called_once_with()
-
- @patch("alerts.metric_alert.build_url_opener")
- def test_collect_warn(self, build_url_opener):
- alert_meta = {
- "definitionId": 1,
- "name": "alert1",
- "label": "label1",
- "serviceName": "service1",
- "componentName": "component1",
- "uuid": "123",
- "enabled": "true",
- }
- alert_source_meta = {
- "ams": {
- "metric_list": ["metric1"],
- "app_id": "APP_ID",
- "interval": 60,
- "minimum_value": -1,
- "compute": "mean",
- "value": "{0}",
- },
- "uri": {
- "http": "192.168.0.10:8080",
- "https_property": "{{ams-site/timeline.metrics.service.http.policy}}",
- "https_property_value": "HTTPS_ONLY",
- },
- "reporting": {
- "ok": {"text": "OK: {0}"},
- "warning": {"text": "Warn: {0}", "value": 3},
- "critical": {"text": "Crit: {0}", "value": 5},
- },
- }
- cluster = "c1"
- host = "host1"
- cluster_id = "0"
- expected_text = "Warn: 4.0"
-
- def collector_side_effect(clus, data):
- self.assertEqual(data["name"], alert_meta["name"])
- self.assertEqual(data["text"], expected_text)
- self.assertEqual(data["clusterId"], cluster_id)
- self.assertEqual(clus, cluster)
-
- response = MagicMock()
- response.getcode.return_value = 200
- build_url_opener.return_value.open.return_value = response
- response.read.return_value = '{"metrics":[{"metricname":"metric1","metrics":{"1459966360838":3,"1459966370838":5}}]}'
-
- mock_collector = MagicMock()
- mock_collector.put = Mock(side_effect=collector_side_effect)
-
- alert = AmsAlert(alert_meta, alert_source_meta, self.config)
- alert.set_helpers(
- mock_collector, MagicMock(), MagicMock()
- ) # {'foo-site/bar': 12, 'foo-site/baz': 'asd'})
- alert.set_cluster(cluster, cluster_id, host)
-
- alert.collect()
-
- @patch("alerts.metric_alert.build_url_opener")
- def test_collect_ok_with_cluster_id(self, build_url_opener):
- alert_meta = {
- "definitionId": 1,
- "name": "alert1",
- "label": "label1",
- "serviceName": "service1",
- "componentName": "component1",
- "uuid": "123",
- "enabled": "true",
- }
- alert_source_meta = {
- "ams": {
- "metric_list": ["metric1"],
- "app_id": "APP_ID",
- "interval": 60,
- "minimum_value": -1,
- "compute": "mean",
- "value": "{0}",
- },
- "uri": {
- "http": "192.168.0.10:8080",
- "https_property": "{{ams-site/timeline.metrics.service.http.policy}}",
- "https_property_value": "HTTPS_ONLY",
- },
- "reporting": {
- "ok": {"text": "OK: {0}"},
- "warning": {"text": "Warn: {0}", "value": 3},
- "critical": {"text": "Crit: {0}", "value": 5},
- },
- }
- cluster = "c1"
- host = "host1"
- cluster_id = "0"
- expected_text = "Crit: 10.0"
-
- def collector_side_effect(clus, data):
- self.assertEqual(data["name"], alert_meta["name"])
- self.assertEqual(data["text"], expected_text)
- self.assertEqual(data["clusterId"], cluster_id)
- self.assertEqual(clus, cluster)
-
- response = MagicMock()
- response.getcode.return_value = 200
- build_url_opener.return_value.open.return_value = response
- response.read.return_value = '{"metrics":[{"metricname":"metric1","metrics":{"1459966360838":10,"1459966370838":10}}]}'
-
- mock_collector = MagicMock()
- mock_collector.put = Mock(side_effect=collector_side_effect)
-
- alert = AmsAlert(alert_meta, alert_source_meta, self.config)
- alert.set_helpers(
- mock_collector, MagicMock(), MagicMock()
- ) # {'foo-site/bar': 12, 'foo-site/baz': 'asd'})
- alert.set_cluster(cluster, cluster_id, host)
-
- alert.collect()
-
- @patch("alerts.metric_alert.build_url_opener")
- def test_load_metric_uses_https_and_request_scoped_proxy_policy(
- self, build_url_opener
- ):
- self.config.set("network", "use_system_proxy_settings", "false")
- response = MagicMock()
- response.getcode.return_value = 200
- response.read.return_value = b'{"metrics":[]}'
- build_url_opener.return_value.open.return_value = response
- alert = AmsAlert(
- {"name": "alert1"},
- {
- "ams": {
- "metric_list": ["metric1"],
- "app_id": "APP_ID",
- "interval": 1,
- "minimum_value": -1,
- }
- },
- self.config,
- )
- alert.set_cluster("c1", "0", "host1")
-
- metrics, status = alert._load_metric(True, "ams.example", 6188, alert.metric_info)
-
- self.assertEqual([], metrics)
- self.assertEqual(200, status)
- build_url_opener.assert_called_once()
- self.assertTrue(build_url_opener.call_args.args[0])
- self.assertIs(self.ssl_context, build_url_opener.call_args.args[1])
- request_url = build_url_opener.return_value.open.call_args.args[0]
- self.assertTrue(request_url.startswith("https://ams.example:6188/"))
- response.close.assert_called_once_with()
-
- @patch("alerts.metric_alert.build_url_opener")
- def test_load_metric_returns_unknown_on_network_failure(self, build_url_opener):
- build_url_opener.return_value.open.side_effect = OSError("connection failed")
- alert = AmsAlert(
- {"name": "alert1"},
- {
- "ams": {
- "metric_list": ["metric1"],
- "app_id": "APP_ID",
- "interval": 1,
- "minimum_value": -1,
- }
- },
- self.config,
- )
- alert.set_cluster("c1", "0", "host1")
-
- metrics, status = alert._load_metric(False, "ams.example", 6188, alert.metric_info)
-
- self.assertIsNone(metrics)
- self.assertIsNone(status)
diff --git a/ambari-agent/src/test/python/ambari_agent/TestHostCleanup.py b/ambari-agent/src/test/python/ambari_agent/TestHostCleanup.py
index a2bed18f0e..8bd59cc50a 100644
--- a/ambari-agent/src/test/python/ambari_agent/TestHostCleanup.py
+++ b/ambari-agent/src/test/python/ambari_agent/TestHostCleanup.py
@@ -52,7 +52,7 @@ proc_list = 323,434
proc_owner_list = abc,efg
[users]
-usr_list = rrdcached,ambari-qa,hive,oozie,hbase,hcat,mysql,mapred,hdfs,zookeeper,sqoop
+usr_list = ambari-qa,hive,oozie,hbase,hcat,mysql,mapred,hdfs,zookeeper,sqoop
[repositories]
repo_list = HDP-1.3.0,HDP-epel
@@ -65,7 +65,7 @@ symlink_list = hcatalog-conf,hadoop-default,hadoop-log,oozie-conf
target_list = /etc/hcatalog/conf.dist,/usr/share/man/man1/hadoop.1.gz,/etc/oozie/conf.dist,/usr/lib/hadoop
[packages]
-pkg_list = sqoop.noarch,hadoop-libhdfs.x86_64,rrdtool.x86_64,ganglia-gmond.x86_64
+pkg_list = sqoop.noarch,hadoop-libhdfs.x86_64
[metadata]
created = 2013-07-02 20:39:22.162757"""
diff --git a/ambari-agent/src/test/python/ambari_agent/TestPrometheusMetrics.py b/ambari-agent/src/test/python/ambari_agent/TestPrometheusMetrics.py
new file mode 100644
index 0000000000..bbc05d2517
--- /dev/null
+++ b/ambari-agent/src/test/python/ambari_agent/TestPrometheusMetrics.py
@@ -0,0 +1,530 @@
+#!/usr/bin/env python3
+
+"""
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+"""
+
+from collections import namedtuple
+from urllib.error import HTTPError
+from urllib.request import urlopen
+import os
+import tempfile
+import threading
+import unittest
+from unittest.mock import patch
+
+from ambari_agent.AmbariConfig import AmbariConfig
+from ambari_agent.metrics.core import CollectorRegistry, MetricFamily, render_metrics
+from ambari_agent.metrics.linux import (
+ CpuCollector,
+ DiskIoCollector,
+ FilesystemCollector,
+ MemoryCollector,
+ NetworkCollector,
+ ProcessCollector,
+ SocketCollector,
+ SystemCollector,
+ TimexCollector,
+ _decode_mount_field,
+ default_collectors,
+)
+from ambari_agent.metrics.server import PrometheusMetricsServer
+
+
+class StaticCollector:
+ name = "static"
+
+ def collect(self):
+ family = MetricFamily("test_metric", "A test metric.", "gauge")
+ family.add_sample(7, {"source": "unit"})
+ return [family]
+
+
+class FailingCollector:
+ name = "failing"
+
+ def collect(self):
+ raise RuntimeError("collection failed")
+
+
+class TestPrometheusMetricCore(unittest.TestCase):
+ def test_render_metrics_escapes_help_and_labels(self):
+ family = MetricFamily("test_metric", "line one\nline two\\", "gauge")
+ family.add_sample(1.5, {"z": 'quoted"', "a": "line\nslash\\"})
+
+ rendered = render_metrics([family]).decode("utf-8")
+
+ self.assertEqual(
+ "# HELP test_metric line one\\nline two\\\\\n"
+ "# TYPE test_metric gauge\n"
+ 'test_metric{a="line\\nslash\\\\",z="quoted\\""} 1.5\n',
+ rendered,
+ )
+
+ def test_metric_and_label_names_are_validated(self):
+ with self.assertRaises(ValueError):
+ MetricFamily("bad-name", "Bad metric.", "gauge")
+
+ family = MetricFamily("valid_metric", "Valid metric.", "gauge")
+ with self.assertRaises(ValueError):
+ family.add_sample(1, {"bad-label": "value"})
+
+ def test_render_metrics_merges_compatible_families(self):
+ first = MetricFamily("shared_metric", "Shared metric.", "gauge")
+ first.add_sample(1, {"component": "first"})
+ second = MetricFamily("shared_metric", "Shared metric.", "gauge")
+ second.add_sample(2, {"component": "second"})
+
+ rendered = render_metrics([first, second]).decode("utf-8")
+
+ self.assertEqual(1, rendered.count("# HELP shared_metric"))
+ self.assertIn('shared_metric{component="first"} 1', rendered)
+ self.assertIn('shared_metric{component="second"} 2', rendered)
+
+ def test_registry_isolates_collector_failures(self):
+ clock_values = iter((1.0, 1.25, 2.0, 2.5))
+ registry = CollectorRegistry(
+ [StaticCollector(), FailingCollector()], clock=lambda: next(clock_values)
+ )
+
+ with self.assertLogs("ambari_agent.metrics.core", level="ERROR"):
+ rendered = registry.render().decode("utf-8")
+
+ self.assertIn('test_metric{source="unit"} 7', rendered)
+ self.assertIn('ambari_agent_metrics_collector_up{collector="static"} 1', rendered)
+ self.assertIn('ambari_agent_metrics_collector_up{collector="failing"} 0', rendered)
+ self.assertIn(
+ 'ambari_agent_metrics_collector_duration_seconds{collector="failing"} 0.5',
+ rendered,
+ )
+
+
+class TestLinuxCollectors(unittest.TestCase):
+ def setUp(self):
+ self.temp_dir = tempfile.TemporaryDirectory()
+ self.proc_root = self.temp_dir.name
+ self._write(
+ "stat",
+ "cpu 100 20 30 400 50 6 7 8 10 2\n"
+ "cpu0 60 10 20 200 30 4 5 6 5 1\n"
+ "intr 100 10 20\n"
+ "ctxt 4321\n"
+ "btime 1700000000\n"
+ "processes 1234\n"
+ "procs_running 3\n"
+ "procs_blocked 2\n",
+ )
+ self._write(
+ "meminfo",
+ "MemTotal: 1000 kB\n"
+ "MemFree: 100 kB\n"
+ "MemAvailable: 600 kB\n"
+ "Buffers: 20 kB\n"
+ "Cached: 300 kB\n"
+ "Active: 400 kB\n"
+ "Inactive: 200 kB\n"
+ "Slab: 50 kB\n"
+ "SwapTotal: 500 kB\n"
+ "SwapFree: 450 kB\n"
+ "Active(anon): 250 kB\n"
+ "Active(file): 150 kB\n"
+ "Inactive(anon): 125 kB\n"
+ "Inactive(file): 75 kB\n"
+ "HugePages_Total: 8\n"
+ "HugePages_Free: 3\n"
+ "Hugepagesize: 2048 kB\n"
+ "Dirty: 12 kB\n"
+ "DirectMap2M: 512 kB\n",
+ )
+ self._write("loadavg", "1.25 0.75 0.50 2/100 123\n")
+ self._write("uptime", "3600.50 7200.00\n")
+ self._write("sys/fs/file-nr", "256 0 9223372036854775807\n")
+ self._write("sys/fs/file-max", "4096\n")
+ self._write("sys/kernel/random/entropy_avail", "192\n")
+ self._write("sys/net/netfilter/nf_conntrack_count", "23\n")
+ self._write("sys/net/netfilter/nf_conntrack_max", "1024\n")
+ self._write("vmstat", "pgfault 500\noom_kill 4\n")
+ self._write(
+ "net/tcp",
+ " sl local_address rem_address st tx_queue rx_queue\n"
+ " 0: 0100007F:0016 00000000:0000 0A 00000000:00000000\n"
+ " 1: 0100007F:1234 0100007F:5678 01 00000000:00000000\n",
+ )
+ self._write(
+ "net/tcp6",
+ " sl local_address rem_address st tx_queue rx_queue\n"
+ " 0: 00000000000000000000000000000000:0016 "
+ "00000000000000000000000000000000:0000 0A 00000000:00000000\n"
+ " 1: 00000000000000000000000000000001:1234 "
+ "00000000000000000000000000000001:5678 06 00000000:00000000\n",
+ )
+ self._write(
+ "diskstats",
+ "8 0 sda 10 1 20 300 40 2 50 600 3 700 800\n",
+ )
+ self._write(
+ "net/dev",
+ "Inter-| Receive | Transmit\n"
+ " face |bytes packets errs drop fifo frame compressed multicast|"
+ "bytes packets errs drop fifo colls carrier compressed\n"
+ " lo: 1 2 3 4 0 0 0 0 5 6 7 8 0 0 0 0\n"
+ "eth0: 100 20 3 4 0 0 0 0 200 30 5 6 0 0 0 0\n",
+ )
+ self._write(
+ "net/sockstat",
+ "sockets: used 42\n"
+ "TCP: inuse 5 orphan 2 tw 7 alloc 11 mem 3\n",
+ )
+ self._write("net/sockstat6", "TCP6: inuse 4\n")
+ self._write(
+ "self/mounts",
+ "proc /proc proc rw 0 0\n"
+ "/dev/sda1 /data\\040disk ext4 rw,relatime 0 0\n"
+ "/dev/sdb1 /missing ext4 rw,relatime 0 0\n",
+ )
+ self._write(
+ "1/stat", "1 (init process) S 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 5\n"
+ )
+ self._write(
+ "2/stat", "2 (worker) R 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3\n"
+ )
+ self._write("3/stat", "3 (exited) Z 0 0 0 0\n")
+
+ def tearDown(self):
+ self.temp_dir.cleanup()
+
+ def _write(self, relative_path, content):
+ path = os.path.join(self.proc_root, relative_path)
+ directory = os.path.dirname(path)
+ if not os.path.isdir(directory):
+ os.makedirs(directory)
+ with open(path, "w", encoding="utf-8") as stream:
+ stream.write(content)
+
+ def _families_by_name(self, collector):
+ return {family.name: family for family in collector.collect()}
+
+ def test_cpu_collector_exports_cumulative_seconds(self):
+ with patch("ambari_agent.metrics.linux.os.sysconf", return_value=100):
+ family = CpuCollector(self.proc_root).collect()[0]
+
+ samples = {
+ (sample.labels["cpu"], sample.labels["mode"]): sample.value
+ for sample in family.samples
+ }
+ self.assertEqual(0.9, samples[("total", "user")])
+ self.assertEqual(0.18, samples[("total", "nice")])
+ self.assertEqual(4.0, samples[("total", "idle")])
+ self.assertEqual(0.55, samples[("0", "user")])
+
+ def test_memory_collector_converts_kibibytes_to_bytes(self):
+ families = self._families_by_name(MemoryCollector(self.proc_root))
+
+ self.assertEqual(
+ 1000 * 1024, families["ambari_agent_memory_total_bytes"].samples[0].value
+ )
+ self.assertEqual(
+ 600 * 1024,
+ families["ambari_agent_memory_available_bytes"].samples[0].value,
+ )
+ self.assertEqual(
+ 250 * 1024,
+ families["ambari_agent_memory_active_anon_bytes"].samples[0].value,
+ )
+ self.assertEqual(
+ 8, families["ambari_agent_memory_hugepages_total"].samples[0].value
+ )
+ self.assertEqual(
+ 2048 * 1024,
+ families["ambari_agent_memory_hugepages_size_bytes"].samples[0].value,
+ )
+
+ def test_system_collector_exports_load_process_and_host_metrics(self):
+ with patch("ambari_agent.metrics.linux.os.cpu_count", return_value=8), patch(
+ "ambari_agent.metrics.linux.socket.getfqdn", return_value="host.example.com"
+ ):
+ families = self._families_by_name(SystemCollector(self.proc_root))
+
+ self.assertEqual(1.25, families["ambari_agent_system_load1"].samples[0].value)
+ self.assertEqual(
+ 1234,
+ families["ambari_agent_system_processes_forked_total"].samples[0].value,
+ )
+ self.assertEqual(
+ 4321,
+ families["ambari_agent_system_context_switches_total"].samples[0].value,
+ )
+ self.assertEqual(
+ 100, families["ambari_agent_system_interrupts_total"].samples[0].value
+ )
+ for metric_name in (
+ "ambari_agent_system_context_switches_total",
+ "ambari_agent_system_interrupts_total",
+ "ambari_agent_file_descriptors_allocated",
+ "ambari_agent_file_descriptors_maximum",
+ "ambari_agent_entropy_available_bits",
+ "ambari_agent_oom_kills_total",
+ "ambari_agent_conntrack_entries",
+ "ambari_agent_conntrack_entries_limit",
+ ):
+ self.assertEqual(1, len(families[metric_name].samples))
+ self.assertEqual(
+ 256, families["ambari_agent_file_descriptors_allocated"].samples[0].value
+ )
+ self.assertEqual(
+ 4096, families["ambari_agent_file_descriptors_maximum"].samples[0].value
+ )
+ self.assertEqual(
+ 192, families["ambari_agent_entropy_available_bits"].samples[0].value
+ )
+ self.assertEqual(4, families["ambari_agent_oom_kills_total"].samples[0].value)
+ self.assertEqual(23, families["ambari_agent_conntrack_entries"].samples[0].value)
+ self.assertEqual(
+ 1024, families["ambari_agent_conntrack_entries_limit"].samples[0].value
+ )
+ tcp_samples = {
+ sample.labels["state"]: sample.value
+ for sample in families["ambari_agent_tcp_connections"].samples
+ }
+ self.assertEqual(1, tcp_samples["established"])
+ self.assertEqual(2, tcp_samples["listen"])
+ self.assertEqual(1, tcp_samples["time_wait"])
+ self.assertEqual(0, tcp_samples["close_wait"])
+ self.assertEqual(
+ {"hostname": "host.example.com"},
+ families["ambari_agent_host_info"].samples[0].labels,
+ )
+
+ def test_system_collector_ignores_missing_optional_kernel_files(self):
+ for relative_path in (
+ "sys/fs/file-nr",
+ "sys/fs/file-max",
+ "sys/kernel/random/entropy_avail",
+ "sys/net/netfilter/nf_conntrack_count",
+ "sys/net/netfilter/nf_conntrack_max",
+ "vmstat",
+ "net/tcp",
+ "net/tcp6",
+ ):
+ os.remove(os.path.join(self.proc_root, relative_path))
+
+ families = self._families_by_name(SystemCollector(self.proc_root))
+
+ self.assertEqual(1.25, families["ambari_agent_system_load1"].samples[0].value)
+ self.assertNotIn("ambari_agent_file_descriptors_allocated", families)
+ self.assertNotIn("ambari_agent_tcp_connections", families)
+
+ def test_filesystem_collector_skips_pseudo_filesystems(self):
+ StatVfs = namedtuple(
+ "StatVfs", "f_blocks f_frsize f_bfree f_bavail f_files f_ffree"
+ )
+
+ def statvfs(path):
+ if path == "/data disk":
+ return StatVfs(100, 4096, 40, 30, 50, 20)
+ raise OSError("stale mount")
+
+ collector = FilesystemCollector(
+ self.proc_root,
+ statvfs=statvfs,
+ )
+
+ with self.assertLogs("ambari_agent.metrics.linux", level="WARNING"):
+ families = self._families_by_name(collector)
+
+ sample = families["ambari_agent_filesystem_size_bytes"].samples[0]
+ self.assertEqual(409600, sample.value)
+ self.assertEqual("/data disk", sample.labels["mountpoint"])
+ self.assertEqual(1, len(families["ambari_agent_filesystem_size_bytes"].samples))
+ errors = {
+ sample.labels["mountpoint"]: sample.value
+ for sample in families["ambari_agent_filesystem_device_error"].samples
+ }
+ self.assertEqual(0, errors["/data disk"])
+ self.assertEqual(1, errors["/missing"])
+ self.assertEqual(
+ 0, families["ambari_agent_filesystem_readonly"].samples[0].value
+ )
+
+ def test_disk_io_collector_exports_kernel_counters(self):
+ families = self._families_by_name(DiskIoCollector(self.proc_root))
+
+ self.assertEqual(
+ 10, families["ambari_agent_disk_reads_completed_total"].samples[0].value
+ )
+ self.assertEqual(
+ 20 * 512, families["ambari_agent_disk_read_bytes_total"].samples[0].value
+ )
+ self.assertEqual(
+ 1, families["ambari_agent_disk_reads_merged_total"].samples[0].value
+ )
+ self.assertEqual(
+ 2, families["ambari_agent_disk_writes_merged_total"].samples[0].value
+ )
+ self.assertEqual(
+ 0.6,
+ families["ambari_agent_disk_write_time_seconds_total"].samples[0].value,
+ )
+
+ def test_network_collector_excludes_loopback(self):
+ families = self._families_by_name(NetworkCollector(self.proc_root))
+
+ sample = families["ambari_agent_network_receive_bytes_total"].samples[0]
+ self.assertEqual(100, sample.value)
+ self.assertEqual({"device": "eth0"}, sample.labels)
+ self.assertEqual(
+ 200,
+ families["ambari_agent_network_transmit_bytes_total"].samples[0].value,
+ )
+
+ def test_process_collector_exports_low_cardinality_state_counts(self):
+ family, thread_family = ProcessCollector(self.proc_root).collect()
+ samples = {sample.labels["state"]: sample.value for sample in family.samples}
+
+ self.assertEqual(3, samples["total"])
+ self.assertEqual(1, samples["running"])
+ self.assertEqual(1, samples["sleeping"])
+ self.assertEqual(1, samples["zombie"])
+ self.assertEqual(0, samples["unknown"])
+ self.assertEqual(8, thread_family.samples[0].value)
+
+ def test_socket_collector_exports_sockstat_values(self):
+ families = self._families_by_name(SocketCollector(self.proc_root))
+
+ self.assertEqual(42, families["ambari_agent_sockets_used"].samples[0].value)
+ self.assertEqual(
+ 11, families["ambari_agent_socket_tcp_allocated"].samples[0].value
+ )
+ self.assertEqual(
+ 5, families["ambari_agent_socket_tcp_in_use"].samples[0].value
+ )
+ self.assertEqual(
+ 7, families["ambari_agent_socket_tcp_time_wait"].samples[0].value
+ )
+
+ def test_timex_collector_exports_offset_in_seconds(self):
+ families = self._families_by_name(TimexCollector(read_timex=lambda: -0.00025))
+
+ self.assertEqual(
+ -0.00025, families["ambari_agent_timex_offset_seconds"].samples[0].value
+ )
+
+ def test_default_collectors_require_proc(self):
+ self.assertEqual([], default_collectors(os.path.join(self.proc_root, "missing")))
+ self.assertEqual(9, len(default_collectors(self.proc_root)))
+
+ def test_mount_field_decoding(self):
+ self.assertEqual("a b\tc\nd\\e", _decode_mount_field("a\\040b\\011c\\012d\\134e"))
+
+
+class TestPrometheusMetricsServer(unittest.TestCase):
+ def _initializer(self, enabled=True, port=0):
+ class Config:
+ prometheus_metrics_enabled = enabled
+ prometheus_metrics_bind_address = "127.0.0.1"
+ prometheus_metrics_port = port
+
+ class Initializer:
+ config = Config()
+ stop_event = threading.Event()
+
+ return Initializer()
+
+ def test_ambari_config_defaults_to_enabled(self):
+ config = AmbariConfig()
+
+ self.assertTrue(config.prometheus_metrics_enabled)
+ self.assertEqual("0.0.0.0", config.prometheus_metrics_bind_address)
+ self.assertEqual(9101, config.prometheus_metrics_port)
+
+ config.set("prometheus", "enabled", "false")
+ self.assertFalse(config.prometheus_metrics_enabled)
+
+ config.set("prometheus", "enabled", "YES")
+ self.assertTrue(config.prometheus_metrics_enabled)
+
+ config.set("prometheus", "port", "invalid")
+ with self.assertLogs("ambari_agent.AmbariConfig", level="WARNING"):
+ self.assertEqual(9101, config.prometheus_metrics_port)
+
+ config.set("prometheus", "port", "70000")
+ with self.assertLogs("ambari_agent.AmbariConfig", level="WARNING"):
+ self.assertEqual(9101, config.prometheus_metrics_port)
+
+ def test_server_exposes_metrics_health_and_not_found(self):
+ server = PrometheusMetricsServer(
+ self._initializer(), registry=CollectorRegistry([StaticCollector()])
+ )
+ server.start()
+ self.assertTrue(server.ready_event.wait(2))
+ self.assertIsNone(server.startup_error)
+ port = server.server_address[1]
+
+ try:
+ with urlopen(f"http://127.0.0.1:{port}/metrics", timeout=2) as response:
+ body = response.read().decode("utf-8")
+ self.assertEqual(200, response.status)
+ self.assertEqual(
+ "text/plain; version=0.0.4; charset=utf-8",
+ response.headers["Content-Type"],
+ )
+ self.assertIn('test_metric{source="unit"} 7', body)
+
+ with urlopen(f"http://127.0.0.1:{port}/-/healthy", timeout=2) as response:
+ self.assertEqual(200, response.status)
+
+ with self.assertRaises(HTTPError) as context:
+ urlopen(f"http://127.0.0.1:{port}/missing", timeout=2)
+ self.assertEqual(404, context.exception.code)
+ finally:
+ server.stop()
+ server.join(2)
+
+ self.assertFalse(server.is_alive())
+
+ def test_disabled_server_does_not_bind(self):
+ server = PrometheusMetricsServer(self._initializer(enabled=False))
+
+ server.start()
+ server.join(2)
+
+ self.assertFalse(server.is_alive())
+ self.assertIsNone(server.server_address)
+ self.assertIsNone(server.startup_error)
+
+ def test_bind_failure_is_contained(self):
+ class FailingServer:
+ def __init__(self, address, handler):
+ raise OSError("address unavailable")
+
+ server = PrometheusMetricsServer(
+ self._initializer(),
+ registry=CollectorRegistry([StaticCollector()]),
+ server_class=FailingServer,
+ )
+
+ with self.assertLogs("ambari_agent.metrics.server", level="ERROR"):
+ server.start()
+ self.assertTrue(server.ready_event.wait(2))
+ server.join(2)
+
+ self.assertIsInstance(server.startup_error, OSError)
+ self.assertFalse(server.is_alive())
+
+
+if __name__ == "__main__":
+ unittest.main()
diff --git a/ambari-agent/src/test/python/ambari_agent/TestTelemetryHeartbeat.py b/ambari-agent/src/test/python/ambari_agent/TestTelemetryHeartbeat.py
new file mode 100644
index 0000000000..a43c25d5b4
--- /dev/null
+++ b/ambari-agent/src/test/python/ambari_agent/TestTelemetryHeartbeat.py
@@ -0,0 +1,93 @@
+#!/usr/bin/env python3
+
+"""
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+"""
+
+import importlib
+import unittest
+from unittest.mock import MagicMock, patch
+
+from ambari_agent import HeartbeatThread
+from ambari_agent.metrics.telemetry import KerberosTelemetryFetcher
+
+
+class TestTelemetryHeartbeat(unittest.TestCase):
+ def test_kerberos_fetcher_disables_redirects_and_verifies_tls(self):
+ curl_module = importlib.import_module(
+ "resource_management.libraries.functions.curl_krb_request"
+ )
+ config = MagicMock()
+ config.get.side_effect = lambda section, key, default=None: {
+ ("agent", "tmp_dir"): "/tmp",
+ ("agent", "run_as_user"): "ambari-agent",
+ }.get((section, key), default)
+ target = {
+ "id": "namenode",
+ "url": "https://host.example.com:9871/prom",
+ "timeoutSeconds": 5,
+ "maxResponseBytes": 4096,
+ "auth": {
+ "type": "kerberos",
+ "principal": "HTTP/host.example.com@EXAMPLE.COM",
+ "keytab": "/etc/security/keytabs/spnego.service.keytab",
+ },
+ }
+
+ with patch.object(curl_module, "curl_krb_request") as request:
+ request.return_value = ("metric_total 1\n", None, 0.1)
+ response = KerberosTelemetryFetcher(config)(target)
+
+ self.assertEqual(b"metric_total 1\n", response.body)
+ request_options = request.call_args[1]
+ self.assertFalse(request_options["follow_redirects"])
+ self.assertTrue(request_options["fail_on_http_error"])
+ self.assertTrue(request_options["verify_ssl"])
+ self.assertEqual(4096, request_options["max_response_bytes"])
+ self.assertIsNone(request_options["ca_certs"])
+
+ def test_registration_capability_gates_telemetry_sync(self):
+ heartbeat = object.__new__(HeartbeatThread.HeartbeatThread)
+
+ heartbeat.handle_registration_response(
+ {"id": 1, "serverCapabilities": ["telemetry-v1"]}
+ )
+ self.assertTrue(heartbeat.telemetry_supported)
+
+ heartbeat.handle_registration_response({"id": 2})
+ self.assertFalse(heartbeat.telemetry_supported)
+
+ @patch("ambari_agent.HeartbeatThread.time.monotonic", return_value=600)
+ def test_reconciliation_failure_keeps_cached_assignment(self, monotonic):
+ heartbeat = object.__new__(HeartbeatThread.HeartbeatThread)
+ heartbeat.telemetry_supported = True
+ heartbeat.last_telemetry_reconciliation = 0
+ heartbeat.initializer_module = MagicMock()
+ heartbeat.initializer_module.telemetry_cache.hash = "assignment-v1"
+ heartbeat.telemetry_events_listener = MagicMock()
+ heartbeat.blocking_request = MagicMock(side_effect=RuntimeError("offline"))
+
+ with self.assertLogs("ambari_agent.HeartbeatThread", level="ERROR"):
+ heartbeat.reconcile_telemetry_if_due()
+
+ heartbeat.telemetry_events_listener.on_event.assert_not_called()
+ self.assertEqual("assignment-v1", heartbeat.initializer_module.telemetry_cache.hash)
+ self.assertEqual(600, heartbeat.last_telemetry_reconciliation)
+
+
+if __name__ == "__main__":
+ unittest.main()
diff --git a/ambari-agent/src/test/python/ambari_agent/TestTelemetryMetrics.py b/ambari-agent/src/test/python/ambari_agent/TestTelemetryMetrics.py
new file mode 100644
index 0000000000..f18dd70d7c
--- /dev/null
+++ b/ambari-agent/src/test/python/ambari_agent/TestTelemetryMetrics.py
@@ -0,0 +1,560 @@
+#!/usr/bin/env python3
+
+"""
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+"""
+
+import json
+import os
+import tempfile
+import threading
+import unittest
+from unittest.mock import MagicMock, patch
+from urllib.error import HTTPError
+from urllib.request import urlopen
+
+from ambari_agent.TelemetryConfigCache import TelemetryConfigCache
+from ambari_agent.listeners.TelemetryEventListener import TelemetryEventListener
+from ambari_agent.metrics.core import CollectorRegistry
+from ambari_agent.metrics.server import PrometheusMetricsServer
+from ambari_agent.metrics.telemetry import (
+ TelemetryConfigError,
+ TelemetryHttpClient,
+ TelemetryResponse,
+ TelemetryRouteRegistry,
+ TelemetryScrapeError,
+ convert_jmx_json,
+ parse_object_name,
+ profile_digest,
+ validate_profile,
+ validate_prometheus_text,
+)
+
+
+TEST_DIR = os.path.dirname(os.path.abspath(__file__))
+REPOSITORY_ROOT = os.path.abspath(os.path.join(TEST_DIR, "../../../../.."))
+TELEMETRY_FIXTURES = os.path.join(TEST_DIR, "fixtures", "telemetry")
+STACK_SERVICES = os.path.join(
+ REPOSITORY_ROOT,
+ "ambari-server",
+ "src",
+ "main",
+ "resources",
+ "stacks",
+ "BIGTOP",
+ "3.2.0",
+ "services",
+)
+
+
+def load_json(path):
+ with open(path, "r", encoding="utf-8") as stream:
+ return json.load(stream)
+
+
+def load_fixture(name):
+ with open(os.path.join(TELEMETRY_FIXTURES, name), "rb") as stream:
+ return stream.read()
+
+
+def load_stack_profile(service, name):
+ return load_json(
+ os.path.join(STACK_SERVICES, service, "telemetry-profiles", name)
+ )
+
+
+def hbase_profile():
+ return {
+ "schemaVersion": 1,
+ "id": "hbase-regionserver-2.4",
+ "maxSeries": 10,
+ "rules": [
+ {
+ "bean": {
+ "domain": "Hadoop",
+ "properties": {
+ "service": "HBase",
+ "name": "RegionServer",
+ "sub": "Server",
+ },
+ },
+ "labels": {"role": {"value": "regionserver"}},
+ "attributes": {
+ "regionCount": {
+ "name": "hbase_regionserver_regions",
+ "type": "gauge",
+ "unit": "regions",
+ "help": "Number of online regions.",
+ },
+ "totalRequestCount": {
+ "name": "hbase_regionserver_requests_total",
+ "type": "counter",
+ "unit": "requests",
+ "help": "Total number of requests.",
+ },
+ "requestTimeMs": {
+ "name": "hbase_regionserver_request_time_seconds",
+ "type": "gauge",
+ "unit": "seconds",
+ "scale": 0.001,
+ "help": "Latest request time in seconds.",
+ },
+ },
+ }
+ ],
+ }
+
+
+def jmx_assignment(profile_hash):
+ return {
+ "schemaVersion": 1,
+ "targets": [
+ {
+ "id": "hbase-regionserver",
+ "service": "HBASE",
+ "component": "HBASE_REGIONSERVER",
+ "format": "jmx_json",
+ "url": "http://host.example.com:16030/jmx",
+ "profileHash": profile_hash,
+ "timeoutSeconds": 5,
+ "maxResponseBytes": 4096,
+ }
+ ],
+ }
+
+
+def jmx_payload():
+ return json.dumps(
+ {
+ "beans": [
+ {
+ "name": "Hadoop:service=HBase,name=RegionServer,sub=Server",
+ "regionCount": 12,
+ "totalRequestCount": 450,
+ "requestTimeMs": 250,
+ "ignoredText": "not numeric",
+ }
+ ]
+ }
+ ).encode("utf-8")
+
+
+class FakeHttpClient:
+ def __init__(self, responses):
+ self.responses = responses
+ self.requests = []
+
+ def fetch(self, target):
+ self.requests.append(target["id"])
+ response = self.responses[target["id"]]
+ if isinstance(response, Exception):
+ raise response
+ return TelemetryResponse(response)
+
+
+class TestTelemetryConfigCache(unittest.TestCase):
+ def test_persists_and_recovers_a_valid_assignment(self):
+ profile = hbase_profile()
+ digest = profile_digest(profile)
+
+ with tempfile.TemporaryDirectory() as cache_dir:
+ cache = TelemetryConfigCache(cache_dir)
+ cache.update(jmx_assignment(digest), {digest: profile}, "assignment-v1")
+
+ recovered = TelemetryConfigCache(cache_dir)
+ snapshot = recovered.snapshot()
+
+ self.assertEqual("assignment-v1", recovered.hash)
+ self.assertEqual("hbase-regionserver", snapshot["assignment"]["targets"][0]["id"])
+ self.assertEqual(profile, snapshot["profiles"][digest])
+ self.assertTrue(recovered.last_reload_successful)
+
+ def test_invalid_update_keeps_last_known_good_assignment(self):
+ profile = hbase_profile()
+ digest = profile_digest(profile)
+
+ with tempfile.TemporaryDirectory() as cache_dir:
+ cache = TelemetryConfigCache(cache_dir)
+ cache.update(jmx_assignment(digest), {digest: profile}, "assignment-v1")
+ before = cache.snapshot()
+
+ invalid = jmx_assignment("sha256:" + "0" * 64)
+ with self.assertRaises(TelemetryConfigError):
+ cache.update(invalid, {}, "assignment-v2")
+
+ self.assertEqual("assignment-v1", cache.hash)
+ self.assertEqual(before, cache.snapshot())
+ self.assertFalse(cache.last_reload_successful)
+
+ def test_profile_digest_is_canonical(self):
+ first = {"schemaVersion": 1, "id": "profile", "rules": []}
+ second = {"rules": [], "id": "profile", "schemaVersion": 1}
+ self.assertEqual(profile_digest(first), profile_digest(second))
+
+ def test_concurrent_updates_keep_memory_and_disk_on_the_same_version(self):
+ first_entered_persist = threading.Event()
+ release_first = threading.Event()
+
+ class CoordinatedCache(TelemetryConfigCache):
+ def _persist_profiles(self, profiles):
+ if not first_entered_persist.is_set():
+ first_entered_persist.set()
+ release_first.wait(2)
+ return super()._persist_profiles(profiles)
+
+ with tempfile.TemporaryDirectory() as cache_dir:
+ cache = CoordinatedCache(cache_dir)
+ first = {"schemaVersion": 1, "targets": []}
+ second = {"schemaVersion": 1, "targets": []}
+ failures = []
+
+ def apply_update(assignment, cache_hash):
+ try:
+ cache.update(assignment, {}, cache_hash)
+ except Exception as err:
+ failures.append(err)
+
+ first_worker = threading.Thread(
+ target=apply_update, args=(first, "assignment-v1")
+ )
+ second_worker = threading.Thread(
+ target=apply_update, args=(second, "assignment-v2")
+ )
+ first_worker.start()
+ self.assertTrue(first_entered_persist.wait(2))
+ second_worker.start()
+ release_first.set()
+ first_worker.join(2)
+ second_worker.join(2)
+
+ recovered = TelemetryConfigCache(cache_dir)
+
+ self.assertFalse(failures)
+ self.assertEqual("assignment-v2", cache.hash)
+ self.assertEqual(cache.hash, recovered.hash)
+
+
+class TestJmxTelemetryConversion(unittest.TestCase):
+ def test_converts_typed_scaled_metrics(self):
+ rendered = convert_jmx_json(jmx_payload(), hbase_profile()).decode("utf-8")
+
+ self.assertIn(
+ 'hbase_regionserver_regions{role="regionserver"} 12', rendered
+ )
+ self.assertIn(
+ 'hbase_regionserver_requests_total{role="regionserver"} 450', rendered
+ )
+ self.assertIn(
+ 'hbase_regionserver_request_time_seconds{role="regionserver"} 0.25',
+ rendered,
+ )
+ self.assertIn("# TYPE hbase_regionserver_requests_total counter", rendered)
+
+ def test_preserves_large_unscaled_integer_counters(self):
+ profile = hbase_profile()
+ payload = json.dumps(
+ {
+ "beans": [
+ {
+ "name": "Hadoop:service=HBase,name=RegionServer,sub=Server",
+ "totalRequestCount": 9007199254740993,
+ }
+ ]
+ }
+ ).encode("utf-8")
+
+ rendered = convert_jmx_json(payload, profile).decode("utf-8")
+
+ self.assertIn(
+ 'hbase_regionserver_requests_total{role="regionserver"} 9007199254740993',
+ rendered,
+ )
+
+ def test_parses_quoted_object_name_properties(self):
+ domain, properties = parse_object_name(
+ 'Hadoop:service=ResourceManager,name=QueueMetrics,queue="root,a"'
+ )
+
+ self.assertEqual("Hadoop", domain)
+ self.assertEqual("root,a", properties["queue"])
+
+ def test_rejects_counter_without_total_suffix(self):
+ profile = hbase_profile()
+ definition = profile["rules"][0]["attributes"]["totalRequestCount"]
+ definition["name"] = "hbase_regionserver_requests"
+
+ with self.assertRaises(TelemetryConfigError):
+ validate_profile(profile)
+
+ def test_no_matching_metrics_marks_scrape_failed(self):
+ payload = b'{"beans": [{"name": "java.lang:type=Runtime"}]}'
+
+ with self.assertRaises(TelemetryScrapeError):
+ convert_jmx_json(payload, hbase_profile())
+
+ def test_converts_live_nodemanager_fixture_with_stack_profile(self):
+ profile = load_stack_profile("YARN", "nodemanager-3.3.json")
+ rendered = convert_jmx_json(
+ load_fixture("nodemanager-jmx.json"), profile
+ ).decode("utf-8")
+
+ self.assertIn("yarn_nodemanager_available_memory_bytes 1073741824", rendered)
+ self.assertIn("yarn_nodemanager_local_directory_utilization_ratio 0.1", rendered)
+
+ def test_converts_live_hbase_master_fixture_with_stack_profile(self):
+ profile = load_stack_profile("HBASE", "hbase-master-2.4.json")
+ rendered = convert_jmx_json(
+ load_fixture("hbase-master-jmx.json"), profile
+ ).decode("utf-8")
+
+ self.assertIn("hbase_master_active 1", rendered)
+ self.assertIn("hbase_master_regionservers 1", rendered)
+ self.assertIn("hbase_master_rpc_process_call_duration_p99_seconds 0.001", rendered)
+ self.assertIn("hbase_master_jvm_gc_time_seconds_total 0.816", rendered)
+
+ def test_converts_live_hbase_regionserver_fixture_with_stack_profile(self):
+ profile = load_stack_profile("HBASE", "hbase-regionserver-2.4.json")
+ rendered = convert_jmx_json(
+ load_fixture("hbase-regionserver-jmx.json"), profile
+ ).decode("utf-8")
+
+ self.assertIn("hbase_regionserver_store_file_locality_ratio 1", rendered)
+ self.assertIn("# TYPE hbase_regionserver_read_requests_per_second gauge", rendered)
+ self.assertIn("hbase_regionserver_read_requests_per_second 0", rendered)
+ self.assertIn("# TYPE hbase_regionserver_write_requests_per_second gauge", rendered)
+ self.assertIn("hbase_regionserver_write_requests_per_second 0", rendered)
+ self.assertIn("hbase_regionserver_scan_duration_p99_seconds 0.001", rendered)
+ self.assertIn("hbase_regionserver_rpc_received_bytes_total 261174", rendered)
+ self.assertIn("hbase_regionserver_jvm_heap_max_bytes 536870912", rendered)
+
+ def test_converts_live_hiveserver2_fixture_with_stack_profile(self):
+ profile = load_stack_profile("HIVE", "hiveserver2-3.1.json")
+ rendered = convert_jmx_json(
+ load_fixture("hiveserver2-jmx.json"), profile
+ ).decode("utf-8")
+
+ self.assertIn("hive_server2_connections_total 28", rendered)
+ self.assertIn("hive_server2_async_pool_size 1", rendered)
+ self.assertIn("hive_server2_open_sessions 0", rendered)
+ self.assertIn("hive_server2_jvm_heap_used_bytes 203948312", rendered)
+ self.assertIn("hive_server2_query_cache_max_bytes 2147483648", rendered)
+
+
+class TestTelemetryHttpClient(unittest.TestCase):
+ class Response:
+ def __init__(self, body):
+ self.body = body
+ self.headers = {"Content-Type": "text/plain"}
+
+ def __enter__(self):
+ return self
+
+ def __exit__(self, exception_type, exception, traceback):
+ return False
+
+ def read(self, limit):
+ return self.body[:limit]
+
+ class Opener:
+ def __init__(self, body):
+ self.body = body
+
+ def open(self, request, timeout):
+ return TestTelemetryHttpClient.Response(self.body)
+
+ def test_rejects_response_larger_than_target_limit(self):
+ target = {
+ "url": "http://host.example.com/prom",
+ "maxResponseBytes": 1024,
+ "timeoutSeconds": 5,
+ }
+ client = TelemetryHttpClient(opener=self.Opener(b"x" * 1025))
+
+ with self.assertRaises(TelemetryScrapeError):
+ client.fetch(target)
+
+ def test_rejects_malformed_native_prometheus_text(self):
+ with self.assertRaises(TelemetryScrapeError):
+ validate_prometheus_text(b"this is not an exposition response\n")
+
+ def test_accepts_prometheus_text_with_labels_and_timestamp(self):
+ validate_prometheus_text(
+ b'# TYPE hadoop_requests_total counter\n'
+ b'hadoop_requests_total{method="get block"} 3.5e2 1700000000\n'
+ )
+
+class TestTelemetryRoutes(unittest.TestCase):
+ def _cache(self, cache_dir):
+ profile = hbase_profile()
+ digest = profile_digest(profile)
+ cache = TelemetryConfigCache(cache_dir)
+ assignment = jmx_assignment(digest)
+ assignment["targets"].append(
+ {
+ "id": "namenode",
+ "format": "prometheus_text",
+ "url": "http://host.example.com:9870/prom",
+ }
+ )
+ cache.update(assignment, {digest: profile}, "assignment-v1")
+ return cache
+
+ def test_routes_native_and_jmx_targets_independently(self):
+ with tempfile.TemporaryDirectory() as cache_dir:
+ cache = self._cache(cache_dir)
+ client = FakeHttpClient(
+ {
+ "namenode": b"# TYPE namenode_capacity gauge\nnamenode_capacity 10\n",
+ "hbase-regionserver": jmx_payload(),
+ }
+ )
+ registry = TelemetryRouteRegistry(cache, http_client=client)
+
+ native = registry.scrape("namenode").body
+ converted = registry.scrape("hbase-regionserver").body
+
+ self.assertIn(b"namenode_capacity 10", native)
+ self.assertIn(b"hbase_regionserver_regions", converted)
+ self.assertEqual(["namenode", "hbase-regionserver"], client.requests)
+
+ def test_malformed_native_response_fails_only_its_route(self):
+ with tempfile.TemporaryDirectory() as cache_dir:
+ cache = self._cache(cache_dir)
+ registry = TelemetryRouteRegistry(
+ cache,
+ http_client=FakeHttpClient(
+ {
+ "namenode": b"upstream login page",
+ "hbase-regionserver": jmx_payload(),
+ }
+ ),
+ )
+
+ with self.assertRaises(TelemetryScrapeError):
+ registry.scrape("namenode")
+ converted = registry.scrape("hbase-regionserver").body
+
+ self.assertIn(b"hbase_regionserver_regions", converted)
+
+ def test_rejects_a_concurrent_scrape_above_the_route_limit(self):
+ entered = threading.Event()
+ release = threading.Event()
+
+ class BlockingClient:
+ def fetch(self, target):
+ entered.set()
+ release.wait(2)
+ return TelemetryResponse(b"namenode_capacity 10\n")
+
+ with tempfile.TemporaryDirectory() as cache_dir:
+ cache = self._cache(cache_dir)
+ assignment = cache.snapshot()["assignment"]
+ assignment["targets"][1]["maxConcurrentRequests"] = 1
+ cache.update(assignment, {}, "assignment-v2")
+ registry = TelemetryRouteRegistry(cache, http_client=BlockingClient())
+ first_error = []
+
+ def scrape_first():
+ try:
+ registry.scrape("namenode")
+ except Exception as err:
+ first_error.append(err)
+
+ worker = threading.Thread(target=scrape_first)
+ worker.start()
+ self.assertTrue(entered.wait(2))
+ try:
+ with self.assertRaises(TelemetryScrapeError) as context:
+ registry.scrape("namenode")
+ self.assertEqual(503, context.exception.status)
+ finally:
+ release.set()
+ worker.join(2)
+
+ self.assertFalse(first_error)
+
+ def test_component_http_path_returns_upstream_failure(self):
+ with tempfile.TemporaryDirectory() as cache_dir:
+ cache = self._cache(cache_dir)
+ routes = TelemetryRouteRegistry(
+ cache,
+ http_client=FakeHttpClient(
+ {
+ "namenode": TelemetryScrapeError("upstream unavailable"),
+ "hbase-regionserver": jmx_payload(),
+ }
+ ),
+ )
+
+ class Config:
+ prometheus_metrics_enabled = True
+ prometheus_metrics_bind_address = "127.0.0.1"
+ prometheus_metrics_port = 0
+
+ class Initializer:
+ config = Config()
+ stop_event = threading.Event()
+
+ server = PrometheusMetricsServer(
+ Initializer(),
+ registry=CollectorRegistry(),
+ telemetry_routes=routes,
+ )
+ server.start()
+ self.assertTrue(server.ready_event.wait(2))
+ port = server.server_address[1]
+ try:
+ with self.assertRaises(HTTPError) as context:
+ urlopen(
+ "http://127.0.0.1:{}/metrics/components/namenode".format(port),
+ timeout=2,
+ )
+ self.assertEqual(502, context.exception.code)
+
+ with urlopen(
+ "http://127.0.0.1:{}/metrics/components/hbase-regionserver".format(
+ port
+ ),
+ timeout=2,
+ ) as response:
+ self.assertIn(b"hbase_regionserver_regions", response.read())
+ finally:
+ server.stop()
+ server.join(2)
+
+
+class TestTelemetryDelivery(unittest.TestCase):
+ def test_listener_applies_complete_updates_and_ignores_empty_updates(self):
+ cache = MagicMock()
+ initializer = MagicMock()
+ initializer.telemetry_cache = cache
+ listener = TelemetryEventListener(initializer)
+ message = {
+ "hash": "assignment-v1",
+ "assignment": {"schemaVersion": 1, "targets": []},
+ "profiles": {},
+ }
+
+ listener.on_event({}, message)
+ listener.on_event({}, {})
+
+ cache.update.assert_called_once_with(
+ message["assignment"], message["profiles"], message["hash"]
+ )
+
+if __name__ == "__main__":
+ unittest.main()
diff --git a/ambari-agent/src/test/python/ambari_agent/dummy_files/dummy_current_stack b/ambari-agent/src/test/python/ambari_agent/dummy_files/dummy_current_stack
index b468638e5c..2cb4be4efd 100644
--- a/ambari-agent/src/test/python/ambari_agent/dummy_files/dummy_current_stack
+++ b/ambari-agent/src/test/python/ambari_agent/dummy_files/dummy_current_stack
@@ -18,4 +18,3 @@ limitations under the License.
DATANODE {"stackName":"HDP","stackVersion":"1.2.0"}
HCATALOG {"stackName":"HDP","stackVersion":"1.2.2"}
-GANGLIA_SERVER {"stackName":"HDP","stackVersion":"1.2.2"}
diff --git a/ambari-agent/src/test/python/ambari_agent/examples/ControllerTester.py b/ambari-agent/src/test/python/ambari_agent/examples/ControllerTester.py
index 26f6da65fa..c106295eee 100644
--- a/ambari-agent/src/test/python/ambari_agent/examples/ControllerTester.py
+++ b/ambari-agent/src/test/python/ambari_agent/examples/ControllerTester.py
@@ -71,9 +71,6 @@ responces = [
"ambari_db_server_host": [
"dev.hortonworks.com"
],
- "ganglia_server_host": [
- "dev.hortonworks.com"
- ],
"namenode_host": [
"dev.hortonworks.com"
],
@@ -206,9 +203,6 @@ if __name__ == "__main__":
# "ambari_db_server_host": [
# "dev.hortonworks.com"
# ],
- # "ganglia_server_host": [
- # "dev.hortonworks.com"
- # ],
# "namenode_host": [
# "dev.hortonworks.com"
# ],
diff --git a/ambari-agent/src/test/python/ambari_agent/fixtures/telemetry/hbase-master-jmx.json b/ambari-agent/src/test/python/ambari_agent/fixtures/telemetry/hbase-master-jmx.json
new file mode 100644
index 0000000000..c6d375f6c2
--- /dev/null
+++ b/ambari-agent/src/test/python/ambari_agent/fixtures/telemetry/hbase-master-jmx.json
@@ -0,0 +1,42 @@
+{
+ "beans": [
+ {
+ "name": "Hadoop:service=HBase,name=Master,sub=Server",
+ "averageLoad": 125.0,
+ "numRegionServers": 1,
+ "numDeadRegionServers": 0,
+ "numDrainingRegionServers": 0,
+ "tag.isActiveMaster": "true",
+ "clusterRequests": 777,
+ "masterStartTime": 1787882801033
+ },
+ {
+ "name": "Hadoop:service=HBase,name=Master,sub=AssignmentManager",
+ "ritCount": 0,
+ "ritCountOverThreshold": 0
+ },
+ {
+ "name": "Hadoop:service=HBase,name=JvmMetrics",
+ "MemHeapUsedM": 182.33856,
+ "MemHeapMaxM": 512.0,
+ "MemNonHeapUsedM": 116.296394,
+ "GcCount": 37,
+ "GcTimeMillis": 816,
+ "ThreadsRunnable": 33,
+ "ThreadsBlocked": 0,
+ "ThreadsWaiting": 103,
+ "ThreadsTimedWaiting": 44
+ },
+ {
+ "name": "Hadoop:service=HBase,name=Master,sub=IPC",
+ "queueSize": 0,
+ "numOpenConnections": 2,
+ "numActiveHandler": 0,
+ "numCallsInGeneralQueue": 0,
+ "numCallsInPriorityQueue": 0,
+ "numCallsInReplicationQueue": 0,
+ "ProcessCallTime_mean": 0,
+ "ProcessCallTime_99th_percentile": 1
+ }
+ ]
+}
diff --git a/ambari-agent/src/test/python/ambari_agent/fixtures/telemetry/hbase-regionserver-jmx.json b/ambari-agent/src/test/python/ambari_agent/fixtures/telemetry/hbase-regionserver-jmx.json
new file mode 100644
index 0000000000..e920019964
--- /dev/null
+++ b/ambari-agent/src/test/python/ambari_agent/fixtures/telemetry/hbase-regionserver-jmx.json
@@ -0,0 +1,59 @@
+{
+ "beans": [
+ {
+ "name": "Hadoop:service=HBase,name=RegionServer,sub=Server",
+ "regionCount": 125,
+ "storeCount": 307,
+ "storeFileCount": 3,
+ "memStoreSize": 0,
+ "storeFileSize": 202650,
+ "totalRequestCount": 777,
+ "readRequestRatePerSecond": 0.0,
+ "writeRequestRatePerSecond": 0.0,
+ "compactionQueueLength": 0,
+ "flushQueueLength": 0,
+ "blockCacheSize": 356120,
+ "blockCacheHitCount": 399,
+ "blockCacheMissCount": 26,
+ "percentFilesLocal": 100.0,
+ "blockedRequestCount": 0,
+ "slowGetCount": 0,
+ "slowPutCount": 0,
+ "slowDeleteCount": 0,
+ "slowIncrementCount": 0,
+ "Get_mean": 0,
+ "Get_99th_percentile": 0,
+ "Put_mean": 0,
+ "Put_99th_percentile": 0,
+ "ScanTime_mean": 0,
+ "ScanTime_99th_percentile": 1
+ },
+ {
+ "name": "Hadoop:service=HBase,name=JvmMetrics",
+ "MemHeapUsedM": 147.86482,
+ "MemHeapMaxM": 512.0,
+ "MemNonHeapUsedM": 115.034004,
+ "GcCount": 49,
+ "GcTimeMillis": 903,
+ "ThreadsRunnable": 31,
+ "ThreadsBlocked": 0,
+ "ThreadsWaiting": 128,
+ "ThreadsTimedWaiting": 31
+ },
+ {
+ "name": "Hadoop:service=HBase,name=RegionServer,sub=IPC",
+ "queueSize": 0,
+ "numOpenConnections": 5,
+ "numActiveHandler": 0,
+ "numCallsInGeneralQueue": 0,
+ "numCallsInPriorityQueue": 0,
+ "numCallsInReplicationQueue": 0,
+ "receivedBytes": 261174,
+ "sentBytes": 1975466,
+ "ProcessCallTime_mean": 1,
+ "ProcessCallTime_99th_percentile": 1,
+ "QueueCallTime_mean": 0,
+ "QueueCallTime_99th_percentile": 0
+ }
+ ]
+}
diff --git a/ambari-agent/src/test/python/ambari_agent/fixtures/telemetry/hiveserver2-jmx.json b/ambari-agent/src/test/python/ambari_agent/fixtures/telemetry/hiveserver2-jmx.json
new file mode 100644
index 0000000000..48c0e8edf0
--- /dev/null
+++ b/ambari-agent/src/test/python/ambari_agent/fixtures/telemetry/hiveserver2-jmx.json
@@ -0,0 +1,48 @@
+{
+ "beans": [
+ {
+ "name": "metrics:name=cumulative_connection_count",
+ "Count": 28
+ },
+ {
+ "name": "metrics:name=open_connections",
+ "Count": 0
+ },
+ {
+ "name": "metrics:name=hs2_active_sessions",
+ "Value": 0
+ },
+ {
+ "name": "metrics:name=exec_async_pool_size",
+ "Value": 1
+ },
+ {
+ "name": "metrics:name=exec_async_queue_size",
+ "Value": 0
+ },
+ {
+ "name": "metrics:name=hs2_open_sessions",
+ "Value": 0
+ },
+ {
+ "name": "metrics:name=memory.heap.used",
+ "Value": 203948312
+ },
+ {
+ "name": "metrics:name=memory.heap.max",
+ "Value": 536870912
+ },
+ {
+ "name": "metrics:name=threads.count",
+ "Value": 39
+ },
+ {
+ "name": "metrics:name=qc_current_size",
+ "Value": 0
+ },
+ {
+ "name": "metrics:name=qc_max_size",
+ "Value": 2147483648
+ }
+ ]
+}
diff --git a/ambari-agent/src/test/python/ambari_agent/fixtures/telemetry/nodemanager-jmx.json b/ambari-agent/src/test/python/ambari_agent/fixtures/telemetry/nodemanager-jmx.json
new file mode 100644
index 0000000000..098f6ab504
--- /dev/null
+++ b/ambari-agent/src/test/python/ambari_agent/fixtures/telemetry/nodemanager-jmx.json
@@ -0,0 +1,24 @@
+{
+ "beans": [
+ {
+ "name": "Hadoop:service=NodeManager,name=NodeManagerMetrics",
+ "ApplicationsRunning": 0,
+ "AllocatedContainers": 0,
+ "ContainersRunning": 0,
+ "ContainersIniting": 0,
+ "ContainersLaunched": 0,
+ "ContainersCompleted": 0,
+ "ContainersFailed": 0,
+ "ContainersKilled": 0,
+ "AllocatedGB": 0,
+ "AvailableGB": 1,
+ "AllocatedVCores": 0,
+ "AvailableVCores": 1,
+ "BadLocalDirs": 0,
+ "BadLogDirs": 0,
+ "GoodLocalDirsDiskUtilizationPerc": 10,
+ "GoodLogDirsDiskUtilizationPerc": 10,
+ "ContainerLaunchDurationAvgTime": 0.0
+ }
+ ]
+}
diff --git a/ambari-common/src/main/python/ambari_commons/ambari_metrics_helper.py b/ambari-common/src/main/python/ambari_commons/ambari_metrics_helper.py
deleted file mode 100644
index 5d8956941a..0000000000
--- a/ambari-common/src/main/python/ambari_commons/ambari_metrics_helper.py
+++ /dev/null
@@ -1,288 +0,0 @@
-#!/usr/bin/env python3
-
-"""
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-"""
-
-import ambari_commons.network as network
-import json
-import logging
-import os
-import random
-import urllib.request, urllib.parse, urllib.error
-from ambari_agent.AmbariConfig import AmbariConfig
-from resource_management.libraries.functions import conf_select
-from resource_management.libraries.functions.default import default
-from resource_management.libraries.functions.is_empty import is_empty
-
-logger = logging.getLogger(__name__)
-
-
-DEFAULT_COLLECTOR_SUFFIX = ".sink.timeline.collector.hosts"
-DEFAULT_METRICS2_PROPERTIES_FILE_NAME = "hadoop-metrics2.properties"
-
-AMS_METRICS_GET_URL = "/ws/v1/timeline/metrics?%s"
-
-METRICS_COLLECTOR_WEBAPP_ADDRESS_KEY = (
- "{{ams-site/timeline.metrics.service.webapp.address}}"
-)
-METRICS_COLLECTOR_VIP_HOST_KEY = "{{cluster-env/metrics_collector_external_hosts}}"
-METRICS_COLLECTOR_VIP_PORT_KEY = "{{cluster-env/metrics_collector_external_port}}"
-AMS_METRICS_COLLECTOR_USE_SSL_KEY = "{{ams-site/timeline.metrics.service.http.policy}}"
-CONNECTION_TIMEOUT_KEY = "http.connection.timeout"
-CONNECTION_TIMEOUT_DEFAULT = 5.0
-
-
-def select_metric_collector_for_sink(sink_name):
- # TODO check '*' sink_name
-
- all_collectors_string = get_metric_collectors_from_properties_file(sink_name)
- return select_metric_collector_hosts_from_hostnames(all_collectors_string)
-
-
-def select_metric_collector_hosts_from_hostnames(comma_separated_hosts):
- if comma_separated_hosts:
- hosts = comma_separated_hosts.split(",")
- return get_random_host(hosts)
- else:
- return "localhost"
-
-
-def get_random_host(hosts):
- return random.choice(hosts)
-
-
-def get_metric_collectors_from_properties_file(sink_name):
- try:
- hadoop_conf_dir = conf_select.get_hadoop_conf_dir()
- except Exception as e:
- raise Exception(f"Couldn't define hadoop_conf_dir: {e}")
- properties_filepath = os.path.join(
- hadoop_conf_dir, DEFAULT_METRICS2_PROPERTIES_FILE_NAME
- )
-
- if not os.path.exists(properties_filepath):
- raise Exception(
- f"Properties file doesn't exist : {properties_filepath}. Can't define metric collector hosts"
- )
- props = load_properties_from_file(properties_filepath)
-
- property_key = sink_name + DEFAULT_COLLECTOR_SUFFIX
- if property_key in props:
- return props.get(property_key)
- else:
- raise Exception(
- f"Properties file doesn't contain {property_key}. Can't define metric collector hosts"
- )
-
-
-def load_properties_from_file(filepath, sep="=", comment_char="#"):
- """
- Read the file passed as parameter as a properties file.
- """
- props = {}
- with open(filepath, "rt") as f:
- for line in f:
- l = line.strip()
- if l and not l.startswith(comment_char):
- key_value = l.split(sep)
- key = key_value[0].strip()
- value = sep.join(key_value[1:]).strip('" \t')
- props[key] = value
- return props
-
-
-def get_ams_tokens():
- return (
- METRICS_COLLECTOR_WEBAPP_ADDRESS_KEY,
- AMS_METRICS_COLLECTOR_USE_SSL_KEY,
- METRICS_COLLECTOR_VIP_HOST_KEY,
- METRICS_COLLECTOR_VIP_PORT_KEY,
- )
-
-
-def create_ams_client(alert_id, ams_app_id, configurations, parameters):
- if (
- METRICS_COLLECTOR_VIP_HOST_KEY in configurations
- and METRICS_COLLECTOR_VIP_PORT_KEY in configurations
- ):
- ams_collector_hosts = configurations[METRICS_COLLECTOR_VIP_HOST_KEY].split(",")
- ams_collector_port = int(configurations[METRICS_COLLECTOR_VIP_PORT_KEY])
- else:
- # ams-site/timeline.metrics.service.webapp.address is required
- if not METRICS_COLLECTOR_WEBAPP_ADDRESS_KEY in configurations:
- raise Exception(
- f"{METRICS_COLLECTOR_WEBAPP_ADDRESS_KEY} is a required parameter for the script"
- )
-
- collector_webapp_address = configurations[
- METRICS_COLLECTOR_WEBAPP_ADDRESS_KEY
- ].split(":")
- if not _valid_collector_webapp_address(collector_webapp_address):
- raise Exception(
- f'{METRICS_COLLECTOR_WEBAPP_ADDRESS_KEY} value should be set as "fqdn_hostname:port", but set to {configurations[METRICS_COLLECTOR_WEBAPP_ADDRESS_KEY]}'
- )
-
- ams_collector_hosts = default("/clusterHostInfo/metrics_collector_hosts", [])
- if not ams_collector_hosts:
- raise Exception("Ambari metrics is not available: ams_collector_hosts is None")
- ams_collector_port = int(collector_webapp_address[1])
-
- use_ssl = False
- if AMS_METRICS_COLLECTOR_USE_SSL_KEY in configurations:
- use_ssl = configurations[AMS_METRICS_COLLECTOR_USE_SSL_KEY] == "HTTPS_ONLY"
-
- connection_timeout = CONNECTION_TIMEOUT_DEFAULT
- if CONNECTION_TIMEOUT_KEY in parameters:
- connection_timeout = float(parameters[CONNECTION_TIMEOUT_KEY])
- return AmsClient(
- alert_id,
- ams_collector_hosts,
- ams_collector_port,
- use_ssl,
- connection_timeout,
- ams_app_id,
- )
-
-
-def _valid_collector_webapp_address(webapp_address):
- if (
- len(webapp_address) == 2
- and webapp_address[0] != "127.0.0.1"
- and webapp_address[1].isdigit()
- ):
- return True
-
- return False
-
-
-class AmsClient:
- def __init__(
- self,
- alert_id,
- ams_collector_hosts,
- ams_collector_port,
- use_ssl,
- connection_timeout,
- ams_app_id,
- ):
- self.alert_id = alert_id
- self.ams_collector_hosts = ams_collector_hosts
- self.ams_collector_port = ams_collector_port
- self.use_ssl = use_ssl
- self.connection_timeout = connection_timeout
- self.ams_app_id = ams_app_id
-
- def load_metric(self, ams_metric, host_filter):
- metric_dict = None
- http_code = None
- for ams_collector_host in self.ams_collector_hosts:
- try:
- metric_dict, http_code = self._load_metric(
- ams_collector_host, ams_metric, host_filter
- )
- if http_code == 200 and metric_dict:
- break
- except Exception as exception:
- if logger.isEnabledFor(logging.DEBUG):
- logger.exception(
- f"[Alert][{self.alert_id}] Unable to retrieve metrics from AMS ({ams_collector_host}:{self.ams_collector_port}): {str(exception)}"
- )
-
- if not http_code:
- raise Exception("Ambari metrics is not available: no response")
- if http_code not in [200, 299]:
- raise Exception(
- "Ambari metrics is not available: http status code = " + str(http_code)
- )
- if not metric_dict:
- raise Exception("Ambari metrics is not available: no metrics were found.")
-
- return metric_dict
-
- def _load_metric(self, ams_collector_host, ams_metric, host_filter):
- get_metrics_parameters = {
- "metricNames": ams_metric,
- "appId": self.ams_app_id,
- "hostname": host_filter,
- "precision": "seconds",
- "grouped": "true",
- }
- encoded_get_metrics_parameters = urllib.parse.urlencode(get_metrics_parameters)
- url = AMS_METRICS_GET_URL % encoded_get_metrics_parameters
-
- _ssl_version = AmbariConfig.get_resolved_config().get_force_https_protocol_value()
-
- ams_monitor_conf_dir = "/etc/ambari-metrics-monitor/conf"
- metric_truststore_ca_certs = "ca.pem"
- ca_certs = os.path.join(ams_monitor_conf_dir, metric_truststore_ca_certs)
-
- conn = None
- response = None
- data = None
- try:
- conn = network.get_http_connection(
- ams_collector_host,
- int(self.ams_collector_port),
- self.use_ssl,
- ca_certs,
- ssl_version=_ssl_version,
- )
- conn.request("GET", url)
- response = conn.getresponse()
- data = response.read()
- except Exception as exception:
- if logger.isEnabledFor(logging.DEBUG):
- logger.exception(
- f"[Alert][{self.alert_id}] Unable to retrieve metrics from AMS: {str(exception)}"
- )
- status = response.status if response else None
- return None, status
- finally:
- if logger.isEnabledFor(logging.DEBUG):
- logger.debug(f"""
- AMS request parameters - {encoded_get_metrics_parameters}
- AMS response - {data}
- """)
- # explicitly close the connection as we've seen python hold onto these
- if conn is not None:
- try:
- conn.close()
- except:
- logger.debug(
- f"[Alert][{self.alert_id}] Unable to close URL connection to {url}"
- )
-
- data_json = None
- try:
- data_json = json.loads(data)
- except Exception as exception:
- if logger.isEnabledFor(logging.DEBUG):
- logger.exception(
- f"[Alert][{self.alert_id}] Convert response to json failed or json doesn't contain needed data: {str(exception)}"
- )
-
- if not data_json:
- return None, response.status
-
- metric_dict = {}
- if "metrics" not in data_json:
- return None, response.status
- for metrics_data in data_json["metrics"]:
- metric_dict[metrics_data["metricname"]] = metrics_data["metrics"]
-
- return metric_dict, response.status
diff --git a/ambari-common/src/main/python/ambari_commons/repo_manager/repo_manager_helper.py b/ambari-common/src/main/python/ambari_commons/repo_manager/repo_manager_helper.py
deleted file mode 100644
index d832afbc60..0000000000
--- a/ambari-common/src/main/python/ambari_commons/repo_manager/repo_manager_helper.py
+++ /dev/null
@@ -1,72 +0,0 @@
-#!/usr/bin/env python3
-"""
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-"""
-
-from . import ManagerFactory
-from resource_management.libraries.functions.version import compare_versions
-from resource_management.core.logger import Logger
-from resource_management.core.exceptions import Fail
-from ambari_commons import OSCheck
-
-
-def check_installed_metrics_hadoop_sink_version(
- hadoop_sink_package_name="ambari-metrics-hadoop-sink",
- checked_version="2.7.0.0",
- less_valid=True,
- equal_valid=False,
-):
- # The default package name is different for ubuntu and debian, so if the dafault one is used change the name
- if (
- hadoop_sink_package_name == "ambari-metrics-hadoop-sink"
- and OSCheck.is_ubuntu_family()
- ):
- hadoop_sink_package_name = "ambari-metrics-assembly"
-
- pkg_provider = ManagerFactory.get()
- hadoop_sink_version = pkg_provider.get_installed_package_version(
- hadoop_sink_package_name
- )
-
- if not hadoop_sink_version:
- Logger.warning(
- "Couldn't determine %s package version, skipping the sink version check"
- % hadoop_sink_package_name
- )
- return
- else:
- if "-" in hadoop_sink_version:
- hadoop_sink_version = hadoop_sink_version.split("-")[0]
- # installed version should be less than next version
- compare_result = compare_versions(hadoop_sink_version, checked_version)
- if equal_valid and compare_result == 0:
- pass
- elif less_valid and compare_result != -1:
- raise Fail(
- "%s installed package version is %s. It should be less than %s due to"
- " incompatibility. Please downgrade the package or upgrade the stack and try again."
- % (hadoop_sink_package_name, hadoop_sink_version, checked_version)
- )
-
- elif not less_valid and compare_result != 1:
- raise Fail(
- "%s installed package version is %s. It should be greater than or equal to %s due to"
- " incompatibility. Please upgrade the package or downgrade the stack and try again."
- % (hadoop_sink_package_name, hadoop_sink_version, checked_version)
- )
-
- Logger.info("ambari-metrics-hadoop-sink package version is OK")
diff --git a/ambari-common/src/main/python/pluggable_stack_definition/configs/ODP.json b/ambari-common/src/main/python/pluggable_stack_definition/configs/ODP.json
index f55b64246f..8e569e4cc9 100644
--- a/ambari-common/src/main/python/pluggable_stack_definition/configs/ODP.json
+++ b/ambari-common/src/main/python/pluggable_stack_definition/configs/ODP.json
@@ -32,9 +32,6 @@
{
"name": "KERBEROS"
},
- {
- "name": "AMBARI_METRICS"
- },
{
"name": "HDFS"
},
@@ -58,9 +55,6 @@
{
"name": "KERBEROS"
},
- {
- "name": "AMBARI_METRICS"
- },
{
"name": "HDFS"
},
@@ -84,9 +78,6 @@
{
"name": "KERBEROS"
},
- {
- "name": "AMBARI_METRICS"
- },
{
"name": "HDFS"
},
@@ -110,9 +101,6 @@
{
"name": "KERBEROS"
},
- {
- "name": "AMBARI_METRICS"
- },
{
"name": "HDFS"
},
diff --git a/ambari-common/src/main/python/pluggable_stack_definition/configs/PHD.json b/ambari-common/src/main/python/pluggable_stack_definition/configs/PHD.json
index 7590e696d1..9d1f98d387 100644
--- a/ambari-common/src/main/python/pluggable_stack_definition/configs/PHD.json
+++ b/ambari-common/src/main/python/pluggable_stack_definition/configs/PHD.json
@@ -26,9 +26,6 @@
{
"name": "KERBEROS"
},
- {
- "name": "AMBARI_METRICS"
- },
{
"name": "HDFS"
@@ -59,9 +56,6 @@
},
{
"name": "PIG"
- },
- {
- "name": "GANGLIA"
}
]
},
@@ -74,9 +68,6 @@
{
"name": "KERBEROS"
},
- {
- "name": "AMBARI_METRICS"
- },
{
"name": "HDFS"
},
@@ -118,9 +109,6 @@
{
"name": "KERBEROS"
},
- {
- "name": "AMBARI_METRICS"
- },
{
"name": "HDFS"
},
@@ -208,9 +196,6 @@
{
"name": "KERBEROS"
},
- {
- "name": "AMBARI_METRICS"
- },
{
"name": "HDFS"
},
@@ -364,9 +349,6 @@
{
"name": "KERBEROS"
},
- {
- "name": "AMBARI_METRICS"
- },
{
"name": "HDFS"
},
@@ -417,9 +399,6 @@
{
"name": "KERBEROS"
},
- {
- "name": "AMBARI_METRICS"
- },
{
"name": "HDFS"
},
@@ -470,9 +449,6 @@
{
"name": "KERBEROS"
},
- {
- "name": "AMBARI_METRICS"
- },
{
"name": "HDFS"
},
diff --git a/ambari-common/src/main/python/pluggable_stack_definition/configs/SAPHD.json b/ambari-common/src/main/python/pluggable_stack_definition/configs/SAPHD.json
index de85aded3a..e268438ba3 100644
--- a/ambari-common/src/main/python/pluggable_stack_definition/configs/SAPHD.json
+++ b/ambari-common/src/main/python/pluggable_stack_definition/configs/SAPHD.json
@@ -109,9 +109,6 @@
{
"name": "KERBEROS"
},
- {
- "name": "AMBARI_METRICS"
- },
{
"name": "HDFS"
@@ -159,9 +156,6 @@
{
"name": "KERBEROS"
},
- {
- "name": "AMBARI_METRICS"
- },
{
"name": "HDFS"
},
@@ -223,9 +217,6 @@
{
"name": "KERBEROS"
},
- {
- "name": "AMBARI_METRICS"
- },
{
"name": "HDFS"
},
@@ -299,9 +290,6 @@
{
"name": "KERBEROS"
},
- {
- "name": "AMBARI_METRICS"
- },
{
"name": "HDFS"
},
diff --git a/ambari-common/src/main/python/resource_management/libraries/functions/constants.py b/ambari-common/src/main/python/resource_management/libraries/functions/constants.py
index a86d775e33..f17e728482 100644
--- a/ambari-common/src/main/python/resource_management/libraries/functions/constants.py
+++ b/ambari-common/src/main/python/resource_management/libraries/functions/constants.py
@@ -66,7 +66,6 @@ class StackFeature:
SPARK_LIVY = "spark_livy"
SPARK_LIVY2 = "spark_livy2"
STORM_KERBEROS = "storm_kerberos"
- STORM_AMS = "storm_ams"
KAFKA_LISTENERS = "kafka_listeners"
KAFKA_KERBEROS = "kafka_kerberos"
PIG_ON_TEZ = "pig_on_tez"
@@ -125,7 +124,6 @@ class StackFeature:
KAFKA_EXTENDED_SASL_SUPPORT = "kafka_extended_sasl_support"
OOZIE_EXTJS_INCLUDED = "oozie_extjs_included"
MULTIPLE_ENV_SH_FILES_SUPPORT = "multiple_env_sh_files_support"
- AMS_LEGACY_HADOOP_SINK = "ams_legacy_hadoop_sink"
RANGER_ALL_ADMIN_CHANGE_DEFAULT_PASSWORD = "ranger_all_admin_change_default_password"
KAFKA_ENV_INCLUDE_RANGER_SCRIPT = "kafka_env_include_ranger_script"
RANGER_SUPPORT_SECURITY_ZONE_FEATURE = "ranger_support_security_zone_feature"
diff --git a/ambari-common/src/main/python/resource_management/libraries/functions/curl_krb_request.py b/ambari-common/src/main/python/resource_management/libraries/functions/curl_krb_request.py
index c3ef8e745a..11047b4cb0 100644
--- a/ambari-common/src/main/python/resource_management/libraries/functions/curl_krb_request.py
+++ b/ambari-common/src/main/python/resource_management/libraries/functions/curl_krb_request.py
@@ -75,6 +75,10 @@ def curl_krb_request(
body="",
header="",
use_system_proxy_settings=True,
+ follow_redirects=True,
+ fail_on_http_error=False,
+ max_response_bytes=None,
+ verify_ssl=True,
):
"""
Makes a curl request using a private, per-request Kerberos credential cache.
@@ -113,8 +117,15 @@ def curl_krb_request(
connection_timeout = max(1, math.ceil(requested_timeout))
maximum_timeout = connection_timeout + 2
- ssl_options = ["--cacert", ca_certs] if ca_certs else []
+ ssl_options = ["--cacert", ca_certs] if ca_certs else ([] if verify_ssl else ["-k"])
proxy_options = [] if use_system_proxy_settings else ["--noproxy", "*"]
+ transfer_options = []
+ if follow_redirects:
+ transfer_options.append("--location")
+ if fail_on_http_error:
+ transfer_options.append("--fail")
+ if max_response_bytes is not None:
+ transfer_options.extend(["--max-filesize", str(max_response_bytes)])
safe_prefix = re.sub(r"[^A-Za-z0-9_.-]", "_", str(cache_file_prefix))[:64]
cache_prefix = f"ambari-curl-{safe_prefix or 'request'}-"
kinit_path_local = get_kinit_path(krb_exec_search_paths)
@@ -139,7 +150,8 @@ def curl_krb_request(
with body_context as request_body_file:
curl_command = (
- ["curl", "--silent", "--show-error", "--location"]
+ ["curl", "--silent", "--show-error"]
+ + transfer_options
+ ssl_options
+ proxy_options
+ [
diff --git a/ambari-common/src/main/python/resource_management/libraries/functions/package_conditions.py b/ambari-common/src/main/python/resource_management/libraries/functions/package_conditions.py
index 842f39cace..c253b58143 100644
--- a/ambari-common/src/main/python/resource_management/libraries/functions/package_conditions.py
+++ b/ambari-common/src/main/python/resource_management/libraries/functions/package_conditions.py
@@ -22,8 +22,6 @@ Ambari Agent
__all__ = [
"should_install_phoenix",
- "should_install_ams_collector",
- "should_install_ams_grafana",
"should_install_infra_solr",
"should_install_infra_solr_client",
"should_install_mysql",
@@ -90,16 +88,6 @@ def should_install_phoenix():
return _strict_configuration_boolean(config, "hbase-env", "phoenix_sql_enabled")
-def should_install_ams_collector():
- config = Script.get_config()
- return _has_applicable_local_component(config, ["METRICS_COLLECTOR"])
-
-
-def should_install_ams_grafana():
- config = Script.get_config()
- return _has_applicable_local_component(config, ["METRICS_GRAFANA"])
-
-
def should_install_yarn_ats_hbase():
config = Script.get_config()
config_type = "yarn-hbase-env"
diff --git a/ambari-funtest/src/test/resources/metrics.properties b/ambari-funtest/src/test/resources/metrics.properties
deleted file mode 100644
index 9236120ecc..0000000000
--- a/ambari-funtest/src/test/resources/metrics.properties
+++ /dev/null
@@ -1,24 +0,0 @@
-# Copyright 2011 The Apache Software Foundation
-#
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-
-#### Source Configs #####
-# Source interval determines how often the metric is sent to sink. Its unit is in seconds
-metric.sources=jvm
-source.jvm.class=org.apache.ambari.server.metrics.system.impl.JvmMetricsSource
-source.stats.class=org.apache.ambari.server.metrics.system.impl.StompStatsMetricsSource
diff --git a/ambari-funtest/src/test/resources/stacks/HDP/1.2.0/services/GANGLIA/metainfo.xml b/ambari-funtest/src/test/resources/stacks/HDP/1.2.0/services/GANGLIA/metainfo.xml
deleted file mode 100644
index ca15c7b1b6..0000000000
--- a/ambari-funtest/src/test/resources/stacks/HDP/1.2.0/services/GANGLIA/metainfo.xml
+++ /dev/null
@@ -1,97 +0,0 @@
-<?xml version="1.0"?>
-<!--
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-<metainfo>
- <schemaVersion>2.0</schemaVersion>
- <services>
- <service>
- <name>GANGLIA</name>
- <comment>Ganglia Metrics Collection system</comment>
- <version>3.2.0</version>
-
- <components>
- <component>
- <name>GANGLIA_SERVER</name>
- <category>MASTER</category>
- <cardinality>1</cardinality>
- <commandScript>
- <script>scripts/ganglia_server.py</script>
- <scriptType>PYTHON</scriptType>
- <timeout>600</timeout>
- </commandScript>
- </component>
-
- <component>
- <name>GANGLIA_MONITOR</name>
- <category>SLAVE</category>
- <cardinality>ALL</cardinality>
- <auto-deploy>
- <enabled>true</enabled>
- </auto-deploy>
- <commandScript>
- <script>scripts/ganglia_monitor.py</script>
- <scriptType>PYTHON</scriptType>
- <timeout>600</timeout>
- </commandScript>
- </component>
- </components>
- <osSpecifics>
- <osSpecific>
- <osFamily>any</osFamily>
- <packages>
- <package>
- <name>libganglia-3.5.0-99</name>
- </package>
- <package>
- <name>ganglia-devel-3.5.0-99</name>
- </package>
- <package>
- <name>ganglia-gmetad-3.5.0-99</name>
- </package>
- <package>
- <name>ganglia-web-3.5.7-99.noarch</name>
- </package>
- <package>
- <name>python-rrdtool.x86_64</name>
- </package>
- <package>
- <name>ganglia-gmond-3.5.0-99</name>
- </package>
- <package>
- <name>ganglia-gmond-modules-python-3.5.0-99</name>
- </package>
- </packages>
- </osSpecific>
- <osSpecific>
- <osFamily>suse</osFamily>
- <package>
- <name>apache2</name>
- </package>
- <package>
- <name>apache2-mod_php5</name>
- </package>
- </osSpecific>
- <osSpecific>
- <osFamily>centos6</osFamily>
- <package>
- <name>httpd</name>
- </package>
- </osSpecific>
- </osSpecifics>
- </service>
- </services>
-</metainfo>
diff --git a/ambari-funtest/src/test/resources/stacks/HDP/1.3.0/services/GANGLIA/configuration/global.xml b/ambari-funtest/src/test/resources/stacks/HDP/1.3.0/services/GANGLIA/configuration/global.xml
deleted file mode 100644
index 16df0b8fbc..0000000000
--- a/ambari-funtest/src/test/resources/stacks/HDP/1.3.0/services/GANGLIA/configuration/global.xml
+++ /dev/null
@@ -1,55 +0,0 @@
-<?xml version="1.0"?>
-<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
-<!--
-/**
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
--->
-
-<configuration>
- <property>
- <name>ganglia_conf_dir</name>
- <value>/etc/ganglia/hdp</value>
- <description>Config directory for Ganglia</description>
- </property>
- <property>
- <name>ganglia_runtime_dir</name>
- <value>/var/run/ganglia/hdp</value>
- <description>Run directories for Ganglia</description>
- </property>
- <property>
- <name>ganglia_runtime_dir</name>
- <value>/var/run/ganglia/hdp</value>
- <description>Run directories for Ganglia</description>
- </property>
- <property>
- <name>gmetad_user</name>
- <value>nobody</value>
- <description>User </description>
- </property>
- <property>
- <name>gmond_user</name>
- <value>nobody</value>
- <description>User </description>
- </property>
- <property>
- <name>rrdcached_base_dir</name>
- <value>/var/lib/ganglia/rrds</value>
- <description>Default directory for saving the rrd files on ganglia server</description>
- </property>
-
-</configuration>
diff --git a/ambari-funtest/src/test/resources/stacks/HDP/1.3.0/services/GANGLIA/metainfo.xml b/ambari-funtest/src/test/resources/stacks/HDP/1.3.0/services/GANGLIA/metainfo.xml
deleted file mode 100644
index ca15c7b1b6..0000000000
--- a/ambari-funtest/src/test/resources/stacks/HDP/1.3.0/services/GANGLIA/metainfo.xml
+++ /dev/null
@@ -1,97 +0,0 @@
-<?xml version="1.0"?>
-<!--
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-<metainfo>
- <schemaVersion>2.0</schemaVersion>
- <services>
- <service>
- <name>GANGLIA</name>
- <comment>Ganglia Metrics Collection system</comment>
- <version>3.2.0</version>
-
- <components>
- <component>
- <name>GANGLIA_SERVER</name>
- <category>MASTER</category>
- <cardinality>1</cardinality>
- <commandScript>
- <script>scripts/ganglia_server.py</script>
- <scriptType>PYTHON</scriptType>
- <timeout>600</timeout>
- </commandScript>
- </component>
-
- <component>
- <name>GANGLIA_MONITOR</name>
- <category>SLAVE</category>
- <cardinality>ALL</cardinality>
- <auto-deploy>
- <enabled>true</enabled>
- </auto-deploy>
- <commandScript>
- <script>scripts/ganglia_monitor.py</script>
- <scriptType>PYTHON</scriptType>
- <timeout>600</timeout>
- </commandScript>
- </component>
- </components>
- <osSpecifics>
- <osSpecific>
- <osFamily>any</osFamily>
- <packages>
- <package>
- <name>libganglia-3.5.0-99</name>
- </package>
- <package>
- <name>ganglia-devel-3.5.0-99</name>
- </package>
- <package>
- <name>ganglia-gmetad-3.5.0-99</name>
- </package>
- <package>
- <name>ganglia-web-3.5.7-99.noarch</name>
- </package>
- <package>
- <name>python-rrdtool.x86_64</name>
- </package>
- <package>
- <name>ganglia-gmond-3.5.0-99</name>
- </package>
- <package>
- <name>ganglia-gmond-modules-python-3.5.0-99</name>
- </package>
- </packages>
- </osSpecific>
- <osSpecific>
- <osFamily>suse</osFamily>
- <package>
- <name>apache2</name>
- </package>
- <package>
- <name>apache2-mod_php5</name>
- </package>
- </osSpecific>
- <osSpecific>
- <osFamily>centos6</osFamily>
- <package>
- <name>httpd</name>
- </package>
- </osSpecific>
- </osSpecifics>
- </service>
- </services>
-</metainfo>
diff --git a/ambari-funtest/src/test/resources/stacks/HDP/1.3.1/services/GANGLIA/metainfo.xml b/ambari-funtest/src/test/resources/stacks/HDP/1.3.1/services/GANGLIA/metainfo.xml
deleted file mode 100644
index ca15c7b1b6..0000000000
--- a/ambari-funtest/src/test/resources/stacks/HDP/1.3.1/services/GANGLIA/metainfo.xml
+++ /dev/null
@@ -1,97 +0,0 @@
-<?xml version="1.0"?>
-<!--
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-<metainfo>
- <schemaVersion>2.0</schemaVersion>
- <services>
- <service>
- <name>GANGLIA</name>
- <comment>Ganglia Metrics Collection system</comment>
- <version>3.2.0</version>
-
- <components>
- <component>
- <name>GANGLIA_SERVER</name>
- <category>MASTER</category>
- <cardinality>1</cardinality>
- <commandScript>
- <script>scripts/ganglia_server.py</script>
- <scriptType>PYTHON</scriptType>
- <timeout>600</timeout>
- </commandScript>
- </component>
-
- <component>
- <name>GANGLIA_MONITOR</name>
- <category>SLAVE</category>
- <cardinality>ALL</cardinality>
- <auto-deploy>
- <enabled>true</enabled>
- </auto-deploy>
- <commandScript>
- <script>scripts/ganglia_monitor.py</script>
- <scriptType>PYTHON</scriptType>
- <timeout>600</timeout>
- </commandScript>
- </component>
- </components>
- <osSpecifics>
- <osSpecific>
- <osFamily>any</osFamily>
- <packages>
- <package>
- <name>libganglia-3.5.0-99</name>
- </package>
- <package>
- <name>ganglia-devel-3.5.0-99</name>
- </package>
- <package>
- <name>ganglia-gmetad-3.5.0-99</name>
- </package>
- <package>
- <name>ganglia-web-3.5.7-99.noarch</name>
- </package>
- <package>
- <name>python-rrdtool.x86_64</name>
- </package>
- <package>
- <name>ganglia-gmond-3.5.0-99</name>
- </package>
- <package>
- <name>ganglia-gmond-modules-python-3.5.0-99</name>
- </package>
- </packages>
- </osSpecific>
- <osSpecific>
- <osFamily>suse</osFamily>
- <package>
- <name>apache2</name>
- </package>
- <package>
- <name>apache2-mod_php5</name>
- </package>
- </osSpecific>
- <osSpecific>
- <osFamily>centos6</osFamily>
- <package>
- <name>httpd</name>
- </package>
- </osSpecific>
- </osSpecifics>
- </service>
- </services>
-</metainfo>
diff --git a/ambari-funtest/src/test/resources/stacks/HDP/1.3.4/services/GANGLIA/metainfo.xml b/ambari-funtest/src/test/resources/stacks/HDP/1.3.4/services/GANGLIA/metainfo.xml
deleted file mode 100644
index 3f9c3452be..0000000000
--- a/ambari-funtest/src/test/resources/stacks/HDP/1.3.4/services/GANGLIA/metainfo.xml
+++ /dev/null
@@ -1,96 +0,0 @@
-<?xml version="1.0"?>
-<!--
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-<metainfo>
- <schemaVersion>2.0</schemaVersion>
- <services>
- <service>
- <name>GANGLIA</name>
- <comment>Ganglia Metrics Collection system</comment>
- <version>3.5.0</version>
- <components>
- <component>
- <name>GANGLIA_SERVER</name>
- <category>MASTER</category>
- <cardinality>1</cardinality>
- <commandScript>
- <script>scripts/ganglia_server.py</script>
- <scriptType>PYTHON</scriptType>
- <timeout>600</timeout>
- </commandScript>
- </component>
-
- <component>
- <name>GANGLIA_MONITOR</name>
- <category>SLAVE</category>
- <cardinality>ALL</cardinality>
- <auto-deploy>
- <enabled>true</enabled>
- </auto-deploy>
- <commandScript>
- <script>scripts/ganglia_monitor.py</script>
- <scriptType>PYTHON</scriptType>
- <timeout>600</timeout>
- </commandScript>
- </component>
- </components>
- <osSpecifics>
- <osSpecific>
- <osFamily>any</osFamily>
- <packages>
- <package>
- <name>libganglia-3.5.0-99</name>
- </package>
- <package>
- <name>ganglia-devel-3.5.0-99</name>
- </package>
- <package>
- <name>ganglia-gmetad-3.5.0-99</name>
- </package>
- <package>
- <name>ganglia-web-3.5.7-99.noarch</name>
- </package>
- <package>
- <name>python-rrdtool.x86_64</name>
- </package>
- <package>
- <name>ganglia-gmond-3.5.0-99</name>
- </package>
- <package>
- <name>ganglia-gmond-modules-python-3.5.0-99</name>
- </package>
- </packages>
- </osSpecific>
- <osSpecific>
- <osFamily>suse</osFamily>
- <package>
- <name>apache2</name>
- </package>
- <package>
- <name>apache2-mod_php5</name>
- </package>
- </osSpecific>
- <osSpecific>
- <osFamily>centos6</osFamily>
- <package>
- <name>httpd</name>
- </package>
- </osSpecific>
- </osSpecifics>
- </service>
- </services>
-</metainfo>
diff --git a/ambari-funtest/src/test/resources/stacks/HDP/2.0.1/services/GANGLIA/metainfo.xml b/ambari-funtest/src/test/resources/stacks/HDP/2.0.1/services/GANGLIA/metainfo.xml
deleted file mode 100644
index ee9b1f2c07..0000000000
--- a/ambari-funtest/src/test/resources/stacks/HDP/2.0.1/services/GANGLIA/metainfo.xml
+++ /dev/null
@@ -1,129 +0,0 @@
-<?xml version="1.0"?>
-<!--
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-<metainfo>
- <schemaVersion>2.0</schemaVersion>
- <services>
- <service>
- <name>GANGLIA</name>
- <comment>Ganglia Metrics Collection system</comment>
- <version>3.2.0</version>
-
- <components>
- <component>
- <name>GANGLIA_SERVER</name>
- <category>MASTER</category>
- <cardinality>1</cardinality>
- <commandScript>
- <script>scripts/ganglia_server.py</script>
- <scriptType>PYTHON</scriptType>
- <timeout>600</timeout>
- </commandScript>
- </component>
-
- <component>
- <name>GANGLIA_MONITOR</name>
- <category>SLAVE</category>
- <cardinality>ALL</cardinality>
- <auto-deploy>
- <enabled>true</enabled>
- </auto-deploy>
- <commandScript>
- <script>scripts/ganglia_monitor.py</script>
- <scriptType>PYTHON</scriptType>
- <timeout>600</timeout>
- </commandScript>
- </component>
- </components>
- <osSpecifics>
- <osSpecific>
- <osFamily>any</osFamily>
- <packages>
- <package>
- <name>python-rrdtool</name>
- </package>
- </packages>
- </osSpecific>
- <osSpecific>
- <osFamily>redhat5,amazon2015,redhat6,suse11</osFamily>
- <packages>
- <package>
- <name>libganglia-3.5.0-99</name>
- </package>
- <package>
- <name>ganglia-devel-3.5.0-99</name>
- </package>
- <package>
- <name>ganglia-gmetad-3.5.0-99</name>
- </package>
- <package>
- <name>ganglia-web-3.5.7-99.noarch</name>
- </package>
- <package>
- <name>python-rrdtool.x86_64</name>
- </package>
- <package>
- <name>ganglia-gmond-3.5.0-99</name>
- </package>
- <package>
- <name>ganglia-gmond-modules-python-3.5.0-99</name>
- </package>
- </packages>
- </osSpecific>
- <osSpecific>
- <osFamily>ubuntu12</osFamily>
- <packages>
- <package>
- <name>gmetad</name>
- </package>
- <package>
- <name>ganglia-webfrontend</name>
- </package>
- <package>
- <name>ganglia-monitor-python</name>
- </package>
- <package>
- <name>rrdcached</name>
- </package>
- </packages>
- </osSpecific>
- <osSpecific>
- <osFamily>suse11</osFamily>
- <packages>
- <package>
- <name>apache2</name>
- </package>
- <package>
- <name>apache2?mod_php*</name>
- </package>
- </packages>
- </osSpecific>
- <osSpecific>
- <osFamily>redhat5,amazon2015,redhat6</osFamily>
- <packages>
- <package>
- <name>httpd</name>
- </package>
- </packages>
- </osSpecific>
- </osSpecifics>
- <configuration-dependencies>
- <config-type>global</config-type>
- </configuration-dependencies>
- </service>
- </services>
-</metainfo>
diff --git a/ambari-funtest/src/test/resources/stacks/HDP/2.0.5/services/GANGLIA/metainfo.xml b/ambari-funtest/src/test/resources/stacks/HDP/2.0.5/services/GANGLIA/metainfo.xml
deleted file mode 100644
index d8d5263975..0000000000
--- a/ambari-funtest/src/test/resources/stacks/HDP/2.0.5/services/GANGLIA/metainfo.xml
+++ /dev/null
@@ -1,130 +0,0 @@
-<?xml version="1.0"?>
-<!--
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-<metainfo>
- <schemaVersion>2.0</schemaVersion>
- <services>
- <service>
- <name>GANGLIA</name>
- <comment>Ganglia Metrics Collection system (<a href="http://oss.oetiker.ch/rrdtool/" target="_blank">RRDTool</a> will be installed too)</comment>
- <version>3.5.0</version>
- <components>
- <component>
- <name>GANGLIA_SERVER</name>
- <category>MASTER</category>
- <cardinality>1</cardinality>
- <versionAdvertised>false</versionAdvertised>
- <commandScript>
- <script>scripts/ganglia_server.py</script>
- <scriptType>PYTHON</scriptType>
- <timeout>600</timeout>
- </commandScript>
- </component>
-
- <component>
- <name>GANGLIA_MONITOR</name>
- <category>SLAVE</category>
- <cardinality>ALL</cardinality>
- <versionAdvertised>false</versionAdvertised>
- <auto-deploy>
- <enabled>true</enabled>
- </auto-deploy>
- <commandScript>
- <script>scripts/ganglia_monitor.py</script>
- <scriptType>PYTHON</scriptType>
- <timeout>600</timeout>
- </commandScript>
- </component>
- </components>
- <osSpecifics>
- <osSpecific>
- <osFamily>any</osFamily>
- <packages>
- <package>
- <name>python-rrdtool</name>
- </package>
- </packages>
- </osSpecific>
- <osSpecific>
- <osFamily>redhat5,amazon2015,redhat6,suse11</osFamily>
- <packages>
- <package>
- <name>libganglia-3.5.0-99</name>
- </package>
- <package>
- <name>ganglia-devel-3.5.0-99</name>
- </package>
- <package>
- <name>ganglia-gmetad-3.5.0-99</name>
- </package>
- <package>
- <name>ganglia-web-3.5.7-99.noarch</name>
- </package>
- <package>
- <name>python-rrdtool.x86_64</name>
- </package>
- <package>
- <name>ganglia-gmond-3.5.0-99</name>
- </package>
- <package>
- <name>ganglia-gmond-modules-python-3.5.0-99</name>
- </package>
- </packages>
- </osSpecific>
- <osSpecific>
- <osFamily>ubuntu12</osFamily>
- <packages>
- <package>
- <name>gmetad</name>
- </package>
- <package>
- <name>ganglia-webfrontend</name>
- </package>
- <package>
- <name>ganglia-monitor-python</name>
- </package>
- <package>
- <name>rrdcached</name>
- </package>
- </packages>
- </osSpecific>
- <osSpecific>
- <osFamily>suse11</osFamily>
- <packages>
- <package>
- <name>apache2</name>
- </package>
- <package>
- <name>apache2?mod_php*</name>
- </package>
- </packages>
- </osSpecific>
- <osSpecific>
- <osFamily>redhat5,amazon2015,redhat6</osFamily>
- <packages>
- <package>
- <name>httpd</name>
- </package>
- </packages>
- </osSpecific>
- </osSpecifics>
- <configuration-dependencies>
- <config-type>global</config-type>
- </configuration-dependencies>
- </service>
- </services>
-</metainfo>
diff --git a/ambari-funtest/src/test/resources/stacks/HDP/2.0.5/services/HBASE/metrics.json b/ambari-funtest/src/test/resources/stacks/HDP/2.0.5/services/HBASE/metrics.json
deleted file mode 100644
index 955c90a778..0000000000
--- a/ambari-funtest/src/test/resources/stacks/HDP/2.0.5/services/HBASE/metrics.json
+++ /dev/null
@@ -1,3206 +0,0 @@
-{
- "HBASE_MASTER": {
- "HostComponent": [
- {
- "type": "ganglia",
- "metrics": {
- "default": {
- "metrics/rpc/deleteTable/aboveOneSec/_num_ops": {
- "metric": "rpc.rpc.deleteTable.aboveOneSec._num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/closeRegion_num_ops": {
- "metric": "rpc.rpc.closeRegion_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/regionServerStartup/aboveOneSec/_avg_time": {
- "metric": "rpc.rpc.regionServerStartup.aboveOneSec._avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/unassign_num_ops": {
- "metric": "rpc.rpc.unassign_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/getClusterStatus/aboveOneSec/_avg_time": {
- "metric": "rpc.rpc.getClusterStatus.aboveOneSec._avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/modifyTable_num_ops": {
- "metric": "rpc.rpc.modifyTable_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/getProtocolVersion_avg_time": {
- "metric": "rpc.rpc.getProtocolVersion_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/unassign/aboveOneSec/_num_ops": {
- "metric": "rpc.rpc.unassign.aboveOneSec._num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/load/load_one": {
- "metric": "load_one",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/getClosestRowBefore_num_ops": {
- "metric": "rpc.rpc.getClosestRowBefore_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/jvm/memNonHeapUsedM": {
- "metric": "jvm.JvmMetrics.MemNonHeapUsedM",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/assign/aboveOneSec/_avg_time": {
- "metric": "rpc.rpc.assign.aboveOneSec._avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/replicateLogEntries_num_ops": {
- "metric": "rpc.rpc.replicateLogEntries_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/createTable/aboveOneSec/_avg_time": {
- "metric": "rpc.rpc.createTable.aboveOneSec._avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/lockRow_num_ops": {
- "metric": "rpc.rpc.lockRow_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/flushRegion_avg_time": {
- "metric": "rpc.rpc.flushRegion_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/multi_avg_time": {
- "metric": "rpc.rpc.multi_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/memory/swap_total": {
- "metric": "swap_total",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/stopMaster_num_ops": {
- "metric": "rpc.rpc.stopMaster_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/process/proc_total": {
- "metric": "proc_total",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/balance_avg_time": {
- "metric": "rpc.rpc.balance_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/disk/part_max_used": {
- "metric": "part_max_used",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/splitRegion_num_ops": {
- "metric": "rpc.rpc.splitRegion_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/modifyColumn_avg_time": {
- "metric": "rpc.rpc.modifyColumn_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/balance_num_ops": {
- "metric": "rpc.rpc.balance_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/shutdown_num_ops": {
- "metric": "rpc.rpc.shutdown_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/flushRegion_num_ops": {
- "metric": "rpc.rpc.flushRegion_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/cpu/cpu_idle": {
- "metric": "cpu_idle",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/modifyTable/aboveOneSec/_num_ops": {
- "metric": "rpc.rpc.modifyTable.aboveOneSec._num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/regionServerReport/aboveOneSec/_avg_time": {
- "metric": "rpc.rpc.regionServerReport.aboveOneSec._avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/getProtocolSignature/aboveOneSec/_num_ops": {
- "metric": "rpc.rpc.getProtocolSignature.aboveOneSec._num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/get_num_ops": {
- "metric": "rpc.rpc.get_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/stopMaster_avg_time": {
- "metric": "rpc.rpc.stopMaster_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/network/bytes_in": {
- "metric": "bytes_in",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/reportRSFatalError/aboveOneSec/_num_ops": {
- "metric": "rpc.rpc.reportRSFatalError.aboveOneSec._num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/jvm/memNonHeapCommittedM": {
- "metric": "jvm.JvmMetrics.MemNonHeapCommittedM",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/shutdown/aboveOneSec/_avg_time": {
- "metric": "rpc.rpc.shutdown.aboveOneSec._avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/getRegionInfo_avg_time": {
- "metric": "rpc.rpc.getRegionInfo_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/balanceSwitch/aboveOneSec/_num_ops": {
- "metric": "rpc.rpc.balanceSwitch.aboveOneSec._num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/disableTable/aboveOneSec/_num_ops": {
- "metric": "rpc.rpc.disableTable.aboveOneSec._num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/reportRSFatalError/aboveOneSec/_avg_time": {
- "metric": "rpc.rpc.reportRSFatalError.aboveOneSec._avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/assign/aboveOneSec/_num_ops": {
- "metric": "rpc.rpc.assign.aboveOneSec._num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/balance/aboveOneSec/_num_ops": {
- "metric": "rpc.rpc.balance.aboveOneSec._num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/enableTable_num_ops": {
- "metric": "rpc.rpc.enableTable_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/lockRow_avg_time": {
- "metric": "rpc.rpc.lockRow_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/addColumn_avg_time": {
- "metric": "rpc.rpc.addColumn_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/reportRSFatalError_num_ops": {
- "metric": "rpc.rpc.reportRSFatalError_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/cpu/cpu_wio": {
- "metric": "cpu_wio",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/hbase/master/splitTime_num_ops": {
- "metric": "master.FileSystem.HlogSplitTime_num_ops",
- "pointInTime": false,
- "temporal": true
- },
- "metrics/rpc/getAlterStatus/aboveOneSec/_num_ops": {
- "metric": "rpc.rpc.getAlterStatus.aboveOneSec._num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/deleteColumn/aboveOneSec/_avg_time": {
- "metric": "rpc.rpc.deleteColumn.aboveOneSec._avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/reportRSFatalError_avg_time": {
- "metric": "rpc.rpc.reportRSFatalError_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/rpcAuthenticationSuccesses": {
- "metric": "rpc.rpc.rpcAuthenticationSuccesses",
- "pointInTime": false,
- "temporal": true
- },
- "metrics/network/pkts_in": {
- "metric": "pkts_in",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/disableTable_avg_time": {
- "metric": "rpc.rpc.disableTable_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/memory/mem_total": {
- "metric": "mem_total",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/jvm/memHeapCommittedM": {
- "metric": "jvm.JvmMetrics.MemHeapCommittedM",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/jvm/threadsRunnable": {
- "metric": "jvm.JvmMetrics.ThreadsRunnable",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/openRegion_avg_time": {
- "metric": "rpc.rpc.openRegion_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/enableTable/aboveOneSec/_num_ops": {
- "metric": "rpc.rpc.enableTable.aboveOneSec._num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/jvm/threadsNew": {
- "metric": "jvm.JvmMetrics.ThreadsNew",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/getClusterStatus_num_ops": {
- "metric": "rpc.rpc.getClusterStatus_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/getHTableDescriptors_avg_time": {
- "metric": "rpc.rpc.getHTableDescriptors_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/rpcAuthorizationFailures": {
- "metric": "rpc.rpc.rpcAuthorizationFailures",
- "pointInTime": false,
- "temporal": true
- },
- "metrics/rpc/regionServerReport_avg_time": {
- "metric": "rpc.rpc.regionServerReport_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/deleteColumn_num_ops": {
- "metric": "rpc.rpc.deleteColumn_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/RpcQueueTime_avg_time": {
- "metric": "rpc.rpc.RpcQueueTime_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/increment_num_ops": {
- "metric": "rpc.rpc.increment_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/getAlterStatus_num_ops": {
- "metric": "rpc.rpc.getAlterStatus_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/modifyColumn_num_ops": {
- "metric": "rpc.rpc.modifyColumn_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/next_avg_time": {
- "metric": "rpc.rpc.next_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/deleteColumn/aboveOneSec/_num_ops": {
- "metric": "rpc.rpc.deleteColumn.aboveOneSec._num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/RpcSlowResponse_avg_time": {
- "metric": "rpc.rpc.RpcSlowResponse_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/getClusterStatus/aboveOneSec/_num_ops": {
- "metric": "rpc.rpc.getClusterStatus.aboveOneSec._num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/ReceivedBytes": {
- "metric": "rpc.rpc.ReceivedBytes",
- "pointInTime": false,
- "temporal": true
- },
- "metrics/cpu/cpu_nice": {
- "metric": "cpu_nice",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/bulkLoadHFiles_num_ops": {
- "metric": "rpc.rpc.bulkLoadHFiles_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/unassign_avg_time": {
- "metric": "rpc.rpc.unassign_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/NumOpenConnections": {
- "metric": "rpc.rpc.NumOpenConnections",
- "pointInTime": false,
- "temporal": true
- },
- "metrics/jvm/gcTimeMillis": {
- "metric": "jvm.JvmMetrics.GcTimeMillis",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/regionServerReport/aboveOneSec/_num_ops": {
- "metric": "rpc.rpc.regionServerReport.aboveOneSec._num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/getProtocolVersion/aboveOneSec/_avg_time": {
- "metric": "rpc.rpc.getProtocolVersion.aboveOneSec._avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/jvm/threadsTerminated": {
- "metric": "jvm.JvmMetrics.ThreadsTerminated",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/network/bytes_out": {
- "metric": "bytes_out",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/unassign/aboveOneSec/_avg_time": {
- "metric": "rpc.rpc.unassign.aboveOneSec._avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/balanceSwitch_avg_time": {
- "metric": "rpc.rpc.balanceSwitch_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/load/load_five": {
- "metric": "load_five",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/boottime": {
- "metric": "boottime",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/compactRegion_avg_time": {
- "metric": "rpc.rpc.compactRegion_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/openRegion_num_ops": {
- "metric": "rpc.rpc.openRegion_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/callQueueLen": {
- "metric": "rpc.rpc.callQueueLen",
- "pointInTime": false,
- "temporal": true
- },
- "metrics/rpc/offline/aboveOneSec/_num_ops": {
- "metric": "rpc.rpc.offline.aboveOneSec._num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/getProtocolSignature/aboveOneSec/_avg_time": {
- "metric": "rpc.rpc.getProtocolSignature.aboveOneSec._avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/execCoprocessor_num_ops": {
- "metric": "rpc.rpc.execCoprocessor_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/getOnlineRegions_avg_time": {
- "metric": "rpc.rpc.getOnlineRegions_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/addColumn/aboveOneSec/_avg_time": {
- "metric": "rpc.rpc.addColumn.aboveOneSec._avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/disableTable/aboveOneSec/_avg_time": {
- "metric": "rpc.rpc.disableTable.aboveOneSec._avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/incrementColumnValue_avg_time": {
- "metric": "rpc.rpc.incrementColumnValue_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/RpcProcessingTime_num_ops": {
- "metric": "rpc.rpc.RpcProcessingTime_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/move_num_ops": {
- "metric": "rpc.rpc.move_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/stop_num_ops": {
- "metric": "rpc.rpc.stop_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/replicateLogEntries_avg_time": {
- "metric": "rpc.rpc.replicateLogEntries_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/deleteTable_num_ops": {
- "metric": "rpc.rpc.deleteTable_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/jvm/logError": {
- "metric": "jvm.JvmMetrics.LogError",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/get_avg_time": {
- "metric": "rpc.rpc.get_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/multi_num_ops": {
- "metric": "rpc.rpc.multi_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/getBlockCacheColumnFamilySummaries_num_ops": {
- "metric": "rpc.rpc.getBlockCacheColumnFamilySummaries_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/deleteColumn_avg_time": {
- "metric": "rpc.rpc.deleteColumn_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/bulkLoadHFiles_avg_time": {
- "metric": "rpc.rpc.bulkLoadHFiles_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/exists_avg_time": {
- "metric": "rpc.rpc.exists_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/offline_avg_time": {
- "metric": "rpc.rpc.offline_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/unlockRow_avg_time": {
- "metric": "rpc.rpc.unlockRow_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/delete_num_ops": {
- "metric": "rpc.rpc.delete_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/exists_num_ops": {
- "metric": "rpc.rpc.exists_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/regionServerStartup_avg_time": {
- "metric": "rpc.rpc.regionServerStartup_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/checkAndDelete_num_ops": {
- "metric": "rpc.rpc.checkAndDelete_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/closeRegion_avg_time": {
- "metric": "rpc.rpc.closeRegion_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/modifyColumn/aboveOneSec/_avg_time": {
- "metric": "rpc.rpc.modifyColumn.aboveOneSec._avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/enableTable/aboveOneSec/_avg_time": {
- "metric": "rpc.rpc.enableTable.aboveOneSec._avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/getProtocolSignature_avg_time": {
- "metric": "rpc.rpc.getProtocolSignature_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/createTable_num_ops": {
- "metric": "rpc.rpc.createTable_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/assign_avg_time": {
- "metric": "rpc.rpc.assign_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/execCoprocessor_avg_time": {
- "metric": "rpc.rpc.execCoprocessor_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/isMasterRunning/aboveOneSec/_avg_time": {
- "metric": "rpc.rpc.isMasterRunning.aboveOneSec._avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/close_avg_time": {
- "metric": "rpc.rpc.close_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/process/proc_run": {
- "metric": "proc_run",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/jvm/threadsBlocked": {
- "metric": "jvm.JvmMetrics.ThreadsBlocked",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/getHServerInfo_num_ops": {
- "metric": "rpc.rpc.getHServerInfo_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/RpcQueueTime_num_ops": {
- "metric": "rpc.rpc.RpcQueueTime_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/stop_avg_time": {
- "metric": "rpc.rpc.stop_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/rollHLogWriter_avg_time": {
- "metric": "rpc.rpc.rollHLogWriter_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/modifyTable/aboveOneSec/_avg_time": {
- "metric": "rpc.rpc.modifyTable.aboveOneSec._avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/balance/aboveOneSec/_avg_time": {
- "metric": "rpc.rpc.balance.aboveOneSec._avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/isMasterRunning_avg_time": {
- "metric": "rpc.rpc.isMasterRunning_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/hbase/master/splitSize_num_ops": {
- "metric": "master.FileSystem.HlogSplitSize_num_ops",
- "pointInTime": false,
- "temporal": true
- },
- "metrics/rpc/incrementColumnValue_num_ops": {
- "metric": "rpc.rpc.incrementColumnValue_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/modifyColumn/aboveOneSec/_num_ops": {
- "metric": "rpc.rpc.modifyColumn.aboveOneSec._num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/hbase/master/splitTime_avg_time": {
- "metric": "master.FileSystem.HlogSplitSize_mean",
- "pointInTime": false,
- "temporal": true
- },
- "metrics/cpu/cpu_aidle": {
- "metric": "cpu_aidle",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/memory/mem_free": {
- "metric": "mem_free",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/assign_num_ops": {
- "metric": "rpc.rpc.assign_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/enableTable_avg_time": {
- "metric": "rpc.rpc.enableTable_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/move/aboveOneSec/_avg_time": {
- "metric": "rpc.rpc.move.aboveOneSec._avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/network/pkts_out": {
- "metric": "pkts_out",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/close_num_ops": {
- "metric": "rpc.rpc.close_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/memory/mem_cached": {
- "metric": "mem_cached",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/addColumn/aboveOneSec/_num_ops": {
- "metric": "rpc.rpc.addColumn.aboveOneSec._num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/disk/disk_total": {
- "metric": "disk_total",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/shutdown/aboveOneSec/_num_ops": {
- "metric": "rpc.rpc.shutdown.aboveOneSec._num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/openScanner_avg_time": {
- "metric": "rpc.rpc.openScanner_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/RpcSlowResponse_num_ops": {
- "metric": "rpc.rpc.RpcSlowResponse_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/deleteTable_avg_time": {
- "metric": "rpc.rpc.deleteTable_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/hbase/master/splitSize_avg_time": {
- "metric": "master.FileSystem.HlogSplitSize_mean",
- "pointInTime": false,
- "temporal": true
- },
- "metrics/rpc/delete_avg_time": {
- "metric": "rpc.rpc.delete_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/cpu/cpu_speed": {
- "metric": "cpu_speed",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/getClosestRowBefore_avg_time": {
- "metric": "rpc.rpc.getClosestRowBefore_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/hbase/master/cluster_requests": {
- "metric": "master.Server.clusterRequests",
- "pointInTime": false,
- "temporal": true
- },
- "metrics/rpc/stopMaster/aboveOneSec/_num_ops": {
- "metric": "rpc.rpc.stopMaster.aboveOneSec._num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/stopMaster/aboveOneSec/_avg_time": {
- "metric": "rpc.rpc.stopMaster.aboveOneSec._avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/RpcProcessingTime_avg_time": {
- "metric": "rpc.rpc.RpcProcessingTime_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/put_num_ops": {
- "metric": "rpc.rpc.put_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/rpcAuthenticationFailures": {
- "metric": "rpc.rpc.rpcAuthenticationFailures",
- "pointInTime": false,
- "temporal": true
- },
- "metrics/rpc/move_avg_time": {
- "metric": "rpc.rpc.move_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/getClusterStatus_avg_time": {
- "metric": "rpc.rpc.getClusterStatus_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/regionServerStartup/aboveOneSec/_num_ops": {
- "metric": "rpc.rpc.regionServerStartup.aboveOneSec._num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/getHTableDescriptors/aboveOneSec/_num_ops": {
- "metric": "rpc.rpc.getHTableDescriptors.aboveOneSec._num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/modifyTable_avg_time": {
- "metric": "rpc.rpc.modifyTable_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/checkAndPut_avg_time": {
- "metric": "rpc.rpc.checkAndPut_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/put_avg_time": {
- "metric": "rpc.rpc.put_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/getHTableDescriptors/aboveOneSec/_avg_time": {
- "metric": "rpc.rpc.getHTableDescriptors.aboveOneSec._avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/SentBytes": {
- "metric": "rpc.rpc.SentBytes",
- "pointInTime": false,
- "temporal": true
- },
- "metrics/rpc/createTable_avg_time": {
- "metric": "rpc.rpc.createTable_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/addColumn_num_ops": {
- "metric": "rpc.rpc.addColumn_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/getAlterStatus_avg_time": {
- "metric": "rpc.rpc.getAlterStatus_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/getHTableDescriptors_num_ops": {
- "metric": "rpc.rpc.getHTableDescriptors_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/getAlterStatus/aboveOneSec/_avg_time": {
- "metric": "rpc.rpc.getAlterStatus.aboveOneSec._avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/getRegionInfo_num_ops": {
- "metric": "rpc.rpc.getRegionInfo_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/jvm/maxMemoryM": {
- "metric": "jvm.metrics.maxMemoryM",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/jvm/logWarn": {
- "metric": "jvm.JvmMetrics.LogWarn",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/compactRegion_num_ops": {
- "metric": "rpc.rpc.compactRegion_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/balanceSwitch/aboveOneSec/_avg_time": {
- "metric": "rpc.rpc.balanceSwitch.aboveOneSec._avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/jvm/threadsTimedWaiting": {
- "metric": "jvm.JvmMetrics.ThreadsTimedWaiting",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/jvm/gcCount": {
- "metric": "jvm.JvmMetrics.GcCount",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/deleteTable/aboveOneSec/_avg_time": {
- "metric": "rpc.rpc.deleteTable.aboveOneSec._avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/getOnlineRegions_num_ops": {
- "metric": "rpc.rpc.getOnlineRegions_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/jvm/memHeapUsedM": {
- "metric": "jvm.JvmMetrics.MemHeapUsedM",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/getProtocolVersion_num_ops": {
- "metric": "rpc.rpc.getProtocolVersion_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/unlockRow_num_ops": {
- "metric": "rpc.rpc.unlockRow_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/disk/disk_free": {
- "metric": "disk_free",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/jvm/threadsWaiting": {
- "metric": "jvm.JvmMetrics.ThreadsWaiting",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/memory/mem_buffers": {
- "metric": "mem_buffers",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/disableTable_num_ops": {
- "metric": "rpc.rpc.disableTable_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/shutdown_avg_time": {
- "metric": "rpc.rpc.shutdown_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/openScanner_num_ops": {
- "metric": "rpc.rpc.openScanner_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/offline/aboveOneSec/_avg_time": {
- "metric": "rpc.rpc.offline.aboveOneSec._avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/regionServerStartup_num_ops": {
- "metric": "rpc.rpc.regionServerStartup_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/cpu/cpu_user": {
- "metric": "cpu_user",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/memory/swap_free": {
- "metric": "swap_free",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/cpu/cpu_system": {
- "metric": "cpu_system",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/regionServerReport_num_ops": {
- "metric": "rpc.rpc.regionServerReport_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/openRegions_avg_time": {
- "metric": "rpc.rpc.openRegions_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/isMasterRunning/aboveOneSec/_num_ops": {
- "metric": "rpc.rpc.isMasterRunning.aboveOneSec._num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/next_num_ops": {
- "metric": "rpc.rpc.next_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/createTable/aboveOneSec/_num_ops": {
- "metric": "rpc.rpc.createTable.aboveOneSec._num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/getBlockCacheColumnFamilySummaries_avg_time": {
- "metric": "rpc.rpc.getBlockCacheColumnFamilySummaries_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/checkAndDelete_avg_time": {
- "metric": "rpc.rpc.checkAndDelete_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/load/load_fifteen": {
- "metric": "load_fifteen",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/getHServerInfo_avg_time": {
- "metric": "rpc.rpc.getHServerInfo_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/jvm/logInfo": {
- "metric": "jvm.JvmMetrics.LogInfo",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/move/aboveOneSec/_num_ops": {
- "metric": "rpc.rpc.move.aboveOneSec._num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/isMasterRunning_num_ops": {
- "metric": "rpc.rpc.isMasterRunning_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/balanceSwitch_num_ops": {
- "metric": "rpc.rpc.balanceSwitch_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/offline_num_ops": {
- "metric": "rpc.rpc.offline_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/memory/mem_shared": {
- "metric": "mem_shared",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/getProtocolSignature_num_ops": {
- "metric": "rpc.rpc.getProtocolSignature_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/getProtocolVersion/aboveOneSec/_num_ops": {
- "metric": "rpc.rpc.getProtocolVersion.aboveOneSec._num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/cpu/cpu_num": {
- "metric": "cpu_num",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/rollHLogWriter_num_ops": {
- "metric": "rpc.rpc.rollHLogWriter_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/rpcAuthorizationSuccesses": {
- "metric": "rpc.rpc.rpcAuthorizationSuccesses",
- "pointInTime": false,
- "temporal": true
- },
- "metrics/rpc/openRegions_num_ops": {
- "metric": "rpc.rpc.openRegions_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/jvm/logFatal": {
- "metric": "jvm.JvmMetrics.LogFatal",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/splitRegion_avg_time": {
- "metric": "rpc.rpc.splitRegion_avg_time",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/checkAndPut_num_ops": {
- "metric": "rpc.rpc.checkAndPut_num_ops",
- "pointInTime": true,
- "temporal": true
- },
- "metrics/rpc/increment_avg_time": {
- "metric": "rpc.rpc.increment_avg_time",
- "pointInTime": true,
- "temporal": true
- }
- }
- }
- },
- {
- "type": "jmx",
- "metrics": {
- "default": {
- "metrics/rpc/regionServerReport.aboveOneSec.MaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.regionServerReport.aboveOneSec.MaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/jvm/memMaxM": {
- "metric": "Hadoop:service=HBase,name=JvmMetrics.MemMaxM",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/reportRSFatalError.aboveOneSec.AvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.reportRSFatalError.aboveOneSec.AvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/closeRegionMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.closeRegionMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/isMasterRunningAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.isMasterRunningAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/RpcQueueTimeAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.RpcQueueTimeAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/deleteTableNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.deleteTableNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/splitRegionAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.splitRegionAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getHTableDescriptorsMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getHTableDescriptorsMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getClusterStatus.aboveOneSec.MaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getClusterStatus.aboveOneSec.MaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/splitRegionNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.splitRegionNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getProtocolVersionAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getProtocolVersionAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/modifyColumnMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.modifyColumnMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getBlockCacheColumnFamilySummariesMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getBlockCacheColumnFamilySummariesMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getClosestRowBeforeMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getClosestRowBeforeMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getProtocolSignatureNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getProtocolSignatureNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/RpcSlowResponseMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.RpcSlowResponseMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/hbase/master/AverageLoad": {
- "metric": "Hadoop:service=HBase,name=Master,sub=Server.averageLoad",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/openScannerNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.openScannerNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/deleteTable.aboveOneSec.MinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.deleteTable.aboveOneSec.MinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/deleteColumn.aboveOneSec.MaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.deleteColumn.aboveOneSec.MaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getCompactionStateMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getCompactionStateMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/stopMaster.aboveOneSec.MaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.stopMaster.aboveOneSec.MaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/regionServerReport.aboveOneSec.MinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.regionServerReport.aboveOneSec.MinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getHServerInfoAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getHServerInfoAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/modifyColumn.aboveOneSec.AvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.modifyColumn.aboveOneSec.AvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/rollHLogWriterMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.rollHLogWriterMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/offlineMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.offlineMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getProtocolSignatureMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getProtocolSignatureMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/hbase/master/ServerName": {
- "metric": "Hadoop:service=HBase,name=Master,sub=Server.tag.serverName",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getHServerInfoMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getHServerInfoMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/hbase/master/splitSizeMaxTime": {
- "metric": "Hadoop:service=HBase,name=Master,sub=Server.HlogSplitSize_max",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/execCoprocessorAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.execCoprocessorAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/RpcProcessingTimeMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.RpcProcessingTimeMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getClusterStatusMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getClusterStatusMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/offline.aboveOneSec.NumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.offline.aboveOneSec.NumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/hbase/master/ZookeeperQuorum": {
- "metric": "Hadoop:service=HBase,name=Master,sub=Server.tag.zookeeperQuorum",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/hbase/master/hdfsDate": {
- "metric": "hadoop:service=HBase,name=Info.hdfsDate",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/createTable.aboveOneSec.MaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.createTable.aboveOneSec.MaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/offlineMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.offlineMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/offline.aboveOneSec.MaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.offline.aboveOneSec.MaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/balanceNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.balanceNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getClosestRowBeforeNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getClosestRowBeforeNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/modifyColumnNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.modifyColumnNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getLastFlushTimeNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getLastFlushTimeNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/stopMaster.aboveOneSec.NumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.stopMaster.aboveOneSec.NumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/deleteTable.aboveOneSec.NumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.deleteTable.aboveOneSec.NumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/hbase/master/hdfsUrl": {
- "metric": "hadoop:service=HBase,name=Info.hdfsUrl",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/jvm/NonHeapMemoryMax": {
- "metric": "java.lang:type=Memory.NonHeapMemoryUsage[max]",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/multiMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.multiMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/hbase/master/revision": {
- "metric": "hadoop:service=HBase,name=Info.revision",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/modifyColumnMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.modifyColumnMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/addColumnMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.addColumnMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getHTableDescriptorsNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getHTableDescriptorsNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/RpcQueueTimeMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.RpcQueueTimeMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/multiNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.multiNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getProtocolVersion.aboveOneSec.AvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getProtocolVersion.aboveOneSec.AvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/offline.aboveOneSec.MinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.offline.aboveOneSec.MinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/deleteTable.aboveOneSec.MaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.deleteTable.aboveOneSec.MaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/enableTableMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.enableTableMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/regionServerReportMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.regionServerReportMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/reportRSFatalErrorNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.reportRSFatalErrorNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/addColumn.aboveOneSec.NumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.addColumn.aboveOneSec.NumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/unassign.aboveOneSec.AvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.unassign.aboveOneSec.AvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/reportRSFatalErrorMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.reportRSFatalErrorMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/disableTable.aboveOneSec.MaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.disableTable.aboveOneSec.MaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/existsAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.existsAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/hbase/master/MasterActiveTime": {
- "metric": "Hadoop:service=HBase,name=Master,sub=Server.masterActiveTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getBlockCacheColumnFamilySummariesNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getBlockCacheColumnFamilySummariesNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/createTable.aboveOneSec.MinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.createTable.aboveOneSec.MinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/rpcAuthorizationFailures": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.rpcAuthorizationFailures",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/hbase/master/hdfsUser": {
- "metric": "hadoop:service=HBase,name=Info.hdfsUser",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/regionServerStartupAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.regionServerStartupAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/deleteMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.deleteMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/regionServerStartupNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.regionServerStartupNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/deleteColumnNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.deleteColumnNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/hbase/master/version": {
- "metric": "hadoop:service=HBase,name=Info.version",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/hbase/master/splitTimeMaxTime": {
- "metric": "Hadoop:service=HBase,name=Master,sub=Server.HlogSplitTime_max",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/synchronousBalanceSwitchNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.synchronousBalanceSwitchNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/stopMasterNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.stopMasterNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/hbase/master/splitTimeNumOps": {
- "metric": "hadoop:service=Master,name=MasterStatistics.splitTimeNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/reportRSFatalErrorAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.reportRSFatalErrorAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/replicateLogEntriesNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.replicateLogEntriesNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/multiMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.multiMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/deleteColumnMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.deleteColumnMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getProtocolSignature.aboveOneSec.AvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getProtocolSignature.aboveOneSec.AvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getLastFlushTimeMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getLastFlushTimeMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/disableTable.aboveOneSec.MinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.disableTable.aboveOneSec.MinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/NumOpenConnections": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.NumOpenConnections",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/RpcQueueTimeNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.RpcQueueTimeNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/regionServerReportMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.regionServerReportMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/hbase/master/IsActiveMaster": {
- "metric": "Hadoop:service=HBase,name=Master,sub=Server.tag.isActiveMaster",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/bulkLoadHFilesMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.bulkLoadHFilesMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/synchronousBalanceSwitch.aboveOneSec.NumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.synchronousBalanceSwitch.aboveOneSec.NumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/hbase/master/MasterStartTime": {
- "metric": "Hadoop:service=HBase,name=Master,sub=Server.masterStartTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/balanceSwitchMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.balanceSwitchMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/unlockRowMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.unlockRowMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/incrementNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.incrementNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/reportRSFatalError.aboveOneSec.MinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.reportRSFatalError.aboveOneSec.MinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/balanceSwitch.aboveOneSec.MinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.balanceSwitch.aboveOneSec.MinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/execCoprocessorMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.execCoprocessorMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/enableTable.aboveOneSec.MaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.enableTable.aboveOneSec.MaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/putMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.putMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/flushRegionMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.flushRegionMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/nextNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.nextNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getOnlineRegionsAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getOnlineRegionsAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/createTableAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.createTableAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/deleteNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.deleteNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/modifyTable.aboveOneSec.AvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.modifyTable.aboveOneSec.AvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getAlterStatusAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getAlterStatusAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/assign.aboveOneSec.NumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.assign.aboveOneSec.NumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/regionServerStartup.aboveOneSec.MinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.regionServerStartup.aboveOneSec.MinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/balanceSwitch.aboveOneSec.AvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.balanceSwitch.aboveOneSec.AvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/isMasterRunningMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.isMasterRunningMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/existsNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.existsNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/compactRegionMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.compactRegionMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/bulkLoadHFilesMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.bulkLoadHFilesMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/rollHLogWriterNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.rollHLogWriterNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/unlockRowAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.unlockRowAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/openRegionsNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.openRegionsNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/checkAndDeleteMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.checkAndDeleteMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/stopMaster.aboveOneSec.AvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.stopMaster.aboveOneSec.AvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/splitRegionMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.splitRegionMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getHTableDescriptorsMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getHTableDescriptorsMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/deleteColumn.aboveOneSec.MinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.deleteColumn.aboveOneSec.MinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/moveMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.moveMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/shutdown.aboveOneSec.AvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.shutdown.aboveOneSec.AvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/appendNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.appendNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/appendAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.appendAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getClusterStatusNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getClusterStatusNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/RpcSlowResponseNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.RpcSlowResponseNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/hbase/master/splitSize_num_ops": {
- "metric": "Hadoop:service=HBase,name=Master,sub=Server.HlogSplitSize_num_ops",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getLastFlushTimeMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getLastFlushTimeMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/checkAndPutNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.checkAndPutNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/hbase/master/splitTime_avg_time": {
- "metric": "Hadoop:service=HBase,name=Master,sub=Server.HlogSplitTime_mean",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/createTableMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.createTableMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/incrementMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.incrementMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getAlterStatus.aboveOneSec.MaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getAlterStatus.aboveOneSec.MaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getCompactionStateMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getCompactionStateMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/hbase/master/splitTimeAvgTime": {
- "metric": "hadoop:service=Master,name=MasterStatistics.splitTimeAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getStoreFileListMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getStoreFileListMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getProtocolSignature.aboveOneSec.MinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getProtocolSignature.aboveOneSec.MinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/RpcProcessingTimeAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.RpcProcessingTimeAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/incrementColumnValueNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.incrementColumnValueNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/enableTable.aboveOneSec.MinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.enableTable.aboveOneSec.MinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/stopMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.stopMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/multiAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.multiAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/createTable.aboveOneSec.NumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.createTable.aboveOneSec.NumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/shutdownAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.shutdownAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/openRegionMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.openRegionMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getBlockCacheColumnFamilySummariesMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getBlockCacheColumnFamilySummariesMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/deleteColumn.aboveOneSec.NumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.deleteColumn.aboveOneSec.NumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/modifyColumn.aboveOneSec.MinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.modifyColumn.aboveOneSec.MinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/disableTableNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.disableTableNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getProtocolVersion.aboveOneSec.MaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getProtocolVersion.aboveOneSec.MaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/replicateLogEntriesAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.replicateLogEntriesAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/hbase/master/cluster_requests": {
- "metric": "Hadoop:service=HBase,name=Master,sub=Server.clusterRequests",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getHServerInfoMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getHServerInfoMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getClusterStatusMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getClusterStatusMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/rpcAuthenticationFailures": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.rpcAuthenticationFailures",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/hbase/master/Coprocessors": {
- "metric": "hadoop:service=Master,name=Master.Coprocessors",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/unlockRowNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.unlockRowNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getAlterStatus.aboveOneSec.AvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getAlterStatus.aboveOneSec.AvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/regionServerStartup.aboveOneSec.AvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.regionServerStartup.aboveOneSec.AvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/balanceMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.balanceMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/modifyColumn.aboveOneSec.MaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.modifyColumn.aboveOneSec.MaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/incrementColumnValueMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.incrementColumnValueMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getProtocolVersionMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getProtocolVersionMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/hbase/master/RegionsInTransition": {
- "metric": "hadoop:service=Master,name=Master.RegionsInTransition",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/balanceSwitchAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.balanceSwitchAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getAlterStatusMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getAlterStatusMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/unassignMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.unassignMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/nextAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.nextAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/rollHLogWriterMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.rollHLogWriterMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getAlterStatus.aboveOneSec.MinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getAlterStatus.aboveOneSec.MinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/stopAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.stopAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/hbase/master/hdfsVersion": {
- "metric": "hadoop:service=HBase,name=Info.hdfsVersion",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/unassignMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.unassignMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/RpcSlowResponseAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.RpcSlowResponseAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/assignNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.assignNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getLastFlushTimeAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getLastFlushTimeAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getClusterStatusAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getClusterStatusAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/mutateRowNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.mutateRowNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getClosestRowBeforeMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getClosestRowBeforeMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/regionServerReport.aboveOneSec.NumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.regionServerReport.aboveOneSec.NumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getClusterStatus.aboveOneSec.NumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getClusterStatus.aboveOneSec.NumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/balanceAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.balanceAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/balanceSwitchNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.balanceSwitchNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/hbase/master/RegionServers": {
- "metric": "Hadoop:service=HBase,name=Master,sub=Server.numRegionServers",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/bulkLoadHFilesAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.bulkLoadHFilesAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/compactRegionAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.compactRegionAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/disableTableMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.disableTableMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/closeMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.closeMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/enableTableNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.enableTableNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/openScannerMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.openScannerMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/moveMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.moveMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/checkAndPutMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.checkAndPutMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/deleteTableAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.deleteTableAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/assign.aboveOneSec.MinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.assign.aboveOneSec.MinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/hbase/master/ClusterId": {
- "metric": "Hadoop:service=HBase,name=Master,sub=Server.tag.clusterId",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getProtocolVersionMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getProtocolVersionMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/regionServerStartupMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.regionServerStartupMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/RpcQueueTimeMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.RpcQueueTimeMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/createTableNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.createTableNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/RpcProcessingTimeMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.RpcProcessingTimeMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getAlterStatus.aboveOneSec.NumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getAlterStatus.aboveOneSec.NumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/hbase/master/splitSizeNumOps": {
- "metric": "hadoop:service=Master,name=MasterStatistics.splitSizeNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/addColumnNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.addColumnNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/unassignNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.unassignNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/balance.aboveOneSec.MinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.balance.aboveOneSec.MinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/closeRegionNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.closeRegionNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getProtocolSignature.aboveOneSec.MaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getProtocolSignature.aboveOneSec.MaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/synchronousBalanceSwitchAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.synchronousBalanceSwitchAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/appendMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.appendMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/unlockRowMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.unlockRowMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getStoreFileListMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getStoreFileListMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/moveAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.moveAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/mutateRowMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.mutateRowMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getHTableDescriptors.aboveOneSec.MinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getHTableDescriptors.aboveOneSec.MinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getCompactionStateAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getCompactionStateAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/closeRegionMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.closeRegionMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/reportRSFatalError.aboveOneSec.NumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.reportRSFatalError.aboveOneSec.NumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/jvm/HeapMemoryMax": {
- "metric": "java.lang:type=Memory.HeapMemoryUsage[max]",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/RpcSlowResponseMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.RpcSlowResponseMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/balanceSwitchMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.balanceSwitchMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/jvm/HeapMemoryUsed": {
- "metric": "java.lang:type=Memory.HeapMemoryUsage[used]",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/balanceMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.balanceMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/lockRowMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.lockRowMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/openScannerMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.openScannerMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getStoreFileListAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getStoreFileListAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/openRegionsAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.openRegionsAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/disableTableMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.disableTableMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getOnlineRegionsNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getOnlineRegionsNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/deleteColumnAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.deleteColumnAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/replicationCallQueueLen": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.replicationCallQueueLen",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/putMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.putMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/modifyTableMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.modifyTableMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/reportRSFatalErrorMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.reportRSFatalErrorMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/flushRegionAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.flushRegionAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/deleteColumn.aboveOneSec.AvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.deleteColumn.aboveOneSec.AvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/offline.aboveOneSec.AvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.offline.aboveOneSec.AvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/hbase/master/splitTimeMinTime": {
- "metric": "Hadoop:service=HBase,name=Master,sub=Server.HlogSplitTime_min",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/createTable.aboveOneSec.AvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.createTable.aboveOneSec.AvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/lockRowAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.lockRowAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/lockRowMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.lockRowMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/synchronousBalanceSwitch.aboveOneSec.AvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.synchronousBalanceSwitch.aboveOneSec.AvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getClusterStatus.aboveOneSec.MinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getClusterStatus.aboveOneSec.MinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/addColumn.aboveOneSec.AvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.addColumn.aboveOneSec.AvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/isMasterRunningNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.isMasterRunningNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/closeAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.closeAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/addColumn.aboveOneSec.MaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.addColumn.aboveOneSec.MaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/closeMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.closeMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/replicateLogEntriesMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.replicateLogEntriesMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/openRegionsMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.openRegionsMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/balance.aboveOneSec.NumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.balance.aboveOneSec.NumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/replicateLogEntriesMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.replicateLogEntriesMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/execCoprocessorMinTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.execCoprocessorMinTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/hbase/master/splitTime_num_ops": {
- "metric": "Hadoop:service=HBase,name=Master,sub=Server.HlogSplitTime_num_ops",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/modifyTableAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.modifyTableAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/assignMaxTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.assignMaxTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/isMasterRunning.aboveOneSec.AvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.isMasterRunning.aboveOneSec.AvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/getAvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.getAvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/regionServerReport.aboveOneSec.AvgTime": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.regionServerReport.aboveOneSec.AvgTime",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/rpcAuthenticationSuccesses": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.rpcAuthenticationSuccesses",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/enableTable.aboveOneSec.NumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.enableTable.aboveOneSec.NumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/execCoprocessorNumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.execCoprocessorNumOps",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/jvm/NonHeapMemoryUsed": {
- "metric": "java.lang:type=Memory.NonHeapMemoryUsage[used]",
- "pointInTime": true,
- "temporal": false
- },
- "metrics/rpc/move.aboveOneSec.NumOps": {
- "metric": "hadoop:service=HBase,name=RPCStatistics.move.aboveOneSec.NumOps",
- "pointInTime": true,
- "temporal": false
... 781700 lines suppressed ...
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@ambari.apache.org
For additional commands, e-mail: commits-help@ambari.apache.org