You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by jh...@apache.org on 2019/08/30 02:27:56 UTC
[hadoop] branch branch-2 updated (f56972b -> 056b54c)
This is an automated email from the ASF dual-hosted git repository.
jhung pushed a change to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hadoop.git.
from f56972b HADOOP-16460: ABFS: fix for Sever Name Indication (SNI)
add afcf9c5 YARN-4081. Add support for multiple resource types in the Resource class. (Varun Vasudev via wangda)
add be2afb4 YARN-4172. Extend DominantResourceCalculator to account for all resources. (Varun Vasudev via wangda)
add 133eb31 YARN-4715. Add support to read resource types from a config file. Contributed by Varun Vasudev.
add 817837e YARN-4829. Add support for binary units in Resource class.(vvasudev via asuresh)
add 7d5050a YARN-4830. Add support for resource types in the nodemanager. Contributed by Varun Vasudev.
add 015a55a YARN-5242. Update DominantResourceCalculator to consider all resource types in calculations. Contributed by Varun Vasudev.
add f99d603 YARN-5586. Update the Resources class to consider all resource types. Contributed by Varun Vasudev.
add 6db8c36 YARN-5587. [Partial backport] Add support for resource profiles. (vvasudev via asuresh)
add c5e77e8 YARN-5588. [Partial backport] [YARN-3926] Add support for resource profiles in distributed shell. Contributed by Varun Vasudev.
add 7f614f7 YARN-9181. Backport YARN-6232 for generic resource type usage to branch-2
add 08923e3 YARN-9177. Use resource map for app metrics in TestCombinedSystemMetricsPublisher for branch-2
add d5963b2 YARN-9182. Backport YARN-6445 resource profile performance improvements to branch-2
add ad6f095 YARN-6761. Fix build for YARN-3926 branch. Contributed by Varun Vasudev.
add 1dc66db YARN-6786. [YARN-3926] ResourcePBImpl imports cleanup. Contributed by Yeliang Cang.
add d8f3886 YARN-6788. [YARN-3926] Improve performance of resource profile branch (Contributed by Sunil Govindan via Daniel Templeton)
add 59cfecd YARN-6994. [YARN-3926] Remove last uses of Long from resource types code. (Daniel Templeton via Yufei Gu)
add 06c38ee YARN-6892. [YARN-3926] Improve API implementation in Resources and DominantResourceCalculator class. Contributed by Sunil G.
add 2fafdc7 YARN-6610. [YARN-3926] DominantResourceCalculator#getResourceAsValue dominant param is updated to handle multiple resources. Contributed by Daniel Templeton.
add 7b1c58b YARN-7030. [YARN-3926] Performance optimizations in Resource and ResourceUtils class. Contributed by Wangda Tan.
add 43030aa YARN-7042. Clean up unit tests after YARN-6610. (Daniel Templeton via wangda)
add c69dd9e YARN-6789. Add Client API to get all supported resource types from RM. (Sunil G via wangda)
add 7c37fbe YARN-6781. [YARN-3926] ResourceUtils#initializeResourcesMap takes an unnecessary Map parameter. Contributed by Yu-Tang Lin.
add 1f05fa0 YARN-7043. [Partial backport] Cleanup ResourceProfileManager. (wangda)
add 74bb2d6 YARN-7067. [YARN-3926] Optimize ResourceType information display in UI. Contributed by Wangda Tan.
add f340726 YARN-7039. Fix javac and javadoc errors in YARN-3926 branch. (Sunil G via wangda)
add 8cea242 YARN-7093. Improve log message in ResourceUtils. (Sunil G via wangda)
add 45b50c4 YARN-6933. [YARN-3926] ResourceUtils.DISALLOWED_NAMES check is duplicated. Contributed by Manikandan R.
add 26e87be YARN-7056. [Partial backport] Document Resource Profiles feature. (Sunil G via wangda)
add d6673c5 YARN-9188. Port YARN-7136 to branch-2
add 72d90e7 YARN-7270 addendum: Reapplied changes after YARN-3926 backports
add 4d30c90 YARN-7137. [YARN-3926] Move newly added APIs to unstable in YARN-3926 branch. Contributed by Wangda Tan.
add 078dfb0 YARN-9175. Null resources check in ResourceInfo for branch-3.0
add 56259bc YARN-9187. Backport YARN-6852 for GPU-specific native changes to branch-2
add 631dfc7 YARN-9180. Port YARN-7033 NM recovery of assigned resources to branch-2
add 7ec4d7c YARN-9280. Backport YARN-6620 to YARN-8200/branch-2 for NodeManager-side GPU isolation
add 16faceb YARN-9174. Backport YARN-7224 for refactoring of GpuDevice class
add 4d82318 YARN-7396. NPE when accessing container logs due to null dirsHandler. Contributed by Jonathan Hung
add 7939ae1 YARN-9289. Backport YARN-7330 for GPU in UI to branch-2
add ca7d0fd YARN-7573. Gpu Information page could be empty for nodes without GPU. (Sunil G via wangda)
add c539427 YARN-7594. TestNMWebServices#testGetNMResourceInfo fails on trunk. Contributed by Gergely Novák.
add 8670956 YARN-7223. Document GPU isolation feature. Contributed by Wangda Tan.
add e102c3d YARN-9397. Fix empty NMResourceInfo object test failures in branch-2
add 22ba693 YARN-9291. Backport YARN-7637 to branch-2
add 7025333 YARN-7345. GPU Isolation: Incorrect minor device numbers written to devices.deny file. (Jonathan Hung via wangda)
add 062fbd0 YARN-7143. FileNotFound handling in ResourceUtils is inconsistent
add 1875231 YARN-7383. Node resource is not parsed correctly for resource names containing dot. Contributed by Gergely Novák.
add 6f4ebe0 YARN-8183. Fix ConcurrentModificationException inside RMAppAttemptMetrics#convertAtomicLongMaptoLongMap. (Suma Shivaprasad via wangda)
add 8e5ce80 YARN-9271. Backport YARN-6927 for resource type support in MapReduce
add 28fbc4e YARN-9272. Backport YARN-7738 for refreshing max allocation for multiple resource types
add 87ad52f YARN-9409. Port resource type changes from YARN-7237 to branch-2
add 056b54c YARN-6909. Use LightWeightedResource when number of resource types more than two. (Sunil G via wangda)
No new revisions were added by this update.
Summary of changes:
.../mapreduce/v2/app/job/impl/TaskAttemptImpl.java | 141 ++++-
.../mapreduce/TestMapreduceConfigFields.java | 11 +
.../mapreduce/v2/app/job/impl/TestTaskAttempt.java | 365 +++++++++++-
.../org/apache/hadoop/mapreduce/MRJobConfig.java | 68 ++-
.../apache/hadoop/mapred/ResourceMgrDelegate.java | 7 +
.../java/org/apache/hadoop/mapred/YARNRunner.java | 86 ++-
.../apache/hadoop/mapred/TestClientRedirect.java | 9 +
.../org/apache/hadoop/mapred/TestYARNRunner.java | 167 ++++++
hadoop-project/src/site/site.xml | 1 +
.../hadoop-yarn/dev-support/findbugs-exclude.xml | 30 +
.../ams/ApplicationMasterServiceProcessor.java | 8 +-
.../hadoop/yarn/api/ApplicationClientProtocol.java | 17 +
.../GetAllResourceTypeInfoRequest.java} | 19 +-
.../GetAllResourceTypeInfoResponse.java | 60 ++
.../yarn/api/protocolrecords/ResourceTypes.java} | 11 +-
.../records/ApplicationResourceUsageReport.java | 57 +-
.../apache/hadoop/yarn/api/records/Resource.java | 434 +++++++++++---
.../yarn/api/records/ResourceInformation.java | 306 ++++++++++
.../hadoop/yarn/api/records/ResourceRequest.java | 1 +
.../hadoop/yarn/api/records/ResourceTypeInfo.java | 197 +++++++
.../yarn/api/records/impl/LightWeightResource.java | 197 +++++++
.../yarn/api/records/impl/package-info.java} | 14 +-
.../apache/hadoop/yarn/conf/YarnConfiguration.java | 62 ++
.../exceptions/ResourceNotFoundException.java} | 29 +-
.../YARNFeatureNotEnabledException.java} | 29 +-
.../hadoop/yarn/util/UnitsConversionUtil.java | 221 ++++++++
.../hadoop/yarn/util/resource/ResourceUtils.java | 621 +++++++++++++++++++++
.../hadoop/yarn/util/resource/package-info.java} | 10 +-
.../main/proto/applicationclient_protocol.proto | 1 +
.../src/main/proto/yarn_protos.proto | 25 +
.../src/main/proto/yarn_service_protos.proto | 7 +
.../hadoop/yarn/conf/TestResourceInformation.java | 73 +++
.../hadoop/yarn/util/TestUnitsConversionUtil.java | 139 +++++
.../yarn/applications/distributedshell/Client.java | 1 +
.../apache/hadoop/yarn/client/api/YarnClient.java | 16 +
.../yarn/client/api/impl/AMRMClientImpl.java | 7 +-
.../yarn/client/api/impl/YarnClientImpl.java | 11 +
.../hadoop/yarn/client/cli/ApplicationCLI.java | 35 +-
.../yarn/client/api/impl/TestAMRMClient.java | 2 +-
.../apache/hadoop/yarn/client/cli/TestYarnCLI.java | 16 +-
.../hadoop-yarn/hadoop-yarn-common/pom.xml | 8 +
.../yarn/FileSystemBasedConfigurationProvider.java | 3 +-
.../hadoop/yarn/LocalConfigurationProvider.java | 3 +-
.../ApplicationClientProtocolPBClientImpl.java | 18 +
.../ApplicationClientProtocolPBServiceImpl.java | 21 +
.../pb/GetAllResourceTypeInfoRequestPBImpl.java | 71 +++
.../pb/GetAllResourceTypeInfoResponsePBImpl.java | 184 ++++++
.../pb/ApplicationResourceUsageReportPBImpl.java | 150 ++++-
.../yarn/api/records/impl/pb/ProtoUtils.java | 46 ++
.../yarn/api/records/impl/pb/ResourcePBImpl.java | 160 +++++-
.../records/impl/pb/ResourceTypeInfoPBImpl.java | 152 +++++
.../org/apache/hadoop/yarn/util/StringHelper.java | 36 ++
.../util/resource/DominantResourceCalculator.java | 577 ++++++++++++++-----
.../hadoop/yarn/util/resource/Resources.java | 283 +++++++---
.../src/main/resources/yarn-default.xml | 49 ++
.../hadoop/yarn/api/BasePBImplRecordsTest.java | 12 +
.../apache/hadoop/yarn/api/TestPBImplRecords.java | 30 +
.../apache/hadoop/yarn/api/TestResourcePBImpl.java | 61 ++
.../yarn/util/resource/TestResourceCalculator.java | 227 +++++++-
.../yarn/util/resource/TestResourceUtils.java | 328 +++++++++++
.../hadoop/yarn/util/resource/TestResources.java | 220 +++++++-
.../resources/resource-types/node-resources-1.xml | 29 +
.../resources/resource-types/node-resources-2.xml | 44 ++
.../resources/resource-types/resource-types-1.xml | 18 +
.../resources/resource-types/resource-types-2.xml | 29 +
.../resources/resource-types/resource-types-3.xml | 24 +
.../resources/resource-types/resource-types-4.xml | 39 ++
.../resource-types/resource-types-error-1.xml | 29 +
.../resource-types/resource-types-error-2.xml | 33 ++
.../resource-types/resource-types-error-3.xml | 29 +
.../resource-types/resource-types-error-4.xml | 24 +
.../ApplicationHistoryManagerOnTimelineStore.java | 18 +-
.../hadoop/yarn/server/utils/BuilderUtils.java | 16 +-
.../yarn/server/MockResourceManagerFacade.java | 8 +
.../src/CMakeLists.txt | 8 +-
.../yarn/server/nodemanager/ContainerExecutor.java | 3 +-
.../hadoop/yarn/server/nodemanager/Context.java | 3 +
.../nodemanager/DefaultContainerExecutor.java | 2 +-
.../nodemanager/DockerContainerExecutor.java | 2 +-
.../server/nodemanager/LinuxContainerExecutor.java | 10 +-
.../yarn/server/nodemanager/NodeManager.java | 95 ++--
.../server/nodemanager/NodeStatusUpdaterImpl.java | 51 +-
.../containermanager/container/Container.java | 7 +
.../containermanager/container/ContainerImpl.java | 13 +
.../container/ResourceMappings.java | 124 ++++
.../linux/privileged/PrivilegedOperation.java | 1 +
.../linux/resources/ResourceHandlerChain.java | 4 +-
.../linux/resources/ResourceHandlerModule.java | 42 +-
.../linux/resources/gpu/GpuResourceAllocator.java | 245 ++++++++
.../resources/gpu/GpuResourceHandlerImpl.java | 160 ++++++
.../resourceplugin/NodeResourceUpdaterPlugin.java | 52 ++
.../resourceplugin/ResourcePlugin.java | 94 ++++
.../resourceplugin/ResourcePluginManager.java | 106 ++++
.../resourceplugin/gpu/AssignedGpuDevice.java | 88 +++
.../resourceplugin/gpu/GpuDevice.java | 88 +++
.../resourceplugin/gpu/GpuDiscoverer.java | 264 +++++++++
.../gpu/GpuNodeResourceUpdateHandler.java | 68 +++
.../resourceplugin/gpu/GpuResourcePlugin.java | 83 +++
.../recovery/NMLeveldbStateStoreService.java | 98 +++-
.../recovery/NMNullStateStoreService.java | 9 +
.../nodemanager/recovery/NMStateStoreService.java | 34 ++
.../nodemanager/util/NodeManagerHardwareUtils.java | 84 ++-
.../server/nodemanager/webapp/NMWebServices.java | 25 +
.../nodemanager/webapp/dao/NMResourceInfo.java} | 16 +-
.../webapp/dao/gpu/GpuDeviceInformation.java | 72 +++
.../webapp/dao/gpu/GpuDeviceInformationParser.java | 87 +++
.../webapp/dao/gpu/NMGpuResourceInfo.java | 80 +++
.../webapp/dao/gpu/PerGpuDeviceInformation.java | 165 ++++++
.../webapp/dao/gpu/PerGpuMemoryUsage.java | 58 ++
.../webapp/dao/gpu/PerGpuTemperature.java | 80 +++
.../webapp/dao/gpu/PerGpuUtilizations.java | 50 ++
.../container-executor/impl/container-executor.h | 2 +
.../src/main/native/container-executor/impl/main.c | 11 +
.../impl/modules/cgroups/cgroups-operations.c | 161 ++++++
.../impl/modules/cgroups/cgroups-operations.h | 55 ++
.../impl/modules/gpu/gpu-module.c | 229 ++++++++
.../impl/modules/gpu/gpu-module.h} | 29 +-
.../test/modules/cgroups/test-cgroups-module.cc | 121 ++++
.../test/modules/gpu/test-gpu-module.cc | 216 +++++++
.../test/test-container-executor.c | 1 -
.../server/nodemanager/NodeManagerTestBase.java | 164 ++++++
.../nodemanager/TestDefaultContainerExecutor.java | 4 +-
.../TestDockerContainerExecutorWithMocks.java | 2 +-
.../nodemanager/TestLinuxContainerExecutor.java | 2 +-
.../TestLinuxContainerExecutorWithMocks.java | 2 +-
.../yarn/server/nodemanager/TestNodeManager.java | 2 +-
.../server/nodemanager/TestNodeStatusUpdater.java | 100 +---
.../nodemanager/amrmproxy/BaseAMRMProxyTest.java | 46 +-
.../TestContainerManagerRecovery.java | 164 ++++--
.../linux/resources/TestResourceHandlerModule.java | 8 +-
.../resources/gpu/TestGpuResourceHandler.java | 480 ++++++++++++++++
.../TestContainersMonitorResourceChange.java | 2 +-
.../resourceplugin/TestResourcePluginManager.java | 261 +++++++++
.../resourceplugin/gpu/TestGpuDiscoverer.java | 141 +++++
.../recovery/NMMemoryStateStoreService.java | 18 +
.../recovery/TestNMLeveldbStateStoreService.java | 136 +++--
.../util/TestNodeManagerHardwareUtils.java | 2 +-
.../server/nodemanager/webapp/MockContainer.java | 6 +
.../nodemanager/webapp/TestNMWebServices.java | 184 +++++-
.../dao/gpu/TestGpuDeviceInformationParser.java | 50 ++
.../test/resources/nvidia-smi-sample-xml-output | 547 ++++++++++++++++++
.../server/resourcemanager/AMSProcessingChain.java | 2 +-
.../yarn/server/resourcemanager/AdminService.java | 22 +-
.../server/resourcemanager/ClientRMService.java | 11 +
.../resourcemanager/DefaultAMSProcessor.java | 3 +-
.../OpportunisticContainerAllocatorAMService.java | 3 +-
.../yarn/server/resourcemanager/RMAppManager.java | 8 +-
.../yarn/server/resourcemanager/RMServerUtils.java | 10 +-
.../resourcemanager/ResourceTrackerService.java | 9 +-
.../resourcemanager/recovery/RMStateStore.java | 7 +-
.../records/ApplicationAttemptStateData.java | 89 ++-
.../impl/pb/ApplicationAttemptStateDataPBImpl.java | 50 ++
.../server/resourcemanager/rmapp/RMAppImpl.java | 43 +-
.../server/resourcemanager/rmapp/RMAppMetrics.java | 41 +-
.../rmapp/attempt/AggregateAppResourceUsage.java | 34 +-
.../rmapp/attempt/RMAppAttemptImpl.java | 40 +-
.../rmapp/attempt/RMAppAttemptMetrics.java | 110 +++-
.../rmcontainer/RMContainerImpl.java | 16 +-
.../scheduler/AbstractYarnScheduler.java | 34 +-
.../scheduler/ClusterNodeTracker.java | 15 +-
.../scheduler/SchedulerApplicationAttempt.java | 46 +-
.../scheduler/capacity/CapacityScheduler.java | 14 +-
.../capacity/CapacitySchedulerConfiguration.java | 22 +-
.../scheduler/capacity/LeafQueue.java | 4 +-
.../scheduler/fair/FairScheduler.java | 4 +-
.../scheduler/fifo/FifoScheduler.java | 13 +-
.../webapp/MetricsOverviewTable.java | 11 +-
.../server/resourcemanager/webapp/RMAppBlock.java | 14 +-
.../server/resourcemanager/webapp/dao/AppInfo.java | 28 +
.../resourcemanager/webapp/dao/ResourceInfo.java | 39 +-
.../resourcemanager/webapp/dao/SchedulerInfo.java | 3 +-
.../yarn_server_resourcemanager_recovery.proto | 2 +
.../hadoop/yarn/server/resourcemanager/MockNM.java | 41 +-
.../hadoop/yarn/server/resourcemanager/MockRM.java | 15 +
.../yarn/server/resourcemanager/RMHATestBase.java | 29 +-
.../server/resourcemanager/TestAppManager.java | 9 +-
.../TestApplicationMasterService.java | 11 +-
.../resourcemanager/TestClientRMService.java | 45 ++
.../TestContainerResourceUsage.java | 7 +-
.../applicationsmanager/MockAsm.java | 9 +-
.../TestCombinedSystemMetricsPublisher.java | 14 +-
.../metrics/TestSystemMetricsPublisher.java | 15 +-
.../metrics/TestSystemMetricsPublisherForV2.java | 22 +-
.../recovery/RMStateStoreTestBase.java | 8 +-
.../recovery/TestZKRMStateStore.java | 23 +-
.../resourcemanager/resource/TestResources.java | 43 --
.../scheduler/capacity/TestCapacityScheduler.java | 148 +----
.../capacity/TestCapacitySchedulerPerf.java | 265 +++++++++
...estCapacitySchedulerWithMultiResourceTypes.java | 190 +++++++
.../scheduler/capacity/TestUtils.java | 41 +-
.../scheduler/fair/TestFairScheduler.java | 4 +
.../server/resourcemanager/webapp/TestAppPage.java | 8 +-
.../webapp/TestRMWebAppFairScheduler.java | 5 +-
.../webapp/TestRMWebServicesApps.java | 2 +-
.../clientrm/DefaultClientRequestInterceptor.java | 8 +
.../clientrm/FederationClientInterceptor.java | 7 +
.../router/clientrm/RouterClientRMService.java | 9 +
.../PassThroughClientRequestInterceptor.java | 8 +
.../apache/hadoop/yarn/server/MiniYARNCluster.java | 7 +
.../src/site/markdown/ResourceProfiles.md | 79 +++
.../src/site/markdown/UsingGpus.md | 230 ++++++++
.../app/{constants.js => adapters/yarn-nm-gpu.js} | 21 +-
.../src/main/webapp/app/components/donut-chart.js | 18 +-
.../main/webapp/app/components/gpu-donut-chart.js | 66 +++
.../src/main/webapp/app/constants.js | 13 +
.../webapp/app/controllers/yarn-nodes/table.js | 2 +-
.../src/main/webapp/app/models/cluster-metric.js | 69 +++
.../app/{constants.js => models/yarn-nm-gpu.js} | 15 +-
.../webapp/app/models/yarn-queue/capacity-queue.js | 3 +-
.../src/main/webapp/app/models/yarn-rm-node.js | 37 ++
.../hadoop-yarn-ui/src/main/webapp/app/router.js | 5 +-
.../src/main/webapp/app/routes/cluster-overview.js | 2 +-
.../src/main/webapp/app/routes/yarn-node.js | 2 +
.../table.js => yarn-node/yarn-nm-gpu.js} | 0
.../yarn-node.js => serializers/yarn-nm-gpu.js} | 34 +-
.../app/serializers/yarn-queue/capacity-queue.js | 1 +
.../main/webapp/app/serializers/yarn-rm-node.js | 4 +-
.../main/webapp/app/templates/cluster-overview.hbs | 88 ++-
.../app/templates/components/node-menu-panel.hbs | 10 +-
.../app/templates/components/yarn-nm-gpu-info.hbs | 69 +++
.../main/webapp/app/templates/yarn-node-apps.hbs | 2 +-
.../webapp/app/templates/yarn-node-containers.hbs | 2 +-
.../src/main/webapp/app/templates/yarn-node.hbs | 125 -----
.../main/webapp/app/templates/yarn-node/info.hbs | 154 +++++
.../yarn-nm-gpu.hbs} | 52 +-
.../src/main/webapp/app/utils/converter.js | 51 ++
226 files changed, 13493 insertions(+), 1508 deletions(-)
copy hadoop-yarn-project/hadoop-yarn/{hadoop-yarn-ui/src/main/webapp/app/constants.js => hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetAllResourceTypeInfoRequest.java} (60%)
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetAllResourceTypeInfoResponse.java
copy hadoop-yarn-project/hadoop-yarn/{hadoop-yarn-ui/src/main/webapp/app/constants.js => hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/ResourceTypes.java} (81%)
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ResourceInformation.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ResourceTypeInfo.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/LightWeightResource.java
copy hadoop-yarn-project/hadoop-yarn/{hadoop-yarn-ui/src/main/webapp/app/constants.js => hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/package-info.java} (71%)
copy hadoop-yarn-project/hadoop-yarn/{hadoop-yarn-ui/src/main/webapp/app/constants.js => hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/exceptions/ResourceNotFoundException.java} (54%)
copy hadoop-yarn-project/hadoop-yarn/{hadoop-yarn-ui/src/main/webapp/app/constants.js => hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/exceptions/YARNFeatureNotEnabledException.java} (53%)
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/util/UnitsConversionUtil.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/util/resource/ResourceUtils.java
copy hadoop-yarn-project/hadoop-yarn/{hadoop-yarn-ui/src/main/webapp/app/constants.js => hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/util/resource/package-info.java} (81%)
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/test/java/org/apache/hadoop/yarn/conf/TestResourceInformation.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/test/java/org/apache/hadoop/yarn/util/TestUnitsConversionUtil.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetAllResourceTypeInfoRequestPBImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetAllResourceTypeInfoResponsePBImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ResourceTypeInfoPBImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/TestResourcePBImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/resource/TestResourceUtils.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/resources/resource-types/node-resources-1.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/resources/resource-types/node-resources-2.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/resources/resource-types/resource-types-1.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/resources/resource-types/resource-types-2.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/resources/resource-types/resource-types-3.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/resources/resource-types/resource-types-4.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/resources/resource-types/resource-types-error-1.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/resources/resource-types/resource-types-error-2.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/resources/resource-types/resource-types-error-3.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/resources/resource-types/resource-types-error-4.xml
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ResourceMappings.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/gpu/GpuResourceAllocator.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/gpu/GpuResourceHandlerImpl.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/NodeResourceUpdaterPlugin.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/ResourcePlugin.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/ResourcePluginManager.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/gpu/AssignedGpuDevice.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/gpu/GpuDevice.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/gpu/GpuDiscoverer.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/gpu/GpuNodeResourceUpdateHandler.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/gpu/GpuResourcePlugin.java
copy hadoop-yarn-project/hadoop-yarn/{hadoop-yarn-ui/src/main/webapp/app/constants.js => hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/dao/NMResourceInfo.java} (73%)
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/dao/gpu/GpuDeviceInformation.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/dao/gpu/GpuDeviceInformationParser.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/dao/gpu/NMGpuResourceInfo.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/dao/gpu/PerGpuDeviceInformation.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/dao/gpu/PerGpuMemoryUsage.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/dao/gpu/PerGpuTemperature.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/dao/gpu/PerGpuUtilizations.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/modules/cgroups/cgroups-operations.c
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/modules/cgroups/cgroups-operations.h
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/modules/gpu/gpu-module.c
copy hadoop-yarn-project/hadoop-yarn/{hadoop-yarn-ui/src/main/webapp/app/constants.js => hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/modules/gpu/gpu-module.h} (54%)
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/modules/cgroups/test-cgroups-module.cc
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/modules/gpu/test-gpu-module.cc
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/NodeManagerTestBase.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/gpu/TestGpuResourceHandler.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/TestResourcePluginManager.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/gpu/TestGpuDiscoverer.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/webapp/dao/gpu/TestGpuDeviceInformationParser.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-sample-xml-output
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/resource/TestResources.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestCapacitySchedulerPerf.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestCapacitySchedulerWithMultiResourceTypes.java
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/ResourceProfiles.md
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/UsingGpus.md
copy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/{constants.js => adapters/yarn-nm-gpu.js} (70%)
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/components/gpu-donut-chart.js
copy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/{constants.js => models/yarn-nm-gpu.js} (81%)
copy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/routes/{yarn-nodes/table.js => yarn-node/yarn-nm-gpu.js} (100%)
copy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/{routes/yarn-node.js => serializers/yarn-nm-gpu.js} (56%)
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/templates/components/yarn-nm-gpu-info.hbs
delete mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/templates/yarn-node.hbs
create mode 100644 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/templates/yarn-node/info.hbs
copy hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/templates/{yarn-node-apps.hbs => yarn-node/yarn-nm-gpu.hbs} (50%)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org