[jira] [Created] (AIRAVATA-3977) Allocation Governance POC Inspired by Cloud Management Platforms

["Lahiru Jayathilake (Jira)" <ji...@apache.org>]

Lahiru Jayathilake created AIRAVATA-3977:
--------------------------------------------

             Summary: Allocation Governance POC Inspired by Cloud Management Platforms
                 Key: AIRAVATA-3977
                 URL: https://issues.apache.org/jira/browse/AIRAVATA-3977
             Project: Airavata
          Issue Type: Task
          Components: Airavata Allocation Manager, Custos Airavata Integration
            Reporter: Lahiru Jayathilake


h3. Summary

Research cloud governance and account management platforms (such as Kion, AWS Control Tower, and similar tools) and build a standalone proof-of-concept that demonstrates how their governance patterns can be applied to HPC compute allocation management. The goal is to explore concepts like hierarchical organizational structures, budget threshold enforcement, funding source prioritization, and self-service allocation with guardrails, and produce a working POC that the Custos team can learn from and adapt for future allocation management work.
h3. Problem

HPC sites manage compute allocations from multiple funding sources (ACCESS-CI, NAIRR, internal discretionary pools). Each source has different credit units, lifecycles, and rules. As allocation management grows more complex, the system needs governance capabilities that go beyond simple balance tracking:

 
 * How do you organize projects into departments or groups and apply default policies to all of them at once?
 * When an allocation is running low, how do you automatically notify the PI at 80%, warn at 95%, and block new job submissions at 100%?
 * If a researcher has credits from both ACCESS-CI and an internal pool for the same resource, which source gets drawn down first?
 * How does a PI self-allocate their awarded credits across CPU, GPU, and storage without needing admin approval for every change, while still staying within the total award?

 

Cloud governance platforms like [Kion|https://kion.io/] have solved analogous problems for cloud account and budget management. Kion provides hierarchical organizational units with policy inheritance, automated budget enforcement with configurable thresholds, multi-source funding management, and self-service account provisioning with built-in guardrails. While the domain is different (cloud spending vs. HPC compute credits), the governance patterns are directly transferable.
h3. Description

This project is an {*}exploratory research and prototyping task{*}. The student will study cloud governance platforms, identify which patterns apply to HPC allocation management, and build a standalone POC demonstrating those patterns. 

 
 #  *Study cloud governance platforms*

 

Research how cloud management platforms handle governance at scale. The primary reference is [Kion|https://kion.io/], but also look at AWS Control Tower, Azure Management Groups, and GCP Organization Policies to get a broad perspective. Read their documentation, watch available product demos, and understand the core concepts:

 
 * {*}Hierarchical organizational model{*}: How organizations, departments, and projects are structured in a tree. How policies and budgets defined at a higher level automatically apply to everything underneath.
 * {*}Budget and funding management{*}: How funding sources (grants, contracts, departmental budgets) are attached to organizational units and disseminated to projects. How budgets can be set at different levels.
 * {*}Threshold-based enforcement{*}: How the system takes automated actions when spending hits configurable thresholds (notifications, warnings, freezes). Not just tracking, but actually enforcing limits.
 * {*}Policy inheritance and exemptions{*}: How governance rules cascade down the organizational tree, and how specific projects can be exempted from inherited rules when needed.
 * {*}Self-service with guardrails{*}: How end users (PIs, project leads) can provision and manage their own resources within limits set by administrators, without needing manual approval for every action.
 * {*}Funding source prioritization{*}: When multiple funding sources apply to the same consumption, how the system determines which source to draw from first.

 
 #  *Map these patterns to HPC allocation management*

 

Produce a mapping document that translates cloud governance concepts to the HPC context:

 
||*Cloud Concept*||*HPC Equivalent*||
|Organization / OU hierarchy|Site > Department > Research Group > Project|
|Cloud account|Slurm account (the thing that actually consumes resources)|
|Funding source|Allocation source (ACCESS-CI, NAIRR, internal pool)|
|Budget with threshold actions|Award balance with notification/enforcement at configurable levels|
|Policy inheritance|Default allocation rules per department (e.g., all ACCESS allocations get Slurm enforcement)|
|Account vending (self-service)|PI self-allocates credits across resource types within their award|

 

Identify where the analogy holds, where it breaks down, and what is unique to HPC (e.g., heterogeneous resource types like CPU vs. GPU vs. storage, external awards arriving pre-approved).

 
 #  *Build a standalone POC*

 

Build a working prototype that demonstrates the key governance patterns in an HPC allocation context. This should be a self-contained application (not plugged into the existing Custos services) that showcases:

 
 * {*}Hierarchical organization{*}: Create sites, departments, and projects in a tree structure. Show how a policy or budget set at the department level propagates to all projects underneath.
 * {*}Threshold enforcement{*}: Configure thresholds on an allocation (e.g., 80%, 95%, 100%) with different actions at each level (notify, warn, block). Simulate consumption and show the enforcement triggering.
 * {*}Multi-source funding{*}: Attach multiple allocation sources to a project. Demonstrate draw-down ordering (e.g., use ACCESS credits first because they expire, then fall back to internal credits).
 * {*}Self-service allocation{*}: A PI receives an award of N credits. They can split those credits across CPU, GPU, and storage allocations without admin approval, as long as the total doesn't exceed the award.
 * {*}Policy management{*}: Define a governance rule at the site level (e.g., "all allocations must have enforcement enabled"). Show it applying automatically to all descendant projects. Demonstrate an exemption for a specific project.

 

The POC should have a simple UI to demonstrate the interactions and a REST API backing it.

 
 #  *Deliver a findings report*

 
 * Summary of governance patterns studied and their sources
 * Mapping of cloud governance concepts to HPC allocation management
 * Recommendations for which patterns Custos should adopt and how
 * What worked well in the POC, what was tricky, and open questions
 * Architecture notes for how these patterns could eventually be integrated into the Custos allocation management layer

h3. Expected Deliverables
 * Research summary of cloud governance platforms (Kion, AWS Control Tower, Azure Management Groups, GCP Org Policies) and their relevant patterns
 * Concept mapping document: cloud governance to HPC allocation governance
 * Standalone POC (Go + React/TypeScript) demonstrating hierarchical organizations, threshold enforcement, multi-source funding, self-service allocation, and policy inheritance
 * Findings report with recommendations for Custos adoption

h3. Required Skills
 * Go (backend)
 * React/TypeScript (POC UI)
 * REST API design
 * Database modeling (relational)
 * Interest in cloud governance and infrastructure management concepts

h3. Resources
 * {*}Custos repository{*}: [github.com/apache/airavata-custos|https://github.com/apache/airavata-custos]
 * {*}Allocation management module{*}: allocations/ directory for context on how Custos currently handles allocations from ACCESS-CI
 * {*}Kion{*}: [https://kion.io/] (primary reference, explore their docs, blog posts, and product videos)
 * {*}Kion higher education use case{*}: [https://kion.io/industries/higher-education/]
 * {*}AWS Control Tower{*}: [https://docs.aws.amazon.com/controltower/] (organizational governance for AWS accounts)
 * {*}Azure Management Groups{*}: [https://learn.microsoft.com/en-us/azure/governance/management-groups/] (hierarchical governance for Azure subscriptions)

{*}GCP Organization Policies{*}: [https://cloud.google.com/resource-manager/docs/organization-policy/overview]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)