You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airavata.apache.org by la...@apache.org on 2026/07/13 19:33:39 UTC
(airavata-custos) branch master updated: Add core and web verify workflows and bind compose services to loopback and parameterize credentials (#525)
This is an automated email from the ASF dual-hosted git repository.
lahirujayathilake pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/airavata-custos.git
The following commit(s) were added to refs/heads/master by this push:
new 0dbe68739 Add core and web verify workflows and bind compose services to loopback and parameterize credentials (#525)
0dbe68739 is described below
commit 0dbe68739aeb9468d9345858fc24bf4b740a6c13
Author: Lahiru Jayathilake <la...@gmail.com>
AuthorDate: Mon Jul 13 15:33:33 2026 -0400
Add core and web verify workflows and bind compose services to loopback and parameterize credentials (#525)
* Add core and web verify workflows
* Point to the packeges.json and updated the node version to 22
* Bind compose services to loopback and parameterize credentials
---
.github/workflows/README.md | 43 ++++++++++++++++++
.github/workflows/core-verify.yml | 91 ++++++++++++++++++++++++++++++++++++++
.github/workflows/web-verify.yml | 77 ++++++++++++++++++++++++++++++++
INSTALL.md | 2 +-
dev-ops/compose/docker-compose.yml | 22 ++++-----
5 files changed, 223 insertions(+), 12 deletions(-)
diff --git a/.github/workflows/README.md b/.github/workflows/README.md
new file mode 100644
index 000000000..cab31badb
--- /dev/null
+++ b/.github/workflows/README.md
@@ -0,0 +1,43 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# Workflows
+
+Files are named `<area>-<purpose>.yml`, where the area matches the part of
+the repo the workflow validates:
+
+| Area prefix | Covers |
+|---|---|
+| `core-` | Go backend: `cmd/`, `internal/`, `pkg/`, and unit tests repo-wide |
+| `web-` | the portal under `web/` |
+| `amie-` | the AMIE connector and its integration stack |
+
+Current workflows:
+
+- `core-verify.yml` — formatting, build, vet, unit tests for all Go packages,
+ and the `./internal/...` integration suite against a MariaDB service
+ container. Runs on Go changes.
+- `web-verify.yml` — frozen-lockfile install, typecheck, lint, unit tests,
+ and a production build. Runs on `web/` changes.
+- `amie-integration-tests.yml` — the AMIE integration suite with its own
+ compose stack (mock AMIE server + MariaDB). Runs on connector changes.
+ Not duplicated by `core-verify.yml`.
+
+When adding a workflow, keep the area prefix, add path filters so unrelated
+PRs skip it, and list it here.
diff --git a/.github/workflows/core-verify.yml b/.github/workflows/core-verify.yml
new file mode 100644
index 000000000..810134266
--- /dev/null
+++ b/.github/workflows/core-verify.yml
@@ -0,0 +1,91 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Core Verify
+
+on:
+ pull_request:
+ paths:
+ - '**.go'
+ - 'go.mod'
+ - 'go.sum'
+ - 'internal/db/migrations/**'
+ - '.github/workflows/core-verify.yml'
+ push:
+ branches: [master]
+ paths:
+ - '**.go'
+ - 'go.mod'
+ - 'go.sum'
+ - 'internal/db/migrations/**'
+ - '.github/workflows/core-verify.yml'
+ workflow_dispatch:
+
+jobs:
+ verify:
+ name: Format, build, vet, test
+ runs-on: ubuntu-latest
+ timeout-minutes: 20
+ services:
+ mariadb:
+ image: mariadb:11.2
+ env:
+ MARIADB_ROOT_PASSWORD: root
+ MARIADB_DATABASE: custos
+ MARIADB_USER: admin
+ MARIADB_PASSWORD: admin
+ ports:
+ - 3306:3306
+ options: >-
+ --health-cmd="healthcheck.sh --connect --innodb_initialized"
+ --health-interval=10s
+ --health-timeout=5s
+ --health-retries=10
+ steps:
+ - name: Checkout
+ uses: actions/checkout@v4
+
+ - name: Set up Go
+ uses: actions/setup-go@v5
+ with:
+ go-version: '1.24'
+ cache: true
+
+ - name: Check formatting
+ run: |
+ files=$(gofmt -l .)
+ if [ -n "$files" ]; then
+ echo "gofmt required on:"
+ echo "$files"
+ exit 1
+ fi
+
+ - name: Build
+ run: go build ./...
+
+ - name: Vet
+ run: go vet ./...
+
+ - name: Unit tests
+ run: go test ./...
+
+ # The AMIE integration suite has its own workflow and compose stack;
+ # this covers the core suite only.
+ - name: Core integration tests
+ env:
+ CORE_TEST_DATABASE_DSN: "admin:admin@tcp(127.0.0.1:3306)/custos?parseTime=true&charset=utf8mb4&multiStatements=true"
+ run: go test -tags integration ./internal/...
diff --git a/.github/workflows/web-verify.yml b/.github/workflows/web-verify.yml
new file mode 100644
index 000000000..c9d6df3f9
--- /dev/null
+++ b/.github/workflows/web-verify.yml
@@ -0,0 +1,77 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Web Verify
+
+on:
+ pull_request:
+ paths:
+ - 'web/**'
+ - '.github/workflows/web-verify.yml'
+ push:
+ branches: [master]
+ paths:
+ - 'web/**'
+ - '.github/workflows/web-verify.yml'
+ workflow_dispatch:
+
+jobs:
+ verify:
+ name: Typecheck, lint, test, build
+ runs-on: ubuntu-latest
+ timeout-minutes: 15
+ defaults:
+ run:
+ working-directory: web
+ # Dummy values that satisfy the env schema; nothing here talks to a
+ # real identity provider or backend.
+ env:
+ NEXTAUTH_SECRET: ci-only-dummy-secret
+ OIDC_ISSUER_URL: https://idp.invalid
+ OIDC_CLIENT_ID: ci
+ OIDC_CLIENT_SECRET: ci
+ steps:
+ - name: Checkout
+ uses: actions/checkout@v4
+
+ - name: Set up pnpm
+ uses: pnpm/action-setup@v4
+ with:
+ # packageManager lives in web/package.json, not the repo root.
+ package_json_file: web/package.json
+
+ - name: Set up Node
+ uses: actions/setup-node@v4
+ with:
+ node-version: 22
+ cache: pnpm
+ cache-dependency-path: web/pnpm-lock.yaml
+
+ - name: Install
+ run: pnpm install --frozen-lockfile
+
+ - name: Typecheck
+ run: pnpm typecheck
+
+ - name: Lint
+ run: pnpm lint
+
+ - name: Unit tests
+ run: pnpm test
+
+ - name: Production build
+ run: pnpm build
diff --git a/INSTALL.md b/INSTALL.md
index 26e31b4df..2ca2403a8 100644
--- a/INSTALL.md
+++ b/INSTALL.md
@@ -25,7 +25,7 @@ conventions and contribution workflow, see [CONTRIBUTING.md](CONTRIBUTING.md).
## Prerequisites
- Go **1.24+**
-- Node **20+** and **pnpm** (for the portal)
+- Node **22+** and **pnpm** (for the portal)
- Docker and Docker Compose
- `git`
diff --git a/dev-ops/compose/docker-compose.yml b/dev-ops/compose/docker-compose.yml
index 98fdba617..03df6c2e5 100644
--- a/dev-ops/compose/docker-compose.yml
+++ b/dev-ops/compose/docker-compose.yml
@@ -24,11 +24,11 @@ services:
restart: always
command: --character-set-server=utf8 --collation-server=utf8_general_ci
ports:
- - "3306:3306"
+ - "127.0.0.1:3306:3306"
environment:
- MARIADB_ROOT_PASSWORD: root
+ MARIADB_ROOT_PASSWORD: ${CUSTOS_DB_ROOT_PASSWORD:-root}
MARIADB_USER: admin
- MARIADB_PASSWORD: admin
+ MARIADB_PASSWORD: ${CUSTOS_DB_PASSWORD:-admin}
MAX_ALLOWED_PACKET: 1073741824
volumes:
- ./dbinit:/docker-entrypoint-initdb.d
@@ -50,14 +50,14 @@ services:
KC_DB: mariadb
KC_DB_URL: jdbc:mariadb://db:3306/keycloak
KC_DB_USERNAME: admin
- KC_DB_PASSWORD: admin
+ KC_DB_PASSWORD: ${CUSTOS_DB_PASSWORD:-admin}
KC_HOSTNAME_URL: http://localhost:8081
KC_HOSTNAME_STRICT: "false"
KC_HTTP_ENABLED: "true"
KC_HEALTH_ENABLED: "true"
ports:
- - "8081:8080"
- - "9001:9000"
+ - "127.0.0.1:8081:8080"
+ - "127.0.0.1:9001:9000"
volumes:
- ./keycloak/import:/opt/keycloak/data/import:ro
depends_on:
@@ -74,14 +74,14 @@ services:
image: adminer
restart: always
ports:
- - 18080:8080
+ - "127.0.0.1:18080:8080"
prometheus:
image: prom/prometheus:latest
container_name: prometheus
restart: unless-stopped
ports:
- - "19090:9090"
+ - "127.0.0.1:19090:9090"
volumes:
- ./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml
extra_hosts:
@@ -92,7 +92,7 @@ services:
container_name: grafana
restart: unless-stopped
ports:
- - "13000:3000"
+ - "127.0.0.1:13000:3000"
environment:
GF_SECURITY_ADMIN_USER: admin
GF_SECURITY_ADMIN_PASSWORD: admin
@@ -106,8 +106,8 @@ services:
container_name: vault
restart: unless-stopped
ports:
- - "8201:8201"
- - "8200:8200"
+ - "127.0.0.1:8201:8201"
+ - "127.0.0.1:8200:8200"
environment:
VAULT_ADDR: http://0.0.0.0:8201
VAULT_API_ADDR: http://127.0.0.1:8200