You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hugegraph.apache.org by "bitflicker64 (via GitHub)" <gi...@apache.org> on 2026/08/10 15:56:00 UTC
[PR] feat(docker): add attachable Hubble Compose add-on for the 3-node cluster [hugegraph]
bitflicker64 opened a new pull request, #3149:
URL: https://github.com/apache/hugegraph/pull/3149
- close #3147
## What this PR does
Adds an attachable Hubble deployment for the distributed Compose topology:
- **`docker/docker-compose-hubble.yml`** (new) — defines only the Hubble
service. It joins the pre-created cluster network and has no `depends_on`
on cluster services, so attaching, upgrading, or removing Hubble never
recreates PD, Store, or Server containers.
- **`docker/docker-compose-3pd-3store-3server.yml`** — adds the Server
PD-registration and auth settings Hubble needs (cluster name, PD mode,
per-replica REST URL, required admin password and shared token secret),
mirroring what #3143 did for the single-node file.
- **`docker/hugegraph-hubble-3x3.properties`** (new) — Hubble topology
config for the 3-node cluster.
- **`docker/README.md`** — documents the three flows: attach to a running
cluster, fresh cluster + Hubble in one command, and a local Hubble image
override for development.
- **`.github/workflows/server-ci.yml`** — extends the existing compose
contract checks to the cluster file and the add-on.
- Doc call-site updates (root/server/pd/store docs) — the old documented
commands no longer work with the required-credential guards.
## Design rationale (why it is shaped this way)
- **Add-on file over duplication:** the add-on defines only Hubble and
merges with the canonical cluster file via `-f` flags, so the topology
is never maintained in two places. This is the pattern Elastic documents
for attaching Kibana to a running Elasticsearch over a shared Docker
network with versions matched through one variable, and the attach model
TiDB Operator provides via its standalone `TidbDashboard` resource.
- **Pre-created external network:** Compose validates `external:` networks
before creating project networks and refuses to adopt unlabeled ones, so
both files declare the shared network as external with a well-known name
(`hugegraph-net`, override via `HUGEGRAPH_NETWORK`) and setup includes a
one-time `docker network create`. Verified empirically; the generated
per-project network name cannot support the one-command fresh flow.
- **Explicit `-p hugegraph-hubble` for the attach flow:** without it the
attach project takes the directory name, colliding with other projects
and making documented lifecycle commands cross project boundaries.
- **`latest` as the current default:** the authenticated PD/Hubble
integration is not in `1.7.x`, so pinning to an existing release would
ship a broken combination. Once the next release publishes, the default
should move to that tag; deployments pin today via `HUGEGRAPH_VERSION`
in `docker/.env`. (Unpinned `latest` with no compatibility story is the
documented failure mode of Dgraph's Ratel — hence the single shared
version variable.)
## Breaking change owned by this PR
Existing 3-node deployments: the first `up -d` after this change recreates
all nine containers (network move; named volumes survive), and the graph
APIs begin requiring authentication (401 for previously anonymous
clients). The README carries an upgrade note covering `.env` creation,
the orphaned old network, and verifying sign-in before decommissioning
existing access paths.
## Validation evidence
- Fresh combined up: 10/10 containers healthy; Hubble login through PD
(HTTP 200, role SUPERADMIN); Operations view shows 1 logical Server +
3 PD + 3 Store, all UP.
- Attach flow: cluster brought up alone, container IDs recorded; one
add-on command attached Hubble; all cluster container IDs unchanged.
- Single-node regression: `docker/docker-compose.yml` path unaffected and
healthy.
- CI checks replicated locally (positive and negative cases).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(docker): rebuild Compose topologies with Hubble [hugegraph]
Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on code in PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149#discussion_r3891968368
##########
docker/docker-compose.yml:
##########
@@ -14,117 +14,59 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
-name: hugegraph-single
+
+name: hugegraph-standalone
Review Comment:
This default-file change is intentional: the PR redesigns Docker Compose into explicit standalone, minimal HStore, and HA topology families with separate project-owned volumes. It is not an in-place upgrade of the previous default HStore stack. Historical migration and rollback compatibility are outside this refactor scope; users who need HStore select docker-compose-hstore.yml explicitly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(docker): add attachable Hubble Compose add-on for the 3-node cluster [hugegraph]
Posted by "bitflicker64 (via GitHub)" <gi...@apache.org>.
bitflicker64 commented on code in PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149#discussion_r3789894486
##########
docker/docker-compose-hubble.yml:
##########
@@ -0,0 +1,53 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+# Hubble add-on for the distributed cluster defined in
+# docker-compose-3pd-3store-3server.yml. Requires the pre-created cluster
+# network (docker network create hugegraph-net). See "Hubble for the
+# 3-Node Cluster" in docker/README.md for the attach and combined flows.
+
+networks:
+ hg-net:
+ external: true
+ name: ${HUGEGRAPH_NETWORK:-hugegraph-net}
+
+volumes:
+ hg-hubble-db:
Review Comment:
I’ll give the Hubble DB/upload volumes explicit stable names so attach and combined share the same physical volumes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(docker): add attachable Hubble Compose add-on for the 3-node cluster [hugegraph]
Posted by "bitflicker64 (via GitHub)" <gi...@apache.org>.
bitflicker64 commented on code in PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149#discussion_r3767825548
##########
docker/README.md:
##########
@@ -168,14 +173,87 @@ To validate local images without Compose replacing them with remote `latest`:
## 3-Node Cluster Quickstart
+The cluster and the Hubble add-on share one named Docker network so Hubble
+can attach to a running cluster without touching it. Treat that network as a
+trust boundary: PD and Store expose unauthenticated control APIs on it (only
+the Server layer authenticates), and any container on the host can join it
+by declaring the well-known name. One-time setup: write the required
+credentials to a mode-600 `docker/.env` and create the network.
+The cluster file requires both credentials — the admin password enables
+authentication, and every Server replica must share one token secret so a
+token issued by any server validates on all of them. The `:?` guards fire
+on every Compose subcommand, including `down`.
+
+```bash
+(
+ set -eu
+ cd docker
+ command -v openssl >/dev/null 2>&1 || { echo "openssl not found" >&2; exit 1; }
+ [ -e .env ] || install -m 600 /dev/null .env
+ chmod 600 .env
+ # Keep appends on their own lines even if the file was hand-edited.
+ [ ! -s .env ] || [ -z "$(tail -c1 .env)" ] || printf '\n' >> .env
+ pat='^[[:space:]]*(export[[:space:]]+)?'
+ if ! grep -Eq "${pat}HUGEGRAPH_ADMIN_PASSWORD=" .env; then
Review Comment:
Addressed in . The validator now isolates unset and empty admin/token cases, checks the error names the intended variable, and the setup block validates existing credentials rather than treating a present-but-empty or short value as complete. Short token values now fail with a repair message.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(docker): rebuild Compose topologies with Hubble [hugegraph]
Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on code in PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149#discussion_r3892317694
##########
docker/README.md:
##########
@@ -1,436 +1,362 @@
-# HugeGraph Docker Deployment
+# HugeGraph Docker Compose
-This directory contains Docker Compose files for running HugeGraph:
+## Users
-| File | Description |
-|------|-------------|
-| `docker-compose.yml` | PD, Store, Server, and Hubble using pre-built images |
-| `docker-compose.dev.yml` | PD, Store, and Server built from source, plus Hubble |
-| `docker-compose-3pd-3store-3server.yml` | 3-node distributed cluster (PD + Store + Server) |
+### Choose a topology
-## Prerequisites
+Run commands in this directory:
-- **Docker Engine** 20.10+ (or Docker Desktop 4.x+)
-- **Docker Compose** v2 (included in Docker Desktop)
-- **OpenSSL CLI** (used to generate the initial administrator password)
-- **Memory**: Allocate at least **12 GB** to Docker Desktop (Settings → Resources → Memory). The 3-node cluster runs 9 JVM processes (3 PD + 3 Store + 3 Server) which are memory-intensive. Insufficient memory causes OOM kills that appear as silent Raft failures.
+```bash
+cd docker
+```
-> [!IMPORTANT]
-> The 12 GB minimum is for Docker Desktop. On Linux with native Docker, ensure the host has at least 12 GB of free memory.
----
+| Topology | Compose file | Services | When to use it |
+| --- | --- | --- | --- |
+| Standalone | `docker-compose.yml` | 1 RocksDB Server + 1 Hubble | Default; start here |
+| Minimal HStore | `docker-compose-hstore.yml` | 1 PD + 1 Store + 1 Server + 1 Hubble | Distributed local development |
+| HA | `docker-compose-3pd-3store-3server.yml` | 3 PD + 3 Store + 3 Server + 1 Hubble | Reference and evaluation |
-## Single-Node Setup
+Standalone uses `hugegraph/hugegraph:${HUGEGRAPH_VERSION:-latest}`. The HStore
+topologies use the matching `hugegraph/pd`, `hugegraph/store`, and
+`hugegraph/server` tags. Hubble is selected independently with
+`${HUBBLE_IMAGE:-hugegraph/hubble:latest}`.
-Two compose files run one PD, one Store, one Server, and one Hubble instance:
+### Create the authentication environment
-Create a Compose environment file once so every lifecycle command can resolve the required administrator password:
+Create `.env` once. Replace `replace-with-your-password` with an administrator
+password that you choose; the command generates and persists a random 32-byte
+JWT secret. For this simple single-quoted format, do not use a password that
+contains a single quote or newline.
```bash
(
set -eu
- cd docker
- if [ -e .env ]; then
- echo "docker/.env already exists; reusing it"
- else
- command -v openssl >/dev/null 2>&1
- admin_password="$(openssl rand -base64 12)"
- if [ "${#admin_password}" -ne 16 ]; then
- echo "Failed to generate a 16-character password" >&2
- exit 1
- fi
- install -m 600 /dev/null .env
- {
- printf "HUGEGRAPH_ADMIN_PASSWORD='%s'\n" "${admin_password}"
- } >> .env
- unset admin_password
- fi
- chmod 600 .env
- if ! env -u HUGEGRAPH_ADMIN_PASSWORD \
- docker compose -f docker-compose.yml config --quiet ||
- ! env -u HUGEGRAPH_ADMIN_PASSWORD \
- docker compose -f docker-compose.dev.yml config --quiet; then
- echo "docker/.env is incomplete; repair or move it, then retry" >&2
+ command -v openssl >/dev/null
+ jwt_secret="$(openssl rand -hex 32)"
+ test "${#jwt_secret}" -eq 64
+ umask 077
+ test ! -e .env || {
+ echo ".env already exists; edit it instead of overwriting it" >&2
exit 1
- fi
+ }
+ printf "HUGEGRAPH_ADMIN_PASSWORD='%s'\nHUGEGRAPH_AUTH_TOKEN_SECRET='%s'\n" \
+ 'replace-with-your-password' "${jwt_secret}" > .env
)
```
-Compose automatically reads `docker/.env` for `up`, `ps`, `stop`, and `down`. The generated password is a 16-character, Compose-safe random value. The file is excluded from Git and Docker build contexts; keep its permissions restricted and source production credentials from your secret manager instead of committing them.
+Do not commit `.env`. Keeping the same JWT secret preserves authentication
+tokens when containers are recreated. For authenticated topologies with
+multiple Server replicas, all replicas receive this same secret. The HA
+topology fails fast if authentication is enabled without this shared secret.
-### Option A: Quick Start (pre-built images)
+A non-empty `HUGEGRAPH_ADMIN_PASSWORD` enables Server authentication, and
+Hubble detects that mode automatically. Omitting the variable or setting it to
+an empty value disables authentication. Auth-off is only suitable for a
+trusted local environment; never expose it to a public or untrusted network.
+Hubble listens on host loopback by default. Set `HUBBLE_PUBLISH_HOST` only
+behind an HTTPS reverse proxy and trusted network controls.
-Uses pre-built images from Docker Hub. Best for **end users** who want to run HugeGraph quickly. Set `HUGEGRAPH_VERSION` to the same published release for PD, Store, Server, and Hubble. The authenticated PD/Hubble integration is not present in `1.7.x`; if no later compatible release is available, use Option B.
+`HUGEGRAPH_ADMIN_PASSWORD` initializes the built-in `admin` account on its
+first authenticated startup. Changing `.env` does not rotate an existing
+administrator password; use the HugeGraph user API for credential changes.
+
+For the verification commands below, set the password in your current shell:
```bash
-(
- cd docker
- HUGEGRAPH_VERSION='<compatible-release-after-1.7.x>' \
- docker compose up -d
-)
+ADMIN_PASSWORD='the-same-password-used-in-.env'
```
-- Images: matching `hugegraph/pd`, `hugegraph/store`, `hugegraph/server`, and `hugegraph/hubble` tags from the selected compatible release
-- `pull_policy: always` — always pulls the specified image tag
-
-> **Note**: Do not use `latest` to claim a reproducible deployment. Pin a compatible release tag and keep it unchanged for later lifecycle commands.
-- PD healthcheck endpoint: `/v1/health`
-- Hubble is available at `http://localhost:8088`; sign in as `admin` with the required `HUGEGRAPH_ADMIN_PASSWORD`
-- Hubble binds to host loopback by default. Set `HUBBLE_PUBLISH_HOST` explicitly only behind an HTTPS reverse proxy and trusted network controls.
-- Hubble uses PD discovery and the Docker-network Server address
-- Server healthcheck endpoint: `/versions`
+### Standalone
-### Option B: Development Build (build from source)
+This is the recommended quickstart.
-Builds images locally from source Dockerfiles. Best for **developers** who want to test local changes. Build the matching `hugegraph-toolchain` Hubble source as `local/hugegraph-hubble:dev` before starting this stack.
+Start:
-The publishing pipeline uses [`docker/bake.hcl`](./bake.hcl) from the repository root to compile the Java reactor once and build the PD, Store, HStore Server, and standalone Server runtime images from that shared result.
+```bash
+docker compose -f docker-compose.yml up -d --wait
+```
-Run Bake commands from the repository root. Use `--print` to inspect the resolved targets without building, or run the default group to build all four amd64/arm64 images. Loading both platforms under the same local tags requires Docker's containerd image store.
+Status:
```bash
-# Inspect the resolved build graph
-docker buildx bake --file docker/bake.hcl --print
+docker compose -f docker-compose.yml ps
+```
+
+Verify Server readiness, authentication, and Hubble:
-# Build the default multi-platform target group
-IMAGE_TAG=local docker buildx bake --file docker/bake.hcl
+```bash
+curl -fsS http://localhost:8080/versions
+test "$(curl -sS -o /dev/null -w '%{http_code}' \
+ http://localhost:8080/graphspaces/DEFAULT/graphs)" = 401
+test "$(curl -sS -u "admin:${ADMIN_PASSWORD}" -o /dev/null -w '%{http_code}' \
+ http://localhost:8080/graphspaces/DEFAULT/graphs)" = 200
+curl -fsS http://localhost:8088/about
```
-Local source changes are included because Bake uses the current repository working tree as its build context. For routine development, override all targets to the host architecture so the four images still share one Maven build without requiring the multi-platform containerd image store.
+Open `http://localhost:8088` and sign in as `admin` with the password from
+`.env`.
+
+Stop containers while keeping them:
```bash
-# x86_64 host
-IMAGE_TAG=local docker buildx bake --file docker/bake.hcl --set '*.platform=linux/amd64'
+docker compose -f docker-compose.yml stop
+```
-# ARM64 host
-IMAGE_TAG=local docker buildx bake --file docker/bake.hcl --set '*.platform=linux/arm64'
+Remove containers and the network while keeping data:
+
+```bash
+docker compose -f docker-compose.yml down
```
-These local commands can read existing Registry caches but do not publish images or write remote caches because `EXPORT_CACHE` defaults to `false`.
+Delete containers, the network, and all topology data:
```bash
-(
- cd docker
- HUBBLE_IMAGE=local/hugegraph-hubble:dev \
- HUBBLE_PULL_POLICY=never \
- docker compose -f docker-compose.dev.yml up -d
-)
+docker compose -f docker-compose.yml down -v
```
Review Comment:
The repetition is intentional user-facing completeness: each topology section remains directly copyable for start, status, verification, stop, down, and destructive cleanup. Consolidating it would save documentation lines but require placeholders or cross-section substitution, so it is not a functional fix and is outside this minimal pass.
##########
docker/README.md:
##########
@@ -1,436 +1,362 @@
-# HugeGraph Docker Deployment
+# HugeGraph Docker Compose
-This directory contains Docker Compose files for running HugeGraph:
+## Users
-| File | Description |
-|------|-------------|
-| `docker-compose.yml` | PD, Store, Server, and Hubble using pre-built images |
-| `docker-compose.dev.yml` | PD, Store, and Server built from source, plus Hubble |
-| `docker-compose-3pd-3store-3server.yml` | 3-node distributed cluster (PD + Store + Server) |
+### Choose a topology
-## Prerequisites
+Run commands in this directory:
-- **Docker Engine** 20.10+ (or Docker Desktop 4.x+)
-- **Docker Compose** v2 (included in Docker Desktop)
-- **OpenSSL CLI** (used to generate the initial administrator password)
-- **Memory**: Allocate at least **12 GB** to Docker Desktop (Settings → Resources → Memory). The 3-node cluster runs 9 JVM processes (3 PD + 3 Store + 3 Server) which are memory-intensive. Insufficient memory causes OOM kills that appear as silent Raft failures.
+```bash
+cd docker
+```
-> [!IMPORTANT]
-> The 12 GB minimum is for Docker Desktop. On Linux with native Docker, ensure the host has at least 12 GB of free memory.
----
+| Topology | Compose file | Services | When to use it |
+| --- | --- | --- | --- |
+| Standalone | `docker-compose.yml` | 1 RocksDB Server + 1 Hubble | Default; start here |
+| Minimal HStore | `docker-compose-hstore.yml` | 1 PD + 1 Store + 1 Server + 1 Hubble | Distributed local development |
+| HA | `docker-compose-3pd-3store-3server.yml` | 3 PD + 3 Store + 3 Server + 1 Hubble | Reference and evaluation |
-## Single-Node Setup
+Standalone uses `hugegraph/hugegraph:${HUGEGRAPH_VERSION:-latest}`. The HStore
+topologies use the matching `hugegraph/pd`, `hugegraph/store`, and
+`hugegraph/server` tags. Hubble is selected independently with
+`${HUBBLE_IMAGE:-hugegraph/hubble:latest}`.
-Two compose files run one PD, one Store, one Server, and one Hubble instance:
+### Create the authentication environment
-Create a Compose environment file once so every lifecycle command can resolve the required administrator password:
+Create `.env` once. Replace `replace-with-your-password` with an administrator
+password that you choose; the command generates and persists a random 32-byte
+JWT secret. For this simple single-quoted format, do not use a password that
+contains a single quote or newline.
```bash
(
set -eu
- cd docker
- if [ -e .env ]; then
- echo "docker/.env already exists; reusing it"
- else
- command -v openssl >/dev/null 2>&1
- admin_password="$(openssl rand -base64 12)"
- if [ "${#admin_password}" -ne 16 ]; then
- echo "Failed to generate a 16-character password" >&2
- exit 1
- fi
- install -m 600 /dev/null .env
- {
- printf "HUGEGRAPH_ADMIN_PASSWORD='%s'\n" "${admin_password}"
- } >> .env
- unset admin_password
- fi
- chmod 600 .env
- if ! env -u HUGEGRAPH_ADMIN_PASSWORD \
- docker compose -f docker-compose.yml config --quiet ||
- ! env -u HUGEGRAPH_ADMIN_PASSWORD \
- docker compose -f docker-compose.dev.yml config --quiet; then
- echo "docker/.env is incomplete; repair or move it, then retry" >&2
+ command -v openssl >/dev/null
+ jwt_secret="$(openssl rand -hex 32)"
+ test "${#jwt_secret}" -eq 64
+ umask 077
+ test ! -e .env || {
+ echo ".env already exists; edit it instead of overwriting it" >&2
exit 1
- fi
+ }
+ printf "HUGEGRAPH_ADMIN_PASSWORD='%s'\nHUGEGRAPH_AUTH_TOKEN_SECRET='%s'\n" \
+ 'replace-with-your-password' "${jwt_secret}" > .env
)
```
-Compose automatically reads `docker/.env` for `up`, `ps`, `stop`, and `down`. The generated password is a 16-character, Compose-safe random value. The file is excluded from Git and Docker build contexts; keep its permissions restricted and source production credentials from your secret manager instead of committing them.
+Do not commit `.env`. Keeping the same JWT secret preserves authentication
+tokens when containers are recreated. For authenticated topologies with
+multiple Server replicas, all replicas receive this same secret. The HA
+topology fails fast if authentication is enabled without this shared secret.
-### Option A: Quick Start (pre-built images)
+A non-empty `HUGEGRAPH_ADMIN_PASSWORD` enables Server authentication, and
+Hubble detects that mode automatically. Omitting the variable or setting it to
+an empty value disables authentication. Auth-off is only suitable for a
+trusted local environment; never expose it to a public or untrusted network.
+Hubble listens on host loopback by default. Set `HUBBLE_PUBLISH_HOST` only
+behind an HTTPS reverse proxy and trusted network controls.
-Uses pre-built images from Docker Hub. Best for **end users** who want to run HugeGraph quickly. Set `HUGEGRAPH_VERSION` to the same published release for PD, Store, Server, and Hubble. The authenticated PD/Hubble integration is not present in `1.7.x`; if no later compatible release is available, use Option B.
+`HUGEGRAPH_ADMIN_PASSWORD` initializes the built-in `admin` account on its
+first authenticated startup. Changing `.env` does not rotate an existing
+administrator password; use the HugeGraph user API for credential changes.
+
+For the verification commands below, set the password in your current shell:
```bash
-(
- cd docker
- HUGEGRAPH_VERSION='<compatible-release-after-1.7.x>' \
- docker compose up -d
-)
+ADMIN_PASSWORD='the-same-password-used-in-.env'
```
-- Images: matching `hugegraph/pd`, `hugegraph/store`, `hugegraph/server`, and `hugegraph/hubble` tags from the selected compatible release
-- `pull_policy: always` — always pulls the specified image tag
-
-> **Note**: Do not use `latest` to claim a reproducible deployment. Pin a compatible release tag and keep it unchanged for later lifecycle commands.
-- PD healthcheck endpoint: `/v1/health`
-- Hubble is available at `http://localhost:8088`; sign in as `admin` with the required `HUGEGRAPH_ADMIN_PASSWORD`
-- Hubble binds to host loopback by default. Set `HUBBLE_PUBLISH_HOST` explicitly only behind an HTTPS reverse proxy and trusted network controls.
-- Hubble uses PD discovery and the Docker-network Server address
-- Server healthcheck endpoint: `/versions`
+### Standalone
-### Option B: Development Build (build from source)
+This is the recommended quickstart.
-Builds images locally from source Dockerfiles. Best for **developers** who want to test local changes. Build the matching `hugegraph-toolchain` Hubble source as `local/hugegraph-hubble:dev` before starting this stack.
+Start:
-The publishing pipeline uses [`docker/bake.hcl`](./bake.hcl) from the repository root to compile the Java reactor once and build the PD, Store, HStore Server, and standalone Server runtime images from that shared result.
+```bash
+docker compose -f docker-compose.yml up -d --wait
+```
-Run Bake commands from the repository root. Use `--print` to inspect the resolved targets without building, or run the default group to build all four amd64/arm64 images. Loading both platforms under the same local tags requires Docker's containerd image store.
+Status:
```bash
-# Inspect the resolved build graph
-docker buildx bake --file docker/bake.hcl --print
+docker compose -f docker-compose.yml ps
+```
+
+Verify Server readiness, authentication, and Hubble:
-# Build the default multi-platform target group
-IMAGE_TAG=local docker buildx bake --file docker/bake.hcl
+```bash
+curl -fsS http://localhost:8080/versions
+test "$(curl -sS -o /dev/null -w '%{http_code}' \
+ http://localhost:8080/graphspaces/DEFAULT/graphs)" = 401
+test "$(curl -sS -u "admin:${ADMIN_PASSWORD}" -o /dev/null -w '%{http_code}' \
+ http://localhost:8080/graphspaces/DEFAULT/graphs)" = 200
+curl -fsS http://localhost:8088/about
```
-Local source changes are included because Bake uses the current repository working tree as its build context. For routine development, override all targets to the host architecture so the four images still share one Maven build without requiring the multi-platform containerd image store.
+Open `http://localhost:8088` and sign in as `admin` with the password from
+`.env`.
+
+Stop containers while keeping them:
```bash
-# x86_64 host
-IMAGE_TAG=local docker buildx bake --file docker/bake.hcl --set '*.platform=linux/amd64'
+docker compose -f docker-compose.yml stop
+```
-# ARM64 host
-IMAGE_TAG=local docker buildx bake --file docker/bake.hcl --set '*.platform=linux/arm64'
+Remove containers and the network while keeping data:
+
+```bash
+docker compose -f docker-compose.yml down
```
-These local commands can read existing Registry caches but do not publish images or write remote caches because `EXPORT_CACHE` defaults to `false`.
+Delete containers, the network, and all topology data:
```bash
-(
- cd docker
- HUBBLE_IMAGE=local/hugegraph-hubble:dev \
- HUBBLE_PULL_POLICY=never \
- docker compose -f docker-compose.dev.yml up -d
-)
+docker compose -f docker-compose.yml down -v
```
-- PD, Store, and Server images are built from this repository
-- Hubble uses `HUBBLE_IMAGE` because its source is in `hugegraph-toolchain`
-- Server entrypoint scripts are baked into the built image; Hubble mounts the Docker-local PD configuration
-- PD healthcheck endpoint: `/v1/health`
-- Otherwise identical env vars and structure to the quickstart file
+### Minimal HStore
-Use the same release tag for Option A lifecycle commands:
+Start:
```bash
-(
- cd docker
- export HUGEGRAPH_VERSION='<same-compatible-release>'
- docker compose ps
- docker compose stop
- docker compose down
-)
+docker compose -f docker-compose-hstore.yml up -d --wait
```
-Use the development Compose file for every Option B lifecycle command:
+Status:
```bash
-(
- cd docker
- docker compose -f docker-compose.dev.yml ps
- docker compose -f docker-compose.dev.yml stop
- docker compose -f docker-compose.dev.yml down
-)
+docker compose -f docker-compose-hstore.yml ps
```
-### Key Differences
-
-| | `docker-compose.yml` (quickstart) | `docker-compose.dev.yml` (dev build) |
-|---|---|---|
-| **Images** | Pull from Docker Hub | Build from source |
-| **Who it's for** | End users | Developers |
-| **Server pull_policy** | `always` | `build` |
-| **Hubble pull_policy** | `always` | `never` in the workflow above (`missing` in the Compose file by default) |
+Verify PD, Store, Server authentication, and Hubble:
-**Verify** (both options):
```bash
-curl http://localhost:8080/versions
+curl -fsS http://localhost:8620/v1/health
+curl -fsS http://localhost:8520/v1/health
+curl -fsS http://localhost:8080/versions
+test "$(curl -sS -o /dev/null -w '%{http_code}' \
+ http://localhost:8080/graphspaces/DEFAULT/graphs)" = 401
+test "$(curl -sS -u "admin:${ADMIN_PASSWORD}" -o /dev/null -w '%{http_code}' \
+ http://localhost:8080/graphspaces/DEFAULT/graphs)" = 200
curl -fsS http://localhost:8088/about
```
-To validate local images without Compose replacing them with remote `latest`:
+Open `http://localhost:8088` and sign in as `admin` with the password from
+`.env`.
+
+Stop containers while keeping them:
```bash
-(
- cd docker
- HUGEGRAPH_SERVER_IMAGE=local/hugegraph-server:test \
- HUGEGRAPH_SERVER_PULL_POLICY=never \
- HUBBLE_IMAGE=local/hugegraph-hubble:test \
- HUBBLE_PULL_POLICY=never \
- docker compose up -d --wait
-)
+docker compose -f docker-compose-hstore.yml stop
```
----
+Remove containers and the network while keeping data:
+
+```bash
+docker compose -f docker-compose-hstore.yml down
+```
-## 3-Node Cluster Quickstart
+Delete containers, the network, and all topology data:
```bash
-cd docker
-HUGEGRAPH_VERSION=1.7.0 docker compose -f docker-compose-3pd-3store-3server.yml up -d
+docker compose -f docker-compose-hstore.yml down -v
+```
-# To stop and remove all data volumes (clean restart)
-docker compose -f docker-compose-3pd-3store-3server.yml down -v
+### HA reference
+
+The HA topology is resource-intensive. Running it locally is not required on
+resource-constrained machines, but its Compose configuration must always render
+successfully. This PR validates HA by rendering and static review only; it does
+not start HA locally or in default CI.
+
+Start:
+
+```bash
+docker compose -f docker-compose-3pd-3store-3server.yml up -d --wait
```
-**Startup ordering** is enforced via `depends_on` with `condition: service_healthy`:
+Status:
-1. **PD nodes** start first and must pass healthchecks (`/v1/health`)
-2. **Store nodes** start after all PD nodes are healthy
-3. **Server nodes** start after all Store nodes are healthy
+```bash
+docker compose -f docker-compose-3pd-3store-3server.yml ps
+```
-This ensures PD and Store are healthy before the server starts. The server entrypoint still performs a best-effort partition wait after launch, so partition assignment may take a little longer.
+Verify all published PD, Store, and Server endpoints, Server authentication,
+and Hubble:
+
+```bash
+for port in 8620 8621 8622; do
+ curl -fsS "http://localhost:${port}/v1/health"
+done
+for port in 8520 8521 8522; do
+ curl -fsS "http://localhost:${port}/v1/health"
+done
+for port in 8080 8081 8082; do
+ curl -fsS "http://localhost:${port}/versions"
+ test "$(curl -sS -o /dev/null -w '%{http_code}' \
+ "http://localhost:${port}/graphspaces/DEFAULT/graphs")" = 401
+ test "$(curl -sS -u "admin:${ADMIN_PASSWORD}" -o /dev/null \
+ -w '%{http_code}' \
+ "http://localhost:${port}/graphspaces/DEFAULT/graphs")" = 200
+done
+curl -fsS http://localhost:8088/about
+```
-**Verify the cluster is healthy**:
+Open `http://localhost:8088` and sign in as `admin` with the password from
+`.env`.
+
+Stop containers while keeping them:
```bash
-# Check PD health
-curl http://localhost:8620/v1/health
+docker compose -f docker-compose-3pd-3store-3server.yml stop
+```
-# Check Store health
-curl http://localhost:8520/v1/health
+Remove containers and the network while keeping data:
-# Check Server (Graph API)
-curl http://localhost:8080/versions
+```bash
+docker compose -f docker-compose-3pd-3store-3server.yml down
+```
-# List registered stores via PD
-curl http://localhost:8620/v1/stores
+Delete containers, the network, and all topology data:
-# List partitions
-curl http://localhost:8620/v1/partitions
+```bash
+docker compose -f docker-compose-3pd-3store-3server.yml down -v
```
----
-
-## Environment Variable Reference
-
-Configuration is injected via environment variables. The old `docker/configs/application-pd*.yml` and `docker/configs/application-store*.yml` files are no longer used.
-
-### PD Environment Variables
-
-| Variable | Required | Default | Maps To (`application.yml`) | Description |
-|----------|----------|---------|-----------------------------|-------------|
-| `HG_PD_GRPC_HOST` | Yes | — | `grpc.host` | This node's hostname/IP for gRPC |
-| `HG_PD_RAFT_ADDRESS` | Yes | — | `raft.address` | This node's Raft address (e.g. `pd0:8610`) |
-| `HG_PD_RAFT_PEERS_LIST` | Yes | — | `raft.peers-list` | All PD peers (e.g. `pd0:8610,pd1:8610,pd2:8610`) |
-| `HG_PD_INITIAL_STORE_LIST` | Yes | — | `pd.initial-store-list` | Expected stores (e.g. `store0:8500,store1:8500,store2:8500`) |
-| `HG_PD_GRPC_PORT` | No | `8686` | `grpc.port` | gRPC server port |
-| `HG_PD_REST_PORT` | No | `8620` | `server.port` | REST API port |
-| `HG_PD_DATA_PATH` | No | `/hugegraph-pd/pd_data` | `pd.data-path` | Metadata storage path |
-| `HG_PD_INITIAL_STORE_COUNT` | No | `1` | `pd.initial-store-count` | Min stores for cluster availability |
-
-**Deprecated aliases** (still work but log a warning):
-
-| Deprecated | Use Instead |
-|------------|-------------|
-| `GRPC_HOST` | `HG_PD_GRPC_HOST` |
-| `RAFT_ADDRESS` | `HG_PD_RAFT_ADDRESS` |
-| `RAFT_PEERS` | `HG_PD_RAFT_PEERS_LIST` |
-| `PD_INITIAL_STORE_LIST` | `HG_PD_INITIAL_STORE_LIST` |
-
-### Store Environment Variables
-
-| Variable | Required | Default | Maps To (`application.yml`) | Description |
-|----------|----------|---------|-----------------------------|-------------|
-| `HG_STORE_PD_ADDRESS` | Yes | — | `pdserver.address` | PD gRPC addresses (e.g. `pd0:8686,pd1:8686,pd2:8686`) |
-| `HG_STORE_GRPC_HOST` | Yes | — | `grpc.host` | This node's hostname (e.g. `store0`) |
-| `HG_STORE_RAFT_ADDRESS` | Yes | — | `raft.address` | This node's Raft address (e.g. `store0:8510`) |
-| `HG_STORE_GRPC_PORT` | No | `8500` | `grpc.port` | gRPC server port |
-| `HG_STORE_REST_PORT` | No | `8520` | `server.port` | REST API port |
-| `HG_STORE_DATA_PATH` | No | `/hugegraph-store/storage` | `app.data-path` | Data storage path |
-
-**Deprecated aliases** (still work but log a warning):
-
-| Deprecated | Use Instead |
-|------------|-------------|
-| `PD_ADDRESS` | `HG_STORE_PD_ADDRESS` |
-| `GRPC_HOST` | `HG_STORE_GRPC_HOST` |
-| `RAFT_ADDRESS` | `HG_STORE_RAFT_ADDRESS` |
-
-### Server Environment Variables
-
-| Variable | Required | Default | Maps To | Description |
-|----------|----------|---------|-----------------------------|-------------|
-| `HG_SERVER_BACKEND` | Yes | — | `backend` in `hugegraph.properties` | Storage backend (e.g. `hstore`) |
-| `HG_SERVER_PD_PEERS` | Yes | — | `pd.peers` | PD cluster addresses (e.g. `pd0:8686,pd1:8686,pd2:8686`) |
-| `HG_SERVER_CLUSTER` | No | — | `cluster` in `rest-server.properties` | PD discovery application name; single-node Compose uses `hg` to match Hubble |
-| `HG_SERVER_USE_PD` | No | — | `usePD` in `rest-server.properties` | Enables Server PD registration and discovery |
-| `HG_SERVER_REST_URL` | No | — | `restserver.url` | Address registered with PD and used by clients |
-| `HG_SERVER_MIN_FREE_MEMORY` | No | — | `restserver.min_free_memory` | Minimum free-memory guard in MB; local Compose uses `0` |
-| `HG_SERVER_AUTH_TOKEN_SECRET` | No | generated in auth mode | `auth.token_secret` | Shared JWT secret for REST and embedded Gremlin authentication; explicit values must be at least 32 bytes |
-| `STORE_REST` | No | — | Used by `wait-partition.sh` | Store REST endpoint for partition verification (e.g. `store0:8520`) |
-| `PASSWORD` | No | — | Enables auth and sets `auth.admin_pa` | Initial administrator password; disabled init-store does not read it from stdin, but the entrypoint still applies it to the PD bootstrap path |
-| `HG_SERVER_INIT_STORE_ENABLED` | No | `true` | `init_store.enabled` in `rest-server.properties` | Set `false` in PD/HStore deployments so init-store skips local backend and admin initialization |
-
-> **The built-in authenticator with `HG_SERVER_INIT_STORE_ENABLED=false` requires `usePD=true` and an HStore-backed `auth.graph_store`, unless `auth.remote_url` delegates auth elsewhere.** With init-store skipped, the server creates the built-in admin in PD metadata, and only an HStore auth graph uses the PD-backed auth manager that can read that account. init-store exits non-zero when the combination is unusable, rather than leaving a server nobody can log in to. A custom `auth.authenticator` is exempt because it manages its own identities.
->
-> `docker/init_complete` is written by init-store itself, and only after it has initialized. A skipped run therefore records nothing, whether it was disabled by the variable or by the property in a mounted `rest-server.properties`, so a later re-enable is still able to initialize. The marker only short-circuits re-initialization: init-store runs on every container start, and a disabled one performs the fail-closed check above first, so a marker left by an earlier release or an earlier enabled run cannot bypass it.
->
-> The entrypoint maps **`PASSWORD` to `auth.admin_pa`** before init-store runs. A disabled init-store does not read the password from standard input, but the PD startup path uses the explicit `auth.admin_pa` value when it first creates the administrator. Changing it later does not rotate an existing password.
-
-The single-node Compose files also accept these deployment-level overrides:
-
-| Variable | Default | Description |
-|----------|---------|-------------|
-| `HUGEGRAPH_SERVER_IMAGE` | `hugegraph/server:<version>` | Complete Server image reference |
-| `HUGEGRAPH_SERVER_PULL_POLICY` | `always` (`build` for dev) | Server pull policy |
-| `HUBBLE_IMAGE` | `hugegraph/hubble:<version>` | Complete Hubble image reference |
-| `HUBBLE_PULL_POLICY` | `always` (`missing` for dev) | Hubble pull policy |
-| `HUBBLE_PUBLISH_HOST` | `127.0.0.1` | Hubble host bind address; remote access requires an HTTPS reverse proxy |
-| `HUGEGRAPH_ADMIN_PASSWORD` | required (`docker/.env`) | Initial admin password; no public default is provided |
-| `HUGEGRAPH_AUTH_TOKEN_SECRET` | generated | JWT signing secret; explicit values must be at least 32 bytes |
-
-When authentication is enabled and no token secret is supplied, the Server entrypoint generates a random secret and writes it to both authentication configurations. The value is reused on container restart while the container filesystem is preserved. To preserve tokens across container recreation, generate a compatible secret once and add it to the mode-600 `docker/.env`:
+### Select image versions
+
+Set a HugeGraph release for Server, PD, and Store without changing Hubble:
```bash
-(
- set -euo pipefail
- cd docker
- secret_pattern='^[[:space:]]*(export[[:space:]]+)?HUGEGRAPH_AUTH_TOKEN_SECRET[[:space:]]*='
- secret_count="$(grep -Ec "${secret_pattern}" .env || true)"
- case "${secret_count}" in
- 0)
- command -v openssl >/dev/null 2>&1
- token_secret="$(openssl rand -hex 32)"
- LC_ALL=C
- if (( ${#token_secret} != 64 )); then
- echo "Failed to generate a 64-character token secret" >&2
- exit 1
- fi
- printf "HUGEGRAPH_AUTH_TOKEN_SECRET='%s'\n" \
- "${token_secret}" >> .env
- unset token_secret
- echo "Generated HUGEGRAPH_AUTH_TOKEN_SECRET"
- ;;
- 1)
- token_secret="$(
- sed -nE \
- "s/${secret_pattern}'([^']*)'[[:space:]]*$/\\2/p" .env
- )"
- LC_ALL=C
- if (( ${#token_secret} < 32 )); then
- echo "Existing token secret must use the documented single-quoted" \
- "format and contain at least 32 bytes; .env was not changed" >&2
- exit 1
- fi
- unset token_secret
- echo "HUGEGRAPH_AUTH_TOKEN_SECRET already exists; reusing it"
- ;;
- *)
- echo "Duplicate HUGEGRAPH_AUTH_TOKEN_SECRET entries; repair .env" >&2
- exit 1
- ;;
- esac
- chmod 600 .env
-)
+HUGEGRAPH_VERSION=1.7.0 \
+docker compose -f docker-compose-hstore.yml up -d
+```
+
+Select Hubble independently:
+
+```bash
+HUBBLE_IMAGE=hugegraph/hubble:latest \
+docker compose -f docker-compose.yml up -d
```
-The entrypoint rejects shorter explicit values before changing either Server configuration file.
+The Hubble `latest` image is expected to work with HugeGraph Server 1.7 and
+Server `latest`; compatibility with versions older than 1.7 is not promised.
+Pin immutable image references when reproducibility is required.
-**Deprecated aliases** (still work but log a warning):
+### Data persistence
-| Deprecated | Use Instead |
-|------------|-------------|
-| `BACKEND` | `HG_SERVER_BACKEND` |
-| `PD_PEERS` | `HG_SERVER_PD_PEERS` |
+Each topology creates its own normal Compose network and named volumes. No
+network or volume needs to be created in advance.
----
+Standalone stores RocksDB data at `/hugegraph-server/rocksdb-data`. The HStore
+topologies keep PD and Store data in topology-local volumes. Hubble uses
+`jdbc:h2:file:/hubble/data/hubble;DB_CLOSE_ON_EXIT=FALSE` and stores uploaded
+files under `/hubble/data/upload-files`.
-## Port Reference
+`docker compose down` keeps named-volume data. `docker compose down -v`
+intentionally deletes it.
-The table below reflects the published host ports in `docker-compose-3pd-3store-3server.yml`. The single-node Compose file publishes `8620`, `8520`, `8080`, and Hubble `8088`; Hubble defaults to host loopback.
+## Developers
-| Service | Container Port | Host Port | Protocol | Purpose |
-|---------|---------------|-----------|----------|---------|
-| pd0 | 8620 | 8620 | HTTP | REST API |
-| pd0 | 8686 | 8686 | gRPC | PD gRPC |
-| pd0 | 8610 | — | TCP | Raft (internal only) |
-| pd1 | 8620 | 8621 | HTTP | REST API |
-| pd1 | 8686 | 8687 | gRPC | PD gRPC |
-| pd2 | 8620 | 8622 | HTTP | REST API |
-| pd2 | 8686 | 8688 | gRPC | PD gRPC |
-| store0 | 8500 | 8500 | gRPC | Store gRPC |
-| store0 | 8510 | 8510 | TCP | Raft |
-| store0 | 8520 | 8520 | HTTP | REST API |
-| store1 | 8500 | 8501 | gRPC | Store gRPC |
-| store1 | 8510 | 8511 | TCP | Raft |
-| store1 | 8520 | 8521 | HTTP | REST API |
-| store2 | 8500 | 8502 | gRPC | Store gRPC |
-| store2 | 8510 | 8512 | TCP | Raft |
-| store2 | 8520 | 8522 | HTTP | REST API |
-| server0 | 8080 | 8080 | HTTP | Graph API |
-| server1 | 8080 | 8081 | HTTP | Graph API |
-| server2 | 8080 | 8082 | HTTP | Graph API |
+### Images and Compose files
----
+| Image | Build file |
+| --- | --- |
+| `hugegraph/hugegraph` (standalone RocksDB Server) | `hugegraph-server/Dockerfile` |
+| `hugegraph/server` (HStore Server) | `hugegraph-server/Dockerfile-hstore` |
+| `hugegraph/pd` | `hugegraph-pd/Dockerfile` |
+| `hugegraph/store` | `hugegraph-store/Dockerfile` |
-## Healthcheck Endpoints
+Hubble is built from the separate HugeGraph Toolchain repository and is
+selected here with `HUBBLE_IMAGE`.
-| Service | Endpoint | Expected |
-|---------|----------|----------|
-| PD | `GET /v1/health` | `200 OK` |
-| Store | `GET /v1/health` | `200 OK` |
-| Server | `GET /versions` | `200 OK` with version JSON |
-| Hubble | `GET /about` | `200` JSON with Hubble name and version |
+The Compose mapping is intentionally small:
----
+- `docker-compose.yml` is the standalone user default.
+- `docker-compose-hstore.yml` is the minimal 1 PD + 1 Store + 1 Server base.
+- `docker-compose-3pd-3store-3server.yml` is the HA reference.
+- `docker-compose.dev.yml` is a thin source-build override for the minimal
+ HStore topology. It does not duplicate runtime services, networks, volumes,
+ health checks, or Hubble.
-## Troubleshooting
+Build and start the minimal topology from local source:
-### Containers Exiting or Restarting (OOM Kills)
+```bash
+docker compose \
+ -f docker-compose-hstore.yml \
+ -f docker-compose.dev.yml \
+ up -d --build --wait
+```
-**Symptom**: Containers exit with code 137, or restart loops. Raft logs show election timeouts.
+Use both files for every later lifecycle command, for example:
-**Cause**: Docker Desktop does not have enough memory. The 9 JVM processes require at least 12 GB.
+```bash
+docker compose \
+ -f docker-compose-hstore.yml \
+ -f docker-compose.dev.yml \
+ down
+```
-**Fix**: Docker Desktop → Settings → Resources → Memory → set to **12 GB** or higher. Restart Docker Desktop.
+The development overlay builds `hugegraph/pd:dev`, `hugegraph/store:dev`, and
+`hugegraph/server:dev`. To reuse those local images and a locally built Hubble
+without pulling replacements:
```bash
-# Check if containers were OOM killed
-docker inspect hg-pd0 | grep -i oom
-docker stats --no-stream
+HUGEGRAPH_VERSION=dev \
+HUGEGRAPH_PULL_POLICY=never \
+HUBBLE_IMAGE=local/hugegraph-hubble:test \
+HUBBLE_PULL_POLICY=never \
+docker compose -f docker-compose-hstore.yml up -d --wait
```
-### Raft Leader Election Failure
+### Hubble configuration
+
+The three small files under `conf/hubble/` contain only topology-specific
+discovery settings and container paths:
-**Symptom**: PD logs show repeated `Leader election timeout`. Store nodes cannot register.
+- `conf/hubble/standalone.properties` uses direct Server mode.
+- `conf/hubble/hstore.properties` uses one PD and one Store REST target.
+- `conf/hubble/hstore-ha.properties` uses all three PD peers and all three
+ allowed Store REST targets.
-**Cause**: PD nodes cannot reach each other on the Raft port (8610), or `HG_PD_RAFT_PEERS_LIST` is misconfigured.
+Hubble detects Server authentication through the Server API. Do not add an
+`auth.enabled` property or duplicate auth-on/auth-off configurations.
-**Fix**:
-1. Verify all PD containers are running: `docker compose -f docker-compose-3pd-3store-3server.yml ps`
-2. Check PD logs: `docker logs hg-pd0`
-3. Verify network connectivity: `docker exec hg-pd0 ping pd1`
-4. Ensure `HG_PD_RAFT_PEERS_LIST` is identical on all PD nodes
+### Render and smoke checks
-### Partition Assignment Not Completing
+Render every topology with auth-on inputs before submitting a change:
-**Symptom**: Server starts but graph operations fail. Store logs show `partition not found`.
+```bash
+for file in \
+ docker-compose.yml \
+ docker-compose-hstore.yml \
+ docker-compose-3pd-3store-3server.yml
+do
+ docker compose -f "${file}" config --quiet
+done
+docker compose \
+ -f docker-compose-hstore.yml \
+ -f docker-compose.dev.yml \
+ config --quiet
Review Comment:
Confirmed and fixed in 9cf8de733. The hand-written loop could render auth-off while claiming auth-on. The README now calls bash test-compose.sh render, which injects deterministic auth-on inputs and checks all three topologies plus the developer overlay with the same contracts used by CI.
##########
docker/README.md:
##########
@@ -1,436 +1,362 @@
-# HugeGraph Docker Deployment
+# HugeGraph Docker Compose
-This directory contains Docker Compose files for running HugeGraph:
+## Users
-| File | Description |
-|------|-------------|
-| `docker-compose.yml` | PD, Store, Server, and Hubble using pre-built images |
-| `docker-compose.dev.yml` | PD, Store, and Server built from source, plus Hubble |
-| `docker-compose-3pd-3store-3server.yml` | 3-node distributed cluster (PD + Store + Server) |
+### Choose a topology
-## Prerequisites
+Run commands in this directory:
-- **Docker Engine** 20.10+ (or Docker Desktop 4.x+)
-- **Docker Compose** v2 (included in Docker Desktop)
-- **OpenSSL CLI** (used to generate the initial administrator password)
-- **Memory**: Allocate at least **12 GB** to Docker Desktop (Settings → Resources → Memory). The 3-node cluster runs 9 JVM processes (3 PD + 3 Store + 3 Server) which are memory-intensive. Insufficient memory causes OOM kills that appear as silent Raft failures.
+```bash
+cd docker
+```
-> [!IMPORTANT]
-> The 12 GB minimum is for Docker Desktop. On Linux with native Docker, ensure the host has at least 12 GB of free memory.
----
+| Topology | Compose file | Services | When to use it |
+| --- | --- | --- | --- |
+| Standalone | `docker-compose.yml` | 1 RocksDB Server + 1 Hubble | Default; start here |
+| Minimal HStore | `docker-compose-hstore.yml` | 1 PD + 1 Store + 1 Server + 1 Hubble | Distributed local development |
+| HA | `docker-compose-3pd-3store-3server.yml` | 3 PD + 3 Store + 3 Server + 1 Hubble | Reference and evaluation |
-## Single-Node Setup
+Standalone uses `hugegraph/hugegraph:${HUGEGRAPH_VERSION:-latest}`. The HStore
+topologies use the matching `hugegraph/pd`, `hugegraph/store`, and
+`hugegraph/server` tags. Hubble is selected independently with
+`${HUBBLE_IMAGE:-hugegraph/hubble:latest}`.
-Two compose files run one PD, one Store, one Server, and one Hubble instance:
+### Create the authentication environment
-Create a Compose environment file once so every lifecycle command can resolve the required administrator password:
+Create `.env` once. Replace `replace-with-your-password` with an administrator
+password that you choose; the command generates and persists a random 32-byte
+JWT secret. For this simple single-quoted format, do not use a password that
+contains a single quote or newline.
```bash
(
set -eu
- cd docker
- if [ -e .env ]; then
- echo "docker/.env already exists; reusing it"
- else
- command -v openssl >/dev/null 2>&1
- admin_password="$(openssl rand -base64 12)"
- if [ "${#admin_password}" -ne 16 ]; then
- echo "Failed to generate a 16-character password" >&2
- exit 1
- fi
- install -m 600 /dev/null .env
- {
- printf "HUGEGRAPH_ADMIN_PASSWORD='%s'\n" "${admin_password}"
- } >> .env
- unset admin_password
- fi
- chmod 600 .env
- if ! env -u HUGEGRAPH_ADMIN_PASSWORD \
- docker compose -f docker-compose.yml config --quiet ||
- ! env -u HUGEGRAPH_ADMIN_PASSWORD \
- docker compose -f docker-compose.dev.yml config --quiet; then
- echo "docker/.env is incomplete; repair or move it, then retry" >&2
+ command -v openssl >/dev/null
+ jwt_secret="$(openssl rand -hex 32)"
+ test "${#jwt_secret}" -eq 64
Review Comment:
Confirmed these lines are defensive rather than strictly necessary under set -eu, but keeping them. They make the OpenSSL dependency and the expected 64-hex-character invariant explicit in the copied bootstrap block, at a cost of two simple lines and without adding another code path.
##########
docker/test-compose.sh:
##########
@@ -0,0 +1,403 @@
+#!/usr/bin/env bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+set -Eeuo pipefail
+
+DOCKER_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+PASSWORD="ci-compose-password"
+SECRET="0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"
+VERSION="ci-version"
+RENDER_HUBBLE_IMAGE="example.invalid/hugegraph/hubble:ci"
+DATASOURCE="jdbc:h2:file:/hubble/data/hubble;DB_CLOSE_ON_EXIT=FALSE"
+CURL_TIMEOUTS=(--connect-timeout 5 --max-time 15)
+ACTIVE_PROJECT=""
+ACTIVE_FILES=()
+RENDER_DIR=""
+
+compose_auth() {
+ env HUGEGRAPH_VERSION="${VERSION}" \
+ HUBBLE_IMAGE="${RENDER_HUBBLE_IMAGE}" \
+ HUGEGRAPH_ADMIN_PASSWORD="${PASSWORD}" \
+ HUGEGRAPH_AUTH_TOKEN_SECRET="${SECRET}" \
+ docker compose "$@"
+}
+
+render() {
+ local output="$1"
+ shift
+ compose_auth "$@" config --format json > "${output}"
+}
+
+assert_file_property() {
+ local file="$1"
+ local property="$2"
+ grep -Fqx "${property}" "${file}"
+}
+
+assert_common() {
+ local rendered="$1"
+ local services="$2"
+ local volumes="$3"
+
+ jq -e \
+ --arg password "${PASSWORD}" \
+ --arg secret "${SECRET}" \
Review Comment:
Thanks for the mutation check; agreed that these contracts and smoke-auth-off should stay. No removal was made. The independent pass also added the missing one-line HG_SERVER_CLUSTER == hg assertion for the minimal HStore render in 9cf8de733.
##########
docker/docker-compose-3pd-3store-3server.yml:
##########
@@ -58,6 +59,18 @@ x-store-common: &store-common
retries: 40
start_period: 120s
+x-server-environment: &server-environment
+ STORE_REST: store0:8520
+ HG_SERVER_BACKEND: hstore
+ HG_SERVER_PD_PEERS: pd0:8686,pd1:8686,pd2:8686
+ HG_SERVER_CLUSTER: hg
+ HG_SERVER_USE_PD: "true"
+ HG_SERVER_MIN_FREE_MEMORY: "0"
+ HG_SERVER_INIT_STORE_ENABLED: "false"
+ HG_SERVER_REQUIRE_AUTH_TOKEN_SECRET: "true"
+ HG_SERVER_AUTH_TOKEN_SECRET: ${HUGEGRAPH_AUTH_TOKEN_SECRET:-}
+ PASSWORD: ${HUGEGRAPH_ADMIN_PASSWORD:-}
Review Comment:
Agreed: this is a possible follow-up cleanup, not a request against this PR. PD/Store anchor refactoring is outside the changed behavior and would expand the review surface for a small line-count gain, so it remains unchanged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(docker): add attachable Hubble Compose add-on for the 3-node cluster [hugegraph]
Posted by "bitflicker64 (via GitHub)" <gi...@apache.org>.
bitflicker64 commented on code in PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149#discussion_r3789870293
##########
docker/README.md:
##########
@@ -168,14 +173,109 @@ To validate local images without Compose replacing them with remote `latest`:
## 3-Node Cluster Quickstart
+The cluster and the Hubble add-on share one named Docker network so Hubble
+can attach to a running cluster without touching it. Treat that network as a
+trust boundary: only the Server layer performs real authentication. Store
+serves its control APIs with no credentials at all, and while PD's REST
+control APIs do require an `Authorization` header, PD only checks that the
+Basic-auth *user* is one of its internal service names and never validates
+the password — so any client on the network can read and drive them. Note
+that this stack depends on that behaviour: Hubble reaches PD as the service
+name `hubble` with an empty password, so tightening PD's credential check
+would also break Hubble's operations view. Any container on the host can
+also join the network by declaring the well-known name. One-time setup:
+write the required credentials to a mode-600 `docker/.env` and create the
+network.
+The cluster file requires both credentials — the admin password enables
+authentication, and every Server replica must share one token secret so a
+token issued by any server validates on all of them. The `:?` guards fire
+on every Compose subcommand, including `down`.
+
+```bash
+(
+ set -eu
+ cd docker
+ command -v openssl >/dev/null 2>&1 || { echo "openssl not found" >&2; exit 1; }
+ [ -e .env ] || install -m 600 /dev/null .env
+ chmod 600 .env
+ # Keep appends on their own lines even if the file was hand-edited.
+ [ ! -s .env ] || [ -z "$(tail -c1 .env)" ] || printf '\n' >> .env
+ pat='^[[:space:]]*(export[[:space:]]+)?'
+ if ! grep -Eq "${pat}HUGEGRAPH_ADMIN_PASSWORD=" .env; then
+ admin_password="$(openssl rand -base64 12)"
+ printf "HUGEGRAPH_ADMIN_PASSWORD='%s'\n" "${admin_password}" >> .env
+ unset admin_password
+ fi
+ if ! grep -Eq "${pat}HUGEGRAPH_AUTH_TOKEN_SECRET=" .env; then
+ token_secret="$(openssl rand -hex 32)"
+ printf "HUGEGRAPH_AUTH_TOKEN_SECRET='%s'\n" "${token_secret}" >> .env
+ unset token_secret
+ fi
+ admin_value="$(sed -nE "s/${pat}HUGEGRAPH_ADMIN_PASSWORD='([^']*)'[[:space:]]*$/\\1/p" .env | tail -n1)"
Review Comment:
Agreed and also in scope. The optional export group made \1 capture the wrong text. I’ll fix the setup parser to read the value group (and keep parsing as data, not by sourcing .env).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(docker): add attachable Hubble Compose add-on for the 3-node cluster [hugegraph]
Posted by "bitflicker64 (via GitHub)" <gi...@apache.org>.
bitflicker64 commented on code in PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149#discussion_r3789871426
##########
docker/docker-compose-hubble.yml:
##########
@@ -0,0 +1,53 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+# Hubble add-on for the distributed cluster defined in
+# docker-compose-3pd-3store-3server.yml. Requires the pre-created cluster
+# network (docker network create hugegraph-net). See "Hubble for the
+# 3-Node Cluster" in docker/README.md for the attach and combined flows.
+
+networks:
+ hg-net:
+ external: true
+ name: ${HUGEGRAPH_NETWORK:-hugegraph-net}
+
+volumes:
+ hg-hubble-db:
+ hg-hubble-upload-files:
+
+services:
+ hubble:
+ # Pin a release via HUGEGRAPH_VERSION in docker/.env; unset, the image
+ # tag defaults to latest.
+ image: ${HUBBLE_IMAGE:-hugegraph/hubble:${HUGEGRAPH_VERSION:-latest}}
+ pull_policy: ${HUBBLE_PULL_POLICY:-missing}
+ container_name: hg-hubble
+ hostname: hubble
+ restart: unless-stopped
+ networks: [hg-net]
+ ports:
+ - "${HUBBLE_PUBLISH_HOST:-127.0.0.1}:8088:8088"
+ volumes:
+ - ./hugegraph-hubble-3x3.properties:/hubble/conf/hugegraph-hubble.properties:ro
+ - hg-hubble-db:/hubble/db
Review Comment:
I’ll fix the persistence boundary so Hubble state survives recreate.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(docker): add attachable Hubble Compose add-on for the 3-node cluster [hugegraph]
Posted by "bitflicker64 (via GitHub)" <gi...@apache.org>.
bitflicker64 commented on code in PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149#discussion_r3767870065
##########
docker/hugegraph-hubble-3x3.properties:
##########
@@ -0,0 +1,33 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+server.host=0.0.0.0
+server.port=8088
+
+cluster=hg
+idc=docker
+
+pd.enabled=true
+# Bootstrap target; the live topology comes from PD discovery. If this
+# replica is down when Hubble starts, point it at another server.
+server.direct_url=http://server0:8080
Review Comment:
Correction: addressed in 1b4f00890. The server.direct_url comment now says it is unused when pd.enabled=true and applies only to standalone pd.enabled=false mode.
##########
.github/workflows/server-ci.yml:
##########
@@ -138,6 +138,107 @@ jobs:
check_compose docker/docker-compose.yml always always
check_compose docker/docker-compose.dev.yml build missing
+ check_cluster_compose() {
+ local cluster="docker/docker-compose-3pd-3store-3server.yml"
+ local addon="docker/docker-compose-hubble.yml"
+ local rendered
+ rendered="$(mktemp)"
+
+ # Both cluster credentials are required and may not be empty.
+ if HUGEGRAPH_AUTH_TOKEN_SECRET=ci-test-secret \
+ env -u HUGEGRAPH_ADMIN_PASSWORD \
+ docker compose -f "$cluster" config -q >/dev/null 2>&1; then
+ echo "$cluster accepted an unset admin password" >&2
+ return 1
+ fi
+ if HUGEGRAPH_ADMIN_PASSWORD= \
+ HUGEGRAPH_AUTH_TOKEN_SECRET=ci-test-secret \
+ docker compose -f "$cluster" config -q >/dev/null 2>&1; then
+ echo "$cluster accepted an empty admin password" >&2
+ return 1
+ fi
+ if HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ env -u HUGEGRAPH_AUTH_TOKEN_SECRET \
+ docker compose -f "$cluster" config -q >/dev/null 2>&1; then
+ echo "$cluster accepted an unset token secret" >&2
+ return 1
+ fi
+ if HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ HUGEGRAPH_AUTH_TOKEN_SECRET= \
+ docker compose -f "$cluster" config -q >/dev/null 2>&1; then
+ echo "$cluster accepted an empty token secret" >&2
+ return 1
+ fi
+
+ # The add-on alone must define Hubble and nothing else, join the
+ # shared external network with its default name, and need no
+ # credentials or overrides.
+ env -u HUGEGRAPH_ADMIN_PASSWORD -u HUGEGRAPH_AUTH_TOKEN_SECRET \
+ -u HUGEGRAPH_NETWORK -u HUGEGRAPH_VERSION \
+ docker compose -f "$addon" config --format json > "$rendered"
+ jq -e '
+ (.services | keys) == ["hubble"] and
+ .networks."hg-net".external == true and
+ .networks."hg-net".name == "hugegraph-net" and
+ (.services.hubble | has("depends_on") | not) and
+ any(.services.hubble.volumes[];
+ .target == "/hubble/conf/hugegraph-hubble.properties" and
+ (.source | endswith("hugegraph-hubble-3x3.properties")))
+ ' "$rendered" >/dev/null
+
+ # The combined render carries the PD-registration and auth
+ # settings on every server replica and keeps Hubble on loopback.
+ # Rendered with non-default HUGEGRAPH_NETWORK/HUGEGRAPH_VERSION so
+ # CI fails if any file stops honoring the overrides (the add-on
+ # render above covers the defaults).
+ HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ HUGEGRAPH_AUTH_TOKEN_SECRET=ci-test-secret \
Review Comment:
Correction: addressed in commit 1b4f00890. The CI fixture is ci-test-token-secret-32-bytes-long (>=32 bytes), and the cluster render uses it for every Server replica. Guard tests run with --env-file /dev/null; the repository entrypoint test complements Compose validation.
##########
docker/docker-compose-hubble.yml:
##########
@@ -0,0 +1,47 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+# Hubble add-on for the distributed cluster defined in
+# docker-compose-3pd-3store-3server.yml. Requires the pre-created cluster
+# network (docker network create hugegraph-net). See "Hubble for the
+# 3-Node Cluster" in docker/README.md for the attach and combined flows.
+
+networks:
+ hg-net:
+ external: true
+ name: ${HUGEGRAPH_NETWORK:-hugegraph-net}
+
+services:
+ hubble:
+ # Pin a release via HUGEGRAPH_VERSION in docker/.env; unset, the image
+ # tag defaults to latest.
+ image: ${HUBBLE_IMAGE:-hugegraph/hubble:${HUGEGRAPH_VERSION:-latest}}
+ pull_policy: ${HUBBLE_PULL_POLICY:-missing}
+ container_name: hg-hubble
+ hostname: hubble
+ restart: unless-stopped
+ networks: [hg-net]
+ ports:
+ - "${HUBBLE_PUBLISH_HOST:-127.0.0.1}:8088:8088"
+ volumes:
+ - ./hugegraph-hubble-3x3.properties:/hubble/conf/hugegraph-hubble.properties:ro
Review Comment:
Correction: addressed in 1b4f00890. The add-on mounts named volumes hg-hubble-db at /hubble/db and hg-hubble-upload-files at /hubble/upload-files; CI asserts both mounts and the README documents persistence across Hubble recreation.
##########
docker/hugegraph-hubble-3x3.properties:
##########
@@ -0,0 +1,33 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+server.host=0.0.0.0
+server.port=8088
+
+cluster=hg
+idc=docker
+
+pd.enabled=true
+# Bootstrap target; the live topology comes from PD discovery. If this
+# replica is down when Hubble starts, point it at another server.
+server.direct_url=http://server0:8080
+pd.peers=pd0:8686,pd1:8686,pd2:8686
+# PD REST endpoint for the operations view (single address).
+pd.server=pd0:8620
Review Comment:
Correction: addressed as an explicit limitation in 1b4f00890. Hubble LiveOperationsCollector accepts one pd.server target for operations HTTP calls; pd.peers is discovery only, not HTTP failover. The docs now state that pd0 loss blinds operations and that operators must repoint pd.server. Automatic failover requires a separate Hubble code change.
##########
docker/README.md:
##########
@@ -168,14 +173,87 @@ To validate local images without Compose replacing them with remote `latest`:
## 3-Node Cluster Quickstart
+The cluster and the Hubble add-on share one named Docker network so Hubble
+can attach to a running cluster without touching it. Treat that network as a
+trust boundary: PD and Store expose unauthenticated control APIs on it (only
+the Server layer authenticates), and any container on the host can join it
+by declaring the well-known name. One-time setup: write the required
+credentials to a mode-600 `docker/.env` and create the network.
+The cluster file requires both credentials — the admin password enables
+authentication, and every Server replica must share one token secret so a
+token issued by any server validates on all of them. The `:?` guards fire
+on every Compose subcommand, including `down`.
+
+```bash
+(
+ set -eu
+ cd docker
+ command -v openssl >/dev/null 2>&1 || { echo "openssl not found" >&2; exit 1; }
+ [ -e .env ] || install -m 600 /dev/null .env
+ chmod 600 .env
+ # Keep appends on their own lines even if the file was hand-edited.
+ [ ! -s .env ] || [ -z "$(tail -c1 .env)" ] || printf '\n' >> .env
+ pat='^[[:space:]]*(export[[:space:]]+)?'
+ if ! grep -Eq "${pat}HUGEGRAPH_ADMIN_PASSWORD=" .env; then
Review Comment:
Correction: addressed in 1b4f00890. The validator isolates unset and empty admin/token cases, checks that the error names the intended variable, and the setup block rejects existing empty or short credentials with a repair message.
##########
docker/README.md:
##########
@@ -205,6 +283,110 @@ curl http://localhost:8620/v1/partitions
---
+## Hubble for the 3-Node Cluster
+
+`docker-compose-hubble.yml` defines only the Hubble service. It joins the
+cluster's external network (`hugegraph-net` by default, override with
+`HUGEGRAPH_NETWORK`) and has no `depends_on` on cluster services, so
+starting, stopping, or upgrading Hubble never recreates or restarts PD,
+Store, or Server containers. Hubble reads the cluster topology from
+`hugegraph-hubble-3x3.properties`; adjust that file when attaching to a
+cluster with different hostnames.
+
+Sign in at `http://localhost:8088` as `admin` with the
+`HUGEGRAPH_ADMIN_PASSWORD` from `docker/.env`. Hubble binds to host
+loopback by default (`HUBBLE_PUBLISH_HOST`, same caveats as the
+single-node setup).
+
+The two flows below create Hubble in different Compose projects, so manage
+Hubble with the same flags you started it with: the attach flow always uses
+`-p hugegraph-hubble -f docker-compose-hubble.yml`, the combined flow always
+uses both `-f` flags. The explicit `-p` keeps the attach project independent
+of the directory name and of other Compose projects.
+
+Run one Hubble per host: the single-node stack and both add-on flows all
+publish `127.0.0.1:8088` and name their container `hg-hubble`. The two
+add-on flows are therefore mutually exclusive — starting one while the
+other's Hubble exists fails with a container-name conflict, so `down` the
+flow you are leaving before switching.
+
+### Attach to a running cluster
+
+With the 3-node cluster already up:
+
+```bash
+cd docker
+docker compose -p hugegraph-hubble -f docker-compose-hubble.yml up -d
Review Comment:
Correction: addressed in 1b4f00890. The README distinguishes attach from combined mode, explains that attach requires an existing cluster with the new PD-registration and auth settings, and documents the upgrade-first path for older clusters.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(docker): add attachable Hubble Compose add-on for the 3-node cluster [hugegraph]
Posted by "bitflicker64 (via GitHub)" <gi...@apache.org>.
bitflicker64 commented on PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149#issuecomment-5410097128
Fixed in 96160375, verified against a real cluster.
The advertised-address point is the important one and you were right: `HG_SERVER_REST_URL` becomes `restserver.url`, which is the bind address. Running the cluster with the documented override crash-looped that replica on `java.net.BindException: Cannot assign requested address` while the two defaults stayed healthy. The three variables are gone and the README documents the limitation instead. My earlier reply on that thread was wrong.
Also fixed: healthcheck no longer pins to the seeded password (401-only, so password rotation cannot brick readiness), Hubble volumes are external so `down -v` from either flow cannot destroy shared state, pull policy is overridable for offline use, the non-auth flow documents network/volumes/teardown, the dotenv reader rejects unquoted values and runs in a subshell, and CI unsets the variables it asserts defaults for, covers the non-auth properties, and gates the smoke test to contract changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(docker): add attachable Hubble Compose add-on for the 3-node cluster [hugegraph]
Posted by "bitflicker64 (via GitHub)" <gi...@apache.org>.
bitflicker64 commented on code in PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149#discussion_r3839620289
##########
docker/docker-compose-3pd-3store-3server.yml:
##########
@@ -187,16 +207,25 @@ services:
<<: *server-common
container_name: hg-server0
hostname: server0
+ environment:
+ <<: *server-env
+ HG_SERVER_REST_URL: http://server0:8080
Review Comment:
Fixed in c3115579 rather than deferred.
Each replica's registered URL is now `${HUGEGRAPH_SERVER0_REST_URL:-http://server0:8080}` and siblings, so a PD-aware client outside Docker can be given addresses it can resolve. The defaults are unchanged, so Hubble and anything else on `hugegraph-net` behave exactly as before.
The README explains when the container-name defaults are wrong and that widening the Server ports is part of that setup.
##########
.github/workflows/server-ci.yml:
##########
@@ -138,6 +140,189 @@ jobs:
check_compose docker/docker-compose.yml always always
check_compose docker/docker-compose.dev.yml build missing
+ check_cluster_compose() {
+ local cluster="docker/docker-compose-3pd-3store-3server.yml"
+ local addon="docker/docker-compose-hubble.yml"
+ local rendered
+ local token_fixture=ci-test-token-secret-32-bytes-long
+ rendered="$(mktemp)"
+ if [ "${#token_fixture}" -lt 32 ]; then
+ echo "CI token fixture must be at least 32 bytes" >&2
+ return 1
+ fi
+
+ # RETURN only: an EXIT trap would fire after this function's
+ # `local rendered` has gone out of scope, which `set -u` turns
+ # into an "unbound variable" error. A hard errexit abort can
+ # therefore still leak one temp file, which is acceptable on an
+ # ephemeral runner.
+ trap 'rm -f "$rendered"' RETURN
+
+ # --env-file /dev/null on every invocation: Compose otherwise reads
+ # docker/.env automatically, and the quickstart tells operators to
+ # create one holding exactly the credentials these assertions
+ # control. Without it the checks pass in CI (which has no .env) but
+ # report false failures for anyone running them locally after
+ # following the quickstart, and the add-on render below — which
+ # deliberately unsets the variables to assert their defaults — would
+ # read whatever HUGEGRAPH_NETWORK/HUGEGRAPH_VERSION that .env holds.
+ # Pinning an empty env file makes both renders depend only on what
+ # each invocation sets explicitly.
+
+ # Both cluster credentials are required and may not be empty.
+ # Each case asserts the guard fired for the *intended* variable:
+ # a bare non-zero exit would also be produced by a YAML error, a
+ # renamed file, or a missing docker binary.
+ assert_guard() { # assert_guard <blamed-var> <description>
+ local var="$1" desc="$2" err
+ if err="$(docker compose --env-file /dev/null -f "$cluster" \
+ config -q 2>&1)"; then
+ echo "$cluster accepted $desc" >&2
+ return 1
+ fi
+ case "$err" in
+ *"$var"*) : ;;
+ *) echo "$cluster rejected $desc, but not because of $var: $err" >&2
+ return 1 ;;
+ esac
+ }
+
+ ( unset HUGEGRAPH_ADMIN_PASSWORD
+ export HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}"
+ assert_guard HUGEGRAPH_ADMIN_PASSWORD "an unset admin password" )
+ ( export HUGEGRAPH_ADMIN_PASSWORD=
+ export HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}"
+ assert_guard HUGEGRAPH_ADMIN_PASSWORD "an empty admin password" )
+ ( unset HUGEGRAPH_AUTH_TOKEN_SECRET
+ export HUGEGRAPH_ADMIN_PASSWORD=ci-test-password
+ assert_guard HUGEGRAPH_AUTH_TOKEN_SECRET "an unset token secret" )
+ ( export HUGEGRAPH_ADMIN_PASSWORD=ci-test-password
+ export HUGEGRAPH_AUTH_TOKEN_SECRET=
+ assert_guard HUGEGRAPH_AUTH_TOKEN_SECRET "an empty token secret" )
+ # The add-on alone must define Hubble and nothing else, join the
+ # shared external network with its default name, and need no
+ # credentials or overrides.
+ env -u HUGEGRAPH_ADMIN_PASSWORD -u HUGEGRAPH_AUTH_TOKEN_SECRET \
+ -u HUGEGRAPH_NETWORK -u HUGEGRAPH_VERSION \
+ -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY -u HUBBLE_PUBLISH_HOST \
+ docker compose --env-file /dev/null -f "$addon" config --format json > "$rendered"
+ jq -e '
+ (.services | keys) == ["hubble"] and
+ .networks."hg-net".external == true and
+ .networks."hg-net".name == "hugegraph-net" and
+ (.services.hubble.networks | has("hg-net")) and
+ (.services.hubble | has("depends_on") | not) and
+ any(.services.hubble.volumes[];
+ .target == "/hubble/conf/hugegraph-hubble.properties" and
+ (.source | endswith("hugegraph-hubble-3x3.properties")))
+ and any(.services.hubble.volumes[];
+ .source == "hg-hubble-db" and
+ .target == "/hubble/db")
+ and any(.services.hubble.volumes[];
+ .source == "hg-hubble-upload-files" and
+ .target == "/hubble/upload-files")
+ ' "$rendered" >/dev/null
+
+ # The cluster's own default network name must match the add-on's,
+ # or the attach flow and the cluster land on different networks.
+ # The combined render below pins an override, so it cannot catch a
+ # drifting default.
+ HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}" \
+ env -u HUGEGRAPH_NETWORK -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY \
+ -u HUBBLE_PUBLISH_HOST \
+ docker compose --env-file /dev/null -f "$cluster" \
+ config --format json > "$rendered"
+ jq -e '.networks."hg-net".name == "hugegraph-net"' \
+ "$rendered" >/dev/null
+
+ # The combined render carries the PD-registration and auth
+ # settings on every server replica and keeps Hubble on loopback.
+ # Rendered with non-default HUGEGRAPH_NETWORK/HUGEGRAPH_VERSION so
+ # CI fails if any file stops honoring the overrides (the add-on
+ # render above covers the defaults).
+ HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}" \
+ HUGEGRAPH_NETWORK=ci-test-net \
+ HUGEGRAPH_VERSION=ci-test-tag \
+ env -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY -u HUBBLE_PUBLISH_HOST \
+ docker compose --env-file /dev/null -f "$cluster" -f "$addon" \
Review Comment:
Fixed in c3115579 and corrected in 10c6c6b5.
CI now starts the cluster, checks 401 unauthenticated and 200 authenticated on all three replicas, checks three PD peers and three Stores `Up`, attaches Hubble, confirms no cluster container is recreated, and proves Hubble state survives recreation by writing through the attach flow and reading back through the combined flow.
Writing it against a real cluster caught two bad assertions of my own: `/v1/cluster` returns PD peers, not graph servers, and the H2 check passed even when the two flows used different volumes. Both fixed in 10c6c6b5, which is the argument for the job existing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(docker): add attachable Hubble Compose add-on for the 3-node cluster [hugegraph]
Posted by "bitflicker64 (via GitHub)" <gi...@apache.org>.
bitflicker64 commented on code in PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149#discussion_r3789894380
##########
docker/docker-compose-3pd-3store-3server.yml:
##########
@@ -31,6 +36,8 @@ volumes:
# ── Shared service defaults ──────────────────────────────────────────
x-pd-common: &pd-common
+ # Pin a release via HUGEGRAPH_VERSION in docker/.env; unset, the image
Review Comment:
Agree this is a real gap for the auth-required 3-node quickstart. I’d like to keep #3149 scoped to the Hubble add-on and the minimum cluster contract Hubble needs. Changing Server readiness to prove 401/200 and changing default pull policy feels like follow-up cluster hardening rather than the add-on itself. README already documents the manual 401/200 check for incompatible images.
##########
docker/docker-compose-3pd-3store-3server.yml:
##########
@@ -17,9 +17,14 @@
name: hugegraph-3x3
+# The cluster network is shared with the Hubble add-on
+# (docker-compose-hubble.yml), so it is external and must exist first:
+# docker network create hugegraph-net
+# Set HUGEGRAPH_NETWORK to use a differently named network.
networks:
hg-net:
- driver: bridge
+ external: true
Review Comment:
Agree on the exposure concern. Default loopback binds / ACL+TLS requirements are a cluster-quickstart security redesign; I’d track that as a follow-up so this PR stays reviewable as a Hubble add-on.
##########
docker/docker-compose-hubble.yml:
##########
@@ -0,0 +1,53 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+# Hubble add-on for the distributed cluster defined in
+# docker-compose-3pd-3store-3server.yml. Requires the pre-created cluster
+# network (docker network create hugegraph-net). See "Hubble for the
+# 3-Node Cluster" in docker/README.md for the attach and combined flows.
+
+networks:
+ hg-net:
+ external: true
+ name: ${HUGEGRAPH_NETWORK:-hugegraph-net}
+
+volumes:
+ hg-hubble-db:
Review Comment:
Agreed — in scope. I’ll give the Hubble DB/upload volumes explicit stable names so attach and combined share the same physical volumes.
##########
docker/README.md:
##########
@@ -196,11 +296,184 @@ curl http://localhost:8520/v1/health
# Check Server (Graph API)
curl http://localhost:8080/versions
-# List registered stores via PD
-curl http://localhost:8620/v1/stores
+# Every PD endpoint except /v1/health, /actuator/* and /v1/prom/targets/*
+# needs an Authorization header. PD only checks that the Basic-auth user is
+# one of its internal service names, so the empty password below is enough
+# — and it grants the full PD control plane, writes included, not just these
+# reads. That is exactly why the shared network must be treated as a trust
+# boundary. Without the header PD answers with an exception body, not data.
+pd_auth="Authorization: Basic $(printf 'hubble:' | base64)"
+
+# List registered stores via PD (expect three, each "state":"Up")
+curl -H "${pd_auth}" http://localhost:8620/v1/stores
# List partitions
-curl http://localhost:8620/v1/partitions
+curl -H "${pd_auth}" http://localhost:8620/v1/partitions
+```
+
+Confirm authentication actually engaged — `/versions` stays open by design,
+so it cannot tell you whether auth is on. A graph read without credentials
+must be rejected:
+
+```bash
+cd docker
+# Expect 401 on all three replicas
+for port in 8080 8081 8082; do
+ curl -s -o /dev/null -w "${port}: %{http_code}\n" \
+ "http://localhost:${port}/graphs/hugegraph/schema/vertexlabels"
+done
+
+# And a signed-in read must succeed. Compose reads docker/.env by itself, but
+# your shell does not — load it first. Passing the credential through
+# --config keeps it out of argv, where `ps` would expose it to other users.
+set -a; . ./.env; set +a
Review Comment:
Agreed — in scope. I’ll replace `set -a; . ./.env; set +a` with a parse-only read of the required key so the verify block does not execute dotenv as shell.
##########
docker/docker-compose-3pd-3store-3server.yml:
##########
@@ -187,16 +207,25 @@ services:
<<: *server-common
container_name: hg-server0
hostname: server0
+ environment:
+ <<: *server-env
+ HG_SERVER_REST_URL: http://server0:8080
Review Comment:
Agree for PD-aware clients outside Docker DNS. Hubble on `hugegraph-net` can use `serverN` names as-is. Configurable advertised addresses feel like a follow-up to external-client addressing, not required for the add-on path. What do you think?
##########
.github/workflows/server-ci.yml:
##########
@@ -138,6 +140,189 @@ jobs:
check_compose docker/docker-compose.yml always always
check_compose docker/docker-compose.dev.yml build missing
+ check_cluster_compose() {
+ local cluster="docker/docker-compose-3pd-3store-3server.yml"
+ local addon="docker/docker-compose-hubble.yml"
+ local rendered
+ local token_fixture=ci-test-token-secret-32-bytes-long
+ rendered="$(mktemp)"
+ if [ "${#token_fixture}" -lt 32 ]; then
+ echo "CI token fixture must be at least 32 bytes" >&2
+ return 1
+ fi
+
+ # RETURN only: an EXIT trap would fire after this function's
+ # `local rendered` has gone out of scope, which `set -u` turns
+ # into an "unbound variable" error. A hard errexit abort can
+ # therefore still leak one temp file, which is acceptable on an
+ # ephemeral runner.
+ trap 'rm -f "$rendered"' RETURN
+
+ # --env-file /dev/null on every invocation: Compose otherwise reads
+ # docker/.env automatically, and the quickstart tells operators to
+ # create one holding exactly the credentials these assertions
+ # control. Without it the checks pass in CI (which has no .env) but
+ # report false failures for anyone running them locally after
+ # following the quickstart, and the add-on render below — which
+ # deliberately unsets the variables to assert their defaults — would
+ # read whatever HUGEGRAPH_NETWORK/HUGEGRAPH_VERSION that .env holds.
+ # Pinning an empty env file makes both renders depend only on what
+ # each invocation sets explicitly.
+
+ # Both cluster credentials are required and may not be empty.
+ # Each case asserts the guard fired for the *intended* variable:
+ # a bare non-zero exit would also be produced by a YAML error, a
+ # renamed file, or a missing docker binary.
+ assert_guard() { # assert_guard <blamed-var> <description>
+ local var="$1" desc="$2" err
+ if err="$(docker compose --env-file /dev/null -f "$cluster" \
+ config -q 2>&1)"; then
+ echo "$cluster accepted $desc" >&2
+ return 1
+ fi
+ case "$err" in
+ *"$var"*) : ;;
+ *) echo "$cluster rejected $desc, but not because of $var: $err" >&2
+ return 1 ;;
+ esac
+ }
+
+ ( unset HUGEGRAPH_ADMIN_PASSWORD
+ export HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}"
+ assert_guard HUGEGRAPH_ADMIN_PASSWORD "an unset admin password" )
+ ( export HUGEGRAPH_ADMIN_PASSWORD=
+ export HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}"
+ assert_guard HUGEGRAPH_ADMIN_PASSWORD "an empty admin password" )
+ ( unset HUGEGRAPH_AUTH_TOKEN_SECRET
+ export HUGEGRAPH_ADMIN_PASSWORD=ci-test-password
+ assert_guard HUGEGRAPH_AUTH_TOKEN_SECRET "an unset token secret" )
+ ( export HUGEGRAPH_ADMIN_PASSWORD=ci-test-password
+ export HUGEGRAPH_AUTH_TOKEN_SECRET=
+ assert_guard HUGEGRAPH_AUTH_TOKEN_SECRET "an empty token secret" )
+ # The add-on alone must define Hubble and nothing else, join the
+ # shared external network with its default name, and need no
+ # credentials or overrides.
+ env -u HUGEGRAPH_ADMIN_PASSWORD -u HUGEGRAPH_AUTH_TOKEN_SECRET \
+ -u HUGEGRAPH_NETWORK -u HUGEGRAPH_VERSION \
+ -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY -u HUBBLE_PUBLISH_HOST \
+ docker compose --env-file /dev/null -f "$addon" config --format json > "$rendered"
+ jq -e '
+ (.services | keys) == ["hubble"] and
+ .networks."hg-net".external == true and
+ .networks."hg-net".name == "hugegraph-net" and
+ (.services.hubble.networks | has("hg-net")) and
+ (.services.hubble | has("depends_on") | not) and
+ any(.services.hubble.volumes[];
+ .target == "/hubble/conf/hugegraph-hubble.properties" and
+ (.source | endswith("hugegraph-hubble-3x3.properties")))
+ and any(.services.hubble.volumes[];
+ .source == "hg-hubble-db" and
+ .target == "/hubble/db")
+ and any(.services.hubble.volumes[];
+ .source == "hg-hubble-upload-files" and
+ .target == "/hubble/upload-files")
+ ' "$rendered" >/dev/null
+
+ # The cluster's own default network name must match the add-on's,
+ # or the attach flow and the cluster land on different networks.
+ # The combined render below pins an override, so it cannot catch a
+ # drifting default.
+ HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}" \
+ env -u HUGEGRAPH_NETWORK -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY \
+ -u HUBBLE_PUBLISH_HOST \
+ docker compose --env-file /dev/null -f "$cluster" \
+ config --format json > "$rendered"
+ jq -e '.networks."hg-net".name == "hugegraph-net"' \
+ "$rendered" >/dev/null
+
+ # The combined render carries the PD-registration and auth
+ # settings on every server replica and keeps Hubble on loopback.
+ # Rendered with non-default HUGEGRAPH_NETWORK/HUGEGRAPH_VERSION so
+ # CI fails if any file stops honoring the overrides (the add-on
+ # render above covers the defaults).
+ HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}" \
+ HUGEGRAPH_NETWORK=ci-test-net \
+ HUGEGRAPH_VERSION=ci-test-tag \
+ env -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY -u HUBBLE_PUBLISH_HOST \
+ docker compose --env-file /dev/null -f "$cluster" -f "$addon" \
+ config --format json > "$rendered"
+ jq -e '
+ .name == "hugegraph-3x3" and
+ (.services | keys | length) == 10 and
+ .networks."hg-net".external == true and
+ .networks."hg-net".name == "ci-test-net" and
+ all(.services[]; .networks | has("hg-net")) and
+ .services.pd0.image == "hugegraph/pd:ci-test-tag" and
+ .services.store0.image == "hugegraph/store:ci-test-tag" and
+ .services.server0.image == "hugegraph/server:ci-test-tag" and
+ .services.hubble.image == "hugegraph/hubble:ci-test-tag" and
+ .services.server0.environment.HG_SERVER_USE_PD == "true" and
Review Comment:
Agreed — in scope. This PR introduced the PD/auth Server env for Hubble discovery, so CI should assert the required values on `server0`/`server1`/`server2`, not only `server0`.
##########
.github/workflows/server-ci.yml:
##########
@@ -138,6 +140,189 @@ jobs:
check_compose docker/docker-compose.yml always always
check_compose docker/docker-compose.dev.yml build missing
+ check_cluster_compose() {
+ local cluster="docker/docker-compose-3pd-3store-3server.yml"
+ local addon="docker/docker-compose-hubble.yml"
+ local rendered
+ local token_fixture=ci-test-token-secret-32-bytes-long
+ rendered="$(mktemp)"
+ if [ "${#token_fixture}" -lt 32 ]; then
+ echo "CI token fixture must be at least 32 bytes" >&2
+ return 1
+ fi
+
+ # RETURN only: an EXIT trap would fire after this function's
+ # `local rendered` has gone out of scope, which `set -u` turns
+ # into an "unbound variable" error. A hard errexit abort can
+ # therefore still leak one temp file, which is acceptable on an
+ # ephemeral runner.
+ trap 'rm -f "$rendered"' RETURN
+
+ # --env-file /dev/null on every invocation: Compose otherwise reads
+ # docker/.env automatically, and the quickstart tells operators to
+ # create one holding exactly the credentials these assertions
+ # control. Without it the checks pass in CI (which has no .env) but
+ # report false failures for anyone running them locally after
+ # following the quickstart, and the add-on render below — which
+ # deliberately unsets the variables to assert their defaults — would
+ # read whatever HUGEGRAPH_NETWORK/HUGEGRAPH_VERSION that .env holds.
+ # Pinning an empty env file makes both renders depend only on what
+ # each invocation sets explicitly.
+
+ # Both cluster credentials are required and may not be empty.
+ # Each case asserts the guard fired for the *intended* variable:
+ # a bare non-zero exit would also be produced by a YAML error, a
+ # renamed file, or a missing docker binary.
+ assert_guard() { # assert_guard <blamed-var> <description>
+ local var="$1" desc="$2" err
+ if err="$(docker compose --env-file /dev/null -f "$cluster" \
+ config -q 2>&1)"; then
+ echo "$cluster accepted $desc" >&2
+ return 1
+ fi
+ case "$err" in
+ *"$var"*) : ;;
+ *) echo "$cluster rejected $desc, but not because of $var: $err" >&2
+ return 1 ;;
+ esac
+ }
+
+ ( unset HUGEGRAPH_ADMIN_PASSWORD
+ export HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}"
+ assert_guard HUGEGRAPH_ADMIN_PASSWORD "an unset admin password" )
+ ( export HUGEGRAPH_ADMIN_PASSWORD=
+ export HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}"
+ assert_guard HUGEGRAPH_ADMIN_PASSWORD "an empty admin password" )
+ ( unset HUGEGRAPH_AUTH_TOKEN_SECRET
+ export HUGEGRAPH_ADMIN_PASSWORD=ci-test-password
+ assert_guard HUGEGRAPH_AUTH_TOKEN_SECRET "an unset token secret" )
+ ( export HUGEGRAPH_ADMIN_PASSWORD=ci-test-password
+ export HUGEGRAPH_AUTH_TOKEN_SECRET=
+ assert_guard HUGEGRAPH_AUTH_TOKEN_SECRET "an empty token secret" )
+ # The add-on alone must define Hubble and nothing else, join the
+ # shared external network with its default name, and need no
+ # credentials or overrides.
+ env -u HUGEGRAPH_ADMIN_PASSWORD -u HUGEGRAPH_AUTH_TOKEN_SECRET \
+ -u HUGEGRAPH_NETWORK -u HUGEGRAPH_VERSION \
+ -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY -u HUBBLE_PUBLISH_HOST \
+ docker compose --env-file /dev/null -f "$addon" config --format json > "$rendered"
+ jq -e '
+ (.services | keys) == ["hubble"] and
+ .networks."hg-net".external == true and
+ .networks."hg-net".name == "hugegraph-net" and
+ (.services.hubble.networks | has("hg-net")) and
+ (.services.hubble | has("depends_on") | not) and
+ any(.services.hubble.volumes[];
+ .target == "/hubble/conf/hugegraph-hubble.properties" and
+ (.source | endswith("hugegraph-hubble-3x3.properties")))
+ and any(.services.hubble.volumes[];
+ .source == "hg-hubble-db" and
+ .target == "/hubble/db")
+ and any(.services.hubble.volumes[];
+ .source == "hg-hubble-upload-files" and
+ .target == "/hubble/upload-files")
+ ' "$rendered" >/dev/null
+
+ # The cluster's own default network name must match the add-on's,
+ # or the attach flow and the cluster land on different networks.
+ # The combined render below pins an override, so it cannot catch a
+ # drifting default.
+ HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}" \
+ env -u HUGEGRAPH_NETWORK -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY \
+ -u HUBBLE_PUBLISH_HOST \
+ docker compose --env-file /dev/null -f "$cluster" \
+ config --format json > "$rendered"
+ jq -e '.networks."hg-net".name == "hugegraph-net"' \
+ "$rendered" >/dev/null
+
+ # The combined render carries the PD-registration and auth
+ # settings on every server replica and keeps Hubble on loopback.
+ # Rendered with non-default HUGEGRAPH_NETWORK/HUGEGRAPH_VERSION so
+ # CI fails if any file stops honoring the overrides (the add-on
+ # render above covers the defaults).
+ HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}" \
+ HUGEGRAPH_NETWORK=ci-test-net \
+ HUGEGRAPH_VERSION=ci-test-tag \
+ env -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY -u HUBBLE_PUBLISH_HOST \
+ docker compose --env-file /dev/null -f "$cluster" -f "$addon" \
+ config --format json > "$rendered"
+ jq -e '
+ .name == "hugegraph-3x3" and
+ (.services | keys | length) == 10 and
+ .networks."hg-net".external == true and
+ .networks."hg-net".name == "ci-test-net" and
+ all(.services[]; .networks | has("hg-net")) and
+ .services.pd0.image == "hugegraph/pd:ci-test-tag" and
+ .services.store0.image == "hugegraph/store:ci-test-tag" and
+ .services.server0.image == "hugegraph/server:ci-test-tag" and
+ .services.hubble.image == "hugegraph/hubble:ci-test-tag" and
+ .services.server0.environment.HG_SERVER_USE_PD == "true" and
+ .services.server0.environment.HG_SERVER_CLUSTER == "hg" and
+ .services.server0.environment.HG_SERVER_REST_URL ==
+ "http://server0:8080" and
+ .services.server1.environment.HG_SERVER_REST_URL ==
+ "http://server1:8080" and
+ .services.server1.environment.HG_SERVER_AUTH_TOKEN_SECRET ==
+ "ci-test-token-secret-32-bytes-long" and
+ .services.server2.environment.HG_SERVER_REST_URL ==
+ "http://server2:8080" and
+ .services.server2.environment.HG_SERVER_AUTH_TOKEN_SECRET ==
+ "ci-test-token-secret-32-bytes-long" and
+ .services.server0.environment.HG_SERVER_INIT_STORE_ENABLED ==
+ "false" and
+ .services.server0.environment.PASSWORD ==
+ "ci-test-password" and
+ .services.server0.environment.HG_SERVER_AUTH_TOKEN_SECRET ==
+ "ci-test-token-secret-32-bytes-long" and
+ (.services.hubble | has("depends_on") | not) and
+ .services.hubble.pull_policy == "missing" and
+ (.services.hubble.healthcheck.test[1] |
+ contains("http://127.0.0.1:8088/about") and
+ contains("\"status\":200") and
+ contains("\"name\":\"hugegraph-hubble\"")) and
+ any(.services.hubble.ports[];
+ .target == 8088 and .published == "8088" and
+ .host_ip == "127.0.0.1")
+ ' "$rendered" >/dev/null
+
+ # Hubble's properties file is mounted, not rendered, so Compose
+ # validation alone cannot catch it drifting from the services it
+ # describes. Tie the two together: renaming a service, changing a
+ # container hostname, or moving a REST port must be reflected in
+ # both places or CI fails here. Values are derived from the
+ # rendered model (hostnames and ports included) so the assertions
+ # cannot silently agree with a stale file.
+ local props="docker/hugegraph-hubble-3x3.properties"
+ local pd_peers store_targets cluster_name pd_rest
+ assert_props() { # assert_props <exact-line> <what-it-must-match>
+ grep -Fqx "$1" "$props" ||
+ { echo "$props: expected line '$1' ($2)" >&2; return 1; }
+ }
+ cluster_name="$(jq -r '.services.server0.environment.HG_SERVER_CLUSTER' \
+ "$rendered")"
+ pd_peers="$(jq -r '.services.server0.environment.HG_SERVER_PD_PEERS' \
+ "$rendered")"
+ store_targets="[$(jq -r '[.services | to_entries[]
+ | select(.key | startswith("store"))
+ | "http://" + .value.hostname + ":"
+ + (.value.environment.HG_STORE_REST_PORT)]
+ | sort | join(",")' "$rendered")]"
+ pd_rest="$(jq -r --arg h "$(printf '%s' "${pd_peers}" | cut -d, -f1 | cut -d: -f1)" \
+ '.services | to_entries[]
+ | select(.value.hostname == $h)
+ | .value.hostname + ":" + .value.environment.HG_PD_REST_PORT' \
+ "$rendered")"
+ assert_props "cluster=${cluster_name}" "matches HG_SERVER_CLUSTER"
Review Comment:
Correction: that last reply was meant for the advertised-URL thread. For this one — agreed and in scope. I’ll add an explicit `assert_props "pd.enabled=true"` so the Hubble properties check cannot pass with PD mode disabled.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(docker): add attachable Hubble Compose add-on for the 3-node cluster [hugegraph]
Posted by "bitflicker64 (via GitHub)" <gi...@apache.org>.
bitflicker64 commented on code in PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149#discussion_r3789881553
##########
.github/workflows/server-ci.yml:
##########
@@ -138,6 +140,189 @@ jobs:
check_compose docker/docker-compose.yml always always
check_compose docker/docker-compose.dev.yml build missing
+ check_cluster_compose() {
+ local cluster="docker/docker-compose-3pd-3store-3server.yml"
+ local addon="docker/docker-compose-hubble.yml"
+ local rendered
+ local token_fixture=ci-test-token-secret-32-bytes-long
+ rendered="$(mktemp)"
+ if [ "${#token_fixture}" -lt 32 ]; then
+ echo "CI token fixture must be at least 32 bytes" >&2
+ return 1
+ fi
+
+ # RETURN only: an EXIT trap would fire after this function's
+ # `local rendered` has gone out of scope, which `set -u` turns
+ # into an "unbound variable" error. A hard errexit abort can
+ # therefore still leak one temp file, which is acceptable on an
+ # ephemeral runner.
+ trap 'rm -f "$rendered"' RETURN
+
+ # --env-file /dev/null on every invocation: Compose otherwise reads
+ # docker/.env automatically, and the quickstart tells operators to
+ # create one holding exactly the credentials these assertions
+ # control. Without it the checks pass in CI (which has no .env) but
+ # report false failures for anyone running them locally after
+ # following the quickstart, and the add-on render below — which
+ # deliberately unsets the variables to assert their defaults — would
+ # read whatever HUGEGRAPH_NETWORK/HUGEGRAPH_VERSION that .env holds.
+ # Pinning an empty env file makes both renders depend only on what
+ # each invocation sets explicitly.
+
+ # Both cluster credentials are required and may not be empty.
+ # Each case asserts the guard fired for the *intended* variable:
+ # a bare non-zero exit would also be produced by a YAML error, a
+ # renamed file, or a missing docker binary.
+ assert_guard() { # assert_guard <blamed-var> <description>
+ local var="$1" desc="$2" err
+ if err="$(docker compose --env-file /dev/null -f "$cluster" \
+ config -q 2>&1)"; then
+ echo "$cluster accepted $desc" >&2
+ return 1
+ fi
+ case "$err" in
+ *"$var"*) : ;;
+ *) echo "$cluster rejected $desc, but not because of $var: $err" >&2
+ return 1 ;;
+ esac
+ }
+
+ ( unset HUGEGRAPH_ADMIN_PASSWORD
+ export HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}"
+ assert_guard HUGEGRAPH_ADMIN_PASSWORD "an unset admin password" )
+ ( export HUGEGRAPH_ADMIN_PASSWORD=
+ export HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}"
+ assert_guard HUGEGRAPH_ADMIN_PASSWORD "an empty admin password" )
+ ( unset HUGEGRAPH_AUTH_TOKEN_SECRET
+ export HUGEGRAPH_ADMIN_PASSWORD=ci-test-password
+ assert_guard HUGEGRAPH_AUTH_TOKEN_SECRET "an unset token secret" )
+ ( export HUGEGRAPH_ADMIN_PASSWORD=ci-test-password
+ export HUGEGRAPH_AUTH_TOKEN_SECRET=
+ assert_guard HUGEGRAPH_AUTH_TOKEN_SECRET "an empty token secret" )
+ # The add-on alone must define Hubble and nothing else, join the
+ # shared external network with its default name, and need no
+ # credentials or overrides.
+ env -u HUGEGRAPH_ADMIN_PASSWORD -u HUGEGRAPH_AUTH_TOKEN_SECRET \
+ -u HUGEGRAPH_NETWORK -u HUGEGRAPH_VERSION \
+ -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY -u HUBBLE_PUBLISH_HOST \
+ docker compose --env-file /dev/null -f "$addon" config --format json > "$rendered"
+ jq -e '
+ (.services | keys) == ["hubble"] and
+ .networks."hg-net".external == true and
+ .networks."hg-net".name == "hugegraph-net" and
+ (.services.hubble.networks | has("hg-net")) and
+ (.services.hubble | has("depends_on") | not) and
+ any(.services.hubble.volumes[];
+ .target == "/hubble/conf/hugegraph-hubble.properties" and
+ (.source | endswith("hugegraph-hubble-3x3.properties")))
+ and any(.services.hubble.volumes[];
+ .source == "hg-hubble-db" and
+ .target == "/hubble/db")
+ and any(.services.hubble.volumes[];
+ .source == "hg-hubble-upload-files" and
+ .target == "/hubble/upload-files")
+ ' "$rendered" >/dev/null
+
+ # The cluster's own default network name must match the add-on's,
+ # or the attach flow and the cluster land on different networks.
+ # The combined render below pins an override, so it cannot catch a
+ # drifting default.
+ HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}" \
+ env -u HUGEGRAPH_NETWORK -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY \
+ -u HUBBLE_PUBLISH_HOST \
+ docker compose --env-file /dev/null -f "$cluster" \
+ config --format json > "$rendered"
+ jq -e '.networks."hg-net".name == "hugegraph-net"' \
+ "$rendered" >/dev/null
+
+ # The combined render carries the PD-registration and auth
+ # settings on every server replica and keeps Hubble on loopback.
+ # Rendered with non-default HUGEGRAPH_NETWORK/HUGEGRAPH_VERSION so
+ # CI fails if any file stops honoring the overrides (the add-on
+ # render above covers the defaults).
+ HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}" \
+ HUGEGRAPH_NETWORK=ci-test-net \
+ HUGEGRAPH_VERSION=ci-test-tag \
+ env -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY -u HUBBLE_PUBLISH_HOST \
+ docker compose --env-file /dev/null -f "$cluster" -f "$addon" \
+ config --format json > "$rendered"
+ jq -e '
+ .name == "hugegraph-3x3" and
+ (.services | keys | length) == 10 and
+ .networks."hg-net".external == true and
+ .networks."hg-net".name == "ci-test-net" and
+ all(.services[]; .networks | has("hg-net")) and
+ .services.pd0.image == "hugegraph/pd:ci-test-tag" and
+ .services.store0.image == "hugegraph/store:ci-test-tag" and
+ .services.server0.image == "hugegraph/server:ci-test-tag" and
+ .services.hubble.image == "hugegraph/hubble:ci-test-tag" and
+ .services.server0.environment.HG_SERVER_USE_PD == "true" and
+ .services.server0.environment.HG_SERVER_CLUSTER == "hg" and
+ .services.server0.environment.HG_SERVER_REST_URL ==
+ "http://server0:8080" and
+ .services.server1.environment.HG_SERVER_REST_URL ==
+ "http://server1:8080" and
+ .services.server1.environment.HG_SERVER_AUTH_TOKEN_SECRET ==
+ "ci-test-token-secret-32-bytes-long" and
+ .services.server2.environment.HG_SERVER_REST_URL ==
+ "http://server2:8080" and
+ .services.server2.environment.HG_SERVER_AUTH_TOKEN_SECRET ==
+ "ci-test-token-secret-32-bytes-long" and
+ .services.server0.environment.HG_SERVER_INIT_STORE_ENABLED ==
+ "false" and
+ .services.server0.environment.PASSWORD ==
+ "ci-test-password" and
+ .services.server0.environment.HG_SERVER_AUTH_TOKEN_SECRET ==
+ "ci-test-token-secret-32-bytes-long" and
+ (.services.hubble | has("depends_on") | not) and
+ .services.hubble.pull_policy == "missing" and
+ (.services.hubble.healthcheck.test[1] |
+ contains("http://127.0.0.1:8088/about") and
+ contains("\"status\":200") and
+ contains("\"name\":\"hugegraph-hubble\"")) and
+ any(.services.hubble.ports[];
+ .target == 8088 and .published == "8088" and
+ .host_ip == "127.0.0.1")
+ ' "$rendered" >/dev/null
+
+ # Hubble's properties file is mounted, not rendered, so Compose
+ # validation alone cannot catch it drifting from the services it
+ # describes. Tie the two together: renaming a service, changing a
+ # container hostname, or moving a REST port must be reflected in
+ # both places or CI fails here. Values are derived from the
+ # rendered model (hostnames and ports included) so the assertions
+ # cannot silently agree with a stale file.
+ local props="docker/hugegraph-hubble-3x3.properties"
+ local pd_peers store_targets cluster_name pd_rest
+ assert_props() { # assert_props <exact-line> <what-it-must-match>
+ grep -Fqx "$1" "$props" ||
+ { echo "$props: expected line '$1' ($2)" >&2; return 1; }
+ }
+ cluster_name="$(jq -r '.services.server0.environment.HG_SERVER_CLUSTER' \
+ "$rendered")"
+ pd_peers="$(jq -r '.services.server0.environment.HG_SERVER_PD_PEERS' \
+ "$rendered")"
+ store_targets="[$(jq -r '[.services | to_entries[]
+ | select(.key | startswith("store"))
+ | "http://" + .value.hostname + ":"
+ + (.value.environment.HG_STORE_REST_PORT)]
+ | sort | join(",")' "$rendered")]"
+ pd_rest="$(jq -r --arg h "$(printf '%s' "${pd_peers}" | cut -d, -f1 | cut -d: -f1)" \
+ '.services | to_entries[]
+ | select(.value.hostname == $h)
+ | .value.hostname + ":" + .value.environment.HG_PD_REST_PORT' \
+ "$rendered")"
+ assert_props "cluster=${cluster_name}" "matches HG_SERVER_CLUSTER"
Review Comment:
Configurable advertised addresses seem to be a follow‑up to external‑client addressing, but they are not required for the add‑on path. What do you think we should do?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(docker): rebuild Compose topologies with Hubble [hugegraph]
Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149#issuecomment-5475601501
The latest HStore failure is independent of this Compose PR: the exact base/master SHA and multiple other master SHAs fail the same `VertexCoreTest.testQueryByJointIndexesWithSearchAndTwoRangeIndexesAndWithin` assertion (`expected 3`, `actual 1`), while other heads on this branch pass. The recurring HStore issue, test history, and investigation boundary are tracked in #3180. I am rerunning the failed HStore job; no PR code change is warranted for this failure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(docker): rebuild Compose topologies with Hubble [hugegraph]
Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on code in PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149#discussion_r3891967372
##########
docker/conf/hubble/standalone.properties:
##########
@@ -0,0 +1,20 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+server.host=0.0.0.0
Review Comment:
Confirmed this is not a startup gap for the container path. The Hubble image entrypoint is ./bin/start-hubble.sh -f; its foreground branch execs Java before the daemon-only /about probe reads server.port, and Spring Boot defaults to 8088. We also started the exact latest image and exercised the Hubble UI in all four required cells: standalone and minimal HStore, each with auth on and off, including data import and readback. Adding server.port would therefore duplicate the runtime default without changing behavior.
##########
docker/docker-compose.yml:
##########
@@ -14,117 +14,59 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
-name: hugegraph-single
+
+name: hugegraph-standalone
networks:
hg-net:
- driver: bridge
volumes:
- hg-pd-data:
- hg-store-data:
+ server-data:
+ hubble-data:
services:
-
- pd:
- image: hugegraph/pd:${HUGEGRAPH_VERSION:-latest}
- pull_policy: always
- container_name: hg-pd
- hostname: pd
- restart: unless-stopped
- networks: [hg-net]
- environment:
- HG_PD_GRPC_HOST: pd
- HG_PD_GRPC_PORT: "8686"
- HG_PD_REST_PORT: "8620"
- HG_PD_RAFT_ADDRESS: pd:8610
- HG_PD_RAFT_PEERS_LIST: pd:8610
- HG_PD_INITIAL_STORE_LIST: store:8500
- HG_PD_DATA_PATH: /hugegraph-pd/pd_data
- ports:
- - "8620:8620"
- volumes:
- - hg-pd-data:/hugegraph-pd/pd_data
- healthcheck:
- test: ["CMD-SHELL", "curl -fsS http://localhost:8620/v1/health >/dev/null || exit 1"]
- interval: 10s
- timeout: 5s
- retries: 12
- start_period: 30s
-
- store:
- image: hugegraph/store:${HUGEGRAPH_VERSION:-latest}
- pull_policy: always
- container_name: hg-store
- hostname: store
- restart: unless-stopped
- networks: [hg-net]
- depends_on:
- pd:
- condition: service_healthy
- environment:
- HG_STORE_PD_ADDRESS: pd:8686
- HG_STORE_GRPC_HOST: store
- HG_STORE_GRPC_PORT: "8500"
- HG_STORE_REST_PORT: "8520"
- HG_STORE_RAFT_ADDRESS: store:8510
- HG_STORE_DATA_PATH: /hugegraph-store/storage
- ports:
- - "8520:8520"
- volumes:
- - hg-store-data:/hugegraph-store/storage
- healthcheck:
- test: ["CMD-SHELL", "curl -fsS http://localhost:8520/v1/health >/dev/null || exit 1"]
- interval: 10s
- timeout: 10s
- retries: 30
- start_period: 60s
-
server:
- image: ${HUGEGRAPH_SERVER_IMAGE:-hugegraph/server:${HUGEGRAPH_VERSION:-latest}}
- pull_policy: ${HUGEGRAPH_SERVER_PULL_POLICY:-always}
- container_name: hg-server
- hostname: server
+ image: hugegraph/hugegraph:${HUGEGRAPH_VERSION:-latest}
+ pull_policy: ${HUGEGRAPH_PULL_POLICY:-missing}
restart: unless-stopped
networks: [hg-net]
- depends_on:
- store:
- condition: service_healthy
environment:
- HG_SERVER_BACKEND: hstore
- HG_SERVER_PD_PEERS: pd:8686
- HG_SERVER_CLUSTER: hg
- HG_SERVER_USE_PD: "true"
- HG_SERVER_REST_URL: http://server:8080
- HG_SERVER_MIN_FREE_MEMORY: "0"
- HG_SERVER_INIT_STORE_ENABLED: "false"
+ PASSWORD: ${HUGEGRAPH_ADMIN_PASSWORD:-}
Review Comment:
Auth-off is an intentional supported mode in this Compose-family redesign, selected by an empty or omitted HUGEGRAPH_ADMIN_PASSWORD. The README limits it to trusted local environments and warns against public or untrusted exposure; Hubble itself publishes on loopback by default. Requiring credentials or changing the broader Server publishing contract would remove the explicitly required auth-off acceptance path, so this PR keeps the documented password-only switch.
##########
docker/docker-compose-3pd-3store-3server.yml:
##########
@@ -58,6 +59,17 @@ x-store-common: &store-common
retries: 40
start_period: 120s
+x-server-environment: &server-environment
+ STORE_REST: store0:8520
+ HG_SERVER_BACKEND: hstore
+ HG_SERVER_PD_PEERS: pd0:8686,pd1:8686,pd2:8686
+ HG_SERVER_CLUSTER: hg
+ HG_SERVER_USE_PD: "true"
+ HG_SERVER_MIN_FREE_MEMORY: "0"
+ HG_SERVER_INIT_STORE_ENABLED: "false"
+ HG_SERVER_AUTH_TOKEN_SECRET: ${HUGEGRAPH_AUTH_TOKEN_SECRET:-}
Review Comment:
Fixed in a02886c68. HA now sets HG_SERVER_REQUIRE_AUTH_TOKEN_SECRET=true. The entrypoint fails before initialization only when PASSWORD is non-empty and the shared secret is missing; auth-off still needs no secret, while standalone and minimal HStore retain their existing single-replica generated-secret behavior. Focused tests cover password-only failure, auth-off success, and authenticated success with a shared secret, and the HA render contract asserts the guard on all three replicas.
##########
docker/test-compose.sh:
##########
@@ -0,0 +1,396 @@
+#!/usr/bin/env bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+set -Eeuo pipefail
+
+DOCKER_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+PASSWORD="ci-compose-password"
+SECRET="0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"
+VERSION="ci-version"
+RENDER_HUBBLE_IMAGE="example.invalid/hugegraph/hubble:ci"
+DATASOURCE="jdbc:h2:file:/hubble/data/hubble;DB_CLOSE_ON_EXIT=FALSE"
+ACTIVE_PROJECT=""
+ACTIVE_FILES=()
+RENDER_DIR=""
+
+compose_auth() {
+ env HUGEGRAPH_VERSION="${VERSION}" \
+ HUBBLE_IMAGE="${RENDER_HUBBLE_IMAGE}" \
+ HUGEGRAPH_ADMIN_PASSWORD="${PASSWORD}" \
+ HUGEGRAPH_AUTH_TOKEN_SECRET="${SECRET}" \
+ docker compose "$@"
+}
+
+render() {
+ local output="$1"
+ shift
+ compose_auth "$@" config --format json > "${output}"
+}
+
+assert_file_property() {
+ local file="$1"
+ local property="$2"
+ grep -Fqx "${property}" "${file}"
+}
+
+assert_common() {
+ local rendered="$1"
+ local services="$2"
+ local volumes="$3"
+
+ jq -e \
+ --arg password "${PASSWORD}" \
+ --arg secret "${SECRET}" \
+ --argjson services "${services}" \
+ --argjson volumes "${volumes}" '
+ (.services | keys) == $services and
+ (.volumes | keys) == $volumes and
+ (.networks | keys) == ["hg-net"] and
+ all(.networks[]; .external != true) and
+ all(.volumes[]; .external != true) and
+ all(.services[];
+ (.networks | keys) == ["hg-net"] and
+ .healthcheck.test[0] == "CMD-SHELL") and
+ all(
+ [.services | to_entries[] |
+ select(.key | startswith("server")) |
+ .value.environment][];
+ .PASSWORD == $password and
+ .HG_SERVER_AUTH_TOKEN_SECRET == $secret)
+ ' "${rendered}" >/dev/null
+}
+
+assert_hubble() {
+ local rendered="$1"
+ local config="$2"
+ shift 2
+ local dependencies
+ dependencies="$(printf '%s\n' "$@" | jq -Rsc 'split("\n")[:-1] | sort')"
+
+ jq -e \
+ --arg datasource "${DATASOURCE}" \
+ --arg config "/docker/conf/hubble/${config}" \
+ --argjson dependencies "${dependencies}" '
+ .services.hubble.image == "example.invalid/hugegraph/hubble:ci" and
+ .services.hubble.pull_policy == "missing" and
+ .services.hubble.environment.SPRING_DATASOURCE_URL == $datasource and
+ any(.services.hubble.ports[];
+ .target == 8088 and .published == "8088" and
+ .host_ip == "127.0.0.1") and
+ (.services.hubble.depends_on | keys | sort) == $dependencies and
+ all(.services.hubble.depends_on[];
+ .condition == "service_healthy") and
+ any(.services.hubble.volumes[];
+ .type == "volume" and .source == "hubble-data" and
+ .target == "/hubble/data") and
+ any(.services.hubble.volumes[];
+ .type == "bind" and (.source | endswith($config)) and
+ .target == "/hubble/conf/hugegraph-hubble.properties" and
+ .read_only == true) and
+ (.services.hubble.healthcheck.test[1] |
+ contains("http://localhost:8088/about") and
+ contains("\"status\":200") and
+ contains("\"name\":\"hugegraph-hubble\""))
+ ' "${rendered}" >/dev/null
+}
+
+assert_standalone() {
+ local rendered="$1"
+ assert_common "${rendered}" \
+ '["hubble","server"]' \
+ '["hubble-data","server-data"]'
+ assert_hubble "${rendered}" "standalone.properties" server
+ jq -e '
+ .services.server.image == "hugegraph/hugegraph:ci-version" and
+ .services.server.pull_policy == "missing" and
+ .services.server.healthcheck.test[1] ==
+ "curl -fsS http://localhost:8080/versions >/dev/null" and
+ any(.services.server.volumes[];
+ .source == "server-data" and
+ .target == "/hugegraph-server/rocksdb-data")
+ ' "${rendered}" >/dev/null
+ assert_file_property "${DOCKER_DIR}/conf/hubble/standalone.properties" \
+ "pd.enabled=false"
+ assert_file_property "${DOCKER_DIR}/conf/hubble/standalone.properties" \
+ "server.direct_url=http://server:8080"
+}
+
+assert_hstore() {
+ local rendered="$1"
+ assert_common "${rendered}" \
+ '["hubble","pd","server","store"]' \
+ '["hubble-data","pd-data","store-data"]'
+ assert_hubble "${rendered}" "hstore.properties" server
+ jq -e '
+ .services.pd.image == "hugegraph/pd:ci-version" and
+ .services.store.image == "hugegraph/store:ci-version" and
+ .services.server.image == "hugegraph/server:ci-version" and
+ all([.services.pd, .services.store, .services.server][];
+ .pull_policy == "missing") and
+ .services.server.environment.HG_SERVER_BACKEND == "hstore" and
+ .services.server.environment.HG_SERVER_PD_PEERS == "pd:8686" and
+ .services.server.environment.HG_SERVER_USE_PD == "true" and
+ .services.server.environment.HG_SERVER_REST_URL ==
+ "http://server:8080" and
+ .services.server.healthcheck.test[1] ==
+ "curl -fsS http://server:8080/versions >/dev/null" and
+ any(.services.pd.volumes[];
+ .source == "pd-data" and
+ .target == "/hugegraph-pd/pd_data") and
+ any(.services.store.volumes[];
+ .source == "store-data" and
+ .target == "/hugegraph-store/storage")
+ ' "${rendered}" >/dev/null
+ assert_file_property "${DOCKER_DIR}/conf/hubble/hstore.properties" \
+ "pd.peers=pd:8686"
+ assert_file_property "${DOCKER_DIR}/conf/hubble/hstore.properties" \
+ "operations.store.allowed_targets=[http://store:8520]"
+}
+
+assert_ha() {
+ local rendered="$1"
+ assert_common "${rendered}" \
+ '["hubble","pd0","pd1","pd2","server0","server1","server2","store0","store1","store2"]' \
+ '["hg-pd0-data","hg-pd1-data","hg-pd2-data","hg-store0-data","hg-store1-data","hg-store2-data","hubble-data"]'
+ assert_hubble "${rendered}" "hstore-ha.properties" \
+ server0 server1 server2
+ jq -e '
+ all([.services.pd0, .services.pd1, .services.pd2][];
+ .image == "hugegraph/pd:ci-version" and
+ .pull_policy == "missing") and
+ all([.services.store0, .services.store1, .services.store2][];
+ .image == "hugegraph/store:ci-version" and
+ .pull_policy == "missing") and
+ all([.services.server0, .services.server1, .services.server2][];
+ .image == "hugegraph/server:ci-version" and
+ .pull_policy == "missing" and
+ .environment.STORE_REST == "store0:8520" and
+ .environment.HG_SERVER_BACKEND == "hstore" and
+ .environment.HG_SERVER_PD_PEERS ==
+ "pd0:8686,pd1:8686,pd2:8686" and
+ .environment.HG_SERVER_CLUSTER == "hg" and
+ .environment.HG_SERVER_USE_PD == "true" and
+ .environment.HG_SERVER_INIT_STORE_ENABLED == "false") and
+ [.services.server0.environment.HG_SERVER_REST_URL,
+ .services.server1.environment.HG_SERVER_REST_URL,
+ .services.server2.environment.HG_SERVER_REST_URL] ==
+ ["http://server0:8080",
+ "http://server1:8080",
+ "http://server2:8080"] and
+ [.services.server0.healthcheck.test[1],
+ .services.server1.healthcheck.test[1],
+ .services.server2.healthcheck.test[1]] ==
+ ["curl -fsS http://server0:8080/versions >/dev/null",
+ "curl -fsS http://server1:8080/versions >/dev/null",
+ "curl -fsS http://server2:8080/versions >/dev/null"]
+ ' "${rendered}" >/dev/null
+ assert_file_property "${DOCKER_DIR}/conf/hubble/hstore-ha.properties" \
+ "pd.peers=pd0:8686,pd1:8686,pd2:8686"
+ assert_file_property "${DOCKER_DIR}/conf/hubble/hstore-ha.properties" \
+ "operations.store.allowed_targets=[http://store0:8520,http://store1:8520,http://store2:8520]"
+}
+
+assert_dev_override() {
+ local rendered="$1"
+ local override="$2"
+ assert_common "${rendered}" \
+ '["hubble","pd","server","store"]' \
+ '["hubble-data","pd-data","store-data"]'
+ jq -e '
+ .services.pd.image == "hugegraph/pd:dev" and
+ .services.store.image == "hugegraph/store:dev" and
+ .services.server.image == "hugegraph/server:dev" and
+ all([.services.pd, .services.store, .services.server][];
+ .pull_policy == "build" and .build != null) and
+ .services.hubble.image == "example.invalid/hugegraph/hubble:ci" and
+ .services.hubble.build == null
+ ' "${rendered}" >/dev/null
+ jq -e '
+ (.services | keys) == ["pd","server","store"] and
+ (.networks | keys) == ["default"] and .volumes == null and
+ all(.services[];
+ .build != null and .image != null and
+ .pull_policy == "build" and
+ .environment == null and
+ (.networks | keys) == ["default"] and
+ .volumes == null)
+ ' "${override}" >/dev/null
+}
+
+cleanup() {
+ if [[ -n "${ACTIVE_PROJECT}" ]]; then
+ compose_active down -v --remove-orphans >/dev/null 2>&1 || true
+ fi
+ [[ -z "${RENDER_DIR}" ]] || rm -rf "${RENDER_DIR}"
+}
+
+run_render() {
+ RENDER_DIR="$(mktemp -d)"
+ trap cleanup EXIT INT TERM
+ render "${RENDER_DIR}/standalone.json" \
+ -f "${DOCKER_DIR}/docker-compose.yml"
+ render "${RENDER_DIR}/hstore.json" \
+ -f "${DOCKER_DIR}/docker-compose-hstore.yml"
+ render "${RENDER_DIR}/ha.json" \
+ -f "${DOCKER_DIR}/docker-compose-3pd-3store-3server.yml"
+ render "${RENDER_DIR}/dev.json" \
+ -f "${DOCKER_DIR}/docker-compose-hstore.yml" \
+ -f "${DOCKER_DIR}/docker-compose.dev.yml"
+ render "${RENDER_DIR}/override.json" \
+ -f "${DOCKER_DIR}/docker-compose.dev.yml"
+
+ assert_standalone "${RENDER_DIR}/standalone.json"
+ assert_hstore "${RENDER_DIR}/hstore.json"
+ assert_ha "${RENDER_DIR}/ha.json"
+ assert_dev_override "${RENDER_DIR}/dev.json" \
+ "${RENDER_DIR}/override.json"
+ echo "Compose render contracts passed"
+}
+
+compose_active() {
+ env HUGEGRAPH_VERSION="${HUGEGRAPH_VERSION:-latest}" \
+ HUBBLE_IMAGE="${HUBBLE_IMAGE:-hugegraph/hubble:latest}" \
+ HUGEGRAPH_ADMIN_PASSWORD="${PASSWORD}" \
+ HUGEGRAPH_AUTH_TOKEN_SECRET="${SECRET}" \
+ COMPOSE_PROGRESS=plain \
+ docker compose -p "${ACTIVE_PROJECT}" "${ACTIVE_FILES[@]}" "$@"
+}
+
+diagnose() {
+ compose_active ps || true
+ compose_active logs --no-color --tail 200 || true
+}
+
+http_status() {
+ curl -sS -o /dev/null -w '%{http_code}' "$@"
+}
+
+wait_hubble_mode() {
+ local expected_pd="$1"
+ local expected_auth="$2"
+ local response=""
+ local _
+ for _ in {1..30}; do
+ response="$(curl -fsS http://localhost:8088/api/v1.3/config || true)"
Review Comment:
Fixed in a02886c68. Compose smoke requests now share --connect-timeout 5 and --max-time 15 across http_status, Hubble mode polling, login, anonymous probes, and the direct Server/Hubble probes. ShellCheck, bash syntax, Compose render contracts, and the focused entrypoint test pass.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(docker): add attachable Hubble Compose add-on for the 3-node cluster [hugegraph]
Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on code in PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149#discussion_r3760132573
##########
docker/README.md:
##########
@@ -168,14 +173,87 @@ To validate local images without Compose replacing them with remote `latest`:
## 3-Node Cluster Quickstart
+The cluster and the Hubble add-on share one named Docker network so Hubble
+can attach to a running cluster without touching it. Treat that network as a
+trust boundary: PD and Store expose unauthenticated control APIs on it (only
+the Server layer authenticates), and any container on the host can join it
+by declaring the well-known name. One-time setup: write the required
+credentials to a mode-600 `docker/.env` and create the network.
+The cluster file requires both credentials — the admin password enables
+authentication, and every Server replica must share one token secret so a
+token issued by any server validates on all of them. The `:?` guards fire
+on every Compose subcommand, including `down`.
+
+```bash
+(
+ set -eu
+ cd docker
+ command -v openssl >/dev/null 2>&1 || { echo "openssl not found" >&2; exit 1; }
+ [ -e .env ] || install -m 600 /dev/null .env
+ chmod 600 .env
+ # Keep appends on their own lines even if the file was hand-edited.
+ [ ! -s .env ] || [ -z "$(tail -c1 .env)" ] || printf '\n' >> .env
+ pat='^[[:space:]]*(export[[:space:]]+)?'
+ if ! grep -Eq "${pat}HUGEGRAPH_ADMIN_PASSWORD=" .env; then
Review Comment:
⚠️ The setup block treats existing invalid credentials as complete. Evidence: the checks only look for HUGEGRAPH_ADMIN_PASSWORD= and HUGEGRAPH_AUTH_TOKEN_SECRET=, so an empty password or an existing short token skips generation; the Compose :? guards reject empty values and docker-entrypoint.sh rejects tokens shorter than 32 bytes. Validate non-empty values and token length, or fail with a repair message before creating the network.
##########
docker/hugegraph-hubble-3x3.properties:
##########
@@ -0,0 +1,33 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+server.host=0.0.0.0
+server.port=8088
+
+cluster=hg
+idc=docker
+
+pd.enabled=true
+# Bootstrap target; the live topology comes from PD discovery. If this
+# replica is down when Hubble starts, point it at another server.
+server.direct_url=http://server0:8080
+pd.peers=pd0:8686,pd1:8686,pd2:8686
+# PD REST endpoint for the operations view (single address).
+pd.server=pd0:8620
Review Comment:
⚠️ Hubble Operations is pinned to one PD REST node even though the cluster has three PD peers. Evidence: Hubble LiveOperationsCollector uses pd.server as the base for /v1/cluster, /v1/stores, and PD Prometheus requests; pd.peers only serves discovery and does not fail over this HTTP endpoint. Use a stable PD REST VIP or add endpoint failover, and cover a pd0 failure while pd1/pd2 retain quorum.
##########
docker/docker-compose-hubble.yml:
##########
@@ -0,0 +1,47 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+# Hubble add-on for the distributed cluster defined in
+# docker-compose-3pd-3store-3server.yml. Requires the pre-created cluster
+# network (docker network create hugegraph-net). See "Hubble for the
+# 3-Node Cluster" in docker/README.md for the attach and combined flows.
+
+networks:
+ hg-net:
+ external: true
+ name: ${HUGEGRAPH_NETWORK:-hugegraph-net}
+
+services:
+ hubble:
+ # Pin a release via HUGEGRAPH_VERSION in docker/.env; unset, the image
+ # tag defaults to latest.
+ image: ${HUBBLE_IMAGE:-hugegraph/hubble:${HUGEGRAPH_VERSION:-latest}}
+ pull_policy: ${HUBBLE_PULL_POLICY:-missing}
+ container_name: hg-hubble
+ hostname: hubble
+ restart: unless-stopped
+ networks: [hg-net]
+ ports:
+ - "${HUBBLE_PUBLISH_HOST:-127.0.0.1}:8088:8088"
+ volumes:
+ - ./hugegraph-hubble-3x3.properties:/hubble/conf/hugegraph-hubble.properties:ro
Review Comment:
⚠️ Recreating Hubble loses its local state and uploaded files. Evidence: the Hubble image uses the file H2 database jdbc:h2:file:./db and the default upload_file.location upload-files, both under /hubble, but this service mounts only the read-only properties file; the documented docker compose down removes the container writable layer. Mount named volumes for the database and upload directory, or explicitly make this add-on stateless.
##########
docker/hugegraph-hubble-3x3.properties:
##########
@@ -0,0 +1,33 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+server.host=0.0.0.0
+server.port=8088
+
+cluster=hg
+idc=docker
+
+pd.enabled=true
+# Bootstrap target; the live topology comes from PD discovery. If this
+# replica is down when Hubble starts, point it at another server.
+server.direct_url=http://server0:8080
Review Comment:
🧹 This fallback comment and value are ineffective in the mode selected above. Evidence: HubbleOptions documents server.direct_url as used only when pd.enabled=false, while this file sets pd.enabled=true and Hubble builds the server client from PD discovery. Remove this bootstrap instruction or state clearly that it applies only to standalone mode.
##########
.github/workflows/server-ci.yml:
##########
@@ -138,6 +138,107 @@ jobs:
check_compose docker/docker-compose.yml always always
check_compose docker/docker-compose.dev.yml build missing
+ check_cluster_compose() {
+ local cluster="docker/docker-compose-3pd-3store-3server.yml"
+ local addon="docker/docker-compose-hubble.yml"
+ local rendered
+ rendered="$(mktemp)"
+
+ # Both cluster credentials are required and may not be empty.
+ if HUGEGRAPH_AUTH_TOKEN_SECRET=ci-test-secret \
+ env -u HUGEGRAPH_ADMIN_PASSWORD \
+ docker compose -f "$cluster" config -q >/dev/null 2>&1; then
+ echo "$cluster accepted an unset admin password" >&2
+ return 1
+ fi
+ if HUGEGRAPH_ADMIN_PASSWORD= \
+ HUGEGRAPH_AUTH_TOKEN_SECRET=ci-test-secret \
+ docker compose -f "$cluster" config -q >/dev/null 2>&1; then
+ echo "$cluster accepted an empty admin password" >&2
+ return 1
+ fi
+ if HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ env -u HUGEGRAPH_AUTH_TOKEN_SECRET \
+ docker compose -f "$cluster" config -q >/dev/null 2>&1; then
+ echo "$cluster accepted an unset token secret" >&2
+ return 1
+ fi
+ if HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ HUGEGRAPH_AUTH_TOKEN_SECRET= \
+ docker compose -f "$cluster" config -q >/dev/null 2>&1; then
+ echo "$cluster accepted an empty token secret" >&2
+ return 1
+ fi
+
+ # The add-on alone must define Hubble and nothing else, join the
+ # shared external network with its default name, and need no
+ # credentials or overrides.
+ env -u HUGEGRAPH_ADMIN_PASSWORD -u HUGEGRAPH_AUTH_TOKEN_SECRET \
+ -u HUGEGRAPH_NETWORK -u HUGEGRAPH_VERSION \
+ docker compose -f "$addon" config --format json > "$rendered"
+ jq -e '
+ (.services | keys) == ["hubble"] and
+ .networks."hg-net".external == true and
+ .networks."hg-net".name == "hugegraph-net" and
+ (.services.hubble | has("depends_on") | not) and
+ any(.services.hubble.volumes[];
+ .target == "/hubble/conf/hugegraph-hubble.properties" and
+ (.source | endswith("hugegraph-hubble-3x3.properties")))
+ ' "$rendered" >/dev/null
+
+ # The combined render carries the PD-registration and auth
+ # settings on every server replica and keeps Hubble on loopback.
+ # Rendered with non-default HUGEGRAPH_NETWORK/HUGEGRAPH_VERSION so
+ # CI fails if any file stops honoring the overrides (the add-on
+ # render above covers the defaults).
+ HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ HUGEGRAPH_AUTH_TOKEN_SECRET=ci-test-secret \
Review Comment:
⚠️ This combined-render check uses a token that every Server container rejects at startup. Evidence: exact-head docker-entrypoint.sh:93-98 exits when HG_SERVER_AUTH_TOKEN_SECRET is shorter than 32 bytes, while this workflow sets ci-test-secret and only runs docker compose config, so entrypoint validation is never exercised. Use a >=32-byte fixture and add a minimal entrypoint or container smoke check.
##########
docker/README.md:
##########
@@ -205,6 +283,110 @@ curl http://localhost:8620/v1/partitions
---
+## Hubble for the 3-Node Cluster
+
+`docker-compose-hubble.yml` defines only the Hubble service. It joins the
+cluster's external network (`hugegraph-net` by default, override with
+`HUGEGRAPH_NETWORK`) and has no `depends_on` on cluster services, so
+starting, stopping, or upgrading Hubble never recreates or restarts PD,
+Store, or Server containers. Hubble reads the cluster topology from
+`hugegraph-hubble-3x3.properties`; adjust that file when attaching to a
+cluster with different hostnames.
+
+Sign in at `http://localhost:8088` as `admin` with the
+`HUGEGRAPH_ADMIN_PASSWORD` from `docker/.env`. Hubble binds to host
+loopback by default (`HUBBLE_PUBLISH_HOST`, same caveats as the
+single-node setup).
+
+The two flows below create Hubble in different Compose projects, so manage
+Hubble with the same flags you started it with: the attach flow always uses
+`-p hugegraph-hubble -f docker-compose-hubble.yml`, the combined flow always
+uses both `-f` flags. The explicit `-p` keeps the attach project independent
+of the directory name and of other Compose projects.
+
+Run one Hubble per host: the single-node stack and both add-on flows all
+publish `127.0.0.1:8088` and name their container `hg-hubble`. The two
+add-on flows are therefore mutually exclusive — starting one while the
+other's Hubble exists fails with a container-name conflict, so `down` the
+flow you are leaving before switching.
+
+### Attach to a running cluster
+
+With the 3-node cluster already up:
+
+```bash
+cd docker
+docker compose -p hugegraph-hubble -f docker-compose-hubble.yml up -d
Review Comment:
⚠️ The documented attach command can produce a healthy Hubble container that discovers no graph server on an older running cluster. Evidence: the pre-change 3-node Compose stack does not set HG_SERVER_USE_PD=true, HG_SERVER_CLUSTER=hg, or per-node HG_SERVER_REST_URL, while this add-on enables PD discovery for cluster hg and only starts Hubble. Require upgrading the cluster with the new Compose file first, or provide a direct-server compatibility configuration.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(docker): add attachable Hubble Compose add-on for the 3-node cluster [hugegraph]
Posted by "bitflicker64 (via GitHub)" <gi...@apache.org>.
bitflicker64 commented on PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149#issuecomment-5388512278
Pushed `6b6c3a3c`. All four blocking findings from the last review are fixed in this PR rather than deferred, and the PR description is updated to match.
- Auth false-green: images pull by default and Server readiness requires 401 unauthenticated plus 200 authenticated, so an image that ignores `PASSWORD` never becomes healthy.
- Control plane: PD, Store, and Server ports bind to `127.0.0.1`, widened only through `HUGEGRAPH_CONTROL_PLANE_HOST` / `HUGEGRAPH_SERVER_PUBLISH_HOST`.
- Advertised addresses: each replica's PD-registered URL is configurable, defaults unchanged.
- Render-only CI: a live smoke job starts the cluster, attaches Hubble, and asserts the above.
The cluster also goes back to being authenticated by default, per your point that this is the file people copy. Running without auth is now an explicit opt-in, `docker-compose-3x3.non-auth.yml` plus a matching Hubble properties file, documented beside the default flow with a copy-paste prompt for people who hand this to an assistant.
Tested on a clean Docker host, from zero images: 9/9 healthy, 401/200 on all three replicas, ports on loopback, three PD peers and three Stores Up, Hubble attached without recreating a single cluster container, H2 state surviving recreation across both flows, and the non-auth path returning 200 with no credentials anywhere in the container environment.
Testing caught three bugs that rendering never would have, one of them a half-authenticated cluster where only `server0` lost its credentials because `!reset` does not survive a YAML anchor on every Compose version. That one is the clearest argument for the smoke job, so I would rather keep it than trim it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(docker): add attachable Hubble Compose add-on for the 3-node cluster [hugegraph]
Posted by "bitflicker64 (via GitHub)" <gi...@apache.org>.
bitflicker64 commented on code in PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149#discussion_r3866718064
##########
docker/README.md:
##########
@@ -196,11 +359,307 @@ curl http://localhost:8520/v1/health
# Check Server (Graph API)
curl http://localhost:8080/versions
-# List registered stores via PD
-curl http://localhost:8620/v1/stores
+# Every PD endpoint except /v1/health, /actuator/* and /v1/prom/targets/*
+# needs an Authorization header. PD only checks that the Basic-auth user is
+# one of its internal service names, so the empty password below is enough
+# — and it grants the full PD control plane, writes included, not just these
+# reads. That is exactly why the shared network must be treated as a trust
+# boundary. Without the header PD answers with an exception body, not data.
+pd_auth="Authorization: Basic $(printf 'hubble:' | base64)"
+
+# List registered stores via PD (expect three, each "state":"Up")
+curl -H "${pd_auth}" http://localhost:8620/v1/stores
# List partitions
-curl http://localhost:8620/v1/partitions
+curl -H "${pd_auth}" http://localhost:8620/v1/partitions
+```
+
+Confirm authentication actually engaged — `/versions` stays open by design,
+so it cannot tell you whether auth is on. A graph read without credentials
+must be rejected:
+
+```bash
+cd docker
+# Expect 401 on all three replicas
+for port in 8080 8081 8082; do
+ curl -s -o /dev/null -w "${port}: %{http_code}\n" \
+ "http://localhost:${port}/graphs/hugegraph/schema/vertexlabels"
+done
+
+# And a signed-in read must succeed. Parse the generated single-quoted
+# value without executing docker/.env as shell. Passing the credential
+# through --config keeps it out of argv, where `ps` would expose it.
+# The subshell keeps the `exit 1` below from closing an interactive shell.
+(
+set -eu
+read_dotenv_value() {
+ key="$1"
+ key_pattern="^[[:space:]]*(export[[:space:]]+)?${key}="
+ count="$(grep -Ec "${key_pattern}" .env || true)"
+ [ "${count}" -eq 1 ] || return 1
+ value="$(sed -nE "s/${key_pattern}'([^']*)'[[:space:]]*$/\\2/p" .env)"
+ # sed prints nothing and still exits 0 when the value is unquoted or
+ # double-quoted, which would otherwise hand back an empty credential.
+ [ -n "${value}" ] || return 1
+ printf '%s\n' "${value}"
+}
+HUGEGRAPH_ADMIN_PASSWORD="$(read_dotenv_value HUGEGRAPH_ADMIN_PASSWORD)" || {
+ echo "HUGEGRAPH_ADMIN_PASSWORD must use the documented single-quoted format" >&2
+ exit 1
+}
+curl -s -o /dev/null -w '%{http_code}\n' \
+ --config <(printf 'user = "admin:%s"\n' "${HUGEGRAPH_ADMIN_PASSWORD}") \
+ http://localhost:8080/graphs/hugegraph/schema/vertexlabels
+)
+```
+
+`200` from the second command with `401` from all three of the first means
+authentication is on and working. If the first command returns `200`, the
+running image ignored `PASSWORD` and the cluster is **unauthenticated** —
+the most likely cause is an image older than the release this integration
+needs (see the version note in the quickstart above).
+
+### Running without authentication
+
+The cluster above authenticates, and that is the right default: it is the
+configuration most people copy, and an open graph database on a reachable
+port is a bad surprise. For a throwaway local trial on a trusted network you
+can opt out with `docker-compose-3x3.non-auth.yml`, which drops the admin
+password and the JWT secret from all three Servers. Anyone who can reach the
+published ports then has full read and write access to every graph, so do not
+use it anywhere else.
+
+```bash
+cd docker
+
+# The network and Hubble's volumes are external in every flow, so create them
+# once even here. This mode skips only the credential half of the setup block.
+docker network inspect hugegraph-net >/dev/null 2>&1 ||
+ docker network create hugegraph-net
+for vol in hugegraph-hubble-db hugegraph-hubble-upload-files; do
+ docker volume inspect "${vol}" >/dev/null 2>&1 ||
+ docker volume create "${vol}" >/dev/null
+done
+
+# No docker/.env is needed. The two placeholders exist only because Compose
+# evaluates the base file's required-variable guards before applying the
+# override, which then drops both variables, so neither value reaches a
+# container. Needs Compose v2.24+.
+#
+# Keep them inline on every command; do not write them into docker/.env. They
+# are published values rather than secrets, and the token placeholder is
+# deliberately shorter than the 32 bytes the Server requires, so if it ever
+# reaches an authenticated Server that Server refuses to start instead of
+# signing tokens with a key anyone can read here.
+HUGEGRAPH_ADMIN_PASSWORD=non-auth-placeholder \
+HUGEGRAPH_AUTH_TOKEN_SECRET=non-auth-placeholder \
+ docker compose -f docker-compose-3pd-3store-3server.yml \
+ -f docker-compose-3x3.non-auth.yml up -d
+
+# Every graph request now succeeds without credentials: expect 200, not 401.
+curl -s -o /dev/null -w '%{http_code}\n' \
+ http://localhost:8080/graphs/hugegraph/schema/vertexlabels
+
+# Teardown. The base file's `:?` guards fire on every subcommand, `down`
+# included, so the same two placeholders are required here.
+HUGEGRAPH_ADMIN_PASSWORD=non-auth-placeholder \
+HUGEGRAPH_AUTH_TOKEN_SECRET=non-auth-placeholder \
+ docker compose -f docker-compose-3pd-3store-3server.yml \
+ -f docker-compose-3x3.non-auth.yml down -v
+```
+
+Hubble needs the matching configuration, because its own default is to demand
+a login. Point it at the non-auth properties file when you attach it:
+
+```bash
+HUBBLE_PROPERTIES=./hugegraph-hubble-3x3.non-auth.properties \
+ docker compose -p hugegraph-hubble -f docker-compose-hubble.yml up -d
+```
+
+`HUBBLE_PROPERTIES` is not remembered between commands. It selects a bind
+mount, so any later Compose command for Hubble that omits it silently
+recreates the container against the authenticated properties file, and Hubble
+comes back demanding a login the Servers cannot serve. Repeat the variable on
+every Hubble command in this mode, including `up -d --no-deps hubble` and the
+local-image commands below.
+
+<details>
+<summary>Prompt for an AI assistant</summary>
+
+Copy this to an assistant instead of transcribing the commands by hand.
+
+```markdown
+Start the Apache HugeGraph 3-node Docker cluster (3 PD, 3 Store, 3 Server)
+with Hubble, from the `docker/` directory of the hugegraph repository.
+
+Use the authenticated default unless I say otherwise:
+1. Follow "3-Node Cluster Quickstart" in docker/README.md to create
+ docker/.env with a generated admin password and JWT token secret, and to
+ create the external hugegraph-net network.
+2. Start the cluster with docker-compose-3pd-3store-3server.yml, then attach
+ Hubble with docker-compose-hubble.yml.
+3. Verify: an unauthenticated graph request returns 401, an authenticated one
+ returns 200, and Hubble answers on http://localhost:8088.
+4. Tell me the admin password so I can sign in to Hubble.
+
+Do not weaken authentication to work around an error. If a Server never
+becomes healthy, the image is probably older than the release this stack
+needs; report that instead of disabling the healthcheck. Only if I explicitly
+ask for an unauthenticated cluster, use the "Running without authentication"
+section instead.
+```
+
+</details>
+
+---
+
+## Hubble for the 3-Node Cluster
+
+`docker-compose-hubble.yml` defines only the Hubble service. It joins the
+cluster's external network (`hugegraph-net` by default, override with
+`HUGEGRAPH_NETWORK`) and has no `depends_on` on cluster services, so
+starting, stopping, or upgrading Hubble never recreates or restarts PD,
+Store, or Server containers. Hubble reads the cluster topology from
+`hugegraph-hubble-3x3.properties`; adjust that file when attaching to a
+cluster with different hostnames. Its `pd.server` is a single PD address
+with no failover, so if that PD is down the operations view goes blind even
+though the cluster still has quorum — repoint it at a surviving PD. The
+operations view also reports one `SERVER` node, not three: it describes the
+replica Hubble is currently talking to, not the whole Server tier. The add-on
+stores Hubble's H2 database and uploaded files in named volumes
+`hugegraph-hubble-db` (mounted at `/hubble/db`) and
+`hugegraph-hubble-upload-files`. Those names are explicit, so the attach
+and combined Compose projects share the same physical volumes. Hubble's
+default H2 URL would write `/hubble/db.mv.db` (outside that mount); the
+add-on sets `SPRING_DATASOURCE_URL` so the database file lives inside
+`/hubble/db`. Recreating the container does not discard that state.
+
+Sign in at `http://localhost:8088` as `admin` with the
+`HUGEGRAPH_ADMIN_PASSWORD` from `docker/.env`. Hubble binds to host
+loopback by default (`HUBBLE_PUBLISH_HOST`, same caveats as the
+single-node setup).
+
+Which flow to use: pick the attach flow when you do not have the cluster's
+`docker/.env` — the add-on carries no `:?` guards, so it is the only flow
+that runs without those credentials, which is what you want against a
+cluster someone else started. Otherwise use the combined flow, including to
+add Hubble to an already-running cluster (`up -d --no-deps hubble`).
+
+The two flows below create Hubble in different Compose projects, so manage
+Hubble with the same flags you started it with: the attach flow always uses
+`-p hugegraph-hubble -f docker-compose-hubble.yml`, the combined flow always
+uses both `-f` flags. The explicit `-p` keeps the attach project independent
+of the directory name and of other Compose projects.
+
+> [!IMPORTANT]
+> Do not drop the `-p` from the attach flow. Without it Compose names the
+> project after the current directory (`docker`), so Hubble starts in a
+> third project that none of the commands below manage: `down` reports
+> nothing to remove while `hg-hubble` keeps running, and every later `up`
+> in either flow fails on the container name. If that happens, find it with
+> `docker ps --filter name=hg-hubble` and remove it with
+> `docker rm -f hg-hubble`.
+
+Run one Hubble per host: the single-node stack and both add-on flows all
+publish `127.0.0.1:8088` and name their container `hg-hubble`. The two
+add-on flows are therefore mutually exclusive — starting one while the
+other's Hubble exists fails with a container-name conflict, so remove the
+Hubble of the flow you are leaving before switching. The two directions are
+not symmetric:
+
+```bash
+cd docker
+# leaving the attach flow (removes only Hubble)
+docker compose -p hugegraph-hubble -f docker-compose-hubble.yml down
+
+# leaving the combined flow — remove ONLY Hubble; a plain `down` with both
+# -f flags would stop all ten containers just to move one.
+docker compose -f docker-compose-3pd-3store-3server.yml \
+ -f docker-compose-hubble.yml rm -sf hubble
+```
+
+### Attach to a running cluster
+
+With the 3-node cluster already up:
+
+```bash
+cd docker
+docker compose -p hugegraph-hubble -f docker-compose-hubble.yml up -d
Review Comment:
Fixed in d2a12bc1, and thanks - this was a regression from making the volumes external.
The attach section now creates both volumes before the `up`, honouring `HUBBLE_DB_VOLUME` and `HUBBLE_UPLOAD_VOLUME` when the cluster owner renamed them. It is inspect-then-create, so it is safe to repeat and never touches an existing volume.
Reproduced your case first: with the volumes absent the attach fails with `external volume "hugegraph-hubble-upload-files" not found`, and with the block it reaches healthy.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(docker): add attachable Hubble Compose add-on for the 3-node cluster [hugegraph]
Posted by "bitflicker64 (via GitHub)" <gi...@apache.org>.
bitflicker64 commented on code in PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149#discussion_r3888191227
##########
.github/workflows/server-ci.yml:
##########
@@ -138,6 +140,366 @@ jobs:
check_compose docker/docker-compose.yml always always
check_compose docker/docker-compose.dev.yml build missing
+ check_cluster_compose() {
+ local cluster="docker/docker-compose-3pd-3store-3server.yml"
+ local addon="docker/docker-compose-hubble.yml"
+ local rendered
+ local token_fixture=ci-test-token-secret-32-bytes-long
+ rendered="$(mktemp)"
+ if [ "${#token_fixture}" -lt 32 ]; then
+ echo "CI token fixture must be at least 32 bytes" >&2
+ return 1
+ fi
+
+ # RETURN only: an EXIT trap would fire after this function's
+ # `local rendered` has gone out of scope, which `set -u` turns
+ # into an "unbound variable" error. A hard errexit abort can
+ # therefore still leak one temp file, which is acceptable on an
+ # ephemeral runner.
+ trap 'rm -f "$rendered"' RETURN
+
+ # --env-file /dev/null on every invocation: Compose otherwise reads
+ # docker/.env automatically, and the quickstart tells operators to
+ # create one holding exactly the credentials these assertions
+ # control. Without it the checks pass in CI (which has no .env) but
+ # report false failures for anyone running them locally after
+ # following the quickstart, and the add-on render below — which
+ # deliberately unsets the variables to assert their defaults — would
+ # read whatever HUGEGRAPH_NETWORK/HUGEGRAPH_VERSION that .env holds.
+ # Pinning an empty env file makes both renders depend only on what
+ # each invocation sets explicitly.
+
+ # Both cluster credentials are required and may not be empty.
+ # Each case asserts the guard fired for the *intended* variable:
+ # a bare non-zero exit would also be produced by a YAML error, a
+ # renamed file, or a missing docker binary.
+ assert_guard() { # assert_guard <blamed-var> <description>
+ local var="$1" desc="$2" err
+ if err="$(docker compose --env-file /dev/null -f "$cluster" \
+ config -q 2>&1)"; then
+ echo "$cluster accepted $desc" >&2
+ return 1
+ fi
+ case "$err" in
+ *"$var"*) : ;;
+ *) echo "$cluster rejected $desc, but not because of $var: $err" >&2
+ return 1 ;;
+ esac
+ }
+
+ ( unset HUGEGRAPH_ADMIN_PASSWORD
+ export HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}"
+ assert_guard HUGEGRAPH_ADMIN_PASSWORD "an unset admin password" )
+ ( export HUGEGRAPH_ADMIN_PASSWORD=
+ export HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}"
+ assert_guard HUGEGRAPH_ADMIN_PASSWORD "an empty admin password" )
+ ( unset HUGEGRAPH_AUTH_TOKEN_SECRET
+ export HUGEGRAPH_ADMIN_PASSWORD=ci-test-password
+ assert_guard HUGEGRAPH_AUTH_TOKEN_SECRET "an unset token secret" )
+ ( export HUGEGRAPH_ADMIN_PASSWORD=ci-test-password
+ export HUGEGRAPH_AUTH_TOKEN_SECRET=
+ assert_guard HUGEGRAPH_AUTH_TOKEN_SECRET "an empty token secret" )
+ # The add-on alone must define Hubble and nothing else, join the
+ # shared external network with its default name, and need no
+ # credentials or overrides.
+ env -u HUGEGRAPH_ADMIN_PASSWORD -u HUGEGRAPH_AUTH_TOKEN_SECRET \
+ -u HUGEGRAPH_NETWORK -u HUGEGRAPH_VERSION \
+ -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY -u HUBBLE_PUBLISH_HOST \
+ -u HUBBLE_DB_VOLUME -u HUBBLE_UPLOAD_VOLUME \
+ docker compose --env-file /dev/null -f "$addon" config --format json > "$rendered"
+ jq -e '
+ (.services | keys) == ["hubble"] and
+ .networks."hg-net".external == true and
+ .networks."hg-net".name == "hugegraph-net" and
+ (.services.hubble.networks | has("hg-net")) and
+ (.services.hubble | has("depends_on") | not) and
+ .volumes."hg-hubble-db".name == "hugegraph-hubble-db" and
+ .volumes."hg-hubble-db".external == true and
+ .volumes."hg-hubble-upload-files".name ==
+ "hugegraph-hubble-upload-files" and
+ .volumes."hg-hubble-upload-files".external == true and
+ .services.hubble.environment.SPRING_DATASOURCE_URL ==
+ "jdbc:h2:file:/hubble/db/hubble;DB_CLOSE_ON_EXIT=FALSE" and
+ any(.services.hubble.volumes[];
+ .target == "/hubble/conf/hugegraph-hubble.properties" and
+ (.source | endswith("hugegraph-hubble-3x3.properties")))
+ and any(.services.hubble.volumes[];
+ .source == "hg-hubble-db" and
+ .target == "/hubble/db")
+ and any(.services.hubble.volumes[];
+ .source == "hg-hubble-upload-files" and
+ .target == "/hubble/upload-files")
+ ' "$rendered" >/dev/null
+
+ # The cluster's own default network name must match the add-on's,
+ # or the attach flow and the cluster land on different networks.
+ # The combined render below pins an override, so it cannot catch a
+ # drifting default.
+ HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}" \
+ env -u HUGEGRAPH_NETWORK -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY \
+ -u HUGEGRAPH_PULL_POLICY -u HUGEGRAPH_CONTROL_PLANE_HOST \
+ -u HUGEGRAPH_SERVER_PUBLISH_HOST -u HUBBLE_PROPERTIES \
+ -u HUBBLE_PUBLISH_HOST \
+ docker compose --env-file /dev/null -f "$cluster" \
+ config --format json > "$rendered"
+ jq -e '.networks."hg-net".name == "hugegraph-net"' \
+ "$rendered" >/dev/null
+
+ # The combined render carries the PD-registration and auth
+ # settings on every server replica and keeps Hubble on loopback.
+ # Rendered with non-default HUGEGRAPH_NETWORK/HUGEGRAPH_VERSION so
+ # CI fails if any file stops honoring the overrides (the add-on
+ # render above covers the defaults).
+ HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}" \
+ HUGEGRAPH_NETWORK=ci-test-net \
+ HUGEGRAPH_VERSION=ci-test-tag \
+ env -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY -u HUBBLE_PUBLISH_HOST \
+ -u HUBBLE_DB_VOLUME -u HUBBLE_UPLOAD_VOLUME \
+ -u HUGEGRAPH_PULL_POLICY -u HUGEGRAPH_CONTROL_PLANE_HOST \
+ -u HUGEGRAPH_SERVER_PUBLISH_HOST -u HUBBLE_PROPERTIES \
+ docker compose --env-file /dev/null -f "$cluster" -f "$addon" \
+ config --format json > "$rendered"
+ jq -e '
+ . as $root |
+ .name == "hugegraph-3x3" and
+ (.services | keys | length) == 10 and
+ .networks."hg-net".external == true and
+ .networks."hg-net".name == "ci-test-net" and
+ all(.services[]; .networks | has("hg-net")) and
+ .services.pd0.image == "hugegraph/pd:ci-test-tag" and
+ .services.store0.image == "hugegraph/store:ci-test-tag" and
+ .services.server0.image == "hugegraph/server:ci-test-tag" and
+ .services.hubble.image == "hugegraph/hubble:ci-test-tag" and
+ all(["server0", "server1", "server2"][];
+ $root.services[.].environment.HG_SERVER_USE_PD == "true" and
+ $root.services[.].environment.HG_SERVER_CLUSTER == "hg" and
+ $root.services[.].environment.HG_SERVER_INIT_STORE_ENABLED ==
+ "false" and
+ $root.services[.].environment.PASSWORD ==
+ "ci-test-password" and
+ $root.services[.].environment.HG_SERVER_AUTH_TOKEN_SECRET ==
+ "ci-test-token-secret-32-bytes-long") and
+ (.services.server0.healthcheck.test[1] |
+ contains("= 401") and (contains("PASSWORD") | not)) and
+ .services.server0.environment.HG_SERVER_REST_URL ==
+ "http://server0:8080" and
+ .services.server1.environment.HG_SERVER_REST_URL ==
+ "http://server1:8080" and
+ .services.server2.environment.HG_SERVER_REST_URL ==
+ "http://server2:8080" and
+ (.services.hubble | has("depends_on") | not) and
+ .services.hubble.pull_policy == "always" and
+ .volumes."hg-hubble-db".name == "hugegraph-hubble-db" and
+ .volumes."hg-hubble-db".external == true and
+ .volumes."hg-hubble-upload-files".name ==
+ "hugegraph-hubble-upload-files" and
+ .volumes."hg-hubble-upload-files".external == true and
+ .services.hubble.environment.SPRING_DATASOURCE_URL ==
+ "jdbc:h2:file:/hubble/db/hubble;DB_CLOSE_ON_EXIT=FALSE" and
+ (.services.hubble.healthcheck.test[1] |
+ contains("http://127.0.0.1:8088/about") and
+ contains("\"status\":200") and
+ contains("\"name\":\"hugegraph-hubble\"")) and
+ any(.services.hubble.ports[];
+ .target == 8088 and .published == "8088" and
+ .host_ip == "127.0.0.1") and
+ all(["pd0", "pd1", "pd2", "store0", "store1", "store2",
+ "server0", "server1", "server2"][];
+ all($root.services[.].ports[]; .host_ip == "127.0.0.1"))
+ ' "$rendered" >/dev/null
+
+ # Hubble's properties file is mounted, not rendered, so Compose
+ # validation alone cannot catch it drifting from the services it
+ # describes. Tie the two together: renaming a service, changing a
+ # container hostname, or moving a REST port must be reflected in
+ # both places or CI fails here. Values are derived from the
+ # rendered model (hostnames and ports included) so the assertions
+ # cannot silently agree with a stale file.
+ local props="docker/hugegraph-hubble-3x3.properties"
+ local pd_peers store_targets cluster_name pd_rest
+ assert_props() { # assert_props <exact-line> <what-it-must-match>
+ grep -Fqx "$1" "$props" ||
+ { echo "$props: expected line '$1' ($2)" >&2; return 1; }
+ }
+ cluster_name="$(jq -r '.services.server0.environment.HG_SERVER_CLUSTER' \
+ "$rendered")"
+ pd_peers="$(jq -r '.services.server0.environment.HG_SERVER_PD_PEERS' \
+ "$rendered")"
+ store_targets="[$(jq -r '[.services | to_entries[]
+ | select(.key | startswith("store"))
+ | "http://" + .value.hostname + ":"
+ + (.value.environment.HG_STORE_REST_PORT)]
+ | sort | join(",")' "$rendered")]"
+ pd_rest="$(jq -r --arg h "$(printf '%s' "${pd_peers}" | cut -d, -f1 | cut -d: -f1)" \
+ '.services | to_entries[]
+ | select(.value.hostname == $h)
+ | .value.hostname + ":" + .value.environment.HG_PD_REST_PORT' \
+ "$rendered")"
+ assert_props "cluster=${cluster_name}" "matches HG_SERVER_CLUSTER"
+ assert_props "pd.enabled=true" "keeps Hubble in PD mode"
+ assert_props "auth.enabled=true" "keeps Hubble in authenticated mode"
+ assert_props "pd.peers=${pd_peers}" "matches HG_SERVER_PD_PEERS"
+ assert_props "pd.server=${pd_rest}" "names a real PD REST endpoint"
+ assert_props "operations.store.allowed_targets=${store_targets}" \
+ "lists every Store REST endpoint"
+
+ # The documented opt-out must actually opt out: no admin password,
+ # no shared token secret, and a healthcheck that does not demand
+ # the 401/200 pair the authenticated stack requires. A silent
+ # failure here would leave users on an unauthenticated cluster
+ # that still advertises itself as authenticated.
+ # Use the exact placeholders the README documents, including the
+ # deliberately short token: it must never reach a container, and
+ # the assertions below are what prove the override drops it.
+ local non_auth="docker/docker-compose-3x3.non-auth.yml"
+ HUGEGRAPH_ADMIN_PASSWORD=non-auth-placeholder \
+ HUGEGRAPH_AUTH_TOKEN_SECRET=non-auth-placeholder \
+ docker compose --env-file /dev/null -f "$cluster" -f "$non_auth" \
+ config --format json > "$rendered"
+ jq -e '
+ . as $root |
+ all(["server0", "server1", "server2"][];
+ ($root.services[.].environment | has("PASSWORD") | not) and
+ ($root.services[.].environment
+ | has("HG_SERVER_AUTH_TOKEN_SECRET") | not) and
+ ($root.services[.].healthcheck.test[1]
+ | contains("/versions") and (contains("401") | not)))
+ ' "$rendered" >/dev/null
+ # The non-auth properties file is never booted by CI, so without
+ # this its topology can drift from the compose files unnoticed and
+ # users become the detection mechanism. Same contract, same source
+ # of truth, only the auth mode differs.
+ props="docker/hugegraph-hubble-3x3.non-auth.properties"
+ assert_props "auth.enabled=false" "runs Hubble without a login"
+ assert_props "cluster=${cluster_name}" "matches HG_SERVER_CLUSTER"
+ assert_props "pd.enabled=true" "keeps Hubble in PD mode"
+ assert_props "pd.peers=${pd_peers}" "matches HG_SERVER_PD_PEERS"
+ assert_props "pd.server=${pd_rest}" "names a real PD REST endpoint"
+ assert_props "operations.store.allowed_targets=${store_targets}" \
+ "lists every Store REST endpoint"
+ }
+
+ check_cluster_compose
+
+ # Only runs when this contract changes. The step boots the published
+ # images to prove the Compose wiring end to end, so leaving it on every
+ # build would turn a slow or rate-limited registry into a red run for
+ # pull requests that touch none of these files.
+ - name: Check whether the Compose contract changed
+ if: ${{ env.BACKEND == 'rocksdb' }}
+ id: compose_changed
+ run: |
+ set -euo pipefail
+ base="${{ github.event.pull_request.base.sha }}"
Review Comment:
⚠️ This gate is inert on `push`, so the ten-container smoke test runs on every master build.
`github.event.pull_request.base.sha` is only populated for `pull_request` events, and this workflow also triggers on `push` to `master`, `release-*` and `test-*` (`server-ci.yml:3-9`). On those runs `base` is the empty string, `[ -n "$base" ]` is false, `changed` stays at its `true` initialiser, and the smoke step's `if` is satisfied — so every merge to master boots the nine-container cluster twice with `--wait-timeout 420` plus the attach flow at 180, and pulls `hugegraph/{pd,store,server,hubble}` from Docker Hub because all four are `pull_policy: always`.
That is the outcome the step's own comment says it exists to avoid ("leaving it on every build would turn a slow or rate-limited registry into a red run"), and on master a red run blocks everyone rather than one PR. The pull_request half is fine: `actions/checkout` with `fetch-depth: 5` (`:50`) leaves the merge commit's parents in the object store, so both `git cat-file -e` and the `git diff` resolve.
Requested change: add `github.event_name == 'pull_request'` to the smoke step's `if`, or give the push case something to diff against — `base="${{ github.event.before }}"` when the pull_request payload is absent — so the gate means the same thing on both event types.
##########
.github/workflows/server-ci.yml:
##########
@@ -138,6 +140,366 @@ jobs:
check_compose docker/docker-compose.yml always always
check_compose docker/docker-compose.dev.yml build missing
+ check_cluster_compose() {
+ local cluster="docker/docker-compose-3pd-3store-3server.yml"
+ local addon="docker/docker-compose-hubble.yml"
+ local rendered
+ local token_fixture=ci-test-token-secret-32-bytes-long
+ rendered="$(mktemp)"
+ if [ "${#token_fixture}" -lt 32 ]; then
+ echo "CI token fixture must be at least 32 bytes" >&2
+ return 1
+ fi
+
+ # RETURN only: an EXIT trap would fire after this function's
+ # `local rendered` has gone out of scope, which `set -u` turns
+ # into an "unbound variable" error. A hard errexit abort can
+ # therefore still leak one temp file, which is acceptable on an
+ # ephemeral runner.
+ trap 'rm -f "$rendered"' RETURN
+
+ # --env-file /dev/null on every invocation: Compose otherwise reads
+ # docker/.env automatically, and the quickstart tells operators to
+ # create one holding exactly the credentials these assertions
+ # control. Without it the checks pass in CI (which has no .env) but
+ # report false failures for anyone running them locally after
+ # following the quickstart, and the add-on render below — which
+ # deliberately unsets the variables to assert their defaults — would
+ # read whatever HUGEGRAPH_NETWORK/HUGEGRAPH_VERSION that .env holds.
+ # Pinning an empty env file makes both renders depend only on what
+ # each invocation sets explicitly.
+
+ # Both cluster credentials are required and may not be empty.
+ # Each case asserts the guard fired for the *intended* variable:
+ # a bare non-zero exit would also be produced by a YAML error, a
+ # renamed file, or a missing docker binary.
+ assert_guard() { # assert_guard <blamed-var> <description>
+ local var="$1" desc="$2" err
+ if err="$(docker compose --env-file /dev/null -f "$cluster" \
+ config -q 2>&1)"; then
+ echo "$cluster accepted $desc" >&2
+ return 1
+ fi
+ case "$err" in
+ *"$var"*) : ;;
+ *) echo "$cluster rejected $desc, but not because of $var: $err" >&2
+ return 1 ;;
+ esac
+ }
+
+ ( unset HUGEGRAPH_ADMIN_PASSWORD
+ export HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}"
+ assert_guard HUGEGRAPH_ADMIN_PASSWORD "an unset admin password" )
+ ( export HUGEGRAPH_ADMIN_PASSWORD=
+ export HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}"
+ assert_guard HUGEGRAPH_ADMIN_PASSWORD "an empty admin password" )
+ ( unset HUGEGRAPH_AUTH_TOKEN_SECRET
+ export HUGEGRAPH_ADMIN_PASSWORD=ci-test-password
+ assert_guard HUGEGRAPH_AUTH_TOKEN_SECRET "an unset token secret" )
+ ( export HUGEGRAPH_ADMIN_PASSWORD=ci-test-password
+ export HUGEGRAPH_AUTH_TOKEN_SECRET=
+ assert_guard HUGEGRAPH_AUTH_TOKEN_SECRET "an empty token secret" )
+ # The add-on alone must define Hubble and nothing else, join the
+ # shared external network with its default name, and need no
+ # credentials or overrides.
+ env -u HUGEGRAPH_ADMIN_PASSWORD -u HUGEGRAPH_AUTH_TOKEN_SECRET \
+ -u HUGEGRAPH_NETWORK -u HUGEGRAPH_VERSION \
+ -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY -u HUBBLE_PUBLISH_HOST \
+ -u HUBBLE_DB_VOLUME -u HUBBLE_UPLOAD_VOLUME \
+ docker compose --env-file /dev/null -f "$addon" config --format json > "$rendered"
+ jq -e '
+ (.services | keys) == ["hubble"] and
+ .networks."hg-net".external == true and
+ .networks."hg-net".name == "hugegraph-net" and
+ (.services.hubble.networks | has("hg-net")) and
+ (.services.hubble | has("depends_on") | not) and
+ .volumes."hg-hubble-db".name == "hugegraph-hubble-db" and
+ .volumes."hg-hubble-db".external == true and
+ .volumes."hg-hubble-upload-files".name ==
+ "hugegraph-hubble-upload-files" and
+ .volumes."hg-hubble-upload-files".external == true and
+ .services.hubble.environment.SPRING_DATASOURCE_URL ==
+ "jdbc:h2:file:/hubble/db/hubble;DB_CLOSE_ON_EXIT=FALSE" and
+ any(.services.hubble.volumes[];
+ .target == "/hubble/conf/hugegraph-hubble.properties" and
+ (.source | endswith("hugegraph-hubble-3x3.properties")))
+ and any(.services.hubble.volumes[];
+ .source == "hg-hubble-db" and
+ .target == "/hubble/db")
+ and any(.services.hubble.volumes[];
+ .source == "hg-hubble-upload-files" and
+ .target == "/hubble/upload-files")
+ ' "$rendered" >/dev/null
+
+ # The cluster's own default network name must match the add-on's,
+ # or the attach flow and the cluster land on different networks.
+ # The combined render below pins an override, so it cannot catch a
+ # drifting default.
+ HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}" \
+ env -u HUGEGRAPH_NETWORK -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY \
+ -u HUGEGRAPH_PULL_POLICY -u HUGEGRAPH_CONTROL_PLANE_HOST \
+ -u HUGEGRAPH_SERVER_PUBLISH_HOST -u HUBBLE_PROPERTIES \
+ -u HUBBLE_PUBLISH_HOST \
+ docker compose --env-file /dev/null -f "$cluster" \
+ config --format json > "$rendered"
+ jq -e '.networks."hg-net".name == "hugegraph-net"' \
+ "$rendered" >/dev/null
+
+ # The combined render carries the PD-registration and auth
+ # settings on every server replica and keeps Hubble on loopback.
+ # Rendered with non-default HUGEGRAPH_NETWORK/HUGEGRAPH_VERSION so
+ # CI fails if any file stops honoring the overrides (the add-on
+ # render above covers the defaults).
+ HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}" \
+ HUGEGRAPH_NETWORK=ci-test-net \
+ HUGEGRAPH_VERSION=ci-test-tag \
+ env -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY -u HUBBLE_PUBLISH_HOST \
+ -u HUBBLE_DB_VOLUME -u HUBBLE_UPLOAD_VOLUME \
+ -u HUGEGRAPH_PULL_POLICY -u HUGEGRAPH_CONTROL_PLANE_HOST \
+ -u HUGEGRAPH_SERVER_PUBLISH_HOST -u HUBBLE_PROPERTIES \
+ docker compose --env-file /dev/null -f "$cluster" -f "$addon" \
+ config --format json > "$rendered"
+ jq -e '
+ . as $root |
+ .name == "hugegraph-3x3" and
+ (.services | keys | length) == 10 and
+ .networks."hg-net".external == true and
+ .networks."hg-net".name == "ci-test-net" and
+ all(.services[]; .networks | has("hg-net")) and
+ .services.pd0.image == "hugegraph/pd:ci-test-tag" and
+ .services.store0.image == "hugegraph/store:ci-test-tag" and
+ .services.server0.image == "hugegraph/server:ci-test-tag" and
+ .services.hubble.image == "hugegraph/hubble:ci-test-tag" and
+ all(["server0", "server1", "server2"][];
+ $root.services[.].environment.HG_SERVER_USE_PD == "true" and
+ $root.services[.].environment.HG_SERVER_CLUSTER == "hg" and
+ $root.services[.].environment.HG_SERVER_INIT_STORE_ENABLED ==
+ "false" and
+ $root.services[.].environment.PASSWORD ==
+ "ci-test-password" and
+ $root.services[.].environment.HG_SERVER_AUTH_TOKEN_SECRET ==
+ "ci-test-token-secret-32-bytes-long") and
+ (.services.server0.healthcheck.test[1] |
+ contains("= 401") and (contains("PASSWORD") | not)) and
+ .services.server0.environment.HG_SERVER_REST_URL ==
+ "http://server0:8080" and
+ .services.server1.environment.HG_SERVER_REST_URL ==
+ "http://server1:8080" and
+ .services.server2.environment.HG_SERVER_REST_URL ==
+ "http://server2:8080" and
+ (.services.hubble | has("depends_on") | not) and
+ .services.hubble.pull_policy == "always" and
+ .volumes."hg-hubble-db".name == "hugegraph-hubble-db" and
+ .volumes."hg-hubble-db".external == true and
+ .volumes."hg-hubble-upload-files".name ==
+ "hugegraph-hubble-upload-files" and
+ .volumes."hg-hubble-upload-files".external == true and
+ .services.hubble.environment.SPRING_DATASOURCE_URL ==
+ "jdbc:h2:file:/hubble/db/hubble;DB_CLOSE_ON_EXIT=FALSE" and
+ (.services.hubble.healthcheck.test[1] |
+ contains("http://127.0.0.1:8088/about") and
+ contains("\"status\":200") and
+ contains("\"name\":\"hugegraph-hubble\"")) and
+ any(.services.hubble.ports[];
+ .target == 8088 and .published == "8088" and
+ .host_ip == "127.0.0.1") and
+ all(["pd0", "pd1", "pd2", "store0", "store1", "store2",
+ "server0", "server1", "server2"][];
+ all($root.services[.].ports[]; .host_ip == "127.0.0.1"))
+ ' "$rendered" >/dev/null
+
+ # Hubble's properties file is mounted, not rendered, so Compose
+ # validation alone cannot catch it drifting from the services it
+ # describes. Tie the two together: renaming a service, changing a
+ # container hostname, or moving a REST port must be reflected in
+ # both places or CI fails here. Values are derived from the
+ # rendered model (hostnames and ports included) so the assertions
+ # cannot silently agree with a stale file.
+ local props="docker/hugegraph-hubble-3x3.properties"
+ local pd_peers store_targets cluster_name pd_rest
+ assert_props() { # assert_props <exact-line> <what-it-must-match>
+ grep -Fqx "$1" "$props" ||
+ { echo "$props: expected line '$1' ($2)" >&2; return 1; }
+ }
+ cluster_name="$(jq -r '.services.server0.environment.HG_SERVER_CLUSTER' \
+ "$rendered")"
+ pd_peers="$(jq -r '.services.server0.environment.HG_SERVER_PD_PEERS' \
+ "$rendered")"
+ store_targets="[$(jq -r '[.services | to_entries[]
+ | select(.key | startswith("store"))
+ | "http://" + .value.hostname + ":"
+ + (.value.environment.HG_STORE_REST_PORT)]
+ | sort | join(",")' "$rendered")]"
+ pd_rest="$(jq -r --arg h "$(printf '%s' "${pd_peers}" | cut -d, -f1 | cut -d: -f1)" \
+ '.services | to_entries[]
+ | select(.value.hostname == $h)
+ | .value.hostname + ":" + .value.environment.HG_PD_REST_PORT' \
+ "$rendered")"
+ assert_props "cluster=${cluster_name}" "matches HG_SERVER_CLUSTER"
+ assert_props "pd.enabled=true" "keeps Hubble in PD mode"
+ assert_props "auth.enabled=true" "keeps Hubble in authenticated mode"
+ assert_props "pd.peers=${pd_peers}" "matches HG_SERVER_PD_PEERS"
+ assert_props "pd.server=${pd_rest}" "names a real PD REST endpoint"
+ assert_props "operations.store.allowed_targets=${store_targets}" \
+ "lists every Store REST endpoint"
+
+ # The documented opt-out must actually opt out: no admin password,
+ # no shared token secret, and a healthcheck that does not demand
+ # the 401/200 pair the authenticated stack requires. A silent
+ # failure here would leave users on an unauthenticated cluster
+ # that still advertises itself as authenticated.
+ # Use the exact placeholders the README documents, including the
+ # deliberately short token: it must never reach a container, and
+ # the assertions below are what prove the override drops it.
+ local non_auth="docker/docker-compose-3x3.non-auth.yml"
+ HUGEGRAPH_ADMIN_PASSWORD=non-auth-placeholder \
+ HUGEGRAPH_AUTH_TOKEN_SECRET=non-auth-placeholder \
+ docker compose --env-file /dev/null -f "$cluster" -f "$non_auth" \
+ config --format json > "$rendered"
+ jq -e '
+ . as $root |
+ all(["server0", "server1", "server2"][];
+ ($root.services[.].environment | has("PASSWORD") | not) and
+ ($root.services[.].environment
+ | has("HG_SERVER_AUTH_TOKEN_SECRET") | not) and
+ ($root.services[.].healthcheck.test[1]
+ | contains("/versions") and (contains("401") | not)))
+ ' "$rendered" >/dev/null
+ # The non-auth properties file is never booted by CI, so without
+ # this its topology can drift from the compose files unnoticed and
+ # users become the detection mechanism. Same contract, same source
+ # of truth, only the auth mode differs.
+ props="docker/hugegraph-hubble-3x3.non-auth.properties"
+ assert_props "auth.enabled=false" "runs Hubble without a login"
+ assert_props "cluster=${cluster_name}" "matches HG_SERVER_CLUSTER"
+ assert_props "pd.enabled=true" "keeps Hubble in PD mode"
+ assert_props "pd.peers=${pd_peers}" "matches HG_SERVER_PD_PEERS"
+ assert_props "pd.server=${pd_rest}" "names a real PD REST endpoint"
+ assert_props "operations.store.allowed_targets=${store_targets}" \
+ "lists every Store REST endpoint"
+ }
+
+ check_cluster_compose
+
+ # Only runs when this contract changes. The step boots the published
+ # images to prove the Compose wiring end to end, so leaving it on every
+ # build would turn a slow or rate-limited registry into a red run for
+ # pull requests that touch none of these files.
+ - name: Check whether the Compose contract changed
+ if: ${{ env.BACKEND == 'rocksdb' }}
+ id: compose_changed
+ run: |
+ set -euo pipefail
+ base="${{ github.event.pull_request.base.sha }}"
+ changed=true
+ if [ -n "$base" ] && git cat-file -e "$base^{commit}" 2>/dev/null; then
+ if git diff --quiet "$base" HEAD -- docker/ .github/workflows/server-ci.yml; then
+ changed=false
+ fi
+ fi
+ echo "changed=$changed" >> "$GITHUB_OUTPUT"
+
+ - name: Run distributed Compose auth and attach smoke test
+ if: ${{ env.BACKEND == 'rocksdb' && steps.compose_changed.outputs.changed == 'true' }}
+ run: |
+ set -euo pipefail
+ command -v docker >/dev/null 2>&1
+ docker compose version >/dev/null
+ docker info >/dev/null
+
+ cluster="docker/docker-compose-3pd-3store-3server.yml"
+ addon="docker/docker-compose-hubble.yml"
+ run_id="${GITHUB_RUN_ID:-local}"
+ network="hugegraph-ci-${run_id}"
+ db_volume="hugegraph-ci-${run_id}-db"
+ upload_volume="hugegraph-ci-${run_id}-uploads"
+ admin_password='ci-smoke-admin-password'
+ token_secret='ci-smoke-token-secret-32-bytes-long'
+
+ # Every Compose call needs the same environment, and cleanup must run
+ # even when an assertion below fails.
+ export HUGEGRAPH_NETWORK="$network"
+ export HUGEGRAPH_ADMIN_PASSWORD="$admin_password"
+ export HUGEGRAPH_AUTH_TOKEN_SECRET="$token_secret"
+ export HUBBLE_DB_VOLUME="$db_volume"
+ export HUBBLE_UPLOAD_VOLUME="$upload_volume"
+
+ cleanup() {
+ docker compose --env-file /dev/null -f "$cluster" -f "$addon" \
+ down -v --remove-orphans >/dev/null 2>&1 || true
+ docker compose --env-file /dev/null -p "hugegraph-ci-attach-${run_id}" \
+ -f "$addon" down -v >/dev/null 2>&1 || true
+ docker network rm "$network" >/dev/null 2>&1 || true
+ docker volume rm "$db_volume" "$upload_volume" >/dev/null 2>&1 || true
+ }
+ trap cleanup EXIT
+ docker network create "$network"
+ # Both Hubble volumes are external, so nothing creates them implicitly.
+ docker volume create "$db_volume" >/dev/null
+ docker volume create "$upload_volume" >/dev/null
+
+ # The cluster must come up healthy on its own. Its Server healthcheck
+ # already requires an unauthenticated graph request to return 401, so
+ # --wait failing here means the images do not enforce the
+ # authentication this stack configures. The authenticated half is
+ # asserted below, where the credentials are known to be current.
+ docker compose --env-file /dev/null -f "$cluster" up -d --wait \
+ --wait-timeout 420
+ cluster_ids_before="$(docker compose --env-file /dev/null -f "$cluster" \
+ ps -q pd0 pd1 pd2 store0 store1 store2 server0 server1 server2 | sort)"
+ for port in 8080 8081 8082; do
+ code="$(curl --retry 30 --retry-delay 2 --retry-all-errors -s -o /dev/null \
+ -w '%{http_code}' "http://127.0.0.1:${port}/graphs/hugegraph/schema/vertexlabels")"
+ test "$code" = 401
+ code="$(curl --retry 30 --retry-delay 2 --retry-all-errors -s -o /dev/null \
+ -w '%{http_code}' -u "admin:${admin_password}" \
+ "http://127.0.0.1:${port}/graphs/hugegraph/schema/vertexlabels")"
+ test "$code" = 200
+ done
+
+ # The distributed topology must really be distributed: three PD peers
+ # and three Stores registered and Up. A partial cluster otherwise
+ # still passes a container-level health check. PD requires an
+ # Authorization header whose user is an internal service name; it
+ # never validates the password, hence the empty one.
+ pd_nodes="$(curl -fsS -u 'hubble:' "http://127.0.0.1:8620/v1/cluster" \
+ | grep -o '"restUrl":"pd[0-2]:8620"' | sort -u | wc -l)"
+ test "$pd_nodes" -eq 3
+ stores_up="$(curl -fsS -u 'hubble:' "http://127.0.0.1:8620/v1/stores" \
+ | grep -o '"state":"Up"' | wc -l)"
+ test "$stores_up" -eq 3
+
+ # Attaching Hubble must not recreate any cluster container.
+ docker compose --env-file /dev/null -p "hugegraph-ci-attach-${run_id}" \
+ -f "$addon" up -d --wait --wait-timeout 180
+ curl --retry 30 --retry-delay 2 --retry-all-errors -fsS \
+ http://127.0.0.1:8088/about | grep -q '"name":"hugegraph-hubble"'
+ cluster_ids_after="$(docker compose --env-file /dev/null -f "$cluster" \
+ ps -q pd0 pd1 pd2 store0 store1 store2 server0 server1 server2 | sort)"
+ test "$cluster_ids_before" = "$cluster_ids_after"
Review Comment:
⚠️ This assertion cannot fail, and the flow where it would matter is not checked.
The attach `up` two lines above runs under `-p hugegraph-ci-attach-${run_id}` (`:475-476`), a different Compose project from the cluster's own `hugegraph-3x3` (`docker-compose-3pd-3store-3server.yml:18`). Compose does not touch containers outside the project it was invoked for, so `cluster_ids_after` equals `cluster_ids_before` no matter what the add-on declares — this comparison would still pass if `docker-compose-hubble.yml` grew a `depends_on` on the cluster services tomorrow, which is exactly the property the comment above it claims to be proving.
The flow where the no-`depends_on` design actually decides the outcome is the combined one at `:494-495`, which merges the add-on into the cluster's own project, and no id comparison is taken around it. Since the PR description lists "attaches Hubble and confirms no cluster container is recreated" as what closes the render-only CI finding, the check should sit where it can observe a regression.
Requested change: re-capture the nine ids after the combined `up -d --wait` at `:495` and compare them against `cluster_ids_before` too. Keeping the attach-flow comparison is harmless, but it should not be the only one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(docker): add attachable Hubble Compose add-on for the 3-node cluster [hugegraph]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149#issuecomment-5244608506
## [Codecov](https://app.codecov.io/gh/apache/hugegraph/pull/3149?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 34.74%. Comparing base ([`4f1a8b3`](https://app.codecov.io/gh/apache/hugegraph/commit/4f1a8b34169e021717906167444c75c4c22d426b?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`620f7ff`](https://app.codecov.io/gh/apache/hugegraph/commit/620f7ffc5cf857d40d1f7ff504a758eed60e4b0b?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #3149 +/- ##
============================================
- Coverage 39.30% 34.74% -4.57%
- Complexity 264 498 +234
============================================
Files 771 782 +11
Lines 65915 67065 +1150
Branches 8754 8951 +197
============================================
- Hits 25910 23302 -2608
- Misses 37241 41163 +3922
+ Partials 2764 2600 -164
```
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/hugegraph/pull/3149?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(docker): add attachable Hubble Compose add-on for the 3-node cluster [hugegraph]
Posted by "bitflicker64 (via GitHub)" <gi...@apache.org>.
bitflicker64 commented on code in PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149#discussion_r3789881553
##########
.github/workflows/server-ci.yml:
##########
@@ -138,6 +140,189 @@ jobs:
check_compose docker/docker-compose.yml always always
check_compose docker/docker-compose.dev.yml build missing
+ check_cluster_compose() {
+ local cluster="docker/docker-compose-3pd-3store-3server.yml"
+ local addon="docker/docker-compose-hubble.yml"
+ local rendered
+ local token_fixture=ci-test-token-secret-32-bytes-long
+ rendered="$(mktemp)"
+ if [ "${#token_fixture}" -lt 32 ]; then
+ echo "CI token fixture must be at least 32 bytes" >&2
+ return 1
+ fi
+
+ # RETURN only: an EXIT trap would fire after this function's
+ # `local rendered` has gone out of scope, which `set -u` turns
+ # into an "unbound variable" error. A hard errexit abort can
+ # therefore still leak one temp file, which is acceptable on an
+ # ephemeral runner.
+ trap 'rm -f "$rendered"' RETURN
+
+ # --env-file /dev/null on every invocation: Compose otherwise reads
+ # docker/.env automatically, and the quickstart tells operators to
+ # create one holding exactly the credentials these assertions
+ # control. Without it the checks pass in CI (which has no .env) but
+ # report false failures for anyone running them locally after
+ # following the quickstart, and the add-on render below — which
+ # deliberately unsets the variables to assert their defaults — would
+ # read whatever HUGEGRAPH_NETWORK/HUGEGRAPH_VERSION that .env holds.
+ # Pinning an empty env file makes both renders depend only on what
+ # each invocation sets explicitly.
+
+ # Both cluster credentials are required and may not be empty.
+ # Each case asserts the guard fired for the *intended* variable:
+ # a bare non-zero exit would also be produced by a YAML error, a
+ # renamed file, or a missing docker binary.
+ assert_guard() { # assert_guard <blamed-var> <description>
+ local var="$1" desc="$2" err
+ if err="$(docker compose --env-file /dev/null -f "$cluster" \
+ config -q 2>&1)"; then
+ echo "$cluster accepted $desc" >&2
+ return 1
+ fi
+ case "$err" in
+ *"$var"*) : ;;
+ *) echo "$cluster rejected $desc, but not because of $var: $err" >&2
+ return 1 ;;
+ esac
+ }
+
+ ( unset HUGEGRAPH_ADMIN_PASSWORD
+ export HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}"
+ assert_guard HUGEGRAPH_ADMIN_PASSWORD "an unset admin password" )
+ ( export HUGEGRAPH_ADMIN_PASSWORD=
+ export HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}"
+ assert_guard HUGEGRAPH_ADMIN_PASSWORD "an empty admin password" )
+ ( unset HUGEGRAPH_AUTH_TOKEN_SECRET
+ export HUGEGRAPH_ADMIN_PASSWORD=ci-test-password
+ assert_guard HUGEGRAPH_AUTH_TOKEN_SECRET "an unset token secret" )
+ ( export HUGEGRAPH_ADMIN_PASSWORD=ci-test-password
+ export HUGEGRAPH_AUTH_TOKEN_SECRET=
+ assert_guard HUGEGRAPH_AUTH_TOKEN_SECRET "an empty token secret" )
+ # The add-on alone must define Hubble and nothing else, join the
+ # shared external network with its default name, and need no
+ # credentials or overrides.
+ env -u HUGEGRAPH_ADMIN_PASSWORD -u HUGEGRAPH_AUTH_TOKEN_SECRET \
+ -u HUGEGRAPH_NETWORK -u HUGEGRAPH_VERSION \
+ -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY -u HUBBLE_PUBLISH_HOST \
+ docker compose --env-file /dev/null -f "$addon" config --format json > "$rendered"
+ jq -e '
+ (.services | keys) == ["hubble"] and
+ .networks."hg-net".external == true and
+ .networks."hg-net".name == "hugegraph-net" and
+ (.services.hubble.networks | has("hg-net")) and
+ (.services.hubble | has("depends_on") | not) and
+ any(.services.hubble.volumes[];
+ .target == "/hubble/conf/hugegraph-hubble.properties" and
+ (.source | endswith("hugegraph-hubble-3x3.properties")))
+ and any(.services.hubble.volumes[];
+ .source == "hg-hubble-db" and
+ .target == "/hubble/db")
+ and any(.services.hubble.volumes[];
+ .source == "hg-hubble-upload-files" and
+ .target == "/hubble/upload-files")
+ ' "$rendered" >/dev/null
+
+ # The cluster's own default network name must match the add-on's,
+ # or the attach flow and the cluster land on different networks.
+ # The combined render below pins an override, so it cannot catch a
+ # drifting default.
+ HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}" \
+ env -u HUGEGRAPH_NETWORK -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY \
+ -u HUBBLE_PUBLISH_HOST \
+ docker compose --env-file /dev/null -f "$cluster" \
+ config --format json > "$rendered"
+ jq -e '.networks."hg-net".name == "hugegraph-net"' \
+ "$rendered" >/dev/null
+
+ # The combined render carries the PD-registration and auth
+ # settings on every server replica and keeps Hubble on loopback.
+ # Rendered with non-default HUGEGRAPH_NETWORK/HUGEGRAPH_VERSION so
+ # CI fails if any file stops honoring the overrides (the add-on
+ # render above covers the defaults).
+ HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}" \
+ HUGEGRAPH_NETWORK=ci-test-net \
+ HUGEGRAPH_VERSION=ci-test-tag \
+ env -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY -u HUBBLE_PUBLISH_HOST \
+ docker compose --env-file /dev/null -f "$cluster" -f "$addon" \
+ config --format json > "$rendered"
+ jq -e '
+ .name == "hugegraph-3x3" and
+ (.services | keys | length) == 10 and
+ .networks."hg-net".external == true and
+ .networks."hg-net".name == "ci-test-net" and
+ all(.services[]; .networks | has("hg-net")) and
+ .services.pd0.image == "hugegraph/pd:ci-test-tag" and
+ .services.store0.image == "hugegraph/store:ci-test-tag" and
+ .services.server0.image == "hugegraph/server:ci-test-tag" and
+ .services.hubble.image == "hugegraph/hubble:ci-test-tag" and
+ .services.server0.environment.HG_SERVER_USE_PD == "true" and
+ .services.server0.environment.HG_SERVER_CLUSTER == "hg" and
+ .services.server0.environment.HG_SERVER_REST_URL ==
+ "http://server0:8080" and
+ .services.server1.environment.HG_SERVER_REST_URL ==
+ "http://server1:8080" and
+ .services.server1.environment.HG_SERVER_AUTH_TOKEN_SECRET ==
+ "ci-test-token-secret-32-bytes-long" and
+ .services.server2.environment.HG_SERVER_REST_URL ==
+ "http://server2:8080" and
+ .services.server2.environment.HG_SERVER_AUTH_TOKEN_SECRET ==
+ "ci-test-token-secret-32-bytes-long" and
+ .services.server0.environment.HG_SERVER_INIT_STORE_ENABLED ==
+ "false" and
+ .services.server0.environment.PASSWORD ==
+ "ci-test-password" and
+ .services.server0.environment.HG_SERVER_AUTH_TOKEN_SECRET ==
+ "ci-test-token-secret-32-bytes-long" and
+ (.services.hubble | has("depends_on") | not) and
+ .services.hubble.pull_policy == "missing" and
+ (.services.hubble.healthcheck.test[1] |
+ contains("http://127.0.0.1:8088/about") and
+ contains("\"status\":200") and
+ contains("\"name\":\"hugegraph-hubble\"")) and
+ any(.services.hubble.ports[];
+ .target == 8088 and .published == "8088" and
+ .host_ip == "127.0.0.1")
+ ' "$rendered" >/dev/null
+
+ # Hubble's properties file is mounted, not rendered, so Compose
+ # validation alone cannot catch it drifting from the services it
+ # describes. Tie the two together: renaming a service, changing a
+ # container hostname, or moving a REST port must be reflected in
+ # both places or CI fails here. Values are derived from the
+ # rendered model (hostnames and ports included) so the assertions
+ # cannot silently agree with a stale file.
+ local props="docker/hugegraph-hubble-3x3.properties"
+ local pd_peers store_targets cluster_name pd_rest
+ assert_props() { # assert_props <exact-line> <what-it-must-match>
+ grep -Fqx "$1" "$props" ||
+ { echo "$props: expected line '$1' ($2)" >&2; return 1; }
+ }
+ cluster_name="$(jq -r '.services.server0.environment.HG_SERVER_CLUSTER' \
+ "$rendered")"
+ pd_peers="$(jq -r '.services.server0.environment.HG_SERVER_PD_PEERS' \
+ "$rendered")"
+ store_targets="[$(jq -r '[.services | to_entries[]
+ | select(.key | startswith("store"))
+ | "http://" + .value.hostname + ":"
+ + (.value.environment.HG_STORE_REST_PORT)]
+ | sort | join(",")' "$rendered")]"
+ pd_rest="$(jq -r --arg h "$(printf '%s' "${pd_peers}" | cut -d, -f1 | cut -d: -f1)" \
+ '.services | to_entries[]
+ | select(.value.hostname == $h)
+ | .value.hostname + ":" + .value.environment.HG_PD_REST_PORT' \
+ "$rendered")"
+ assert_props "cluster=${cluster_name}" "matches HG_SERVER_CLUSTER"
Review Comment:
Configurable advertised addresses seem to be a follow‑up to external‑client addressing, but they are not required for the add‑on path. What do you think we should do?
##########
.github/workflows/server-ci.yml:
##########
@@ -138,6 +140,189 @@ jobs:
check_compose docker/docker-compose.yml always always
check_compose docker/docker-compose.dev.yml build missing
+ check_cluster_compose() {
+ local cluster="docker/docker-compose-3pd-3store-3server.yml"
+ local addon="docker/docker-compose-hubble.yml"
+ local rendered
+ local token_fixture=ci-test-token-secret-32-bytes-long
+ rendered="$(mktemp)"
+ if [ "${#token_fixture}" -lt 32 ]; then
+ echo "CI token fixture must be at least 32 bytes" >&2
+ return 1
+ fi
+
+ # RETURN only: an EXIT trap would fire after this function's
+ # `local rendered` has gone out of scope, which `set -u` turns
+ # into an "unbound variable" error. A hard errexit abort can
+ # therefore still leak one temp file, which is acceptable on an
+ # ephemeral runner.
+ trap 'rm -f "$rendered"' RETURN
+
+ # --env-file /dev/null on every invocation: Compose otherwise reads
+ # docker/.env automatically, and the quickstart tells operators to
+ # create one holding exactly the credentials these assertions
+ # control. Without it the checks pass in CI (which has no .env) but
+ # report false failures for anyone running them locally after
+ # following the quickstart, and the add-on render below — which
+ # deliberately unsets the variables to assert their defaults — would
+ # read whatever HUGEGRAPH_NETWORK/HUGEGRAPH_VERSION that .env holds.
+ # Pinning an empty env file makes both renders depend only on what
+ # each invocation sets explicitly.
+
+ # Both cluster credentials are required and may not be empty.
+ # Each case asserts the guard fired for the *intended* variable:
+ # a bare non-zero exit would also be produced by a YAML error, a
+ # renamed file, or a missing docker binary.
+ assert_guard() { # assert_guard <blamed-var> <description>
+ local var="$1" desc="$2" err
+ if err="$(docker compose --env-file /dev/null -f "$cluster" \
+ config -q 2>&1)"; then
+ echo "$cluster accepted $desc" >&2
+ return 1
+ fi
+ case "$err" in
+ *"$var"*) : ;;
+ *) echo "$cluster rejected $desc, but not because of $var: $err" >&2
+ return 1 ;;
+ esac
+ }
+
+ ( unset HUGEGRAPH_ADMIN_PASSWORD
+ export HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}"
+ assert_guard HUGEGRAPH_ADMIN_PASSWORD "an unset admin password" )
+ ( export HUGEGRAPH_ADMIN_PASSWORD=
+ export HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}"
+ assert_guard HUGEGRAPH_ADMIN_PASSWORD "an empty admin password" )
+ ( unset HUGEGRAPH_AUTH_TOKEN_SECRET
+ export HUGEGRAPH_ADMIN_PASSWORD=ci-test-password
+ assert_guard HUGEGRAPH_AUTH_TOKEN_SECRET "an unset token secret" )
+ ( export HUGEGRAPH_ADMIN_PASSWORD=ci-test-password
+ export HUGEGRAPH_AUTH_TOKEN_SECRET=
+ assert_guard HUGEGRAPH_AUTH_TOKEN_SECRET "an empty token secret" )
+ # The add-on alone must define Hubble and nothing else, join the
+ # shared external network with its default name, and need no
+ # credentials or overrides.
+ env -u HUGEGRAPH_ADMIN_PASSWORD -u HUGEGRAPH_AUTH_TOKEN_SECRET \
+ -u HUGEGRAPH_NETWORK -u HUGEGRAPH_VERSION \
+ -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY -u HUBBLE_PUBLISH_HOST \
+ docker compose --env-file /dev/null -f "$addon" config --format json > "$rendered"
+ jq -e '
+ (.services | keys) == ["hubble"] and
+ .networks."hg-net".external == true and
+ .networks."hg-net".name == "hugegraph-net" and
+ (.services.hubble.networks | has("hg-net")) and
+ (.services.hubble | has("depends_on") | not) and
+ any(.services.hubble.volumes[];
+ .target == "/hubble/conf/hugegraph-hubble.properties" and
+ (.source | endswith("hugegraph-hubble-3x3.properties")))
+ and any(.services.hubble.volumes[];
+ .source == "hg-hubble-db" and
+ .target == "/hubble/db")
+ and any(.services.hubble.volumes[];
+ .source == "hg-hubble-upload-files" and
+ .target == "/hubble/upload-files")
+ ' "$rendered" >/dev/null
+
+ # The cluster's own default network name must match the add-on's,
+ # or the attach flow and the cluster land on different networks.
+ # The combined render below pins an override, so it cannot catch a
+ # drifting default.
+ HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}" \
+ env -u HUGEGRAPH_NETWORK -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY \
+ -u HUBBLE_PUBLISH_HOST \
+ docker compose --env-file /dev/null -f "$cluster" \
+ config --format json > "$rendered"
+ jq -e '.networks."hg-net".name == "hugegraph-net"' \
+ "$rendered" >/dev/null
+
+ # The combined render carries the PD-registration and auth
+ # settings on every server replica and keeps Hubble on loopback.
+ # Rendered with non-default HUGEGRAPH_NETWORK/HUGEGRAPH_VERSION so
+ # CI fails if any file stops honoring the overrides (the add-on
+ # render above covers the defaults).
+ HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}" \
+ HUGEGRAPH_NETWORK=ci-test-net \
+ HUGEGRAPH_VERSION=ci-test-tag \
+ env -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY -u HUBBLE_PUBLISH_HOST \
+ docker compose --env-file /dev/null -f "$cluster" -f "$addon" \
+ config --format json > "$rendered"
+ jq -e '
+ .name == "hugegraph-3x3" and
+ (.services | keys | length) == 10 and
+ .networks."hg-net".external == true and
+ .networks."hg-net".name == "ci-test-net" and
+ all(.services[]; .networks | has("hg-net")) and
+ .services.pd0.image == "hugegraph/pd:ci-test-tag" and
+ .services.store0.image == "hugegraph/store:ci-test-tag" and
+ .services.server0.image == "hugegraph/server:ci-test-tag" and
+ .services.hubble.image == "hugegraph/hubble:ci-test-tag" and
+ .services.server0.environment.HG_SERVER_USE_PD == "true" and
+ .services.server0.environment.HG_SERVER_CLUSTER == "hg" and
+ .services.server0.environment.HG_SERVER_REST_URL ==
+ "http://server0:8080" and
+ .services.server1.environment.HG_SERVER_REST_URL ==
+ "http://server1:8080" and
+ .services.server1.environment.HG_SERVER_AUTH_TOKEN_SECRET ==
+ "ci-test-token-secret-32-bytes-long" and
+ .services.server2.environment.HG_SERVER_REST_URL ==
+ "http://server2:8080" and
+ .services.server2.environment.HG_SERVER_AUTH_TOKEN_SECRET ==
+ "ci-test-token-secret-32-bytes-long" and
+ .services.server0.environment.HG_SERVER_INIT_STORE_ENABLED ==
+ "false" and
+ .services.server0.environment.PASSWORD ==
+ "ci-test-password" and
+ .services.server0.environment.HG_SERVER_AUTH_TOKEN_SECRET ==
+ "ci-test-token-secret-32-bytes-long" and
+ (.services.hubble | has("depends_on") | not) and
+ .services.hubble.pull_policy == "missing" and
+ (.services.hubble.healthcheck.test[1] |
+ contains("http://127.0.0.1:8088/about") and
+ contains("\"status\":200") and
+ contains("\"name\":\"hugegraph-hubble\"")) and
+ any(.services.hubble.ports[];
+ .target == 8088 and .published == "8088" and
+ .host_ip == "127.0.0.1")
+ ' "$rendered" >/dev/null
+
+ # Hubble's properties file is mounted, not rendered, so Compose
+ # validation alone cannot catch it drifting from the services it
+ # describes. Tie the two together: renaming a service, changing a
+ # container hostname, or moving a REST port must be reflected in
+ # both places or CI fails here. Values are derived from the
+ # rendered model (hostnames and ports included) so the assertions
+ # cannot silently agree with a stale file.
+ local props="docker/hugegraph-hubble-3x3.properties"
+ local pd_peers store_targets cluster_name pd_rest
+ assert_props() { # assert_props <exact-line> <what-it-must-match>
+ grep -Fqx "$1" "$props" ||
+ { echo "$props: expected line '$1' ($2)" >&2; return 1; }
+ }
+ cluster_name="$(jq -r '.services.server0.environment.HG_SERVER_CLUSTER' \
+ "$rendered")"
+ pd_peers="$(jq -r '.services.server0.environment.HG_SERVER_PD_PEERS' \
+ "$rendered")"
+ store_targets="[$(jq -r '[.services | to_entries[]
+ | select(.key | startswith("store"))
+ | "http://" + .value.hostname + ":"
+ + (.value.environment.HG_STORE_REST_PORT)]
+ | sort | join(",")' "$rendered")]"
+ pd_rest="$(jq -r --arg h "$(printf '%s' "${pd_peers}" | cut -d, -f1 | cut -d: -f1)" \
+ '.services | to_entries[]
+ | select(.value.hostname == $h)
+ | .value.hostname + ":" + .value.environment.HG_PD_REST_PORT' \
+ "$rendered")"
+ assert_props "cluster=${cluster_name}" "matches HG_SERVER_CLUSTER"
Review Comment:
I’ll add an explicit `assert_props "pd.enabled=true"` so the Hubble properties check cannot pass with PD mode disabled.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(docker): add attachable Hubble Compose add-on for the 3-node cluster [hugegraph]
Posted by "bitflicker64 (via GitHub)" <gi...@apache.org>.
bitflicker64 commented on code in PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149#discussion_r3789877450
##########
.github/workflows/server-ci.yml:
##########
@@ -138,6 +140,189 @@ jobs:
check_compose docker/docker-compose.yml always always
check_compose docker/docker-compose.dev.yml build missing
+ check_cluster_compose() {
+ local cluster="docker/docker-compose-3pd-3store-3server.yml"
+ local addon="docker/docker-compose-hubble.yml"
+ local rendered
+ local token_fixture=ci-test-token-secret-32-bytes-long
+ rendered="$(mktemp)"
+ if [ "${#token_fixture}" -lt 32 ]; then
+ echo "CI token fixture must be at least 32 bytes" >&2
+ return 1
+ fi
+
+ # RETURN only: an EXIT trap would fire after this function's
+ # `local rendered` has gone out of scope, which `set -u` turns
+ # into an "unbound variable" error. A hard errexit abort can
+ # therefore still leak one temp file, which is acceptable on an
+ # ephemeral runner.
+ trap 'rm -f "$rendered"' RETURN
+
+ # --env-file /dev/null on every invocation: Compose otherwise reads
+ # docker/.env automatically, and the quickstart tells operators to
+ # create one holding exactly the credentials these assertions
+ # control. Without it the checks pass in CI (which has no .env) but
+ # report false failures for anyone running them locally after
+ # following the quickstart, and the add-on render below — which
+ # deliberately unsets the variables to assert their defaults — would
+ # read whatever HUGEGRAPH_NETWORK/HUGEGRAPH_VERSION that .env holds.
+ # Pinning an empty env file makes both renders depend only on what
+ # each invocation sets explicitly.
+
+ # Both cluster credentials are required and may not be empty.
+ # Each case asserts the guard fired for the *intended* variable:
+ # a bare non-zero exit would also be produced by a YAML error, a
+ # renamed file, or a missing docker binary.
+ assert_guard() { # assert_guard <blamed-var> <description>
+ local var="$1" desc="$2" err
+ if err="$(docker compose --env-file /dev/null -f "$cluster" \
+ config -q 2>&1)"; then
+ echo "$cluster accepted $desc" >&2
+ return 1
+ fi
+ case "$err" in
+ *"$var"*) : ;;
+ *) echo "$cluster rejected $desc, but not because of $var: $err" >&2
+ return 1 ;;
+ esac
+ }
+
+ ( unset HUGEGRAPH_ADMIN_PASSWORD
+ export HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}"
+ assert_guard HUGEGRAPH_ADMIN_PASSWORD "an unset admin password" )
+ ( export HUGEGRAPH_ADMIN_PASSWORD=
+ export HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}"
+ assert_guard HUGEGRAPH_ADMIN_PASSWORD "an empty admin password" )
+ ( unset HUGEGRAPH_AUTH_TOKEN_SECRET
+ export HUGEGRAPH_ADMIN_PASSWORD=ci-test-password
+ assert_guard HUGEGRAPH_AUTH_TOKEN_SECRET "an unset token secret" )
+ ( export HUGEGRAPH_ADMIN_PASSWORD=ci-test-password
+ export HUGEGRAPH_AUTH_TOKEN_SECRET=
+ assert_guard HUGEGRAPH_AUTH_TOKEN_SECRET "an empty token secret" )
+ # The add-on alone must define Hubble and nothing else, join the
+ # shared external network with its default name, and need no
+ # credentials or overrides.
+ env -u HUGEGRAPH_ADMIN_PASSWORD -u HUGEGRAPH_AUTH_TOKEN_SECRET \
+ -u HUGEGRAPH_NETWORK -u HUGEGRAPH_VERSION \
+ -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY -u HUBBLE_PUBLISH_HOST \
+ docker compose --env-file /dev/null -f "$addon" config --format json > "$rendered"
+ jq -e '
+ (.services | keys) == ["hubble"] and
+ .networks."hg-net".external == true and
+ .networks."hg-net".name == "hugegraph-net" and
+ (.services.hubble.networks | has("hg-net")) and
+ (.services.hubble | has("depends_on") | not) and
+ any(.services.hubble.volumes[];
+ .target == "/hubble/conf/hugegraph-hubble.properties" and
+ (.source | endswith("hugegraph-hubble-3x3.properties")))
+ and any(.services.hubble.volumes[];
+ .source == "hg-hubble-db" and
+ .target == "/hubble/db")
+ and any(.services.hubble.volumes[];
+ .source == "hg-hubble-upload-files" and
+ .target == "/hubble/upload-files")
+ ' "$rendered" >/dev/null
+
+ # The cluster's own default network name must match the add-on's,
+ # or the attach flow and the cluster land on different networks.
+ # The combined render below pins an override, so it cannot catch a
+ # drifting default.
+ HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}" \
+ env -u HUGEGRAPH_NETWORK -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY \
+ -u HUBBLE_PUBLISH_HOST \
+ docker compose --env-file /dev/null -f "$cluster" \
+ config --format json > "$rendered"
+ jq -e '.networks."hg-net".name == "hugegraph-net"' \
+ "$rendered" >/dev/null
+
+ # The combined render carries the PD-registration and auth
+ # settings on every server replica and keeps Hubble on loopback.
+ # Rendered with non-default HUGEGRAPH_NETWORK/HUGEGRAPH_VERSION so
+ # CI fails if any file stops honoring the overrides (the add-on
+ # render above covers the defaults).
+ HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}" \
+ HUGEGRAPH_NETWORK=ci-test-net \
+ HUGEGRAPH_VERSION=ci-test-tag \
+ env -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY -u HUBBLE_PUBLISH_HOST \
+ docker compose --env-file /dev/null -f "$cluster" -f "$addon" \
Review Comment:
Render‑only cannot catch runtime auth/DNS failures. For this PR I’ll keep and strengthen the compose contract assertions (including the in‑scope ones above). A full combined or attached Docker smoke job is useful but adds heavier CI overhead; I’d prefer a follow‑up rather than blocking the add‑on here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(docker): add attachable Hubble Compose add-on for the 3-node cluster [hugegraph]
Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on code in PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149#discussion_r3869121552
##########
docker/hugegraph-hubble-3x3.non-auth.properties:
##########
@@ -0,0 +1,51 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Hubble configuration for the 3-node cluster running WITHOUT authentication,
+# i.e. started with docker-compose-3x3.non-auth.yml. It is identical to
+# hugegraph-hubble-3x3.properties except for auth.enabled below. Use it only
+# on a trusted local network; the default stack keeps authentication on.
+#
+# This file REPLACES the image's /hubble/conf/hugegraph-hubble.properties
+# wholesale: it is bind-mounted over it, not merged with it. Keys absent
+# here therefore fall back to the code defaults in HubbleOptions, not to the
+# values in the shipped conf file. Re-check against the shipped conf when
+# upgrading Hubble.
+
+server.host=0.0.0.0
+server.port=8088
+
+cluster=hg
+idc=docker
+
+pd.enabled=true
+# Hubble's code default is auth.enabled=true. Without this line the UI would
+# demand a login against Servers that have no users configured.
+auth.enabled=false
Review Comment:
⚠️ The documented no-auth flow cannot disable Hubble login. Evidence: current apache/hugegraph-toolchain master still registers LoginInterceptor for /api/** and rejects requests without a session token; it has no auth.enabled option or consumer, while this file only adds auth.enabled=false. Starting with this properties file therefore still leaves Hubble's API behind its login, so the advertised no-auth flow cannot operate as documented. Implement a supported Hubble auth-disable setting or remove and document this unsupported Hubble no-auth path.
##########
docker/README.md:
##########
@@ -168,14 +175,170 @@ To validate local images without Compose replacing them with remote `latest`:
## 3-Node Cluster Quickstart
+The cluster and the Hubble add-on share one named Docker network so Hubble
+can attach to a running cluster without touching it. Treat that network as a
+trust boundary: only the Server layer performs real authentication. Store
+serves its control APIs with no credentials at all, and while PD's REST
+control APIs do require an `Authorization` header, PD only checks that the
+Basic-auth *user* is one of its internal service names and never validates
+the password — so any client on the network can read and drive them. Note
+that this stack depends on that behaviour: Hubble reaches PD as the service
+name `hubble` with an empty password, so tightening PD's credential check
+would also break Hubble's operations view. Because of that, PD and Store
+control-plane ports and the three Server REST ports bind to `127.0.0.1` by
+default, so an unauthenticated control plane is not reachable from outside the
+host. Publish them more widely only by setting `HUGEGRAPH_CONTROL_PLANE_HOST`
+or `HUGEGRAPH_SERVER_PUBLISH_HOST`, and only behind a network ACL or TLS
+terminator you control. Any container on the host can still join the network
+by declaring the well-known name. One-time setup:
+write the required credentials to a mode-600 `docker/.env` and create the
+network.
+The cluster file requires both credentials — the admin password enables
+authentication, and every Server replica must share one token secret so a
+token issued by any server validates on all of them. The `:?` guards fire
+on every Compose subcommand, including `down`.
+
+```bash
+(
+ set -eu
+ cd docker
+ command -v openssl >/dev/null 2>&1 || { echo "openssl not found" >&2; exit 1; }
+ [ -e .env ] || install -m 600 /dev/null .env
+ chmod 600 .env
+ # Keep appends on their own lines even if the file was hand-edited.
+ [ ! -s .env ] || [ -z "$(tail -c1 .env)" ] || printf '\n' >> .env
+ pat='^[[:space:]]*(export[[:space:]]+)?'
+ if ! grep -Eq "${pat}HUGEGRAPH_ADMIN_PASSWORD=" .env; then
+ admin_password="$(openssl rand -base64 12)"
+ printf "HUGEGRAPH_ADMIN_PASSWORD='%s'\n" "${admin_password}" >> .env
+ unset admin_password
+ fi
+ if ! grep -Eq "${pat}HUGEGRAPH_AUTH_TOKEN_SECRET=" .env; then
+ token_secret="$(openssl rand -hex 32)"
+ printf "HUGEGRAPH_AUTH_TOKEN_SECRET='%s'\n" "${token_secret}" >> .env
+ unset token_secret
+ fi
+ # Parse values as data. Do not source .env as shell: a value such as
+ # HUGEGRAPH_ADMIN_PASSWORD=$(...) would run on the operator host.
+ # The optional export group is capture 1; the quoted value is capture 2.
+ read_dotenv_value() {
+ key="$1"
+ key_pattern="^[[:space:]]*(export[[:space:]]+)?${key}="
+ count="$(grep -Ec "${key_pattern}" .env || true)"
+ [ "${count}" -eq 1 ] || return 1
+ value="$(sed -nE "s/${key_pattern}'([^']*)'[[:space:]]*$/\\2/p" .env)"
+ # sed prints nothing and still exits 0 when the value is unquoted or
+ # double-quoted, which would otherwise hand back an empty credential.
+ [ -n "${value}" ] || return 1
+ printf '%s\n' "${value}"
+ }
+ if ! admin_value="$(read_dotenv_value HUGEGRAPH_ADMIN_PASSWORD)"; then
+ echo "HUGEGRAPH_ADMIN_PASSWORD must use the documented single-quoted format" >&2
+ exit 1
+ fi
+ if ! token_value="$(read_dotenv_value HUGEGRAPH_AUTH_TOKEN_SECRET)"; then
+ echo "HUGEGRAPH_AUTH_TOKEN_SECRET must use the documented single-quoted format" >&2
+ exit 1
+ fi
+ [ -n "${admin_value}" ] || {
+ echo "HUGEGRAPH_ADMIN_PASSWORD must be non-empty in docker/.env" >&2
+ exit 1
+ }
+ [ "${#token_value}" -ge 32 ] || {
+ echo "HUGEGRAPH_AUTH_TOKEN_SECRET must be at least 32 bytes in docker/.env" >&2
+ exit 1
+ }
+ unset admin_value token_value
+ # The shared cluster network. To override the name, export
+ # HUGEGRAPH_NETWORK in this shell before running the block — a value
+ # in docker/.env is read by Compose, not by this script.
+ net="${HUGEGRAPH_NETWORK:-hugegraph-net}"
+ docker network inspect "${net}" >/dev/null 2>&1 ||
+ docker network create "${net}"
+ # Hubble's volumes are external for the same reason the network is: both
+ # documented flows share them, and an external volume is not destroyed by a
+ # `down -v` from either one.
+ for vol in "${HUBBLE_DB_VOLUME:-hugegraph-hubble-db}" \
Review Comment:
⚠️ The quickstart volume setup ignores custom names stored in docker/.env. Evidence: this shell loop expands HUBBLE_DB_VOLUME and HUBBLE_UPLOAD_VOLUME from the process environment, but Compose later reads those variables from .env; setting either there leaves only the default volume created and combined up fails with an external-volume-not-found error. Export these variables before the block or parse the .env values here.
##########
docker/README.md:
##########
@@ -287,17 +759,28 @@ Configuration is injected via environment variables. The old `docker/configs/app
> PD startup path uses the explicit `auth.admin_pa` value when it first creates
> the administrator. Changing it later does not rotate an existing password.
-The single-node Compose files also accept these deployment-level overrides:
-
-| Variable | Default | Description |
-|----------|---------|-------------|
-| `HUGEGRAPH_SERVER_IMAGE` | `hugegraph/server:<version>` | Complete Server image reference |
-| `HUGEGRAPH_SERVER_PULL_POLICY` | `always` (`build` for dev) | Server pull policy |
-| `HUBBLE_IMAGE` | `hugegraph/hubble:<version>` | Complete Hubble image reference |
-| `HUBBLE_PULL_POLICY` | `always` (`missing` for dev) | Hubble pull policy |
-| `HUBBLE_PUBLISH_HOST` | `127.0.0.1` | Hubble host bind address; remote access requires an HTTPS reverse proxy |
-| `HUGEGRAPH_ADMIN_PASSWORD` | required (`docker/.env`) | Initial admin password; no public default is provided |
-| `HUGEGRAPH_AUTH_TOKEN_SECRET` | generated | JWT signing secret; explicit values must be at least 32 bytes |
+The Compose files also accept these deployment-level overrides; the
+"Used by" column names the files that read each variable (single = the
+single-node files, cluster = `docker-compose-3pd-3store-3server.yml`,
+add-on = `docker-compose-hubble.yml`):
+
+| Variable | Used by | Default | Description |
+|----------|---------|---------|-------------|
+| `HUGEGRAPH_VERSION` | single (quickstart), cluster, add-on | `latest` | Shared image tag for PD, Store, Server, and Hubble; pin it in `docker/.env` so these files resolve the same release. The dev file builds from source and defaults Hubble to `latest`; set `HUBBLE_IMAGE` to pin it |
+| `HUGEGRAPH_SERVER_IMAGE` | single | `hugegraph/server:<version>` | Complete Server image reference |
+| `HUGEGRAPH_SERVER_PULL_POLICY` | single | `always` (`build` for dev) | Server pull policy |
+| `HUBBLE_IMAGE` | single, add-on | `hugegraph/hubble:<version>` | Complete Hubble image reference |
+| `HUBBLE_PULL_POLICY` | single, add-on | `always` (`missing` for dev) | Hubble pull policy |
+| `HUBBLE_PUBLISH_HOST` | single, add-on | `127.0.0.1` | Hubble host bind address; remote access requires an HTTPS reverse proxy |
+| `HUBBLE_PROPERTIES` | add-on | `./hugegraph-hubble-3x3.properties` | Hubble properties file bind-mounted into the container; set it to `./hugegraph-hubble-3x3.non-auth.properties` when the cluster runs without authentication |
+| `HUBBLE_DB_VOLUME` | add-on | `hugegraph-hubble-db` | Volume holding Hubble's H2 database; the explicit name is what lets the attach and combined flows share state, so change it only to run a second independent Hubble |
+| `HUBBLE_UPLOAD_VOLUME` | add-on | `hugegraph-hubble-upload-files` | Volume holding Hubble's uploaded files; same naming caveat as `HUBBLE_DB_VOLUME` |
+| `HUGEGRAPH_NETWORK` | cluster, add-on | `hugegraph-net` | Pre-created external Docker network shared by the 3-node cluster and the Hubble add-on; the single-node files use their own project bridge instead |
+| `HUGEGRAPH_PULL_POLICY` | cluster | `always` | Pull policy for the PD, Store, and Server images. The default refreshes a stale cached `latest`; set it to `missing` to run offline or against locally built tags |
Review Comment:
🧹 `missing` does not guarantee offline use. Evidence: Docker Compose pulls an image when it is absent under `pull_policy: missing`, while this table says `HUGEGRAPH_PULL_POLICY=missing` runs offline; the same guide uses `never` for local images. Document `never` for strict offline or local use and reserve `missing` for pull-if-absent.
##########
docker/README.md:
##########
@@ -196,11 +359,320 @@ curl http://localhost:8520/v1/health
# Check Server (Graph API)
curl http://localhost:8080/versions
-# List registered stores via PD
-curl http://localhost:8620/v1/stores
+# Every PD endpoint except /v1/health, /actuator/* and /v1/prom/targets/*
+# needs an Authorization header. PD only checks that the Basic-auth user is
+# one of its internal service names, so the empty password below is enough
+# — and it grants the full PD control plane, writes included, not just these
+# reads. That is exactly why the shared network must be treated as a trust
+# boundary. Without the header PD answers with an exception body, not data.
+pd_auth="Authorization: Basic $(printf 'hubble:' | base64)"
+
+# List registered stores via PD (expect three, each "state":"Up")
+curl -H "${pd_auth}" http://localhost:8620/v1/stores
# List partitions
-curl http://localhost:8620/v1/partitions
+curl -H "${pd_auth}" http://localhost:8620/v1/partitions
+```
+
+Confirm authentication actually engaged — `/versions` stays open by design,
+so it cannot tell you whether auth is on. A graph read without credentials
+must be rejected:
+
+```bash
+cd docker
+# Expect 401 on all three replicas
+for port in 8080 8081 8082; do
+ curl -s -o /dev/null -w "${port}: %{http_code}\n" \
+ "http://localhost:${port}/graphs/hugegraph/schema/vertexlabels"
+done
+
+# And a signed-in read must succeed. Parse the generated single-quoted
+# value without executing docker/.env as shell. Passing the credential
+# through --config keeps it out of argv, where `ps` would expose it.
+# The subshell keeps the `exit 1` below from closing an interactive shell.
+(
+set -eu
+read_dotenv_value() {
+ key="$1"
+ key_pattern="^[[:space:]]*(export[[:space:]]+)?${key}="
+ count="$(grep -Ec "${key_pattern}" .env || true)"
+ [ "${count}" -eq 1 ] || return 1
+ value="$(sed -nE "s/${key_pattern}'([^']*)'[[:space:]]*$/\\2/p" .env)"
+ # sed prints nothing and still exits 0 when the value is unquoted or
+ # double-quoted, which would otherwise hand back an empty credential.
+ [ -n "${value}" ] || return 1
+ printf '%s\n' "${value}"
+}
+HUGEGRAPH_ADMIN_PASSWORD="$(read_dotenv_value HUGEGRAPH_ADMIN_PASSWORD)" || {
+ echo "HUGEGRAPH_ADMIN_PASSWORD must use the documented single-quoted format" >&2
+ exit 1
+}
+curl -s -o /dev/null -w '%{http_code}\n' \
+ --config <(printf 'user = "admin:%s"\n' "${HUGEGRAPH_ADMIN_PASSWORD}") \
+ http://localhost:8080/graphs/hugegraph/schema/vertexlabels
+)
+```
+
+`200` from the second command with `401` from all three of the first means
+authentication is on and working. If the first command returns `200`, the
+running image ignored `PASSWORD` and the cluster is **unauthenticated** —
+the most likely cause is an image older than the release this integration
+needs (see the version note in the quickstart above).
+
+### Running without authentication
+
+The cluster above authenticates, and that is the right default: it is the
+configuration most people copy, and an open graph database on a reachable
+port is a bad surprise. For a throwaway local trial on a trusted network you
+can opt out with `docker-compose-3x3.non-auth.yml`, which drops the admin
+password and the JWT secret from all three Servers. Anyone who can reach the
+published ports then has full read and write access to every graph, so do not
+use it anywhere else.
+
+```bash
+cd docker
+
+# The network and Hubble's volumes are external in every flow, so create them
+# once even here. This mode skips only the credential half of the setup block.
+docker network inspect hugegraph-net >/dev/null 2>&1 ||
+ docker network create hugegraph-net
+for vol in hugegraph-hubble-db hugegraph-hubble-upload-files; do
+ docker volume inspect "${vol}" >/dev/null 2>&1 ||
+ docker volume create "${vol}" >/dev/null
+done
+
+# No docker/.env is needed. The two placeholders exist only because Compose
+# evaluates the base file's required-variable guards before applying the
+# override, which then drops both variables, so neither value reaches a
+# container. Needs Compose v2.24+.
+#
+# Keep them inline on every command; do not write them into docker/.env. They
+# are published values rather than secrets, and the token placeholder is
+# deliberately shorter than the 32 bytes the Server requires, so if it ever
+# reaches an authenticated Server that Server refuses to start instead of
+# signing tokens with a key anyone can read here.
+HUGEGRAPH_ADMIN_PASSWORD=non-auth-placeholder \
+HUGEGRAPH_AUTH_TOKEN_SECRET=non-auth-placeholder \
+ docker compose -f docker-compose-3pd-3store-3server.yml \
+ -f docker-compose-3x3.non-auth.yml up -d
+
+# Every graph request now succeeds without credentials: expect 200, not 401.
+curl -s -o /dev/null -w '%{http_code}\n' \
+ http://localhost:8080/graphs/hugegraph/schema/vertexlabels
+
+# Teardown. The base file's `:?` guards fire on every subcommand, `down`
+# included, so the same two placeholders are required here.
+HUGEGRAPH_ADMIN_PASSWORD=non-auth-placeholder \
+HUGEGRAPH_AUTH_TOKEN_SECRET=non-auth-placeholder \
+ docker compose -f docker-compose-3pd-3store-3server.yml \
+ -f docker-compose-3x3.non-auth.yml down -v
+```
+
+Hubble needs the matching configuration, because its own default is to demand
+a login. Point it at the non-auth properties file when you attach it:
+
+```bash
+HUBBLE_PROPERTIES=./hugegraph-hubble-3x3.non-auth.properties \
+ docker compose -p hugegraph-hubble -f docker-compose-hubble.yml up -d
+```
+
+`HUBBLE_PROPERTIES` is not remembered between commands. It selects a bind
+mount, so any later Compose command for Hubble that omits it silently
+recreates the container against the authenticated properties file, and Hubble
+comes back demanding a login the Servers cannot serve. Repeat the variable on
+every Hubble command in this mode, including `up -d --no-deps hubble` and the
+local-image commands below.
+
+<details>
+<summary>Prompt for an AI assistant</summary>
+
+Copy this to an assistant instead of transcribing the commands by hand.
+
+```markdown
+Start the Apache HugeGraph 3-node Docker cluster (3 PD, 3 Store, 3 Server)
+with Hubble, from the `docker/` directory of the hugegraph repository.
+
+Use the authenticated default unless I say otherwise:
+1. Follow "3-Node Cluster Quickstart" in docker/README.md to create
+ docker/.env with a generated admin password and JWT token secret, and to
+ create the external hugegraph-net network.
+2. Start the cluster with docker-compose-3pd-3store-3server.yml, then attach
+ Hubble with docker-compose-hubble.yml.
+3. Verify: an unauthenticated graph request returns 401, an authenticated one
+ returns 200, and Hubble answers on http://localhost:8088.
+4. Tell me the admin password so I can sign in to Hubble.
+
+Do not weaken authentication to work around an error. If a Server never
+becomes healthy, the image is probably older than the release this stack
+needs; report that instead of disabling the healthcheck. Only if I explicitly
+ask for an unauthenticated cluster, use the "Running without authentication"
+section instead.
+```
+
+</details>
+
+---
+
+## Hubble for the 3-Node Cluster
+
+`docker-compose-hubble.yml` defines only the Hubble service. It joins the
+cluster's external network (`hugegraph-net` by default, override with
+`HUGEGRAPH_NETWORK`) and has no `depends_on` on cluster services, so
+starting, stopping, or upgrading Hubble never recreates or restarts PD,
+Store, or Server containers. Hubble reads the cluster topology from
+`hugegraph-hubble-3x3.properties`; adjust that file when attaching to a
+cluster with different hostnames. Its `pd.server` is a single PD address
+with no failover, so if that PD is down the operations view goes blind even
+though the cluster still has quorum — repoint it at a surviving PD. The
+operations view also reports one `SERVER` node, not three: it describes the
+replica Hubble is currently talking to, not the whole Server tier. The add-on
+stores Hubble's H2 database and uploaded files in named volumes
+`hugegraph-hubble-db` (mounted at `/hubble/db`) and
+`hugegraph-hubble-upload-files`. Those names are explicit, so the attach
+and combined Compose projects share the same physical volumes. Hubble's
+default H2 URL would write `/hubble/db.mv.db` (outside that mount); the
+add-on sets `SPRING_DATASOURCE_URL` so the database file lives inside
+`/hubble/db`. Recreating the container does not discard that state.
+
+Sign in at `http://localhost:8088` as `admin` with the
+`HUGEGRAPH_ADMIN_PASSWORD` from `docker/.env`. Hubble binds to host
+loopback by default (`HUBBLE_PUBLISH_HOST`, same caveats as the
+single-node setup).
+
+Which flow to use: pick the attach flow when you do not have the cluster's
+`docker/.env` — the add-on carries no `:?` guards, so it is the only flow
+that runs without those credentials, which is what you want against a
+cluster someone else started. Otherwise use the combined flow, including to
+add Hubble to an already-running cluster (`up -d --no-deps hubble`).
+
+The two flows below create Hubble in different Compose projects, so manage
+Hubble with the same flags you started it with: the attach flow always uses
+`-p hugegraph-hubble -f docker-compose-hubble.yml`, the combined flow always
+uses both `-f` flags. The explicit `-p` keeps the attach project independent
+of the directory name and of other Compose projects.
+
+> [!IMPORTANT]
+> Do not drop the `-p` from the attach flow. Without it Compose names the
+> project after the current directory (`docker`), so Hubble starts in a
+> third project that none of the commands below manage: `down` reports
+> nothing to remove while `hg-hubble` keeps running, and every later `up`
+> in either flow fails on the container name. If that happens, find it with
+> `docker ps --filter name=hg-hubble` and remove it with
+> `docker rm -f hg-hubble`.
+
+Run one Hubble per host: the single-node stack and both add-on flows all
+publish `127.0.0.1:8088` and name their container `hg-hubble`. The two
+add-on flows are therefore mutually exclusive — starting one while the
+other's Hubble exists fails with a container-name conflict, so remove the
+Hubble of the flow you are leaving before switching. The two directions are
+not symmetric:
+
+```bash
+cd docker
+# leaving the attach flow (removes only Hubble)
+docker compose -p hugegraph-hubble -f docker-compose-hubble.yml down
+
+# leaving the combined flow — remove ONLY Hubble; a plain `down` with both
+# -f flags would stop all ten containers just to move one.
+docker compose -f docker-compose-3pd-3store-3server.yml \
+ -f docker-compose-hubble.yml rm -sf hubble
+```
+
+### Attach to a running cluster
+
+With the 3-node cluster already up. Hubble's two volumes are `external`, so
+Compose refuses to start until they exist. Whoever ran the quickstart setup
+block already created them; create them here too when attaching to a cluster
+someone else started, which is the case this flow exists for. Both commands
+are safe to repeat.
+
+```bash
+cd docker
+
+# Match HUBBLE_DB_VOLUME and HUBBLE_UPLOAD_VOLUME if the cluster owner set
+# them to something other than these defaults.
+for vol in "${HUBBLE_DB_VOLUME:-hugegraph-hubble-db}" \
+ "${HUBBLE_UPLOAD_VOLUME:-hugegraph-hubble-upload-files}"; do
+ docker volume inspect "${vol}" >/dev/null 2>&1 ||
+ docker volume create "${vol}" >/dev/null
+done
+
+docker compose -p hugegraph-hubble -f docker-compose-hubble.yml up -d
+```
+
+Lifecycle commands in this flow operate on Hubble alone and leave the
+cluster and the external network in place:
+
+```bash
+cd docker
+docker compose -p hugegraph-hubble -f docker-compose-hubble.yml ps
+docker compose -p hugegraph-hubble -f docker-compose-hubble.yml down
+```
+
+To remove everything in this flow, take down Hubble first, then the cluster.
+The cluster file's `:?` guards fire on `down` too, so this needs `docker/.env`
+or the same two variables in the environment. If you attached to a cluster
+someone else started and never had those credentials, remove only Hubble with
+the command above and leave the cluster to whoever owns it.
+
+```bash
+cd docker
+docker compose -p hugegraph-hubble -f docker-compose-hubble.yml down
+docker compose -f docker-compose-3pd-3store-3server.yml down -v
+
+# Hubble's volumes are external, so no `down` removes them. Drop them
+# explicitly when you want the H2 database and uploaded files gone, or a
+# later deployment reattaches the old state.
+docker volume rm hugegraph-hubble-db hugegraph-hubble-upload-files
Review Comment:
🧹 This teardown only removes the default volume names. Evidence: the add-on supports HUBBLE_DB_VOLUME and HUBBLE_UPLOAD_VOLUME, but this command hard-codes the defaults, so a customized deployment retains its H2 and upload data despite the instruction to drop them. Use the same variables with defaults or say custom volumes must be removed separately.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(docker): add attachable Hubble Compose add-on for the 3-node cluster [hugegraph]
Posted by "bitflicker64 (via GitHub)" <gi...@apache.org>.
bitflicker64 commented on code in PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149#discussion_r3839619911
##########
docker/docker-compose-3pd-3store-3server.yml:
##########
@@ -31,6 +36,8 @@ volumes:
# ── Shared service defaults ──────────────────────────────────────────
x-pd-common: &pd-common
+ # Pin a release via HUGEGRAPH_VERSION in docker/.env; unset, the image
Review Comment:
Fixed in c3115579, and you were right that this belonged here rather than in a follow-up.
Both halves of your suggestion are in: all four images now use `pull_policy: always`, so a stale cached `latest` is refreshed, and Server readiness no longer probes public `/versions`. It requires an unauthenticated graph request to return 401 and an authenticated one to return 200. An image that ignores `PASSWORD` never reports healthy, so `up -d --wait` fails instead of handing back an open cluster.
Verified on a clean Docker host: 9/9 healthy against `latest` (1.7.0), 401/200 on all three replicas, wrong password 401.
##########
docker/docker-compose-3pd-3store-3server.yml:
##########
@@ -17,9 +17,14 @@
name: hugegraph-3x3
+# The cluster network is shared with the Hubble add-on
+# (docker-compose-hubble.yml), so it is external and must exist first:
+# docker network create hugegraph-net
+# Set HUGEGRAPH_NETWORK to use a differently named network.
networks:
hg-net:
- driver: bridge
+ external: true
Review Comment:
Fixed in c3115579.
PD, Store, and Server host ports now bind to `127.0.0.1` by default, so the unauthenticated control plane is not reachable from outside the host. `HUGEGRAPH_CONTROL_PLANE_HOST` and `HUGEGRAPH_SERVER_PUBLISH_HOST` widen them, and the README says to do that only behind a network ACL or TLS.
I had asked to defer this as a quickstart redesign. It was not one: the services already talk over `hugegraph-net`, so nothing needed those host bindings.
Verified live: `ss -ltn` shows `127.0.0.1:8080`, `127.0.0.1:8500`, `127.0.0.1:8620`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(docker): add attachable Hubble Compose add-on for the 3-node cluster [hugegraph]
Posted by "bitflicker64 (via GitHub)" <gi...@apache.org>.
bitflicker64 commented on PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149#issuecomment-5431096561
Pushed `d2a12bc1`. Commit-by-commit, since this branch has grown a lot since the last review.
**`0d1e67ba`: back to authenticated by default.** The two commits before it had made the 3-node cluster anonymous. Reverted on top rather than rewritten, so every thread here keeps its anchor. Anonymous is still supported, as an explicit opt-in rather than the default.
**`c3115579`: the four blocking findings.** Images pull by default and readiness proves the running image enforces auth, so a cached older image cannot look healthy while serving the graph APIs open. PD, Store and Server ports bind to loopback, widened only through `HUGEGRAPH_CONTROL_PLANE_HOST` / `HUGEGRAPH_SERVER_PUBLISH_HOST`. And a live smoke job replaces the render-only checks.
**`10c6c6b5`: corrections the smoke job found in itself.** Running it against a real cluster showed two of its own assertions were wrong: `/v1/cluster` returns PD peers, not graph servers, so the registration check could never pass; and the persistence check passed even when the two flows used different volumes. Now it asserts three PD peers and three Stores `Up`, and proves shared state by writing a marker through one flow and reading it back through the other.
**`7ea0efe2`: the port reference contradicted the code.** It still said cluster ports bind every interface after they had moved to loopback. Also, the documented no-auth placeholder token was long enough to pass validation, so copying those two lines into `docker/.env` would produce a cluster that looked authenticated while signing tokens with a key printed in this repo. The placeholder is now deliberately too short for the Server to accept.
**`6b6c3a3c`: the no-auth override only disabled auth on one Server.** The three services shared a YAML anchor, and `!reset` does not survive an alias on every Compose version: on 5.1.2 only `server0` lost its credentials, so the cluster came up half authenticated, while 5.1.4 reset all three. The healthcheck override propagated either way, which is what hid it. Spelled out per service.
**`96160375`: the seams around the contract.** Chiefly: `HG_SERVER_REST_URL` becomes `restserver.url`, which is the *bind* address, so the advertised-address option crash-looped a replica on `java.net.BindException`; those variables are gone and the limitation is documented. Readiness no longer pins to the seeded admin password, which would have bricked every replica after a password rotation. Hubble's volumes became `external`, because Compose deletes a fixed-name volume on `down -v` from any project declaring the name, so leaving one flow destroyed the other's database. Plus the offline pull escape hatch, the no-auth flow's missing network and teardown, a dotenv reader that returned an empty password for malformed values, and a verification block whose `exit 1` closed the operator's shell.
**`05715714`: a claim that covered files it did not.** The version-pinning paragraph promised an image ignoring `PASSWORD` never reports healthy across three Compose files. Only the cluster has that check; the single-node files probe `/versions`, which stays open either way.
**`d2a12bc1`: your latest finding.** Making the volumes external fixed data loss but broke the flow that needs them most: `external volume "hugegraph-hubble-upload-files" not found`. The attach section now creates both first, honouring `HUBBLE_DB_VOLUME` and `HUBBLE_UPLOAD_VOLUME`, inspect-then-create so it is safe to repeat. Reproduced the failure and confirmed the fix against a running cluster.
Everything above was verified by running it, not by reading. On the current head: ten containers healthy, 401 unauthenticated and 200 authenticated on all three replicas, wrong password 401, three PD peers and three Stores `Up`, the H2 database inside its mount, Hubble attaching without recreating a cluster container, and shared state surviving the teardown that used to destroy it.
One note on CI: `docker-build (Dockerfile-hstore)` is red, and it is a Maven build inside that Dockerfile. This branch changes no Java, no pom and no Dockerfile, and the same job passes on current master, so I believe it is unrelated to this diff. This push re-runs it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(docker): add attachable Hubble Compose add-on for the 3-node cluster [hugegraph]
Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on code in PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149#discussion_r3855668715
##########
docker/README.md:
##########
@@ -196,11 +359,307 @@ curl http://localhost:8520/v1/health
# Check Server (Graph API)
curl http://localhost:8080/versions
-# List registered stores via PD
-curl http://localhost:8620/v1/stores
+# Every PD endpoint except /v1/health, /actuator/* and /v1/prom/targets/*
+# needs an Authorization header. PD only checks that the Basic-auth user is
+# one of its internal service names, so the empty password below is enough
+# — and it grants the full PD control plane, writes included, not just these
+# reads. That is exactly why the shared network must be treated as a trust
+# boundary. Without the header PD answers with an exception body, not data.
+pd_auth="Authorization: Basic $(printf 'hubble:' | base64)"
+
+# List registered stores via PD (expect three, each "state":"Up")
+curl -H "${pd_auth}" http://localhost:8620/v1/stores
# List partitions
-curl http://localhost:8620/v1/partitions
+curl -H "${pd_auth}" http://localhost:8620/v1/partitions
+```
+
+Confirm authentication actually engaged — `/versions` stays open by design,
+so it cannot tell you whether auth is on. A graph read without credentials
+must be rejected:
+
+```bash
+cd docker
+# Expect 401 on all three replicas
+for port in 8080 8081 8082; do
+ curl -s -o /dev/null -w "${port}: %{http_code}\n" \
+ "http://localhost:${port}/graphs/hugegraph/schema/vertexlabels"
+done
+
+# And a signed-in read must succeed. Parse the generated single-quoted
+# value without executing docker/.env as shell. Passing the credential
+# through --config keeps it out of argv, where `ps` would expose it.
+# The subshell keeps the `exit 1` below from closing an interactive shell.
+(
+set -eu
+read_dotenv_value() {
+ key="$1"
+ key_pattern="^[[:space:]]*(export[[:space:]]+)?${key}="
+ count="$(grep -Ec "${key_pattern}" .env || true)"
+ [ "${count}" -eq 1 ] || return 1
+ value="$(sed -nE "s/${key_pattern}'([^']*)'[[:space:]]*$/\\2/p" .env)"
+ # sed prints nothing and still exits 0 when the value is unquoted or
+ # double-quoted, which would otherwise hand back an empty credential.
+ [ -n "${value}" ] || return 1
+ printf '%s\n' "${value}"
+}
+HUGEGRAPH_ADMIN_PASSWORD="$(read_dotenv_value HUGEGRAPH_ADMIN_PASSWORD)" || {
+ echo "HUGEGRAPH_ADMIN_PASSWORD must use the documented single-quoted format" >&2
+ exit 1
+}
+curl -s -o /dev/null -w '%{http_code}\n' \
+ --config <(printf 'user = "admin:%s"\n' "${HUGEGRAPH_ADMIN_PASSWORD}") \
+ http://localhost:8080/graphs/hugegraph/schema/vertexlabels
+)
+```
+
+`200` from the second command with `401` from all three of the first means
+authentication is on and working. If the first command returns `200`, the
+running image ignored `PASSWORD` and the cluster is **unauthenticated** —
+the most likely cause is an image older than the release this integration
+needs (see the version note in the quickstart above).
+
+### Running without authentication
+
+The cluster above authenticates, and that is the right default: it is the
+configuration most people copy, and an open graph database on a reachable
+port is a bad surprise. For a throwaway local trial on a trusted network you
+can opt out with `docker-compose-3x3.non-auth.yml`, which drops the admin
+password and the JWT secret from all three Servers. Anyone who can reach the
+published ports then has full read and write access to every graph, so do not
+use it anywhere else.
+
+```bash
+cd docker
+
+# The network and Hubble's volumes are external in every flow, so create them
+# once even here. This mode skips only the credential half of the setup block.
+docker network inspect hugegraph-net >/dev/null 2>&1 ||
+ docker network create hugegraph-net
+for vol in hugegraph-hubble-db hugegraph-hubble-upload-files; do
+ docker volume inspect "${vol}" >/dev/null 2>&1 ||
+ docker volume create "${vol}" >/dev/null
+done
+
+# No docker/.env is needed. The two placeholders exist only because Compose
+# evaluates the base file's required-variable guards before applying the
+# override, which then drops both variables, so neither value reaches a
+# container. Needs Compose v2.24+.
+#
+# Keep them inline on every command; do not write them into docker/.env. They
+# are published values rather than secrets, and the token placeholder is
+# deliberately shorter than the 32 bytes the Server requires, so if it ever
+# reaches an authenticated Server that Server refuses to start instead of
+# signing tokens with a key anyone can read here.
+HUGEGRAPH_ADMIN_PASSWORD=non-auth-placeholder \
+HUGEGRAPH_AUTH_TOKEN_SECRET=non-auth-placeholder \
+ docker compose -f docker-compose-3pd-3store-3server.yml \
+ -f docker-compose-3x3.non-auth.yml up -d
+
+# Every graph request now succeeds without credentials: expect 200, not 401.
+curl -s -o /dev/null -w '%{http_code}\n' \
+ http://localhost:8080/graphs/hugegraph/schema/vertexlabels
+
+# Teardown. The base file's `:?` guards fire on every subcommand, `down`
+# included, so the same two placeholders are required here.
+HUGEGRAPH_ADMIN_PASSWORD=non-auth-placeholder \
+HUGEGRAPH_AUTH_TOKEN_SECRET=non-auth-placeholder \
+ docker compose -f docker-compose-3pd-3store-3server.yml \
+ -f docker-compose-3x3.non-auth.yml down -v
+```
+
+Hubble needs the matching configuration, because its own default is to demand
+a login. Point it at the non-auth properties file when you attach it:
+
+```bash
+HUBBLE_PROPERTIES=./hugegraph-hubble-3x3.non-auth.properties \
+ docker compose -p hugegraph-hubble -f docker-compose-hubble.yml up -d
+```
+
+`HUBBLE_PROPERTIES` is not remembered between commands. It selects a bind
+mount, so any later Compose command for Hubble that omits it silently
+recreates the container against the authenticated properties file, and Hubble
+comes back demanding a login the Servers cannot serve. Repeat the variable on
+every Hubble command in this mode, including `up -d --no-deps hubble` and the
+local-image commands below.
+
+<details>
+<summary>Prompt for an AI assistant</summary>
+
+Copy this to an assistant instead of transcribing the commands by hand.
+
+```markdown
+Start the Apache HugeGraph 3-node Docker cluster (3 PD, 3 Store, 3 Server)
+with Hubble, from the `docker/` directory of the hugegraph repository.
+
+Use the authenticated default unless I say otherwise:
+1. Follow "3-Node Cluster Quickstart" in docker/README.md to create
+ docker/.env with a generated admin password and JWT token secret, and to
+ create the external hugegraph-net network.
+2. Start the cluster with docker-compose-3pd-3store-3server.yml, then attach
+ Hubble with docker-compose-hubble.yml.
+3. Verify: an unauthenticated graph request returns 401, an authenticated one
+ returns 200, and Hubble answers on http://localhost:8088.
+4. Tell me the admin password so I can sign in to Hubble.
+
+Do not weaken authentication to work around an error. If a Server never
+becomes healthy, the image is probably older than the release this stack
+needs; report that instead of disabling the healthcheck. Only if I explicitly
+ask for an unauthenticated cluster, use the "Running without authentication"
+section instead.
+```
+
+</details>
+
+---
+
+## Hubble for the 3-Node Cluster
+
+`docker-compose-hubble.yml` defines only the Hubble service. It joins the
+cluster's external network (`hugegraph-net` by default, override with
+`HUGEGRAPH_NETWORK`) and has no `depends_on` on cluster services, so
+starting, stopping, or upgrading Hubble never recreates or restarts PD,
+Store, or Server containers. Hubble reads the cluster topology from
+`hugegraph-hubble-3x3.properties`; adjust that file when attaching to a
+cluster with different hostnames. Its `pd.server` is a single PD address
+with no failover, so if that PD is down the operations view goes blind even
+though the cluster still has quorum — repoint it at a surviving PD. The
+operations view also reports one `SERVER` node, not three: it describes the
+replica Hubble is currently talking to, not the whole Server tier. The add-on
+stores Hubble's H2 database and uploaded files in named volumes
+`hugegraph-hubble-db` (mounted at `/hubble/db`) and
+`hugegraph-hubble-upload-files`. Those names are explicit, so the attach
+and combined Compose projects share the same physical volumes. Hubble's
+default H2 URL would write `/hubble/db.mv.db` (outside that mount); the
+add-on sets `SPRING_DATASOURCE_URL` so the database file lives inside
+`/hubble/db`. Recreating the container does not discard that state.
+
+Sign in at `http://localhost:8088` as `admin` with the
+`HUGEGRAPH_ADMIN_PASSWORD` from `docker/.env`. Hubble binds to host
+loopback by default (`HUBBLE_PUBLISH_HOST`, same caveats as the
+single-node setup).
+
+Which flow to use: pick the attach flow when you do not have the cluster's
+`docker/.env` — the add-on carries no `:?` guards, so it is the only flow
+that runs without those credentials, which is what you want against a
+cluster someone else started. Otherwise use the combined flow, including to
+add Hubble to an already-running cluster (`up -d --no-deps hubble`).
+
+The two flows below create Hubble in different Compose projects, so manage
+Hubble with the same flags you started it with: the attach flow always uses
+`-p hugegraph-hubble -f docker-compose-hubble.yml`, the combined flow always
+uses both `-f` flags. The explicit `-p` keeps the attach project independent
+of the directory name and of other Compose projects.
+
+> [!IMPORTANT]
+> Do not drop the `-p` from the attach flow. Without it Compose names the
+> project after the current directory (`docker`), so Hubble starts in a
+> third project that none of the commands below manage: `down` reports
+> nothing to remove while `hg-hubble` keeps running, and every later `up`
+> in either flow fails on the container name. If that happens, find it with
+> `docker ps --filter name=hg-hubble` and remove it with
+> `docker rm -f hg-hubble`.
+
+Run one Hubble per host: the single-node stack and both add-on flows all
+publish `127.0.0.1:8088` and name their container `hg-hubble`. The two
+add-on flows are therefore mutually exclusive — starting one while the
+other's Hubble exists fails with a container-name conflict, so remove the
+Hubble of the flow you are leaving before switching. The two directions are
+not symmetric:
+
+```bash
+cd docker
+# leaving the attach flow (removes only Hubble)
+docker compose -p hugegraph-hubble -f docker-compose-hubble.yml down
+
+# leaving the combined flow — remove ONLY Hubble; a plain `down` with both
+# -f flags would stop all ten containers just to move one.
+docker compose -f docker-compose-3pd-3store-3server.yml \
+ -f docker-compose-hubble.yml rm -sf hubble
+```
+
+### Attach to a running cluster
+
+With the 3-node cluster already up:
+
+```bash
+cd docker
+docker compose -p hugegraph-hubble -f docker-compose-hubble.yml up -d
Review Comment:
⚠️ The documented attach path fails when the existing cluster was started by someone else and the Hubble volumes do not exist. Evidence: `docker-compose-hubble.yml:34-39` declares `hg-hubble-db` and `hg-hubble-upload-files` as `external: true`; Compose refuses this `up` until both named volumes already exist, but this section only assumes the network and gives no volume-creation step. Add a non-destructive `docker volume inspect || docker volume create` block before this command, or state that the cluster owner must pre-create both volumes, including custom `HUBBLE_DB_VOLUME` and `HUBBLE_UPLOAD_VOLUME` values.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(docker): add attachable Hubble Compose add-on for the 3-node cluster [hugegraph]
Posted by "bitflicker64 (via GitHub)" <gi...@apache.org>.
bitflicker64 commented on code in PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149#discussion_r3789894535
##########
docker/README.md:
##########
@@ -196,11 +296,184 @@ curl http://localhost:8520/v1/health
# Check Server (Graph API)
curl http://localhost:8080/versions
-# List registered stores via PD
-curl http://localhost:8620/v1/stores
+# Every PD endpoint except /v1/health, /actuator/* and /v1/prom/targets/*
+# needs an Authorization header. PD only checks that the Basic-auth user is
+# one of its internal service names, so the empty password below is enough
+# — and it grants the full PD control plane, writes included, not just these
+# reads. That is exactly why the shared network must be treated as a trust
+# boundary. Without the header PD answers with an exception body, not data.
+pd_auth="Authorization: Basic $(printf 'hubble:' | base64)"
+
+# List registered stores via PD (expect three, each "state":"Up")
+curl -H "${pd_auth}" http://localhost:8620/v1/stores
# List partitions
-curl http://localhost:8620/v1/partitions
+curl -H "${pd_auth}" http://localhost:8620/v1/partitions
+```
+
+Confirm authentication actually engaged — `/versions` stays open by design,
+so it cannot tell you whether auth is on. A graph read without credentials
+must be rejected:
+
+```bash
+cd docker
+# Expect 401 on all three replicas
+for port in 8080 8081 8082; do
+ curl -s -o /dev/null -w "${port}: %{http_code}\n" \
+ "http://localhost:${port}/graphs/hugegraph/schema/vertexlabels"
+done
+
+# And a signed-in read must succeed. Compose reads docker/.env by itself, but
+# your shell does not — load it first. Passing the credential through
+# --config keeps it out of argv, where `ps` would expose it to other users.
+set -a; . ./.env; set +a
Review Comment:
I’ll replace `set -a; . ./.env; set +a` with a parse‑only read of the required key, so the verify block does not execute dotenv as a shell.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] chore(docker): refactor docker-compose topologies with Hubble [hugegraph]
Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin merged PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(docker): add attachable Hubble Compose add-on for the 3-node cluster [hugegraph]
Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on code in PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149#discussion_r3770576251
##########
docker/README.md:
##########
@@ -168,14 +173,109 @@ To validate local images without Compose replacing them with remote `latest`:
## 3-Node Cluster Quickstart
+The cluster and the Hubble add-on share one named Docker network so Hubble
+can attach to a running cluster without touching it. Treat that network as a
+trust boundary: only the Server layer performs real authentication. Store
+serves its control APIs with no credentials at all, and while PD's REST
+control APIs do require an `Authorization` header, PD only checks that the
+Basic-auth *user* is one of its internal service names and never validates
+the password — so any client on the network can read and drive them. Note
+that this stack depends on that behaviour: Hubble reaches PD as the service
+name `hubble` with an empty password, so tightening PD's credential check
+would also break Hubble's operations view. Any container on the host can
+also join the network by declaring the well-known name. One-time setup:
+write the required credentials to a mode-600 `docker/.env` and create the
+network.
+The cluster file requires both credentials — the admin password enables
+authentication, and every Server replica must share one token secret so a
+token issued by any server validates on all of them. The `:?` guards fire
+on every Compose subcommand, including `down`.
+
+```bash
+(
+ set -eu
+ cd docker
+ command -v openssl >/dev/null 2>&1 || { echo "openssl not found" >&2; exit 1; }
+ [ -e .env ] || install -m 600 /dev/null .env
+ chmod 600 .env
+ # Keep appends on their own lines even if the file was hand-edited.
+ [ ! -s .env ] || [ -z "$(tail -c1 .env)" ] || printf '\n' >> .env
+ pat='^[[:space:]]*(export[[:space:]]+)?'
+ if ! grep -Eq "${pat}HUGEGRAPH_ADMIN_PASSWORD=" .env; then
+ admin_password="$(openssl rand -base64 12)"
+ printf "HUGEGRAPH_ADMIN_PASSWORD='%s'\n" "${admin_password}" >> .env
+ unset admin_password
+ fi
+ if ! grep -Eq "${pat}HUGEGRAPH_AUTH_TOKEN_SECRET=" .env; then
+ token_secret="$(openssl rand -hex 32)"
+ printf "HUGEGRAPH_AUTH_TOKEN_SECRET='%s'\n" "${token_secret}" >> .env
+ unset token_secret
+ fi
+ admin_value="$(sed -nE "s/${pat}HUGEGRAPH_ADMIN_PASSWORD='([^']*)'[[:space:]]*$/\\1/p" .env | tail -n1)"
Review Comment:
‼️ The cluster quickstart exits immediately after generating its own credentials. Evidence: the regex has an optional `export` capture before the value, but both sed commands substitute capture group 1, so `HUGEGRAPH_ADMIN_PASSWORD='...'` yields an empty value (and an `export` form yields the word `export`); lines 216/220 then fail. Use the value capture group and parse or explicitly reject all supported Compose dotenv formats consistently.
##########
docker/docker-compose-3pd-3store-3server.yml:
##########
@@ -187,16 +207,25 @@ services:
<<: *server-common
container_name: hg-server0
hostname: server0
+ environment:
+ <<: *server-env
+ HG_SERVER_REST_URL: http://server0:8080
Review Comment:
⚠️ Each Server registers `http://serverN:8080` with PD while the host publishes ports 8080/8081/8082. Evidence: Server registration uses `restserver.url`, so a PD-aware client outside the Docker network receives `server0`/`server1`/`server2`, names that only resolve inside `hugegraph-net`. Provide a separate configurable advertised address or make the external client path use addresses it can resolve.
##########
docker/docker-compose-3pd-3store-3server.yml:
##########
@@ -31,6 +36,8 @@ volumes:
# ── Shared service defaults ──────────────────────────────────────────
x-pd-common: &pd-common
+ # Pin a release via HUGEGRAPH_VERSION in docker/.env; unset, the image
Review Comment:
‼️ This auth-required cluster can report healthy while remaining unauthenticated. Evidence: the documented compatible integration is newer than 1.7.x, but the image defaults to floating `latest` with `pull_policy: missing`; an already-cached older image can ignore `PASSWORD` and the token secret, while the healthcheck probes public `/versions`. Pin or fail closed on incompatible images, or add an authenticated readiness smoke check that proves graph requests are rejected without credentials and succeed with them.
##########
docker/docker-compose-hubble.yml:
##########
@@ -0,0 +1,53 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+# Hubble add-on for the distributed cluster defined in
+# docker-compose-3pd-3store-3server.yml. Requires the pre-created cluster
+# network (docker network create hugegraph-net). See "Hubble for the
+# 3-Node Cluster" in docker/README.md for the attach and combined flows.
+
+networks:
+ hg-net:
+ external: true
+ name: ${HUGEGRAPH_NETWORK:-hugegraph-net}
+
+volumes:
+ hg-hubble-db:
+ hg-hubble-upload-files:
+
+services:
+ hubble:
+ # Pin a release via HUGEGRAPH_VERSION in docker/.env; unset, the image
+ # tag defaults to latest.
+ image: ${HUBBLE_IMAGE:-hugegraph/hubble:${HUGEGRAPH_VERSION:-latest}}
+ pull_policy: ${HUBBLE_PULL_POLICY:-missing}
+ container_name: hg-hubble
+ hostname: hubble
+ restart: unless-stopped
+ networks: [hg-net]
+ ports:
+ - "${HUBBLE_PUBLISH_HOST:-127.0.0.1}:8088:8088"
+ volumes:
+ - ./hugegraph-hubble-3x3.properties:/hubble/conf/hugegraph-hubble.properties:ro
+ - hg-hubble-db:/hubble/db
Review Comment:
‼️ The H2 database is mounted at the wrong persistence boundary. Evidence: the Hubble image works from `/hubble` and its datasource is `jdbc:h2:file:./db`; mounting a volume at `/hubble/db` leaves the actual `/hubble/db.mv.db` outside that volume (or makes startup fail because `/hubble/db` is a directory). Point H2 at a file inside the mounted directory or mount the parent directory, then verify data survives container recreation.
##########
.github/workflows/server-ci.yml:
##########
@@ -138,6 +140,189 @@ jobs:
check_compose docker/docker-compose.yml always always
check_compose docker/docker-compose.dev.yml build missing
+ check_cluster_compose() {
+ local cluster="docker/docker-compose-3pd-3store-3server.yml"
+ local addon="docker/docker-compose-hubble.yml"
+ local rendered
+ local token_fixture=ci-test-token-secret-32-bytes-long
+ rendered="$(mktemp)"
+ if [ "${#token_fixture}" -lt 32 ]; then
+ echo "CI token fixture must be at least 32 bytes" >&2
+ return 1
+ fi
+
+ # RETURN only: an EXIT trap would fire after this function's
+ # `local rendered` has gone out of scope, which `set -u` turns
+ # into an "unbound variable" error. A hard errexit abort can
+ # therefore still leak one temp file, which is acceptable on an
+ # ephemeral runner.
+ trap 'rm -f "$rendered"' RETURN
+
+ # --env-file /dev/null on every invocation: Compose otherwise reads
+ # docker/.env automatically, and the quickstart tells operators to
+ # create one holding exactly the credentials these assertions
+ # control. Without it the checks pass in CI (which has no .env) but
+ # report false failures for anyone running them locally after
+ # following the quickstart, and the add-on render below — which
+ # deliberately unsets the variables to assert their defaults — would
+ # read whatever HUGEGRAPH_NETWORK/HUGEGRAPH_VERSION that .env holds.
+ # Pinning an empty env file makes both renders depend only on what
+ # each invocation sets explicitly.
+
+ # Both cluster credentials are required and may not be empty.
+ # Each case asserts the guard fired for the *intended* variable:
+ # a bare non-zero exit would also be produced by a YAML error, a
+ # renamed file, or a missing docker binary.
+ assert_guard() { # assert_guard <blamed-var> <description>
+ local var="$1" desc="$2" err
+ if err="$(docker compose --env-file /dev/null -f "$cluster" \
+ config -q 2>&1)"; then
+ echo "$cluster accepted $desc" >&2
+ return 1
+ fi
+ case "$err" in
+ *"$var"*) : ;;
+ *) echo "$cluster rejected $desc, but not because of $var: $err" >&2
+ return 1 ;;
+ esac
+ }
+
+ ( unset HUGEGRAPH_ADMIN_PASSWORD
+ export HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}"
+ assert_guard HUGEGRAPH_ADMIN_PASSWORD "an unset admin password" )
+ ( export HUGEGRAPH_ADMIN_PASSWORD=
+ export HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}"
+ assert_guard HUGEGRAPH_ADMIN_PASSWORD "an empty admin password" )
+ ( unset HUGEGRAPH_AUTH_TOKEN_SECRET
+ export HUGEGRAPH_ADMIN_PASSWORD=ci-test-password
+ assert_guard HUGEGRAPH_AUTH_TOKEN_SECRET "an unset token secret" )
+ ( export HUGEGRAPH_ADMIN_PASSWORD=ci-test-password
+ export HUGEGRAPH_AUTH_TOKEN_SECRET=
+ assert_guard HUGEGRAPH_AUTH_TOKEN_SECRET "an empty token secret" )
+ # The add-on alone must define Hubble and nothing else, join the
+ # shared external network with its default name, and need no
+ # credentials or overrides.
+ env -u HUGEGRAPH_ADMIN_PASSWORD -u HUGEGRAPH_AUTH_TOKEN_SECRET \
+ -u HUGEGRAPH_NETWORK -u HUGEGRAPH_VERSION \
+ -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY -u HUBBLE_PUBLISH_HOST \
+ docker compose --env-file /dev/null -f "$addon" config --format json > "$rendered"
+ jq -e '
+ (.services | keys) == ["hubble"] and
+ .networks."hg-net".external == true and
+ .networks."hg-net".name == "hugegraph-net" and
+ (.services.hubble.networks | has("hg-net")) and
+ (.services.hubble | has("depends_on") | not) and
+ any(.services.hubble.volumes[];
+ .target == "/hubble/conf/hugegraph-hubble.properties" and
+ (.source | endswith("hugegraph-hubble-3x3.properties")))
+ and any(.services.hubble.volumes[];
+ .source == "hg-hubble-db" and
+ .target == "/hubble/db")
+ and any(.services.hubble.volumes[];
+ .source == "hg-hubble-upload-files" and
+ .target == "/hubble/upload-files")
+ ' "$rendered" >/dev/null
+
+ # The cluster's own default network name must match the add-on's,
+ # or the attach flow and the cluster land on different networks.
+ # The combined render below pins an override, so it cannot catch a
+ # drifting default.
+ HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}" \
+ env -u HUGEGRAPH_NETWORK -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY \
+ -u HUBBLE_PUBLISH_HOST \
+ docker compose --env-file /dev/null -f "$cluster" \
+ config --format json > "$rendered"
+ jq -e '.networks."hg-net".name == "hugegraph-net"' \
+ "$rendered" >/dev/null
+
+ # The combined render carries the PD-registration and auth
+ # settings on every server replica and keeps Hubble on loopback.
+ # Rendered with non-default HUGEGRAPH_NETWORK/HUGEGRAPH_VERSION so
+ # CI fails if any file stops honoring the overrides (the add-on
+ # render above covers the defaults).
+ HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}" \
+ HUGEGRAPH_NETWORK=ci-test-net \
+ HUGEGRAPH_VERSION=ci-test-tag \
+ env -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY -u HUBBLE_PUBLISH_HOST \
+ docker compose --env-file /dev/null -f "$cluster" -f "$addon" \
Review Comment:
⚠️ The new cluster contract remains render-only. Evidence: the combined check stops at `docker compose config` plus jq/grep and never creates the external network, starts the services, probes Hubble `/about`, verifies PD registration, or checks authenticated graph access. Add a Docker-capable smoke job for the combined and attach flows so entrypoint, DNS, registration, and auth failures cannot remain green.
##########
.github/workflows/server-ci.yml:
##########
@@ -138,6 +140,189 @@ jobs:
check_compose docker/docker-compose.yml always always
check_compose docker/docker-compose.dev.yml build missing
+ check_cluster_compose() {
+ local cluster="docker/docker-compose-3pd-3store-3server.yml"
+ local addon="docker/docker-compose-hubble.yml"
+ local rendered
+ local token_fixture=ci-test-token-secret-32-bytes-long
+ rendered="$(mktemp)"
+ if [ "${#token_fixture}" -lt 32 ]; then
+ echo "CI token fixture must be at least 32 bytes" >&2
+ return 1
+ fi
+
+ # RETURN only: an EXIT trap would fire after this function's
+ # `local rendered` has gone out of scope, which `set -u` turns
+ # into an "unbound variable" error. A hard errexit abort can
+ # therefore still leak one temp file, which is acceptable on an
+ # ephemeral runner.
+ trap 'rm -f "$rendered"' RETURN
+
+ # --env-file /dev/null on every invocation: Compose otherwise reads
+ # docker/.env automatically, and the quickstart tells operators to
+ # create one holding exactly the credentials these assertions
+ # control. Without it the checks pass in CI (which has no .env) but
+ # report false failures for anyone running them locally after
+ # following the quickstart, and the add-on render below — which
+ # deliberately unsets the variables to assert their defaults — would
+ # read whatever HUGEGRAPH_NETWORK/HUGEGRAPH_VERSION that .env holds.
+ # Pinning an empty env file makes both renders depend only on what
+ # each invocation sets explicitly.
+
+ # Both cluster credentials are required and may not be empty.
+ # Each case asserts the guard fired for the *intended* variable:
+ # a bare non-zero exit would also be produced by a YAML error, a
+ # renamed file, or a missing docker binary.
+ assert_guard() { # assert_guard <blamed-var> <description>
+ local var="$1" desc="$2" err
+ if err="$(docker compose --env-file /dev/null -f "$cluster" \
+ config -q 2>&1)"; then
+ echo "$cluster accepted $desc" >&2
+ return 1
+ fi
+ case "$err" in
+ *"$var"*) : ;;
+ *) echo "$cluster rejected $desc, but not because of $var: $err" >&2
+ return 1 ;;
+ esac
+ }
+
+ ( unset HUGEGRAPH_ADMIN_PASSWORD
+ export HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}"
+ assert_guard HUGEGRAPH_ADMIN_PASSWORD "an unset admin password" )
+ ( export HUGEGRAPH_ADMIN_PASSWORD=
+ export HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}"
+ assert_guard HUGEGRAPH_ADMIN_PASSWORD "an empty admin password" )
+ ( unset HUGEGRAPH_AUTH_TOKEN_SECRET
+ export HUGEGRAPH_ADMIN_PASSWORD=ci-test-password
+ assert_guard HUGEGRAPH_AUTH_TOKEN_SECRET "an unset token secret" )
+ ( export HUGEGRAPH_ADMIN_PASSWORD=ci-test-password
+ export HUGEGRAPH_AUTH_TOKEN_SECRET=
+ assert_guard HUGEGRAPH_AUTH_TOKEN_SECRET "an empty token secret" )
+ # The add-on alone must define Hubble and nothing else, join the
+ # shared external network with its default name, and need no
+ # credentials or overrides.
+ env -u HUGEGRAPH_ADMIN_PASSWORD -u HUGEGRAPH_AUTH_TOKEN_SECRET \
+ -u HUGEGRAPH_NETWORK -u HUGEGRAPH_VERSION \
+ -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY -u HUBBLE_PUBLISH_HOST \
+ docker compose --env-file /dev/null -f "$addon" config --format json > "$rendered"
+ jq -e '
+ (.services | keys) == ["hubble"] and
+ .networks."hg-net".external == true and
+ .networks."hg-net".name == "hugegraph-net" and
+ (.services.hubble.networks | has("hg-net")) and
+ (.services.hubble | has("depends_on") | not) and
+ any(.services.hubble.volumes[];
+ .target == "/hubble/conf/hugegraph-hubble.properties" and
+ (.source | endswith("hugegraph-hubble-3x3.properties")))
+ and any(.services.hubble.volumes[];
+ .source == "hg-hubble-db" and
+ .target == "/hubble/db")
+ and any(.services.hubble.volumes[];
+ .source == "hg-hubble-upload-files" and
+ .target == "/hubble/upload-files")
+ ' "$rendered" >/dev/null
+
+ # The cluster's own default network name must match the add-on's,
+ # or the attach flow and the cluster land on different networks.
+ # The combined render below pins an override, so it cannot catch a
+ # drifting default.
+ HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}" \
+ env -u HUGEGRAPH_NETWORK -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY \
+ -u HUBBLE_PUBLISH_HOST \
+ docker compose --env-file /dev/null -f "$cluster" \
+ config --format json > "$rendered"
+ jq -e '.networks."hg-net".name == "hugegraph-net"' \
+ "$rendered" >/dev/null
+
+ # The combined render carries the PD-registration and auth
+ # settings on every server replica and keeps Hubble on loopback.
+ # Rendered with non-default HUGEGRAPH_NETWORK/HUGEGRAPH_VERSION so
+ # CI fails if any file stops honoring the overrides (the add-on
+ # render above covers the defaults).
+ HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}" \
+ HUGEGRAPH_NETWORK=ci-test-net \
+ HUGEGRAPH_VERSION=ci-test-tag \
+ env -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY -u HUBBLE_PUBLISH_HOST \
+ docker compose --env-file /dev/null -f "$cluster" -f "$addon" \
+ config --format json > "$rendered"
+ jq -e '
+ .name == "hugegraph-3x3" and
+ (.services | keys | length) == 10 and
+ .networks."hg-net".external == true and
+ .networks."hg-net".name == "ci-test-net" and
+ all(.services[]; .networks | has("hg-net")) and
+ .services.pd0.image == "hugegraph/pd:ci-test-tag" and
+ .services.store0.image == "hugegraph/store:ci-test-tag" and
+ .services.server0.image == "hugegraph/server:ci-test-tag" and
+ .services.hubble.image == "hugegraph/hubble:ci-test-tag" and
+ .services.server0.environment.HG_SERVER_USE_PD == "true" and
+ .services.server0.environment.HG_SERVER_CLUSTER == "hg" and
+ .services.server0.environment.HG_SERVER_REST_URL ==
+ "http://server0:8080" and
+ .services.server1.environment.HG_SERVER_REST_URL ==
+ "http://server1:8080" and
+ .services.server1.environment.HG_SERVER_AUTH_TOKEN_SECRET ==
+ "ci-test-token-secret-32-bytes-long" and
+ .services.server2.environment.HG_SERVER_REST_URL ==
+ "http://server2:8080" and
+ .services.server2.environment.HG_SERVER_AUTH_TOKEN_SECRET ==
+ "ci-test-token-secret-32-bytes-long" and
+ .services.server0.environment.HG_SERVER_INIT_STORE_ENABLED ==
+ "false" and
+ .services.server0.environment.PASSWORD ==
+ "ci-test-password" and
+ .services.server0.environment.HG_SERVER_AUTH_TOKEN_SECRET ==
+ "ci-test-token-secret-32-bytes-long" and
+ (.services.hubble | has("depends_on") | not) and
+ .services.hubble.pull_policy == "missing" and
+ (.services.hubble.healthcheck.test[1] |
+ contains("http://127.0.0.1:8088/about") and
+ contains("\"status\":200") and
+ contains("\"name\":\"hugegraph-hubble\"")) and
+ any(.services.hubble.ports[];
+ .target == 8088 and .published == "8088" and
+ .host_ip == "127.0.0.1")
+ ' "$rendered" >/dev/null
+
+ # Hubble's properties file is mounted, not rendered, so Compose
+ # validation alone cannot catch it drifting from the services it
+ # describes. Tie the two together: renaming a service, changing a
+ # container hostname, or moving a REST port must be reflected in
+ # both places or CI fails here. Values are derived from the
+ # rendered model (hostnames and ports included) so the assertions
+ # cannot silently agree with a stale file.
+ local props="docker/hugegraph-hubble-3x3.properties"
+ local pd_peers store_targets cluster_name pd_rest
+ assert_props() { # assert_props <exact-line> <what-it-must-match>
+ grep -Fqx "$1" "$props" ||
+ { echo "$props: expected line '$1' ($2)" >&2; return 1; }
+ }
+ cluster_name="$(jq -r '.services.server0.environment.HG_SERVER_CLUSTER' \
+ "$rendered")"
+ pd_peers="$(jq -r '.services.server0.environment.HG_SERVER_PD_PEERS' \
+ "$rendered")"
+ store_targets="[$(jq -r '[.services | to_entries[]
+ | select(.key | startswith("store"))
+ | "http://" + .value.hostname + ":"
+ + (.value.environment.HG_STORE_REST_PORT)]
+ | sort | join(",")' "$rendered")]"
+ pd_rest="$(jq -r --arg h "$(printf '%s' "${pd_peers}" | cut -d, -f1 | cut -d: -f1)" \
+ '.services | to_entries[]
+ | select(.value.hostname == $h)
+ | .value.hostname + ":" + .value.environment.HG_PD_REST_PORT' \
+ "$rendered")"
+ assert_props "cluster=${cluster_name}" "matches HG_SERVER_CLUSTER"
Review Comment:
⚠️ The Hubble properties consistency check can pass while PD mode is disabled. Evidence: `assert_props` validates `cluster`, `pd.peers`, `pd.server`, and Store targets, but never asserts `pd.enabled=true`; the current `server.direct_url` is only meaningful when PD mode is false. Add the explicit mode assertion and a runtime smoke check so Hubble cannot silently fall back to one direct Server.
##########
docker/docker-compose-hubble.yml:
##########
@@ -0,0 +1,53 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+# Hubble add-on for the distributed cluster defined in
+# docker-compose-3pd-3store-3server.yml. Requires the pre-created cluster
+# network (docker network create hugegraph-net). See "Hubble for the
+# 3-Node Cluster" in docker/README.md for the attach and combined flows.
+
+networks:
+ hg-net:
+ external: true
+ name: ${HUGEGRAPH_NETWORK:-hugegraph-net}
+
+volumes:
+ hg-hubble-db:
Review Comment:
⚠️ The two documented Hubble flows do not share these physical volumes. Evidence: attach runs under project `hugegraph-hubble`, while the combined flow runs under `hugegraph-3x3`; Compose therefore creates different project-prefixed H2 and upload volumes. Switching flows makes existing state and uploads appear to disappear. Give both volumes stable explicit names (with a multi-cluster override if needed) or document a supported volume migration.
##########
docker/README.md:
##########
@@ -196,11 +296,184 @@ curl http://localhost:8520/v1/health
# Check Server (Graph API)
curl http://localhost:8080/versions
-# List registered stores via PD
-curl http://localhost:8620/v1/stores
+# Every PD endpoint except /v1/health, /actuator/* and /v1/prom/targets/*
+# needs an Authorization header. PD only checks that the Basic-auth user is
+# one of its internal service names, so the empty password below is enough
+# — and it grants the full PD control plane, writes included, not just these
+# reads. That is exactly why the shared network must be treated as a trust
+# boundary. Without the header PD answers with an exception body, not data.
+pd_auth="Authorization: Basic $(printf 'hubble:' | base64)"
+
+# List registered stores via PD (expect three, each "state":"Up")
+curl -H "${pd_auth}" http://localhost:8620/v1/stores
# List partitions
-curl http://localhost:8620/v1/partitions
+curl -H "${pd_auth}" http://localhost:8620/v1/partitions
+```
+
+Confirm authentication actually engaged — `/versions` stays open by design,
+so it cannot tell you whether auth is on. A graph read without credentials
+must be rejected:
+
+```bash
+cd docker
+# Expect 401 on all three replicas
+for port in 8080 8081 8082; do
+ curl -s -o /dev/null -w "${port}: %{http_code}\n" \
+ "http://localhost:${port}/graphs/hugegraph/schema/vertexlabels"
+done
+
+# And a signed-in read must succeed. Compose reads docker/.env by itself, but
+# your shell does not — load it first. Passing the credential through
+# --config keeps it out of argv, where `ps` would expose it to other users.
+set -a; . ./.env; set +a
Review Comment:
⚠️ This verification block executes the entire Compose dotenv file as shell code. Evidence: `set -a; . ./.env; set +a` evaluates command substitutions and other shell syntax before the curl; a value such as `HUGEGRAPH_ADMIN_PASSWORD=$(...)` runs on the operator host. Parse only the required key with a dotenv-aware parser or avoid sourcing `.env` directly.
##########
docker/docker-compose-3pd-3store-3server.yml:
##########
@@ -17,9 +17,14 @@
name: hugegraph-3x3
+# The cluster network is shared with the Hubble add-on
+# (docker-compose-hubble.yml), so it is external and must exist first:
+# docker network create hugegraph-net
+# Set HUGEGRAPH_NETWORK to use a differently named network.
networks:
hg-net:
- driver: bridge
+ external: true
Review Comment:
‼️ The predictable external network and host-published control-plane ports expose unauthenticated PD/Store administration by default. Evidence: `hugegraph-net` is external and the file publishes PD/Store REST, gRPC, and Raft ports on all host interfaces; Store has no authentication and PD only checks the internal service username, not the password. Bind these ports to loopback/remove them by default, or require real authentication/TLS and explicit network ACLs before shipping this as the default quickstart.
##########
.github/workflows/server-ci.yml:
##########
@@ -138,6 +140,189 @@ jobs:
check_compose docker/docker-compose.yml always always
check_compose docker/docker-compose.dev.yml build missing
+ check_cluster_compose() {
+ local cluster="docker/docker-compose-3pd-3store-3server.yml"
+ local addon="docker/docker-compose-hubble.yml"
+ local rendered
+ local token_fixture=ci-test-token-secret-32-bytes-long
+ rendered="$(mktemp)"
+ if [ "${#token_fixture}" -lt 32 ]; then
+ echo "CI token fixture must be at least 32 bytes" >&2
+ return 1
+ fi
+
+ # RETURN only: an EXIT trap would fire after this function's
+ # `local rendered` has gone out of scope, which `set -u` turns
+ # into an "unbound variable" error. A hard errexit abort can
+ # therefore still leak one temp file, which is acceptable on an
+ # ephemeral runner.
+ trap 'rm -f "$rendered"' RETURN
+
+ # --env-file /dev/null on every invocation: Compose otherwise reads
+ # docker/.env automatically, and the quickstart tells operators to
+ # create one holding exactly the credentials these assertions
+ # control. Without it the checks pass in CI (which has no .env) but
+ # report false failures for anyone running them locally after
+ # following the quickstart, and the add-on render below — which
+ # deliberately unsets the variables to assert their defaults — would
+ # read whatever HUGEGRAPH_NETWORK/HUGEGRAPH_VERSION that .env holds.
+ # Pinning an empty env file makes both renders depend only on what
+ # each invocation sets explicitly.
+
+ # Both cluster credentials are required and may not be empty.
+ # Each case asserts the guard fired for the *intended* variable:
+ # a bare non-zero exit would also be produced by a YAML error, a
+ # renamed file, or a missing docker binary.
+ assert_guard() { # assert_guard <blamed-var> <description>
+ local var="$1" desc="$2" err
+ if err="$(docker compose --env-file /dev/null -f "$cluster" \
+ config -q 2>&1)"; then
+ echo "$cluster accepted $desc" >&2
+ return 1
+ fi
+ case "$err" in
+ *"$var"*) : ;;
+ *) echo "$cluster rejected $desc, but not because of $var: $err" >&2
+ return 1 ;;
+ esac
+ }
+
+ ( unset HUGEGRAPH_ADMIN_PASSWORD
+ export HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}"
+ assert_guard HUGEGRAPH_ADMIN_PASSWORD "an unset admin password" )
+ ( export HUGEGRAPH_ADMIN_PASSWORD=
+ export HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}"
+ assert_guard HUGEGRAPH_ADMIN_PASSWORD "an empty admin password" )
+ ( unset HUGEGRAPH_AUTH_TOKEN_SECRET
+ export HUGEGRAPH_ADMIN_PASSWORD=ci-test-password
+ assert_guard HUGEGRAPH_AUTH_TOKEN_SECRET "an unset token secret" )
+ ( export HUGEGRAPH_ADMIN_PASSWORD=ci-test-password
+ export HUGEGRAPH_AUTH_TOKEN_SECRET=
+ assert_guard HUGEGRAPH_AUTH_TOKEN_SECRET "an empty token secret" )
+ # The add-on alone must define Hubble and nothing else, join the
+ # shared external network with its default name, and need no
+ # credentials or overrides.
+ env -u HUGEGRAPH_ADMIN_PASSWORD -u HUGEGRAPH_AUTH_TOKEN_SECRET \
+ -u HUGEGRAPH_NETWORK -u HUGEGRAPH_VERSION \
+ -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY -u HUBBLE_PUBLISH_HOST \
+ docker compose --env-file /dev/null -f "$addon" config --format json > "$rendered"
+ jq -e '
+ (.services | keys) == ["hubble"] and
+ .networks."hg-net".external == true and
+ .networks."hg-net".name == "hugegraph-net" and
+ (.services.hubble.networks | has("hg-net")) and
+ (.services.hubble | has("depends_on") | not) and
+ any(.services.hubble.volumes[];
+ .target == "/hubble/conf/hugegraph-hubble.properties" and
+ (.source | endswith("hugegraph-hubble-3x3.properties")))
+ and any(.services.hubble.volumes[];
+ .source == "hg-hubble-db" and
+ .target == "/hubble/db")
+ and any(.services.hubble.volumes[];
+ .source == "hg-hubble-upload-files" and
+ .target == "/hubble/upload-files")
+ ' "$rendered" >/dev/null
+
+ # The cluster's own default network name must match the add-on's,
+ # or the attach flow and the cluster land on different networks.
+ # The combined render below pins an override, so it cannot catch a
+ # drifting default.
+ HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}" \
+ env -u HUGEGRAPH_NETWORK -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY \
+ -u HUBBLE_PUBLISH_HOST \
+ docker compose --env-file /dev/null -f "$cluster" \
+ config --format json > "$rendered"
+ jq -e '.networks."hg-net".name == "hugegraph-net"' \
+ "$rendered" >/dev/null
+
+ # The combined render carries the PD-registration and auth
+ # settings on every server replica and keeps Hubble on loopback.
+ # Rendered with non-default HUGEGRAPH_NETWORK/HUGEGRAPH_VERSION so
+ # CI fails if any file stops honoring the overrides (the add-on
+ # render above covers the defaults).
+ HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}" \
+ HUGEGRAPH_NETWORK=ci-test-net \
+ HUGEGRAPH_VERSION=ci-test-tag \
+ env -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY -u HUBBLE_PUBLISH_HOST \
+ docker compose --env-file /dev/null -f "$cluster" -f "$addon" \
+ config --format json > "$rendered"
+ jq -e '
+ .name == "hugegraph-3x3" and
+ (.services | keys | length) == 10 and
+ .networks."hg-net".external == true and
+ .networks."hg-net".name == "ci-test-net" and
+ all(.services[]; .networks | has("hg-net")) and
+ .services.pd0.image == "hugegraph/pd:ci-test-tag" and
+ .services.store0.image == "hugegraph/store:ci-test-tag" and
+ .services.server0.image == "hugegraph/server:ci-test-tag" and
+ .services.hubble.image == "hugegraph/hubble:ci-test-tag" and
+ .services.server0.environment.HG_SERVER_USE_PD == "true" and
Review Comment:
⚠️ The contract comment says PD-registration and auth settings are carried by every Server replica, but the assertions only check `server0` for `HG_SERVER_USE_PD`, `HG_SERVER_CLUSTER`, `HG_SERVER_INIT_STORE_ENABLED`, and `PASSWORD`; `server1`/`server2` only check REST URL and token. A per-replica override can therefore pass CI while breaking discovery or initialization. Apply the required-value assertions to all three Server services.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(docker): add attachable Hubble Compose add-on for the 3-node cluster [hugegraph]
Posted by "bitflicker64 (via GitHub)" <gi...@apache.org>.
bitflicker64 commented on code in PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149#discussion_r3866731744
##########
docker/docker-compose-3pd-3store-3server.yml:
##########
@@ -187,16 +207,25 @@ services:
<<: *server-common
container_name: hg-server0
hostname: server0
+ environment:
+ <<: *server-env
+ HG_SERVER_REST_URL: http://server0:8080
Review Comment:
Leaving this one open deliberately, since it is the only finding I did not actually satisfy.
You asked for a configurable advertised address, or for the external client path to use addresses it can resolve. Neither is possible from the Compose files: `HG_SERVER_REST_URL` becomes `restserver.url`, which is also the address the REST server binds, so an externally resolvable value crash-loops the replica. Splitting bind from advertise needs a Server-side setting.
What shipped is the limitation documented, with external clients pointed at the published ports instead of PD discovery. Your call whether that is an acceptable close or whether it should become a Server issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(docker): add attachable Hubble Compose add-on for the 3-node cluster [hugegraph]
Posted by "bitflicker64 (via GitHub)" <gi...@apache.org>.
bitflicker64 commented on code in PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149#discussion_r3866735042
##########
docker/docker-compose-3pd-3store-3server.yml:
##########
@@ -187,16 +207,25 @@ services:
<<: *server-common
container_name: hg-server0
hostname: server0
+ environment:
+ <<: *server-env
+ HG_SERVER_REST_URL: http://server0:8080
Review Comment:
@imbajin which way do you want this one closed?
**A.** Accept it as a documented limitation and resolve the thread. External clients use the published Server ports; PD discovery stays correct for anything on `hugegraph-net`, which is what Hubble does.
**B.** I open a Server issue for a real advertised-address option, separate from `restserver.url` so bind and advertise can differ, and link it here.
I lean A for this PR, since B is a Server change and would hold up the add-on. Happy to file B either way if you want it tracked.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(docker): rebuild Compose topologies with Hubble [hugegraph]
Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on code in PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149#discussion_r3890888850
##########
docker/docker-compose.yml:
##########
@@ -14,117 +14,59 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
-name: hugegraph-single
+
+name: hugegraph-standalone
networks:
hg-net:
- driver: bridge
volumes:
- hg-pd-data:
- hg-store-data:
+ server-data:
+ hubble-data:
services:
-
- pd:
- image: hugegraph/pd:${HUGEGRAPH_VERSION:-latest}
- pull_policy: always
- container_name: hg-pd
- hostname: pd
- restart: unless-stopped
- networks: [hg-net]
- environment:
- HG_PD_GRPC_HOST: pd
- HG_PD_GRPC_PORT: "8686"
- HG_PD_REST_PORT: "8620"
- HG_PD_RAFT_ADDRESS: pd:8610
- HG_PD_RAFT_PEERS_LIST: pd:8610
- HG_PD_INITIAL_STORE_LIST: store:8500
- HG_PD_DATA_PATH: /hugegraph-pd/pd_data
- ports:
- - "8620:8620"
- volumes:
- - hg-pd-data:/hugegraph-pd/pd_data
- healthcheck:
- test: ["CMD-SHELL", "curl -fsS http://localhost:8620/v1/health >/dev/null || exit 1"]
- interval: 10s
- timeout: 5s
- retries: 12
- start_period: 30s
-
- store:
- image: hugegraph/store:${HUGEGRAPH_VERSION:-latest}
- pull_policy: always
- container_name: hg-store
- hostname: store
- restart: unless-stopped
- networks: [hg-net]
- depends_on:
- pd:
- condition: service_healthy
- environment:
- HG_STORE_PD_ADDRESS: pd:8686
- HG_STORE_GRPC_HOST: store
- HG_STORE_GRPC_PORT: "8500"
- HG_STORE_REST_PORT: "8520"
- HG_STORE_RAFT_ADDRESS: store:8510
- HG_STORE_DATA_PATH: /hugegraph-store/storage
- ports:
- - "8520:8520"
- volumes:
- - hg-store-data:/hugegraph-store/storage
- healthcheck:
- test: ["CMD-SHELL", "curl -fsS http://localhost:8520/v1/health >/dev/null || exit 1"]
- interval: 10s
- timeout: 10s
- retries: 30
- start_period: 60s
-
server:
- image: ${HUGEGRAPH_SERVER_IMAGE:-hugegraph/server:${HUGEGRAPH_VERSION:-latest}}
- pull_policy: ${HUGEGRAPH_SERVER_PULL_POLICY:-always}
- container_name: hg-server
- hostname: server
+ image: hugegraph/hugegraph:${HUGEGRAPH_VERSION:-latest}
+ pull_policy: ${HUGEGRAPH_PULL_POLICY:-missing}
restart: unless-stopped
networks: [hg-net]
- depends_on:
- store:
- condition: service_healthy
environment:
- HG_SERVER_BACKEND: hstore
- HG_SERVER_PD_PEERS: pd:8686
- HG_SERVER_CLUSTER: hg
- HG_SERVER_USE_PD: "true"
- HG_SERVER_REST_URL: http://server:8080
- HG_SERVER_MIN_FREE_MEMORY: "0"
- HG_SERVER_INIT_STORE_ENABLED: "false"
+ PASSWORD: ${HUGEGRAPH_ADMIN_PASSWORD:-}
Review Comment:
‼️ Critical. With no `.env`, `HUGEGRAPH_ADMIN_PASSWORD` expands to an empty value, and the server entrypoint treats that as authentication disabled, while `8080:8080` publishes the API on all host interfaces. A default `docker compose up` therefore exposes an unauthenticated Server to the LAN. Fail closed with required credentials, or bind the auth-off default to loopback and make the unauthenticated mode explicit; apply the same policy to the HStore and HA files.
##########
docker/conf/hubble/standalone.properties:
##########
@@ -0,0 +1,20 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+server.host=0.0.0.0
Review Comment:
‼️ Critical. This mounted Hubble properties file, along with the HStore and HStore-HA variants, omits `server.port`. The Hubble image startup script reads `server.host` and `server.port` directly to build its `/about` probe URL; because this file replaces the image config, the image default of 8088 is not preserved and the probe can be built with an empty port. Add `server.port=8088` to all three files and assert it in the render contract.
##########
docker/docker-compose.yml:
##########
@@ -14,117 +14,59 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
-name: hugegraph-single
+
+name: hugegraph-standalone
Review Comment:
‼️ Critical. This changes the default Compose project from the previous PD/Store/HStore topology to a standalone RocksDB Server with a different project name and volume (`server-data`). An existing user who upgrades the file and runs the documented default command will no longer mount the old HStore data; the old stack and its ports remain separate, with no migration, backup, or rollback warning. Preserve the legacy entrypoint or document an explicit migration and safe rollback path before making this the default.
##########
docker/docker-compose-3pd-3store-3server.yml:
##########
@@ -58,6 +59,17 @@ x-store-common: &store-common
retries: 40
start_period: 120s
+x-server-environment: &server-environment
+ STORE_REST: store0:8520
+ HG_SERVER_BACKEND: hstore
+ HG_SERVER_PD_PEERS: pd0:8686,pd1:8686,pd2:8686
+ HG_SERVER_CLUSTER: hg
+ HG_SERVER_USE_PD: "true"
+ HG_SERVER_MIN_FREE_MEMORY: "0"
+ HG_SERVER_INIT_STORE_ENABLED: "false"
+ HG_SERVER_AUTH_TOKEN_SECRET: ${HUGEGRAPH_AUTH_TOKEN_SECRET:-}
Review Comment:
‼️ Critical. HA leaves `HG_SERVER_AUTH_TOKEN_SECRET` empty by default. When a password is set, the Server entrypoint generates a random secret if this variable is absent; with three Server replicas, each container can then sign JWTs with a different key and requests routed across replicas intermittently fail with 401. Require one non-empty shared secret for HA (or generate and inject it once) and add a password-only multi-replica check.
##########
docker/test-compose.sh:
##########
@@ -0,0 +1,396 @@
+#!/usr/bin/env bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+set -Eeuo pipefail
+
+DOCKER_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+PASSWORD="ci-compose-password"
+SECRET="0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"
+VERSION="ci-version"
+RENDER_HUBBLE_IMAGE="example.invalid/hugegraph/hubble:ci"
+DATASOURCE="jdbc:h2:file:/hubble/data/hubble;DB_CLOSE_ON_EXIT=FALSE"
+ACTIVE_PROJECT=""
+ACTIVE_FILES=()
+RENDER_DIR=""
+
+compose_auth() {
+ env HUGEGRAPH_VERSION="${VERSION}" \
+ HUBBLE_IMAGE="${RENDER_HUBBLE_IMAGE}" \
+ HUGEGRAPH_ADMIN_PASSWORD="${PASSWORD}" \
+ HUGEGRAPH_AUTH_TOKEN_SECRET="${SECRET}" \
+ docker compose "$@"
+}
+
+render() {
+ local output="$1"
+ shift
+ compose_auth "$@" config --format json > "${output}"
+}
+
+assert_file_property() {
+ local file="$1"
+ local property="$2"
+ grep -Fqx "${property}" "${file}"
+}
+
+assert_common() {
+ local rendered="$1"
+ local services="$2"
+ local volumes="$3"
+
+ jq -e \
+ --arg password "${PASSWORD}" \
+ --arg secret "${SECRET}" \
+ --argjson services "${services}" \
+ --argjson volumes "${volumes}" '
+ (.services | keys) == $services and
+ (.volumes | keys) == $volumes and
+ (.networks | keys) == ["hg-net"] and
+ all(.networks[]; .external != true) and
+ all(.volumes[]; .external != true) and
+ all(.services[];
+ (.networks | keys) == ["hg-net"] and
+ .healthcheck.test[0] == "CMD-SHELL") and
+ all(
+ [.services | to_entries[] |
+ select(.key | startswith("server")) |
+ .value.environment][];
+ .PASSWORD == $password and
+ .HG_SERVER_AUTH_TOKEN_SECRET == $secret)
+ ' "${rendered}" >/dev/null
+}
+
+assert_hubble() {
+ local rendered="$1"
+ local config="$2"
+ shift 2
+ local dependencies
+ dependencies="$(printf '%s\n' "$@" | jq -Rsc 'split("\n")[:-1] | sort')"
+
+ jq -e \
+ --arg datasource "${DATASOURCE}" \
+ --arg config "/docker/conf/hubble/${config}" \
+ --argjson dependencies "${dependencies}" '
+ .services.hubble.image == "example.invalid/hugegraph/hubble:ci" and
+ .services.hubble.pull_policy == "missing" and
+ .services.hubble.environment.SPRING_DATASOURCE_URL == $datasource and
+ any(.services.hubble.ports[];
+ .target == 8088 and .published == "8088" and
+ .host_ip == "127.0.0.1") and
+ (.services.hubble.depends_on | keys | sort) == $dependencies and
+ all(.services.hubble.depends_on[];
+ .condition == "service_healthy") and
+ any(.services.hubble.volumes[];
+ .type == "volume" and .source == "hubble-data" and
+ .target == "/hubble/data") and
+ any(.services.hubble.volumes[];
+ .type == "bind" and (.source | endswith($config)) and
+ .target == "/hubble/conf/hugegraph-hubble.properties" and
+ .read_only == true) and
+ (.services.hubble.healthcheck.test[1] |
+ contains("http://localhost:8088/about") and
+ contains("\"status\":200") and
+ contains("\"name\":\"hugegraph-hubble\""))
+ ' "${rendered}" >/dev/null
+}
+
+assert_standalone() {
+ local rendered="$1"
+ assert_common "${rendered}" \
+ '["hubble","server"]' \
+ '["hubble-data","server-data"]'
+ assert_hubble "${rendered}" "standalone.properties" server
+ jq -e '
+ .services.server.image == "hugegraph/hugegraph:ci-version" and
+ .services.server.pull_policy == "missing" and
+ .services.server.healthcheck.test[1] ==
+ "curl -fsS http://localhost:8080/versions >/dev/null" and
+ any(.services.server.volumes[];
+ .source == "server-data" and
+ .target == "/hugegraph-server/rocksdb-data")
+ ' "${rendered}" >/dev/null
+ assert_file_property "${DOCKER_DIR}/conf/hubble/standalone.properties" \
+ "pd.enabled=false"
+ assert_file_property "${DOCKER_DIR}/conf/hubble/standalone.properties" \
+ "server.direct_url=http://server:8080"
+}
+
+assert_hstore() {
+ local rendered="$1"
+ assert_common "${rendered}" \
+ '["hubble","pd","server","store"]' \
+ '["hubble-data","pd-data","store-data"]'
+ assert_hubble "${rendered}" "hstore.properties" server
+ jq -e '
+ .services.pd.image == "hugegraph/pd:ci-version" and
+ .services.store.image == "hugegraph/store:ci-version" and
+ .services.server.image == "hugegraph/server:ci-version" and
+ all([.services.pd, .services.store, .services.server][];
+ .pull_policy == "missing") and
+ .services.server.environment.HG_SERVER_BACKEND == "hstore" and
+ .services.server.environment.HG_SERVER_PD_PEERS == "pd:8686" and
+ .services.server.environment.HG_SERVER_USE_PD == "true" and
+ .services.server.environment.HG_SERVER_REST_URL ==
+ "http://server:8080" and
+ .services.server.healthcheck.test[1] ==
+ "curl -fsS http://server:8080/versions >/dev/null" and
+ any(.services.pd.volumes[];
+ .source == "pd-data" and
+ .target == "/hugegraph-pd/pd_data") and
+ any(.services.store.volumes[];
+ .source == "store-data" and
+ .target == "/hugegraph-store/storage")
+ ' "${rendered}" >/dev/null
+ assert_file_property "${DOCKER_DIR}/conf/hubble/hstore.properties" \
+ "pd.peers=pd:8686"
+ assert_file_property "${DOCKER_DIR}/conf/hubble/hstore.properties" \
+ "operations.store.allowed_targets=[http://store:8520]"
+}
+
+assert_ha() {
+ local rendered="$1"
+ assert_common "${rendered}" \
+ '["hubble","pd0","pd1","pd2","server0","server1","server2","store0","store1","store2"]' \
+ '["hg-pd0-data","hg-pd1-data","hg-pd2-data","hg-store0-data","hg-store1-data","hg-store2-data","hubble-data"]'
+ assert_hubble "${rendered}" "hstore-ha.properties" \
+ server0 server1 server2
+ jq -e '
+ all([.services.pd0, .services.pd1, .services.pd2][];
+ .image == "hugegraph/pd:ci-version" and
+ .pull_policy == "missing") and
+ all([.services.store0, .services.store1, .services.store2][];
+ .image == "hugegraph/store:ci-version" and
+ .pull_policy == "missing") and
+ all([.services.server0, .services.server1, .services.server2][];
+ .image == "hugegraph/server:ci-version" and
+ .pull_policy == "missing" and
+ .environment.STORE_REST == "store0:8520" and
+ .environment.HG_SERVER_BACKEND == "hstore" and
+ .environment.HG_SERVER_PD_PEERS ==
+ "pd0:8686,pd1:8686,pd2:8686" and
+ .environment.HG_SERVER_CLUSTER == "hg" and
+ .environment.HG_SERVER_USE_PD == "true" and
+ .environment.HG_SERVER_INIT_STORE_ENABLED == "false") and
+ [.services.server0.environment.HG_SERVER_REST_URL,
+ .services.server1.environment.HG_SERVER_REST_URL,
+ .services.server2.environment.HG_SERVER_REST_URL] ==
+ ["http://server0:8080",
+ "http://server1:8080",
+ "http://server2:8080"] and
+ [.services.server0.healthcheck.test[1],
+ .services.server1.healthcheck.test[1],
+ .services.server2.healthcheck.test[1]] ==
+ ["curl -fsS http://server0:8080/versions >/dev/null",
+ "curl -fsS http://server1:8080/versions >/dev/null",
+ "curl -fsS http://server2:8080/versions >/dev/null"]
+ ' "${rendered}" >/dev/null
+ assert_file_property "${DOCKER_DIR}/conf/hubble/hstore-ha.properties" \
+ "pd.peers=pd0:8686,pd1:8686,pd2:8686"
+ assert_file_property "${DOCKER_DIR}/conf/hubble/hstore-ha.properties" \
+ "operations.store.allowed_targets=[http://store0:8520,http://store1:8520,http://store2:8520]"
+}
+
+assert_dev_override() {
+ local rendered="$1"
+ local override="$2"
+ assert_common "${rendered}" \
+ '["hubble","pd","server","store"]' \
+ '["hubble-data","pd-data","store-data"]'
+ jq -e '
+ .services.pd.image == "hugegraph/pd:dev" and
+ .services.store.image == "hugegraph/store:dev" and
+ .services.server.image == "hugegraph/server:dev" and
+ all([.services.pd, .services.store, .services.server][];
+ .pull_policy == "build" and .build != null) and
+ .services.hubble.image == "example.invalid/hugegraph/hubble:ci" and
+ .services.hubble.build == null
+ ' "${rendered}" >/dev/null
+ jq -e '
+ (.services | keys) == ["pd","server","store"] and
+ (.networks | keys) == ["default"] and .volumes == null and
+ all(.services[];
+ .build != null and .image != null and
+ .pull_policy == "build" and
+ .environment == null and
+ (.networks | keys) == ["default"] and
+ .volumes == null)
+ ' "${override}" >/dev/null
+}
+
+cleanup() {
+ if [[ -n "${ACTIVE_PROJECT}" ]]; then
+ compose_active down -v --remove-orphans >/dev/null 2>&1 || true
+ fi
+ [[ -z "${RENDER_DIR}" ]] || rm -rf "${RENDER_DIR}"
+}
+
+run_render() {
+ RENDER_DIR="$(mktemp -d)"
+ trap cleanup EXIT INT TERM
+ render "${RENDER_DIR}/standalone.json" \
+ -f "${DOCKER_DIR}/docker-compose.yml"
+ render "${RENDER_DIR}/hstore.json" \
+ -f "${DOCKER_DIR}/docker-compose-hstore.yml"
+ render "${RENDER_DIR}/ha.json" \
+ -f "${DOCKER_DIR}/docker-compose-3pd-3store-3server.yml"
+ render "${RENDER_DIR}/dev.json" \
+ -f "${DOCKER_DIR}/docker-compose-hstore.yml" \
+ -f "${DOCKER_DIR}/docker-compose.dev.yml"
+ render "${RENDER_DIR}/override.json" \
+ -f "${DOCKER_DIR}/docker-compose.dev.yml"
+
+ assert_standalone "${RENDER_DIR}/standalone.json"
+ assert_hstore "${RENDER_DIR}/hstore.json"
+ assert_ha "${RENDER_DIR}/ha.json"
+ assert_dev_override "${RENDER_DIR}/dev.json" \
+ "${RENDER_DIR}/override.json"
+ echo "Compose render contracts passed"
+}
+
+compose_active() {
+ env HUGEGRAPH_VERSION="${HUGEGRAPH_VERSION:-latest}" \
+ HUBBLE_IMAGE="${HUBBLE_IMAGE:-hugegraph/hubble:latest}" \
+ HUGEGRAPH_ADMIN_PASSWORD="${PASSWORD}" \
+ HUGEGRAPH_AUTH_TOKEN_SECRET="${SECRET}" \
+ COMPOSE_PROGRESS=plain \
+ docker compose -p "${ACTIVE_PROJECT}" "${ACTIVE_FILES[@]}" "$@"
+}
+
+diagnose() {
+ compose_active ps || true
+ compose_active logs --no-color --tail 200 || true
+}
+
+http_status() {
+ curl -sS -o /dev/null -w '%{http_code}' "$@"
+}
+
+wait_hubble_mode() {
+ local expected_pd="$1"
+ local expected_auth="$2"
+ local response=""
+ local _
+ for _ in {1..30}; do
+ response="$(curl -fsS http://localhost:8088/api/v1.3/config || true)"
Review Comment:
⚠️ Important. The retry loop is bounded, but each `curl` has no connect or total-request timeout. If Hubble accepts a connection and its handler hangs, this iteration never returns, so the 30-iteration bound and trap cleanup are ineffective. Add shared `--connect-timeout` and `--max-time` options to `wait_hubble_mode`, `http_status`, login, and anonymous probes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(docker): rebuild Compose topologies with Hubble [hugegraph]
Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on code in PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149#discussion_r3892316598
##########
docker/docker-compose-3pd-3store-3server.yml:
##########
@@ -68,11 +81,9 @@ x-server-common: &server-common
store1: { condition: service_healthy }
store2: { condition: service_healthy }
environment:
- STORE_REST: store0:8520
- HG_SERVER_BACKEND: hstore
- HG_SERVER_PD_PEERS: pd0:8686,pd1:8686,pd2:8686
+ <<: *server-environment
healthcheck:
- test: ["CMD-SHELL", "curl -fsS http://localhost:8080/versions >/dev/null || exit 1"]
+ test: ["CMD-SHELL", "curl -fsS http://server0:8080/versions >/dev/null"]
Review Comment:
Confirmed and fixed in 9cf8de733. YAML merge was shallow and the three overrides dropped interval, timeout, retries, and start_period. The shared healthcheck now uses $$(hostname), the per-server overrides are removed, and the render contract asserts the complete timing map for all three Servers. The latest server image contains hostname and resolves its configured hostname.
##########
docker/docker-compose-hstore.yml:
##########
@@ -0,0 +1,131 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+name: hugegraph-hstore
+
+networks:
+ hg-net:
+
+volumes:
+ pd-data:
+ store-data:
+ hubble-data:
+
+services:
+ pd:
+ image: hugegraph/pd:${HUGEGRAPH_VERSION:-latest}
+ pull_policy: ${HUGEGRAPH_PULL_POLICY:-missing}
+ restart: unless-stopped
+ networks: [hg-net]
+ environment:
+ HG_PD_GRPC_HOST: pd
+ HG_PD_GRPC_PORT: "8686"
+ HG_PD_REST_PORT: "8620"
+ HG_PD_RAFT_ADDRESS: pd:8610
+ HG_PD_RAFT_PEERS_LIST: pd:8610
+ HG_PD_INITIAL_STORE_LIST: store:8500
+ HG_PD_DATA_PATH: /hugegraph-pd/pd_data
+ ports:
+ - "8620:8620"
+ volumes:
+ - pd-data:/hugegraph-pd/pd_data
+ healthcheck:
+ test: ["CMD-SHELL", "curl -fsS http://localhost:8620/v1/health >/dev/null"]
+ interval: 10s
+ timeout: 5s
+ retries: 12
+ start_period: 30s
+
+ store:
+ image: hugegraph/store:${HUGEGRAPH_VERSION:-latest}
+ pull_policy: ${HUGEGRAPH_PULL_POLICY:-missing}
+ restart: unless-stopped
+ networks: [hg-net]
+ depends_on:
+ pd:
+ condition: service_healthy
+ environment:
+ HG_STORE_PD_ADDRESS: pd:8686
+ HG_STORE_GRPC_HOST: store
+ HG_STORE_GRPC_PORT: "8500"
+ HG_STORE_REST_PORT: "8520"
+ HG_STORE_RAFT_ADDRESS: store:8510
+ HG_STORE_DATA_PATH: /hugegraph-store/storage
+ ports:
+ - "8520:8520"
+ volumes:
+ - store-data:/hugegraph-store/storage
+ healthcheck:
+ test: ["CMD-SHELL", "curl -fsS http://localhost:8520/v1/health >/dev/null"]
+ interval: 10s
+ timeout: 10s
+ retries: 30
+ start_period: 60s
+
+ server:
+ image: hugegraph/server:${HUGEGRAPH_VERSION:-latest}
+ pull_policy: ${HUGEGRAPH_PULL_POLICY:-missing}
+ restart: unless-stopped
+ networks: [hg-net]
+ depends_on:
+ store:
+ condition: service_healthy
+ environment:
+ HG_SERVER_BACKEND: hstore
+ HG_SERVER_PD_PEERS: pd:8686
+ HG_SERVER_CLUSTER: hg
+ HG_SERVER_USE_PD: "true"
+ HG_SERVER_REST_URL: http://server:8080
+ HG_SERVER_MIN_FREE_MEMORY: "0"
+ HG_SERVER_INIT_STORE_ENABLED: "false"
+ HG_SERVER_AUTH_TOKEN_SECRET: ${HUGEGRAPH_AUTH_TOKEN_SECRET:-}
+ PASSWORD: ${HUGEGRAPH_ADMIN_PASSWORD:-}
+ ports:
+ - "8080:8080"
Review Comment:
The exposure risk is real, but this is the same policy question already decided in discussion_r3891967579. Auth-off is an explicitly required local mode, the README limits it to trusted environments, and broad Server/PD/Store publishing hardening remains outside this focused redesign. Applying it only to the new minimal file would also leave the topology family inconsistent, so no change here.
##########
docker/docker-compose.yml:
##########
@@ -14,117 +14,59 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
-name: hugegraph-single
+
+name: hugegraph-standalone
networks:
hg-net:
- driver: bridge
volumes:
- hg-pd-data:
- hg-store-data:
+ server-data:
+ hubble-data:
services:
-
- pd:
- image: hugegraph/pd:${HUGEGRAPH_VERSION:-latest}
- pull_policy: always
- container_name: hg-pd
- hostname: pd
- restart: unless-stopped
- networks: [hg-net]
- environment:
- HG_PD_GRPC_HOST: pd
- HG_PD_GRPC_PORT: "8686"
- HG_PD_REST_PORT: "8620"
- HG_PD_RAFT_ADDRESS: pd:8610
- HG_PD_RAFT_PEERS_LIST: pd:8610
- HG_PD_INITIAL_STORE_LIST: store:8500
- HG_PD_DATA_PATH: /hugegraph-pd/pd_data
- ports:
- - "8620:8620"
- volumes:
- - hg-pd-data:/hugegraph-pd/pd_data
- healthcheck:
- test: ["CMD-SHELL", "curl -fsS http://localhost:8620/v1/health >/dev/null || exit 1"]
- interval: 10s
- timeout: 5s
- retries: 12
- start_period: 30s
-
- store:
- image: hugegraph/store:${HUGEGRAPH_VERSION:-latest}
- pull_policy: always
- container_name: hg-store
- hostname: store
- restart: unless-stopped
- networks: [hg-net]
- depends_on:
- pd:
- condition: service_healthy
- environment:
- HG_STORE_PD_ADDRESS: pd:8686
- HG_STORE_GRPC_HOST: store
- HG_STORE_GRPC_PORT: "8500"
- HG_STORE_REST_PORT: "8520"
- HG_STORE_RAFT_ADDRESS: store:8510
- HG_STORE_DATA_PATH: /hugegraph-store/storage
- ports:
- - "8520:8520"
- volumes:
- - hg-store-data:/hugegraph-store/storage
- healthcheck:
- test: ["CMD-SHELL", "curl -fsS http://localhost:8520/v1/health >/dev/null || exit 1"]
- interval: 10s
- timeout: 10s
- retries: 30
- start_period: 60s
-
server:
- image: ${HUGEGRAPH_SERVER_IMAGE:-hugegraph/server:${HUGEGRAPH_VERSION:-latest}}
- pull_policy: ${HUGEGRAPH_SERVER_PULL_POLICY:-always}
- container_name: hg-server
- hostname: server
+ image: hugegraph/hugegraph:${HUGEGRAPH_VERSION:-latest}
+ pull_policy: ${HUGEGRAPH_PULL_POLICY:-missing}
restart: unless-stopped
networks: [hg-net]
- depends_on:
- store:
- condition: service_healthy
environment:
- HG_SERVER_BACKEND: hstore
- HG_SERVER_PD_PEERS: pd:8686
- HG_SERVER_CLUSTER: hg
- HG_SERVER_USE_PD: "true"
- HG_SERVER_REST_URL: http://server:8080
- HG_SERVER_MIN_FREE_MEMORY: "0"
- HG_SERVER_INIT_STORE_ENABLED: "false"
+ PASSWORD: ${HUGEGRAPH_ADMIN_PASSWORD:-}
HG_SERVER_AUTH_TOKEN_SECRET: ${HUGEGRAPH_AUTH_TOKEN_SECRET:-}
- PASSWORD: ${HUGEGRAPH_ADMIN_PASSWORD:?Set a non-default admin password}
+ HG_SERVER_MIN_FREE_MEMORY: "0"
ports:
- "8080:8080"
+ volumes:
+ - server-data:/hugegraph-server/rocksdb-data
healthcheck:
- test: ["CMD-SHELL", "curl -fsS http://server:8080/versions >/dev/null || exit 1"]
+ test: ["CMD-SHELL", "curl -fsS http://localhost:8080/versions >/dev/null"]
interval: 10s
timeout: 5s
retries: 30
start_period: 60s
hubble:
- image: ${HUBBLE_IMAGE:-hugegraph/hubble:${HUGEGRAPH_VERSION:-latest}}
- pull_policy: ${HUBBLE_PULL_POLICY:-always}
- container_name: hg-hubble
- hostname: hubble
+ image: ${HUBBLE_IMAGE:-hugegraph/hubble:latest}
+ pull_policy: ${HUBBLE_PULL_POLICY:-missing}
restart: unless-stopped
networks: [hg-net]
depends_on:
server:
condition: service_healthy
+ environment:
+ SPRING_DATASOURCE_URL: jdbc:h2:file:/hubble/data/hubble;DB_CLOSE_ON_EXIT=FALSE
ports:
- "${HUBBLE_PUBLISH_HOST:-127.0.0.1}:8088:8088"
volumes:
- - ./hugegraph-hubble.properties:/hubble/conf/hugegraph-hubble.properties:ro
+ - hubble-data:/hubble/data
+ - ./conf/hubble/standalone.properties:/hubble/conf/hugegraph-hubble.properties:ro
healthcheck:
- test: ["CMD-SHELL", "body=$$(curl -fsS http://127.0.0.1:8088/about) && printf '%s' \"$$body\" | grep -q '\"status\":200' && printf '%s' \"$$body\" | grep -q '\"name\":\"hugegraph-hubble\"'"]
+ test:
+ - CMD-SHELL
+ - >-
+ body=$$(curl -fsS http://localhost:8088/about) &&
+ printf '%s' "$$body" | grep -q '"status":200' &&
+ printf '%s' "$$body" | grep -q '"name":"hugegraph-hubble"'
Review Comment:
Keeping the current check. The one-line pipeline is not strictly equivalent under CMD-SHELL: without pipefail, grep can return success after curl returns a nonzero status with partial matching output. The current body=$(curl ...) && checks preserve curl failure independently and retain both application status and product identity.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(docker): rebuild Compose topologies with Hubble [hugegraph]
Posted by "bitflicker64 (via GitHub)" <gi...@apache.org>.
bitflicker64 commented on code in PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149#discussion_r3892095050
##########
docker/docker-compose-3pd-3store-3server.yml:
##########
@@ -68,11 +81,9 @@ x-server-common: &server-common
store1: { condition: service_healthy }
store2: { condition: service_healthy }
environment:
- STORE_REST: store0:8520
- HG_SERVER_BACKEND: hstore
- HG_SERVER_PD_PEERS: pd0:8686,pd1:8686,pd2:8686
+ <<: *server-environment
healthcheck:
- test: ["CMD-SHELL", "curl -fsS http://localhost:8080/versions >/dev/null || exit 1"]
+ test: ["CMD-SHELL", "curl -fsS http://server0:8080/versions >/dev/null"]
Review Comment:
**These three per-server `healthcheck` overrides are new in this PR, and rendering suggests they also drop the anchor timings.**
`<<: *server-common` merges shallowly, so a per-service `healthcheck:` key replaces the whole map from the anchor rather than merging into it. Rendered on this head:
```console
$ docker compose -f docker-compose-3pd-3store-3server.yml config --format json \
| jq -c '.services.server0.healthcheck, .services.store0.healthcheck'
{"test":["CMD-SHELL","curl -fsS http://server0:8080/versions >/dev/null"]}
{"test":["CMD-SHELL","curl -fsS http://localhost:8520/v1/health >/dev/null || exit 1"],"timeout":"15s","interval":"15s","retries":40,"start_period":"2m0s"}
```
`store0` keeps its timings. `server0/1/2` keep only `test`, losing `interval: 10s`, `timeout: 5s`, `retries: 30` and, most importantly, `start_period: 60s`, so they fall back to engine defaults (30s interval, 3 retries, no start period). That marks a Server unhealthy roughly 90s after container start, while the entrypoint still has `wait-storage`, `init-store` and `start-hugegraph.sh -t 120` to get through, so `up -d --wait` on HA can fail before the Server is ready. Before this PR all three inherited the anchor timings, and HA is never started in CI, so nothing catches it today.
If you agree, the fix is a deletion: keep one healthcheck in `x-server-common` and remove the overrides at 205-206, 216-217 and 227-228. The anchor `test` on line 86 then needs to work for all three, which `hostname` gives you:
```suggestion
test: ["CMD-SHELL", "curl -fsS http://$$(hostname):8080/versions >/dev/null"]
```
`hostname:` is already set per service, `$$` unescapes to `$` in the container exactly like the existing Hubble healthcheck, and I confirmed the image has `/usr/bin/hostname` and that the container self-resolves that name. Plain `localhost` would not work, since `restserver.url` is the Grizzly bind address. Applied to a copy, all three Servers then render with the full timings.
Coupled edit: `assert_ha` lines 197-202 collapse from the three element array comparison to one `all(...)`, also a net deletion. Note `config --format json` re-escapes `$` as `$$`, so the expected string there is the literal `curl -fsS http://$$(hostname):8080/versions >/dev/null`.
##########
docker/docker-compose-hstore.yml:
##########
@@ -0,0 +1,131 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+name: hugegraph-hstore
+
+networks:
+ hg-net:
+
+volumes:
+ pd-data:
+ store-data:
+ hubble-data:
+
+services:
+ pd:
+ image: hugegraph/pd:${HUGEGRAPH_VERSION:-latest}
+ pull_policy: ${HUGEGRAPH_PULL_POLICY:-missing}
+ restart: unless-stopped
+ networks: [hg-net]
+ environment:
+ HG_PD_GRPC_HOST: pd
+ HG_PD_GRPC_PORT: "8686"
+ HG_PD_REST_PORT: "8620"
+ HG_PD_RAFT_ADDRESS: pd:8610
+ HG_PD_RAFT_PEERS_LIST: pd:8610
+ HG_PD_INITIAL_STORE_LIST: store:8500
+ HG_PD_DATA_PATH: /hugegraph-pd/pd_data
+ ports:
+ - "8620:8620"
+ volumes:
+ - pd-data:/hugegraph-pd/pd_data
+ healthcheck:
+ test: ["CMD-SHELL", "curl -fsS http://localhost:8620/v1/health >/dev/null"]
+ interval: 10s
+ timeout: 5s
+ retries: 12
+ start_period: 30s
+
+ store:
+ image: hugegraph/store:${HUGEGRAPH_VERSION:-latest}
+ pull_policy: ${HUGEGRAPH_PULL_POLICY:-missing}
+ restart: unless-stopped
+ networks: [hg-net]
+ depends_on:
+ pd:
+ condition: service_healthy
+ environment:
+ HG_STORE_PD_ADDRESS: pd:8686
+ HG_STORE_GRPC_HOST: store
+ HG_STORE_GRPC_PORT: "8500"
+ HG_STORE_REST_PORT: "8520"
+ HG_STORE_RAFT_ADDRESS: store:8510
+ HG_STORE_DATA_PATH: /hugegraph-store/storage
+ ports:
+ - "8520:8520"
+ volumes:
+ - store-data:/hugegraph-store/storage
+ healthcheck:
+ test: ["CMD-SHELL", "curl -fsS http://localhost:8520/v1/health >/dev/null"]
+ interval: 10s
+ timeout: 10s
+ retries: 30
+ start_period: 60s
+
+ server:
+ image: hugegraph/server:${HUGEGRAPH_VERSION:-latest}
+ pull_policy: ${HUGEGRAPH_PULL_POLICY:-missing}
+ restart: unless-stopped
+ networks: [hg-net]
+ depends_on:
+ store:
+ condition: service_healthy
+ environment:
+ HG_SERVER_BACKEND: hstore
+ HG_SERVER_PD_PEERS: pd:8686
+ HG_SERVER_CLUSTER: hg
+ HG_SERVER_USE_PD: "true"
+ HG_SERVER_REST_URL: http://server:8080
+ HG_SERVER_MIN_FREE_MEMORY: "0"
+ HG_SERVER_INIT_STORE_ENABLED: "false"
+ HG_SERVER_AUTH_TOKEN_SECRET: ${HUGEGRAPH_AUTH_TOKEN_SECRET:-}
+ PASSWORD: ${HUGEGRAPH_ADMIN_PASSWORD:-}
+ ports:
+ - "8080:8080"
Review Comment:
**Should the Server REST port be loopback-gated the way Hubble is?**
Hubble gets `${HUBBLE_PUBLISH_HOST:-127.0.0.1}` on line 117, but Server here, PD on line 43 and Store on line 69 all publish to `0.0.0.0` by default. The README warns at some length about exposing Hubble and documents a supported auth-off mode, and `assert_hubble` asserts `host_ip == "127.0.0.1"` for Hubble and nothing for the others. The graph API is the actual data and auth boundary though, so in auth-off mode this is an unauthenticated graph API on every interface, reachable from the local network, while the UI in front of it is the part that is locked down.
Since this file is new in the PR it seemed worth raising here rather than later. The same shape exists in the HA file, but those port lines are untouched, so that one is a follow-up. Would a `${HUGEGRAPH_PUBLISH_HOST:-127.0.0.1}` prefix on the three ports, matching the Hubble pattern you already have, be reasonable? It is one variable rather than three if you would rather keep it minimal.
##########
docker/README.md:
##########
@@ -1,436 +1,362 @@
-# HugeGraph Docker Deployment
+# HugeGraph Docker Compose
-This directory contains Docker Compose files for running HugeGraph:
+## Users
-| File | Description |
-|------|-------------|
-| `docker-compose.yml` | PD, Store, Server, and Hubble using pre-built images |
-| `docker-compose.dev.yml` | PD, Store, and Server built from source, plus Hubble |
-| `docker-compose-3pd-3store-3server.yml` | 3-node distributed cluster (PD + Store + Server) |
+### Choose a topology
-## Prerequisites
+Run commands in this directory:
-- **Docker Engine** 20.10+ (or Docker Desktop 4.x+)
-- **Docker Compose** v2 (included in Docker Desktop)
-- **OpenSSL CLI** (used to generate the initial administrator password)
-- **Memory**: Allocate at least **12 GB** to Docker Desktop (Settings → Resources → Memory). The 3-node cluster runs 9 JVM processes (3 PD + 3 Store + 3 Server) which are memory-intensive. Insufficient memory causes OOM kills that appear as silent Raft failures.
+```bash
+cd docker
+```
-> [!IMPORTANT]
-> The 12 GB minimum is for Docker Desktop. On Linux with native Docker, ensure the host has at least 12 GB of free memory.
----
+| Topology | Compose file | Services | When to use it |
+| --- | --- | --- | --- |
+| Standalone | `docker-compose.yml` | 1 RocksDB Server + 1 Hubble | Default; start here |
+| Minimal HStore | `docker-compose-hstore.yml` | 1 PD + 1 Store + 1 Server + 1 Hubble | Distributed local development |
+| HA | `docker-compose-3pd-3store-3server.yml` | 3 PD + 3 Store + 3 Server + 1 Hubble | Reference and evaluation |
-## Single-Node Setup
+Standalone uses `hugegraph/hugegraph:${HUGEGRAPH_VERSION:-latest}`. The HStore
+topologies use the matching `hugegraph/pd`, `hugegraph/store`, and
+`hugegraph/server` tags. Hubble is selected independently with
+`${HUBBLE_IMAGE:-hugegraph/hubble:latest}`.
-Two compose files run one PD, one Store, one Server, and one Hubble instance:
+### Create the authentication environment
-Create a Compose environment file once so every lifecycle command can resolve the required administrator password:
+Create `.env` once. Replace `replace-with-your-password` with an administrator
+password that you choose; the command generates and persists a random 32-byte
+JWT secret. For this simple single-quoted format, do not use a password that
+contains a single quote or newline.
```bash
(
set -eu
- cd docker
- if [ -e .env ]; then
- echo "docker/.env already exists; reusing it"
- else
- command -v openssl >/dev/null 2>&1
- admin_password="$(openssl rand -base64 12)"
- if [ "${#admin_password}" -ne 16 ]; then
- echo "Failed to generate a 16-character password" >&2
- exit 1
- fi
- install -m 600 /dev/null .env
- {
- printf "HUGEGRAPH_ADMIN_PASSWORD='%s'\n" "${admin_password}"
- } >> .env
- unset admin_password
- fi
- chmod 600 .env
- if ! env -u HUGEGRAPH_ADMIN_PASSWORD \
- docker compose -f docker-compose.yml config --quiet ||
- ! env -u HUGEGRAPH_ADMIN_PASSWORD \
- docker compose -f docker-compose.dev.yml config --quiet; then
- echo "docker/.env is incomplete; repair or move it, then retry" >&2
+ command -v openssl >/dev/null
+ jwt_secret="$(openssl rand -hex 32)"
+ test "${#jwt_secret}" -eq 64
+ umask 077
+ test ! -e .env || {
+ echo ".env already exists; edit it instead of overwriting it" >&2
exit 1
- fi
+ }
+ printf "HUGEGRAPH_ADMIN_PASSWORD='%s'\nHUGEGRAPH_AUTH_TOKEN_SECRET='%s'\n" \
+ 'replace-with-your-password' "${jwt_secret}" > .env
)
```
-Compose automatically reads `docker/.env` for `up`, `ps`, `stop`, and `down`. The generated password is a 16-character, Compose-safe random value. The file is excluded from Git and Docker build contexts; keep its permissions restricted and source production credentials from your secret manager instead of committing them.
+Do not commit `.env`. Keeping the same JWT secret preserves authentication
+tokens when containers are recreated. For authenticated topologies with
+multiple Server replicas, all replicas receive this same secret. The HA
+topology fails fast if authentication is enabled without this shared secret.
-### Option A: Quick Start (pre-built images)
+A non-empty `HUGEGRAPH_ADMIN_PASSWORD` enables Server authentication, and
+Hubble detects that mode automatically. Omitting the variable or setting it to
+an empty value disables authentication. Auth-off is only suitable for a
+trusted local environment; never expose it to a public or untrusted network.
+Hubble listens on host loopback by default. Set `HUBBLE_PUBLISH_HOST` only
+behind an HTTPS reverse proxy and trusted network controls.
-Uses pre-built images from Docker Hub. Best for **end users** who want to run HugeGraph quickly. Set `HUGEGRAPH_VERSION` to the same published release for PD, Store, Server, and Hubble. The authenticated PD/Hubble integration is not present in `1.7.x`; if no later compatible release is available, use Option B.
+`HUGEGRAPH_ADMIN_PASSWORD` initializes the built-in `admin` account on its
+first authenticated startup. Changing `.env` does not rotate an existing
+administrator password; use the HugeGraph user API for credential changes.
+
+For the verification commands below, set the password in your current shell:
```bash
-(
- cd docker
- HUGEGRAPH_VERSION='<compatible-release-after-1.7.x>' \
- docker compose up -d
-)
+ADMIN_PASSWORD='the-same-password-used-in-.env'
```
-- Images: matching `hugegraph/pd`, `hugegraph/store`, `hugegraph/server`, and `hugegraph/hubble` tags from the selected compatible release
-- `pull_policy: always` — always pulls the specified image tag
-
-> **Note**: Do not use `latest` to claim a reproducible deployment. Pin a compatible release tag and keep it unchanged for later lifecycle commands.
-- PD healthcheck endpoint: `/v1/health`
-- Hubble is available at `http://localhost:8088`; sign in as `admin` with the required `HUGEGRAPH_ADMIN_PASSWORD`
-- Hubble binds to host loopback by default. Set `HUBBLE_PUBLISH_HOST` explicitly only behind an HTTPS reverse proxy and trusted network controls.
-- Hubble uses PD discovery and the Docker-network Server address
-- Server healthcheck endpoint: `/versions`
+### Standalone
-### Option B: Development Build (build from source)
+This is the recommended quickstart.
-Builds images locally from source Dockerfiles. Best for **developers** who want to test local changes. Build the matching `hugegraph-toolchain` Hubble source as `local/hugegraph-hubble:dev` before starting this stack.
+Start:
-The publishing pipeline uses [`docker/bake.hcl`](./bake.hcl) from the repository root to compile the Java reactor once and build the PD, Store, HStore Server, and standalone Server runtime images from that shared result.
+```bash
+docker compose -f docker-compose.yml up -d --wait
+```
-Run Bake commands from the repository root. Use `--print` to inspect the resolved targets without building, or run the default group to build all four amd64/arm64 images. Loading both platforms under the same local tags requires Docker's containerd image store.
+Status:
```bash
-# Inspect the resolved build graph
-docker buildx bake --file docker/bake.hcl --print
+docker compose -f docker-compose.yml ps
+```
+
+Verify Server readiness, authentication, and Hubble:
-# Build the default multi-platform target group
-IMAGE_TAG=local docker buildx bake --file docker/bake.hcl
+```bash
+curl -fsS http://localhost:8080/versions
+test "$(curl -sS -o /dev/null -w '%{http_code}' \
+ http://localhost:8080/graphspaces/DEFAULT/graphs)" = 401
+test "$(curl -sS -u "admin:${ADMIN_PASSWORD}" -o /dev/null -w '%{http_code}' \
+ http://localhost:8080/graphspaces/DEFAULT/graphs)" = 200
+curl -fsS http://localhost:8088/about
```
-Local source changes are included because Bake uses the current repository working tree as its build context. For routine development, override all targets to the host architecture so the four images still share one Maven build without requiring the multi-platform containerd image store.
+Open `http://localhost:8088` and sign in as `admin` with the password from
+`.env`.
+
+Stop containers while keeping them:
```bash
-# x86_64 host
-IMAGE_TAG=local docker buildx bake --file docker/bake.hcl --set '*.platform=linux/amd64'
+docker compose -f docker-compose.yml stop
+```
-# ARM64 host
-IMAGE_TAG=local docker buildx bake --file docker/bake.hcl --set '*.platform=linux/arm64'
+Remove containers and the network while keeping data:
+
+```bash
+docker compose -f docker-compose.yml down
```
-These local commands can read existing Registry caches but do not publish images or write remote caches because `EXPORT_CACHE` defaults to `false`.
+Delete containers, the network, and all topology data:
```bash
-(
- cd docker
- HUBBLE_IMAGE=local/hugegraph-hubble:dev \
- HUBBLE_PULL_POLICY=never \
- docker compose -f docker-compose.dev.yml up -d
-)
+docker compose -f docker-compose.yml down -v
```
Review Comment:
**Do the lifecycle blocks need to be repeated per topology?**
This Status / Open Hubble / stop / down / `down -v` tail is byte identical in the Standalone, Minimal HStore and HA sections once the `-f` filename is normalised, and `down` versus `down -v` is restated a fourth time under Data persistence. That is roughly 78 lines saying one thing three times.
Since each Compose file declares its own project `name` (`hugegraph-standalone`, `hugegraph-hstore`, `hugegraph-3x3`), `-f <file>` alone fully scopes `ps`, `stop`, `down` and `down -v`, so a single shared Lifecycle subsection reading "use the same `-f` as your Start command" is exact rather than approximate. All volumes in all three renders are named and non external, so the data sentence is uniformly true too, and Hubble is `127.0.0.1:8088` everywhere, so "Open Hubble" only needs saying once.
Worth keeping per topology: Start and Verify, since the ports genuinely differ. Would you take that trade, or do you prefer each section to stay copy-paste complete?
##########
docker/docker-compose.yml:
##########
@@ -14,117 +14,59 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
-name: hugegraph-single
+
+name: hugegraph-standalone
networks:
hg-net:
- driver: bridge
volumes:
- hg-pd-data:
- hg-store-data:
+ server-data:
+ hubble-data:
services:
-
- pd:
- image: hugegraph/pd:${HUGEGRAPH_VERSION:-latest}
- pull_policy: always
- container_name: hg-pd
- hostname: pd
- restart: unless-stopped
- networks: [hg-net]
- environment:
- HG_PD_GRPC_HOST: pd
- HG_PD_GRPC_PORT: "8686"
- HG_PD_REST_PORT: "8620"
- HG_PD_RAFT_ADDRESS: pd:8610
- HG_PD_RAFT_PEERS_LIST: pd:8610
- HG_PD_INITIAL_STORE_LIST: store:8500
- HG_PD_DATA_PATH: /hugegraph-pd/pd_data
- ports:
- - "8620:8620"
- volumes:
- - hg-pd-data:/hugegraph-pd/pd_data
- healthcheck:
- test: ["CMD-SHELL", "curl -fsS http://localhost:8620/v1/health >/dev/null || exit 1"]
- interval: 10s
- timeout: 5s
- retries: 12
- start_period: 30s
-
- store:
- image: hugegraph/store:${HUGEGRAPH_VERSION:-latest}
- pull_policy: always
- container_name: hg-store
- hostname: store
- restart: unless-stopped
- networks: [hg-net]
- depends_on:
- pd:
- condition: service_healthy
- environment:
- HG_STORE_PD_ADDRESS: pd:8686
- HG_STORE_GRPC_HOST: store
- HG_STORE_GRPC_PORT: "8500"
- HG_STORE_REST_PORT: "8520"
- HG_STORE_RAFT_ADDRESS: store:8510
- HG_STORE_DATA_PATH: /hugegraph-store/storage
- ports:
- - "8520:8520"
- volumes:
- - hg-store-data:/hugegraph-store/storage
- healthcheck:
- test: ["CMD-SHELL", "curl -fsS http://localhost:8520/v1/health >/dev/null || exit 1"]
- interval: 10s
- timeout: 10s
- retries: 30
- start_period: 60s
-
server:
- image: ${HUGEGRAPH_SERVER_IMAGE:-hugegraph/server:${HUGEGRAPH_VERSION:-latest}}
- pull_policy: ${HUGEGRAPH_SERVER_PULL_POLICY:-always}
- container_name: hg-server
- hostname: server
+ image: hugegraph/hugegraph:${HUGEGRAPH_VERSION:-latest}
+ pull_policy: ${HUGEGRAPH_PULL_POLICY:-missing}
restart: unless-stopped
networks: [hg-net]
- depends_on:
- store:
- condition: service_healthy
environment:
- HG_SERVER_BACKEND: hstore
- HG_SERVER_PD_PEERS: pd:8686
- HG_SERVER_CLUSTER: hg
- HG_SERVER_USE_PD: "true"
- HG_SERVER_REST_URL: http://server:8080
- HG_SERVER_MIN_FREE_MEMORY: "0"
- HG_SERVER_INIT_STORE_ENABLED: "false"
+ PASSWORD: ${HUGEGRAPH_ADMIN_PASSWORD:-}
HG_SERVER_AUTH_TOKEN_SECRET: ${HUGEGRAPH_AUTH_TOKEN_SECRET:-}
- PASSWORD: ${HUGEGRAPH_ADMIN_PASSWORD:?Set a non-default admin password}
+ HG_SERVER_MIN_FREE_MEMORY: "0"
ports:
- "8080:8080"
+ volumes:
+ - server-data:/hugegraph-server/rocksdb-data
healthcheck:
- test: ["CMD-SHELL", "curl -fsS http://server:8080/versions >/dev/null || exit 1"]
+ test: ["CMD-SHELL", "curl -fsS http://localhost:8080/versions >/dev/null"]
interval: 10s
timeout: 5s
retries: 30
start_period: 60s
hubble:
- image: ${HUBBLE_IMAGE:-hugegraph/hubble:${HUGEGRAPH_VERSION:-latest}}
- pull_policy: ${HUBBLE_PULL_POLICY:-always}
- container_name: hg-hubble
- hostname: hubble
+ image: ${HUBBLE_IMAGE:-hugegraph/hubble:latest}
+ pull_policy: ${HUBBLE_PULL_POLICY:-missing}
restart: unless-stopped
networks: [hg-net]
depends_on:
server:
condition: service_healthy
+ environment:
+ SPRING_DATASOURCE_URL: jdbc:h2:file:/hubble/data/hubble;DB_CLOSE_ON_EXIT=FALSE
ports:
- "${HUBBLE_PUBLISH_HOST:-127.0.0.1}:8088:8088"
volumes:
- - ./hugegraph-hubble.properties:/hubble/conf/hugegraph-hubble.properties:ro
+ - hubble-data:/hubble/data
+ - ./conf/hubble/standalone.properties:/hubble/conf/hugegraph-hubble.properties:ro
healthcheck:
- test: ["CMD-SHELL", "body=$$(curl -fsS http://127.0.0.1:8088/about) && printf '%s' \"$$body\" | grep -q '\"status\":200' && printf '%s' \"$$body\" | grep -q '\"name\":\"hugegraph-hubble\"'"]
+ test:
+ - CMD-SHELL
+ - >-
+ body=$$(curl -fsS http://localhost:8088/about) &&
+ printf '%s' "$$body" | grep -q '"status":200' &&
+ printf '%s' "$$body" | grep -q '"name":"hugegraph-hubble"'
Review Comment:
**Is the body capture plus double grep doing anything a one line pipeline would not?**
This same 6 line block appears in all three Compose files. As far as I can tell it collapses to:
```suggestion
test: ["CMD-SHELL", "curl -fsS http://localhost:8088/about | grep -q '\"name\":\"hugegraph-hubble\"'"]
```
Gating looks equivalent: connection refused gives `grep` empty stdin and exit 1, `curl -f` suppresses bodies on HTTP >= 400, and an error envelope has no `"name"` field, so the same cases stay unhealthy. The `"status":200` grep is belt and braces on a static info endpoint, and `test-compose.sh` already asserts `.status == 200 and .data.name == "hugegraph-hubble"` against the live endpoint during smoke, so that check is not lost.
Both versions need only `curl` and `grep`, and the one liner is 108 chars, inside the `.editorconfig` 120 limit. Coupled edit: drop the `contains("\"status\":200")` line from `assert_hubble`, otherwise `render` fails. Applies identically at `docker-compose-hstore.yml:122-127` and `docker-compose-3pd-3store-3server.yml:247-252`.
##########
docker/docker-compose-3pd-3store-3server.yml:
##########
@@ -58,6 +59,18 @@ x-store-common: &store-common
retries: 40
start_period: 120s
+x-server-environment: &server-environment
+ STORE_REST: store0:8520
+ HG_SERVER_BACKEND: hstore
+ HG_SERVER_PD_PEERS: pd0:8686,pd1:8686,pd2:8686
+ HG_SERVER_CLUSTER: hg
+ HG_SERVER_USE_PD: "true"
+ HG_SERVER_MIN_FREE_MEMORY: "0"
+ HG_SERVER_INIT_STORE_ENABLED: "false"
+ HG_SERVER_REQUIRE_AUTH_TOKEN_SECRET: "true"
+ HG_SERVER_AUTH_TOKEN_SECRET: ${HUGEGRAPH_AUTH_TOKEN_SECRET:-}
+ PASSWORD: ${HUGEGRAPH_ADMIN_PASSWORD:-}
Review Comment:
**Follow-up thought, not a request against this PR:** `pd0/1/2` and `store0/1/2` further down could reuse this exact anchor pattern. They are untouched by this diff, so this belongs in a separate issue if you think it is worth doing at all.
Each `pd` repeats the same 6 env vars with only `HG_PD_GRPC_HOST` and `HG_PD_RAFT_ADDRESS` differing, and each `store` repeats 4 with the same two exceptions. An `x-pd-environment` and `x-store-environment` pair nets about 10 lines, 13 if the redundant `networks: [ hg-net ]` at lines 100, 118 and 136 go too, since `x-pd-common` already provides them. I applied it to a copy: `.services`, `.networks`, `.volumes` and `.name` hash identical, and `test-compose.sh render` still passes.
##########
docker/README.md:
##########
@@ -1,436 +1,362 @@
-# HugeGraph Docker Deployment
+# HugeGraph Docker Compose
-This directory contains Docker Compose files for running HugeGraph:
+## Users
-| File | Description |
-|------|-------------|
-| `docker-compose.yml` | PD, Store, Server, and Hubble using pre-built images |
-| `docker-compose.dev.yml` | PD, Store, and Server built from source, plus Hubble |
-| `docker-compose-3pd-3store-3server.yml` | 3-node distributed cluster (PD + Store + Server) |
+### Choose a topology
-## Prerequisites
+Run commands in this directory:
-- **Docker Engine** 20.10+ (or Docker Desktop 4.x+)
-- **Docker Compose** v2 (included in Docker Desktop)
-- **OpenSSL CLI** (used to generate the initial administrator password)
-- **Memory**: Allocate at least **12 GB** to Docker Desktop (Settings → Resources → Memory). The 3-node cluster runs 9 JVM processes (3 PD + 3 Store + 3 Server) which are memory-intensive. Insufficient memory causes OOM kills that appear as silent Raft failures.
+```bash
+cd docker
+```
-> [!IMPORTANT]
-> The 12 GB minimum is for Docker Desktop. On Linux with native Docker, ensure the host has at least 12 GB of free memory.
----
+| Topology | Compose file | Services | When to use it |
+| --- | --- | --- | --- |
+| Standalone | `docker-compose.yml` | 1 RocksDB Server + 1 Hubble | Default; start here |
+| Minimal HStore | `docker-compose-hstore.yml` | 1 PD + 1 Store + 1 Server + 1 Hubble | Distributed local development |
+| HA | `docker-compose-3pd-3store-3server.yml` | 3 PD + 3 Store + 3 Server + 1 Hubble | Reference and evaluation |
-## Single-Node Setup
+Standalone uses `hugegraph/hugegraph:${HUGEGRAPH_VERSION:-latest}`. The HStore
+topologies use the matching `hugegraph/pd`, `hugegraph/store`, and
+`hugegraph/server` tags. Hubble is selected independently with
+`${HUBBLE_IMAGE:-hugegraph/hubble:latest}`.
-Two compose files run one PD, one Store, one Server, and one Hubble instance:
+### Create the authentication environment
-Create a Compose environment file once so every lifecycle command can resolve the required administrator password:
+Create `.env` once. Replace `replace-with-your-password` with an administrator
+password that you choose; the command generates and persists a random 32-byte
+JWT secret. For this simple single-quoted format, do not use a password that
+contains a single quote or newline.
```bash
(
set -eu
- cd docker
- if [ -e .env ]; then
- echo "docker/.env already exists; reusing it"
- else
- command -v openssl >/dev/null 2>&1
- admin_password="$(openssl rand -base64 12)"
- if [ "${#admin_password}" -ne 16 ]; then
- echo "Failed to generate a 16-character password" >&2
- exit 1
- fi
- install -m 600 /dev/null .env
- {
- printf "HUGEGRAPH_ADMIN_PASSWORD='%s'\n" "${admin_password}"
- } >> .env
- unset admin_password
- fi
- chmod 600 .env
- if ! env -u HUGEGRAPH_ADMIN_PASSWORD \
- docker compose -f docker-compose.yml config --quiet ||
- ! env -u HUGEGRAPH_ADMIN_PASSWORD \
- docker compose -f docker-compose.dev.yml config --quiet; then
- echo "docker/.env is incomplete; repair or move it, then retry" >&2
+ command -v openssl >/dev/null
+ jwt_secret="$(openssl rand -hex 32)"
+ test "${#jwt_secret}" -eq 64
Review Comment:
**Are lines 34 and 36 doing anything under `set -eu`?** Line 35 is the one that matters.
- If `openssl` is missing or fails, `jwt_secret="$(openssl rand -hex 32)"` already exits the subshell non-zero. I checked in bash, zsh, sh and dash with `openssl` off `PATH`: exit 127, no `.env` written, and the shell prints a command-not-found error, which is better diagnostics than the silent `set -e` failure of the guard line.
- `openssl rand -hex 32` cannot exit 0 while printing anything other than 64 hex chars. An `openssl` old enough to lack `-hex` exits non-zero with empty stdout, so the length test never runs on a bad value anyway.
The protections that matter are untouched: `umask 077` before the write, the no-clobber guard, and the single quoted `printf`. The entrypoint's own `>= 32` byte check at `docker-entrypoint.sh:95` still backstops the value.
Not a blocker, just two lines of ceremony in the one block every user copies.
##########
docker/test-compose.sh:
##########
@@ -0,0 +1,403 @@
+#!/usr/bin/env bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+set -Eeuo pipefail
+
+DOCKER_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+PASSWORD="ci-compose-password"
+SECRET="0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"
+VERSION="ci-version"
+RENDER_HUBBLE_IMAGE="example.invalid/hugegraph/hubble:ci"
+DATASOURCE="jdbc:h2:file:/hubble/data/hubble;DB_CLOSE_ON_EXIT=FALSE"
+CURL_TIMEOUTS=(--connect-timeout 5 --max-time 15)
+ACTIVE_PROJECT=""
+ACTIVE_FILES=()
+RENDER_DIR=""
+
+compose_auth() {
+ env HUGEGRAPH_VERSION="${VERSION}" \
+ HUBBLE_IMAGE="${RENDER_HUBBLE_IMAGE}" \
+ HUGEGRAPH_ADMIN_PASSWORD="${PASSWORD}" \
+ HUGEGRAPH_AUTH_TOKEN_SECRET="${SECRET}" \
+ docker compose "$@"
+}
+
+render() {
+ local output="$1"
+ shift
+ compose_auth "$@" config --format json > "${output}"
+}
+
+assert_file_property() {
+ local file="$1"
+ local property="$2"
+ grep -Fqx "${property}" "${file}"
+}
+
+assert_common() {
+ local rendered="$1"
+ local services="$2"
+ local volumes="$3"
+
+ jq -e \
+ --arg password "${PASSWORD}" \
+ --arg secret "${SECRET}" \
Review Comment:
**A correction I owe you here, rather than a request.**
I came in expecting to argue that most of these jq contracts are change detectors that the smoke stage already proves behaviourally. I mutation-tested that and it does not hold, so recording the result instead:
- Deleting the inner `<<: *server-environment` from `server0` collapses its env to just `HG_SERVER_REST_URL`, silently dropping `PASSWORD` and the JWT secret. Only `assert_common`'s per-`server*` assertion catches it, and HA is never smoked.
- Hardcoding `image: hugegraph/server:latest` in the hstore file passes smoke, because CI resolves `${HUGEGRAPH_VERSION:-latest}` to `latest` anyway. So `VERSION="ci-version"` and `RENDER_HUBBLE_IMAGE` are the only proof that interpolation still reaches each image field.
- Typing `server:` as `sever:` in the dev overlay still passes `config -q` on the pair: the real service reverts to the base image while an orphan `sever` gets `:dev`. Only `assert_dev_override` notices.
- Changing `pd.peers=pd:8686` to `pd:9999` passes a hostname derived check, so the literal `assert_file_property` greps earn their keep on the port dimension.
So no change requested here. The only part my redundancy argument survives for is roughly 20 lines of assertion the smoke stage re-proves each run on the two smoked topologies, and since `assert_common` is shared with HA and the dev overlay, carving those out would mean splitting the helper, which adds code. Not worth it.
One related note: `smoke-auth-off` looks worth keeping. It never runs in CI so it reads like dead code at first, but the README documents it as a required local check with a stated trust boundary reason, and it is the only verification of the entrypoint's non-auth fork.
##########
docker/README.md:
##########
@@ -1,436 +1,362 @@
-# HugeGraph Docker Deployment
+# HugeGraph Docker Compose
-This directory contains Docker Compose files for running HugeGraph:
+## Users
-| File | Description |
-|------|-------------|
-| `docker-compose.yml` | PD, Store, Server, and Hubble using pre-built images |
-| `docker-compose.dev.yml` | PD, Store, and Server built from source, plus Hubble |
-| `docker-compose-3pd-3store-3server.yml` | 3-node distributed cluster (PD + Store + Server) |
+### Choose a topology
-## Prerequisites
+Run commands in this directory:
-- **Docker Engine** 20.10+ (or Docker Desktop 4.x+)
-- **Docker Compose** v2 (included in Docker Desktop)
-- **OpenSSL CLI** (used to generate the initial administrator password)
-- **Memory**: Allocate at least **12 GB** to Docker Desktop (Settings → Resources → Memory). The 3-node cluster runs 9 JVM processes (3 PD + 3 Store + 3 Server) which are memory-intensive. Insufficient memory causes OOM kills that appear as silent Raft failures.
+```bash
+cd docker
+```
-> [!IMPORTANT]
-> The 12 GB minimum is for Docker Desktop. On Linux with native Docker, ensure the host has at least 12 GB of free memory.
----
+| Topology | Compose file | Services | When to use it |
+| --- | --- | --- | --- |
+| Standalone | `docker-compose.yml` | 1 RocksDB Server + 1 Hubble | Default; start here |
+| Minimal HStore | `docker-compose-hstore.yml` | 1 PD + 1 Store + 1 Server + 1 Hubble | Distributed local development |
+| HA | `docker-compose-3pd-3store-3server.yml` | 3 PD + 3 Store + 3 Server + 1 Hubble | Reference and evaluation |
-## Single-Node Setup
+Standalone uses `hugegraph/hugegraph:${HUGEGRAPH_VERSION:-latest}`. The HStore
+topologies use the matching `hugegraph/pd`, `hugegraph/store`, and
+`hugegraph/server` tags. Hubble is selected independently with
+`${HUBBLE_IMAGE:-hugegraph/hubble:latest}`.
-Two compose files run one PD, one Store, one Server, and one Hubble instance:
+### Create the authentication environment
-Create a Compose environment file once so every lifecycle command can resolve the required administrator password:
+Create `.env` once. Replace `replace-with-your-password` with an administrator
+password that you choose; the command generates and persists a random 32-byte
+JWT secret. For this simple single-quoted format, do not use a password that
+contains a single quote or newline.
```bash
(
set -eu
- cd docker
- if [ -e .env ]; then
- echo "docker/.env already exists; reusing it"
- else
- command -v openssl >/dev/null 2>&1
- admin_password="$(openssl rand -base64 12)"
- if [ "${#admin_password}" -ne 16 ]; then
- echo "Failed to generate a 16-character password" >&2
- exit 1
- fi
- install -m 600 /dev/null .env
- {
- printf "HUGEGRAPH_ADMIN_PASSWORD='%s'\n" "${admin_password}"
- } >> .env
- unset admin_password
- fi
- chmod 600 .env
- if ! env -u HUGEGRAPH_ADMIN_PASSWORD \
- docker compose -f docker-compose.yml config --quiet ||
- ! env -u HUGEGRAPH_ADMIN_PASSWORD \
- docker compose -f docker-compose.dev.yml config --quiet; then
- echo "docker/.env is incomplete; repair or move it, then retry" >&2
+ command -v openssl >/dev/null
+ jwt_secret="$(openssl rand -hex 32)"
+ test "${#jwt_secret}" -eq 64
+ umask 077
+ test ! -e .env || {
+ echo ".env already exists; edit it instead of overwriting it" >&2
exit 1
- fi
+ }
+ printf "HUGEGRAPH_ADMIN_PASSWORD='%s'\nHUGEGRAPH_AUTH_TOKEN_SECRET='%s'\n" \
+ 'replace-with-your-password' "${jwt_secret}" > .env
)
```
-Compose automatically reads `docker/.env` for `up`, `ps`, `stop`, and `down`. The generated password is a 16-character, Compose-safe random value. The file is excluded from Git and Docker build contexts; keep its permissions restricted and source production credentials from your secret manager instead of committing them.
+Do not commit `.env`. Keeping the same JWT secret preserves authentication
+tokens when containers are recreated. For authenticated topologies with
+multiple Server replicas, all replicas receive this same secret. The HA
+topology fails fast if authentication is enabled without this shared secret.
-### Option A: Quick Start (pre-built images)
+A non-empty `HUGEGRAPH_ADMIN_PASSWORD` enables Server authentication, and
+Hubble detects that mode automatically. Omitting the variable or setting it to
+an empty value disables authentication. Auth-off is only suitable for a
+trusted local environment; never expose it to a public or untrusted network.
+Hubble listens on host loopback by default. Set `HUBBLE_PUBLISH_HOST` only
+behind an HTTPS reverse proxy and trusted network controls.
-Uses pre-built images from Docker Hub. Best for **end users** who want to run HugeGraph quickly. Set `HUGEGRAPH_VERSION` to the same published release for PD, Store, Server, and Hubble. The authenticated PD/Hubble integration is not present in `1.7.x`; if no later compatible release is available, use Option B.
+`HUGEGRAPH_ADMIN_PASSWORD` initializes the built-in `admin` account on its
+first authenticated startup. Changing `.env` does not rotate an existing
+administrator password; use the HugeGraph user API for credential changes.
+
+For the verification commands below, set the password in your current shell:
```bash
-(
- cd docker
- HUGEGRAPH_VERSION='<compatible-release-after-1.7.x>' \
- docker compose up -d
-)
+ADMIN_PASSWORD='the-same-password-used-in-.env'
```
-- Images: matching `hugegraph/pd`, `hugegraph/store`, `hugegraph/server`, and `hugegraph/hubble` tags from the selected compatible release
-- `pull_policy: always` — always pulls the specified image tag
-
-> **Note**: Do not use `latest` to claim a reproducible deployment. Pin a compatible release tag and keep it unchanged for later lifecycle commands.
-- PD healthcheck endpoint: `/v1/health`
-- Hubble is available at `http://localhost:8088`; sign in as `admin` with the required `HUGEGRAPH_ADMIN_PASSWORD`
-- Hubble binds to host loopback by default. Set `HUBBLE_PUBLISH_HOST` explicitly only behind an HTTPS reverse proxy and trusted network controls.
-- Hubble uses PD discovery and the Docker-network Server address
-- Server healthcheck endpoint: `/versions`
+### Standalone
-### Option B: Development Build (build from source)
+This is the recommended quickstart.
-Builds images locally from source Dockerfiles. Best for **developers** who want to test local changes. Build the matching `hugegraph-toolchain` Hubble source as `local/hugegraph-hubble:dev` before starting this stack.
+Start:
-The publishing pipeline uses [`docker/bake.hcl`](./bake.hcl) from the repository root to compile the Java reactor once and build the PD, Store, HStore Server, and standalone Server runtime images from that shared result.
+```bash
+docker compose -f docker-compose.yml up -d --wait
+```
-Run Bake commands from the repository root. Use `--print` to inspect the resolved targets without building, or run the default group to build all four amd64/arm64 images. Loading both platforms under the same local tags requires Docker's containerd image store.
+Status:
```bash
-# Inspect the resolved build graph
-docker buildx bake --file docker/bake.hcl --print
+docker compose -f docker-compose.yml ps
+```
+
+Verify Server readiness, authentication, and Hubble:
-# Build the default multi-platform target group
-IMAGE_TAG=local docker buildx bake --file docker/bake.hcl
+```bash
+curl -fsS http://localhost:8080/versions
+test "$(curl -sS -o /dev/null -w '%{http_code}' \
+ http://localhost:8080/graphspaces/DEFAULT/graphs)" = 401
+test "$(curl -sS -u "admin:${ADMIN_PASSWORD}" -o /dev/null -w '%{http_code}' \
+ http://localhost:8080/graphspaces/DEFAULT/graphs)" = 200
+curl -fsS http://localhost:8088/about
```
-Local source changes are included because Bake uses the current repository working tree as its build context. For routine development, override all targets to the host architecture so the four images still share one Maven build without requiring the multi-platform containerd image store.
+Open `http://localhost:8088` and sign in as `admin` with the password from
+`.env`.
+
+Stop containers while keeping them:
```bash
-# x86_64 host
-IMAGE_TAG=local docker buildx bake --file docker/bake.hcl --set '*.platform=linux/amd64'
+docker compose -f docker-compose.yml stop
+```
-# ARM64 host
-IMAGE_TAG=local docker buildx bake --file docker/bake.hcl --set '*.platform=linux/arm64'
+Remove containers and the network while keeping data:
+
+```bash
+docker compose -f docker-compose.yml down
```
-These local commands can read existing Registry caches but do not publish images or write remote caches because `EXPORT_CACHE` defaults to `false`.
+Delete containers, the network, and all topology data:
```bash
-(
- cd docker
- HUBBLE_IMAGE=local/hugegraph-hubble:dev \
- HUBBLE_PULL_POLICY=never \
- docker compose -f docker-compose.dev.yml up -d
-)
+docker compose -f docker-compose.yml down -v
```
-- PD, Store, and Server images are built from this repository
-- Hubble uses `HUBBLE_IMAGE` because its source is in `hugegraph-toolchain`
-- Server entrypoint scripts are baked into the built image; Hubble mounts the Docker-local PD configuration
-- PD healthcheck endpoint: `/v1/health`
-- Otherwise identical env vars and structure to the quickstart file
+### Minimal HStore
-Use the same release tag for Option A lifecycle commands:
+Start:
```bash
-(
- cd docker
- export HUGEGRAPH_VERSION='<same-compatible-release>'
- docker compose ps
- docker compose stop
- docker compose down
-)
+docker compose -f docker-compose-hstore.yml up -d --wait
```
-Use the development Compose file for every Option B lifecycle command:
+Status:
```bash
-(
- cd docker
- docker compose -f docker-compose.dev.yml ps
- docker compose -f docker-compose.dev.yml stop
- docker compose -f docker-compose.dev.yml down
-)
+docker compose -f docker-compose-hstore.yml ps
```
-### Key Differences
-
-| | `docker-compose.yml` (quickstart) | `docker-compose.dev.yml` (dev build) |
-|---|---|---|
-| **Images** | Pull from Docker Hub | Build from source |
-| **Who it's for** | End users | Developers |
-| **Server pull_policy** | `always` | `build` |
-| **Hubble pull_policy** | `always` | `never` in the workflow above (`missing` in the Compose file by default) |
+Verify PD, Store, Server authentication, and Hubble:
-**Verify** (both options):
```bash
-curl http://localhost:8080/versions
+curl -fsS http://localhost:8620/v1/health
+curl -fsS http://localhost:8520/v1/health
+curl -fsS http://localhost:8080/versions
+test "$(curl -sS -o /dev/null -w '%{http_code}' \
+ http://localhost:8080/graphspaces/DEFAULT/graphs)" = 401
+test "$(curl -sS -u "admin:${ADMIN_PASSWORD}" -o /dev/null -w '%{http_code}' \
+ http://localhost:8080/graphspaces/DEFAULT/graphs)" = 200
curl -fsS http://localhost:8088/about
```
-To validate local images without Compose replacing them with remote `latest`:
+Open `http://localhost:8088` and sign in as `admin` with the password from
+`.env`.
+
+Stop containers while keeping them:
```bash
-(
- cd docker
- HUGEGRAPH_SERVER_IMAGE=local/hugegraph-server:test \
- HUGEGRAPH_SERVER_PULL_POLICY=never \
- HUBBLE_IMAGE=local/hugegraph-hubble:test \
- HUBBLE_PULL_POLICY=never \
- docker compose up -d --wait
-)
+docker compose -f docker-compose-hstore.yml stop
```
----
+Remove containers and the network while keeping data:
+
+```bash
+docker compose -f docker-compose-hstore.yml down
+```
-## 3-Node Cluster Quickstart
+Delete containers, the network, and all topology data:
```bash
-cd docker
-HUGEGRAPH_VERSION=1.7.0 docker compose -f docker-compose-3pd-3store-3server.yml up -d
+docker compose -f docker-compose-hstore.yml down -v
+```
-# To stop and remove all data volumes (clean restart)
-docker compose -f docker-compose-3pd-3store-3server.yml down -v
+### HA reference
+
+The HA topology is resource-intensive. Running it locally is not required on
+resource-constrained machines, but its Compose configuration must always render
+successfully. This PR validates HA by rendering and static review only; it does
+not start HA locally or in default CI.
+
+Start:
+
+```bash
+docker compose -f docker-compose-3pd-3store-3server.yml up -d --wait
```
-**Startup ordering** is enforced via `depends_on` with `condition: service_healthy`:
+Status:
-1. **PD nodes** start first and must pass healthchecks (`/v1/health`)
-2. **Store nodes** start after all PD nodes are healthy
-3. **Server nodes** start after all Store nodes are healthy
+```bash
+docker compose -f docker-compose-3pd-3store-3server.yml ps
+```
-This ensures PD and Store are healthy before the server starts. The server entrypoint still performs a best-effort partition wait after launch, so partition assignment may take a little longer.
+Verify all published PD, Store, and Server endpoints, Server authentication,
+and Hubble:
+
+```bash
+for port in 8620 8621 8622; do
+ curl -fsS "http://localhost:${port}/v1/health"
+done
+for port in 8520 8521 8522; do
+ curl -fsS "http://localhost:${port}/v1/health"
+done
+for port in 8080 8081 8082; do
+ curl -fsS "http://localhost:${port}/versions"
+ test "$(curl -sS -o /dev/null -w '%{http_code}' \
+ "http://localhost:${port}/graphspaces/DEFAULT/graphs")" = 401
+ test "$(curl -sS -u "admin:${ADMIN_PASSWORD}" -o /dev/null \
+ -w '%{http_code}' \
+ "http://localhost:${port}/graphspaces/DEFAULT/graphs")" = 200
+done
+curl -fsS http://localhost:8088/about
+```
-**Verify the cluster is healthy**:
+Open `http://localhost:8088` and sign in as `admin` with the password from
+`.env`.
+
+Stop containers while keeping them:
```bash
-# Check PD health
-curl http://localhost:8620/v1/health
+docker compose -f docker-compose-3pd-3store-3server.yml stop
+```
-# Check Store health
-curl http://localhost:8520/v1/health
+Remove containers and the network while keeping data:
-# Check Server (Graph API)
-curl http://localhost:8080/versions
+```bash
+docker compose -f docker-compose-3pd-3store-3server.yml down
+```
-# List registered stores via PD
-curl http://localhost:8620/v1/stores
+Delete containers, the network, and all topology data:
-# List partitions
-curl http://localhost:8620/v1/partitions
+```bash
+docker compose -f docker-compose-3pd-3store-3server.yml down -v
```
----
-
-## Environment Variable Reference
-
-Configuration is injected via environment variables. The old `docker/configs/application-pd*.yml` and `docker/configs/application-store*.yml` files are no longer used.
-
-### PD Environment Variables
-
-| Variable | Required | Default | Maps To (`application.yml`) | Description |
-|----------|----------|---------|-----------------------------|-------------|
-| `HG_PD_GRPC_HOST` | Yes | — | `grpc.host` | This node's hostname/IP for gRPC |
-| `HG_PD_RAFT_ADDRESS` | Yes | — | `raft.address` | This node's Raft address (e.g. `pd0:8610`) |
-| `HG_PD_RAFT_PEERS_LIST` | Yes | — | `raft.peers-list` | All PD peers (e.g. `pd0:8610,pd1:8610,pd2:8610`) |
-| `HG_PD_INITIAL_STORE_LIST` | Yes | — | `pd.initial-store-list` | Expected stores (e.g. `store0:8500,store1:8500,store2:8500`) |
-| `HG_PD_GRPC_PORT` | No | `8686` | `grpc.port` | gRPC server port |
-| `HG_PD_REST_PORT` | No | `8620` | `server.port` | REST API port |
-| `HG_PD_DATA_PATH` | No | `/hugegraph-pd/pd_data` | `pd.data-path` | Metadata storage path |
-| `HG_PD_INITIAL_STORE_COUNT` | No | `1` | `pd.initial-store-count` | Min stores for cluster availability |
-
-**Deprecated aliases** (still work but log a warning):
-
-| Deprecated | Use Instead |
-|------------|-------------|
-| `GRPC_HOST` | `HG_PD_GRPC_HOST` |
-| `RAFT_ADDRESS` | `HG_PD_RAFT_ADDRESS` |
-| `RAFT_PEERS` | `HG_PD_RAFT_PEERS_LIST` |
-| `PD_INITIAL_STORE_LIST` | `HG_PD_INITIAL_STORE_LIST` |
-
-### Store Environment Variables
-
-| Variable | Required | Default | Maps To (`application.yml`) | Description |
-|----------|----------|---------|-----------------------------|-------------|
-| `HG_STORE_PD_ADDRESS` | Yes | — | `pdserver.address` | PD gRPC addresses (e.g. `pd0:8686,pd1:8686,pd2:8686`) |
-| `HG_STORE_GRPC_HOST` | Yes | — | `grpc.host` | This node's hostname (e.g. `store0`) |
-| `HG_STORE_RAFT_ADDRESS` | Yes | — | `raft.address` | This node's Raft address (e.g. `store0:8510`) |
-| `HG_STORE_GRPC_PORT` | No | `8500` | `grpc.port` | gRPC server port |
-| `HG_STORE_REST_PORT` | No | `8520` | `server.port` | REST API port |
-| `HG_STORE_DATA_PATH` | No | `/hugegraph-store/storage` | `app.data-path` | Data storage path |
-
-**Deprecated aliases** (still work but log a warning):
-
-| Deprecated | Use Instead |
-|------------|-------------|
-| `PD_ADDRESS` | `HG_STORE_PD_ADDRESS` |
-| `GRPC_HOST` | `HG_STORE_GRPC_HOST` |
-| `RAFT_ADDRESS` | `HG_STORE_RAFT_ADDRESS` |
-
-### Server Environment Variables
-
-| Variable | Required | Default | Maps To | Description |
-|----------|----------|---------|-----------------------------|-------------|
-| `HG_SERVER_BACKEND` | Yes | — | `backend` in `hugegraph.properties` | Storage backend (e.g. `hstore`) |
-| `HG_SERVER_PD_PEERS` | Yes | — | `pd.peers` | PD cluster addresses (e.g. `pd0:8686,pd1:8686,pd2:8686`) |
-| `HG_SERVER_CLUSTER` | No | — | `cluster` in `rest-server.properties` | PD discovery application name; single-node Compose uses `hg` to match Hubble |
-| `HG_SERVER_USE_PD` | No | — | `usePD` in `rest-server.properties` | Enables Server PD registration and discovery |
-| `HG_SERVER_REST_URL` | No | — | `restserver.url` | Address registered with PD and used by clients |
-| `HG_SERVER_MIN_FREE_MEMORY` | No | — | `restserver.min_free_memory` | Minimum free-memory guard in MB; local Compose uses `0` |
-| `HG_SERVER_AUTH_TOKEN_SECRET` | No | generated in auth mode | `auth.token_secret` | Shared JWT secret for REST and embedded Gremlin authentication; explicit values must be at least 32 bytes |
-| `STORE_REST` | No | — | Used by `wait-partition.sh` | Store REST endpoint for partition verification (e.g. `store0:8520`) |
-| `PASSWORD` | No | — | Enables auth and sets `auth.admin_pa` | Initial administrator password; disabled init-store does not read it from stdin, but the entrypoint still applies it to the PD bootstrap path |
-| `HG_SERVER_INIT_STORE_ENABLED` | No | `true` | `init_store.enabled` in `rest-server.properties` | Set `false` in PD/HStore deployments so init-store skips local backend and admin initialization |
-
-> **The built-in authenticator with `HG_SERVER_INIT_STORE_ENABLED=false` requires `usePD=true` and an HStore-backed `auth.graph_store`, unless `auth.remote_url` delegates auth elsewhere.** With init-store skipped, the server creates the built-in admin in PD metadata, and only an HStore auth graph uses the PD-backed auth manager that can read that account. init-store exits non-zero when the combination is unusable, rather than leaving a server nobody can log in to. A custom `auth.authenticator` is exempt because it manages its own identities.
->
-> `docker/init_complete` is written by init-store itself, and only after it has initialized. A skipped run therefore records nothing, whether it was disabled by the variable or by the property in a mounted `rest-server.properties`, so a later re-enable is still able to initialize. The marker only short-circuits re-initialization: init-store runs on every container start, and a disabled one performs the fail-closed check above first, so a marker left by an earlier release or an earlier enabled run cannot bypass it.
->
-> The entrypoint maps **`PASSWORD` to `auth.admin_pa`** before init-store runs. A disabled init-store does not read the password from standard input, but the PD startup path uses the explicit `auth.admin_pa` value when it first creates the administrator. Changing it later does not rotate an existing password.
-
-The single-node Compose files also accept these deployment-level overrides:
-
-| Variable | Default | Description |
-|----------|---------|-------------|
-| `HUGEGRAPH_SERVER_IMAGE` | `hugegraph/server:<version>` | Complete Server image reference |
-| `HUGEGRAPH_SERVER_PULL_POLICY` | `always` (`build` for dev) | Server pull policy |
-| `HUBBLE_IMAGE` | `hugegraph/hubble:<version>` | Complete Hubble image reference |
-| `HUBBLE_PULL_POLICY` | `always` (`missing` for dev) | Hubble pull policy |
-| `HUBBLE_PUBLISH_HOST` | `127.0.0.1` | Hubble host bind address; remote access requires an HTTPS reverse proxy |
-| `HUGEGRAPH_ADMIN_PASSWORD` | required (`docker/.env`) | Initial admin password; no public default is provided |
-| `HUGEGRAPH_AUTH_TOKEN_SECRET` | generated | JWT signing secret; explicit values must be at least 32 bytes |
-
-When authentication is enabled and no token secret is supplied, the Server entrypoint generates a random secret and writes it to both authentication configurations. The value is reused on container restart while the container filesystem is preserved. To preserve tokens across container recreation, generate a compatible secret once and add it to the mode-600 `docker/.env`:
+### Select image versions
+
+Set a HugeGraph release for Server, PD, and Store without changing Hubble:
```bash
-(
- set -euo pipefail
- cd docker
- secret_pattern='^[[:space:]]*(export[[:space:]]+)?HUGEGRAPH_AUTH_TOKEN_SECRET[[:space:]]*='
- secret_count="$(grep -Ec "${secret_pattern}" .env || true)"
- case "${secret_count}" in
- 0)
- command -v openssl >/dev/null 2>&1
- token_secret="$(openssl rand -hex 32)"
- LC_ALL=C
- if (( ${#token_secret} != 64 )); then
- echo "Failed to generate a 64-character token secret" >&2
- exit 1
- fi
- printf "HUGEGRAPH_AUTH_TOKEN_SECRET='%s'\n" \
- "${token_secret}" >> .env
- unset token_secret
- echo "Generated HUGEGRAPH_AUTH_TOKEN_SECRET"
- ;;
- 1)
- token_secret="$(
- sed -nE \
- "s/${secret_pattern}'([^']*)'[[:space:]]*$/\\2/p" .env
- )"
- LC_ALL=C
- if (( ${#token_secret} < 32 )); then
- echo "Existing token secret must use the documented single-quoted" \
- "format and contain at least 32 bytes; .env was not changed" >&2
- exit 1
- fi
- unset token_secret
- echo "HUGEGRAPH_AUTH_TOKEN_SECRET already exists; reusing it"
- ;;
- *)
- echo "Duplicate HUGEGRAPH_AUTH_TOKEN_SECRET entries; repair .env" >&2
- exit 1
- ;;
- esac
- chmod 600 .env
-)
+HUGEGRAPH_VERSION=1.7.0 \
+docker compose -f docker-compose-hstore.yml up -d
+```
+
+Select Hubble independently:
+
+```bash
+HUBBLE_IMAGE=hugegraph/hubble:latest \
+docker compose -f docker-compose.yml up -d
```
-The entrypoint rejects shorter explicit values before changing either Server configuration file.
+The Hubble `latest` image is expected to work with HugeGraph Server 1.7 and
+Server `latest`; compatibility with versions older than 1.7 is not promised.
+Pin immutable image references when reproducibility is required.
-**Deprecated aliases** (still work but log a warning):
+### Data persistence
-| Deprecated | Use Instead |
-|------------|-------------|
-| `BACKEND` | `HG_SERVER_BACKEND` |
-| `PD_PEERS` | `HG_SERVER_PD_PEERS` |
+Each topology creates its own normal Compose network and named volumes. No
+network or volume needs to be created in advance.
----
+Standalone stores RocksDB data at `/hugegraph-server/rocksdb-data`. The HStore
+topologies keep PD and Store data in topology-local volumes. Hubble uses
+`jdbc:h2:file:/hubble/data/hubble;DB_CLOSE_ON_EXIT=FALSE` and stores uploaded
+files under `/hubble/data/upload-files`.
-## Port Reference
+`docker compose down` keeps named-volume data. `docker compose down -v`
+intentionally deletes it.
-The table below reflects the published host ports in `docker-compose-3pd-3store-3server.yml`. The single-node Compose file publishes `8620`, `8520`, `8080`, and Hubble `8088`; Hubble defaults to host loopback.
+## Developers
-| Service | Container Port | Host Port | Protocol | Purpose |
-|---------|---------------|-----------|----------|---------|
-| pd0 | 8620 | 8620 | HTTP | REST API |
-| pd0 | 8686 | 8686 | gRPC | PD gRPC |
-| pd0 | 8610 | — | TCP | Raft (internal only) |
-| pd1 | 8620 | 8621 | HTTP | REST API |
-| pd1 | 8686 | 8687 | gRPC | PD gRPC |
-| pd2 | 8620 | 8622 | HTTP | REST API |
-| pd2 | 8686 | 8688 | gRPC | PD gRPC |
-| store0 | 8500 | 8500 | gRPC | Store gRPC |
-| store0 | 8510 | 8510 | TCP | Raft |
-| store0 | 8520 | 8520 | HTTP | REST API |
-| store1 | 8500 | 8501 | gRPC | Store gRPC |
-| store1 | 8510 | 8511 | TCP | Raft |
-| store1 | 8520 | 8521 | HTTP | REST API |
-| store2 | 8500 | 8502 | gRPC | Store gRPC |
-| store2 | 8510 | 8512 | TCP | Raft |
-| store2 | 8520 | 8522 | HTTP | REST API |
-| server0 | 8080 | 8080 | HTTP | Graph API |
-| server1 | 8080 | 8081 | HTTP | Graph API |
-| server2 | 8080 | 8082 | HTTP | Graph API |
+### Images and Compose files
----
+| Image | Build file |
+| --- | --- |
+| `hugegraph/hugegraph` (standalone RocksDB Server) | `hugegraph-server/Dockerfile` |
+| `hugegraph/server` (HStore Server) | `hugegraph-server/Dockerfile-hstore` |
+| `hugegraph/pd` | `hugegraph-pd/Dockerfile` |
+| `hugegraph/store` | `hugegraph-store/Dockerfile` |
-## Healthcheck Endpoints
+Hubble is built from the separate HugeGraph Toolchain repository and is
+selected here with `HUBBLE_IMAGE`.
-| Service | Endpoint | Expected |
-|---------|----------|----------|
-| PD | `GET /v1/health` | `200 OK` |
-| Store | `GET /v1/health` | `200 OK` |
-| Server | `GET /versions` | `200 OK` with version JSON |
-| Hubble | `GET /about` | `200` JSON with Hubble name and version |
+The Compose mapping is intentionally small:
----
+- `docker-compose.yml` is the standalone user default.
+- `docker-compose-hstore.yml` is the minimal 1 PD + 1 Store + 1 Server base.
+- `docker-compose-3pd-3store-3server.yml` is the HA reference.
+- `docker-compose.dev.yml` is a thin source-build override for the minimal
+ HStore topology. It does not duplicate runtime services, networks, volumes,
+ health checks, or Hubble.
-## Troubleshooting
+Build and start the minimal topology from local source:
-### Containers Exiting or Restarting (OOM Kills)
+```bash
+docker compose \
+ -f docker-compose-hstore.yml \
+ -f docker-compose.dev.yml \
+ up -d --build --wait
+```
-**Symptom**: Containers exit with code 137, or restart loops. Raft logs show election timeouts.
+Use both files for every later lifecycle command, for example:
-**Cause**: Docker Desktop does not have enough memory. The 9 JVM processes require at least 12 GB.
+```bash
+docker compose \
+ -f docker-compose-hstore.yml \
+ -f docker-compose.dev.yml \
+ down
+```
-**Fix**: Docker Desktop → Settings → Resources → Memory → set to **12 GB** or higher. Restart Docker Desktop.
+The development overlay builds `hugegraph/pd:dev`, `hugegraph/store:dev`, and
+`hugegraph/server:dev`. To reuse those local images and a locally built Hubble
+without pulling replacements:
```bash
-# Check if containers were OOM killed
-docker inspect hg-pd0 | grep -i oom
-docker stats --no-stream
+HUGEGRAPH_VERSION=dev \
+HUGEGRAPH_PULL_POLICY=never \
+HUBBLE_IMAGE=local/hugegraph-hubble:test \
+HUBBLE_PULL_POLICY=never \
+docker compose -f docker-compose-hstore.yml up -d --wait
```
-### Raft Leader Election Failure
+### Hubble configuration
+
+The three small files under `conf/hubble/` contain only topology-specific
+discovery settings and container paths:
-**Symptom**: PD logs show repeated `Leader election timeout`. Store nodes cannot register.
+- `conf/hubble/standalone.properties` uses direct Server mode.
+- `conf/hubble/hstore.properties` uses one PD and one Store REST target.
+- `conf/hubble/hstore-ha.properties` uses all three PD peers and all three
+ allowed Store REST targets.
-**Cause**: PD nodes cannot reach each other on the Raft port (8610), or `HG_PD_RAFT_PEERS_LIST` is misconfigured.
+Hubble detects Server authentication through the Server API. Do not add an
+`auth.enabled` property or duplicate auth-on/auth-off configurations.
-**Fix**:
-1. Verify all PD containers are running: `docker compose -f docker-compose-3pd-3store-3server.yml ps`
-2. Check PD logs: `docker logs hg-pd0`
-3. Verify network connectivity: `docker exec hg-pd0 ping pd1`
-4. Ensure `HG_PD_RAFT_PEERS_LIST` is identical on all PD nodes
+### Render and smoke checks
-### Partition Assignment Not Completing
+Render every topology with auth-on inputs before submitting a change:
-**Symptom**: Server starts but graph operations fail. Store logs show `partition not found`.
+```bash
+for file in \
+ docker-compose.yml \
+ docker-compose-hstore.yml \
+ docker-compose-3pd-3store-3server.yml
+do
+ docker compose -f "${file}" config --quiet
+done
+docker compose \
+ -f docker-compose-hstore.yml \
+ -f docker-compose.dev.yml \
+ config --quiet
Review Comment:
**Can this loop just be `bash test-compose.sh render`?**
`run_render` in the same PR renders these exact four file combinations, plus the dev overlay on its own, with pinned auth-on env and the jq contracts on top. It is a strict superset of this block, and it is what CI runs, so the two cannot drift.
One thing I noticed while checking: in a clean checkout this loop does not do what the lead-in says. Every Compose file uses `:-` empty defaults for `HUGEGRAPH_ADMIN_PASSWORD` and `HUGEGRAPH_AUTH_TOKEN_SECRET`, so with no `.env` present it exits 0 having rendered auth-off, despite "Render every topology with auth-on inputs". With a user created `.env` it is auth-on. The script pins its inputs through OS env, which takes precedence over `.env`, so it is deterministic either way.
```suggestion
bash test-compose.sh render
```
`test-compose.sh smoke` a few lines below already needs `jq`, so this adds no new dependency for anyone following this section. The three line description of what smoke covers could shrink the same way.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(docker): add attachable Hubble Compose add-on for the 3-node cluster [hugegraph]
Posted by "bitflicker64 (via GitHub)" <gi...@apache.org>.
bitflicker64 commented on code in PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149#discussion_r3852385993
##########
docker/docker-compose-hubble.yml:
##########
@@ -0,0 +1,64 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+# Hubble add-on for the distributed cluster defined in
+# docker-compose-3pd-3store-3server.yml. Requires the pre-created cluster
+# network (docker network create hugegraph-net). See "Hubble for the
+# 3-Node Cluster" in docker/README.md for the attach and combined flows.
+
+networks:
+ hg-net:
+ external: true
+ name: ${HUGEGRAPH_NETWORK:-hugegraph-net}
+
+volumes:
+ hg-hubble-db:
+ # Explicit names so attach (-p hugegraph-hubble) and combined
+ # (project hugegraph-3x3) share the same physical volumes.
+ name: ${HUBBLE_DB_VOLUME:-hugegraph-hubble-db}
Review Comment:
These volumes are shared across both flows by fixed name but are not `external`, while the
equally shared network is. That asymmetry is the bug: `down -v` from either project
removes them. I checked this against Compose directly. A fixed-name non-external volume is
removed by `down -v` even from a different project that merely declares the same name, and
an `external: true` volume survives.
So leaving the attach flow with `down -v` silently destroys the combined flow's H2
database and uploads, which contradicts "Recreating the container does not discard that
state" in the README.
Either declare them `external` (pre-created like the network) or warn about `down -v`
explicitly. The current combination gives neither protection nor warning.
##########
docker/README.md:
##########
@@ -168,14 +175,142 @@ To validate local images without Compose replacing them with remote `latest`:
## 3-Node Cluster Quickstart
+The cluster and the Hubble add-on share one named Docker network so Hubble
+can attach to a running cluster without touching it. Treat that network as a
+trust boundary: only the Server layer performs real authentication. Store
+serves its control APIs with no credentials at all, and while PD's REST
+control APIs do require an `Authorization` header, PD only checks that the
+Basic-auth *user* is one of its internal service names and never validates
+the password — so any client on the network can read and drive them. Note
+that this stack depends on that behaviour: Hubble reaches PD as the service
+name `hubble` with an empty password, so tightening PD's credential check
+would also break Hubble's operations view. Because of that, PD and Store
+control-plane ports and the three Server REST ports bind to `127.0.0.1` by
+default, so an unauthenticated control plane is not reachable from outside the
+host. Publish them more widely only by setting `HUGEGRAPH_CONTROL_PLANE_HOST`
+or `HUGEGRAPH_SERVER_PUBLISH_HOST`, and only behind a network ACL or TLS
+terminator you control. Any container on the host can still join the network
+by declaring the well-known name. One-time setup:
+write the required credentials to a mode-600 `docker/.env` and create the
+network.
+The cluster file requires both credentials — the admin password enables
+authentication, and every Server replica must share one token secret so a
+token issued by any server validates on all of them. The `:?` guards fire
+on every Compose subcommand, including `down`.
+
+```bash
+(
+ set -eu
+ cd docker
+ command -v openssl >/dev/null 2>&1 || { echo "openssl not found" >&2; exit 1; }
+ [ -e .env ] || install -m 600 /dev/null .env
+ chmod 600 .env
+ # Keep appends on their own lines even if the file was hand-edited.
+ [ ! -s .env ] || [ -z "$(tail -c1 .env)" ] || printf '\n' >> .env
+ pat='^[[:space:]]*(export[[:space:]]+)?'
+ if ! grep -Eq "${pat}HUGEGRAPH_ADMIN_PASSWORD=" .env; then
+ admin_password="$(openssl rand -base64 12)"
+ printf "HUGEGRAPH_ADMIN_PASSWORD='%s'\n" "${admin_password}" >> .env
+ unset admin_password
+ fi
+ if ! grep -Eq "${pat}HUGEGRAPH_AUTH_TOKEN_SECRET=" .env; then
+ token_secret="$(openssl rand -hex 32)"
+ printf "HUGEGRAPH_AUTH_TOKEN_SECRET='%s'\n" "${token_secret}" >> .env
+ unset token_secret
+ fi
+ # Parse values as data. Do not source .env as shell: a value such as
+ # HUGEGRAPH_ADMIN_PASSWORD=$(...) would run on the operator host.
+ # The optional export group is capture 1; the quoted value is capture 2.
+ read_dotenv_value() {
+ key="$1"
+ key_pattern="^[[:space:]]*(export[[:space:]]+)?${key}="
+ count="$(grep -Ec "${key_pattern}" .env || true)"
+ [ "${count}" -eq 1 ] || return 1
+ sed -nE "s/${key_pattern}'([^']*)'[[:space:]]*$/\\2/p" .env
+ }
+ if ! admin_value="$(read_dotenv_value HUGEGRAPH_ADMIN_PASSWORD)"; then
+ echo "HUGEGRAPH_ADMIN_PASSWORD must use the documented single-quoted format" >&2
+ exit 1
+ fi
+ if ! token_value="$(read_dotenv_value HUGEGRAPH_AUTH_TOKEN_SECRET)"; then
+ echo "HUGEGRAPH_AUTH_TOKEN_SECRET must use the documented single-quoted format" >&2
+ exit 1
+ fi
+ [ -n "${admin_value}" ] || {
+ echo "HUGEGRAPH_ADMIN_PASSWORD must be non-empty in docker/.env" >&2
+ exit 1
+ }
+ [ "${#token_value}" -ge 32 ] || {
+ echo "HUGEGRAPH_AUTH_TOKEN_SECRET must be at least 32 bytes in docker/.env" >&2
+ exit 1
+ }
+ unset admin_value token_value
+ # The shared cluster network. To override the name, export
+ # HUGEGRAPH_NETWORK in this shell before running the block — a value
+ # in docker/.env is read by Compose, not by this script.
+ net="${HUGEGRAPH_NETWORK:-hugegraph-net}"
+ docker network inspect "${net}" >/dev/null 2>&1 ||
+ docker network create "${net}"
+ env -u HUGEGRAPH_ADMIN_PASSWORD -u HUGEGRAPH_AUTH_TOKEN_SECRET \
+ docker compose -f docker-compose-3pd-3store-3server.yml config --quiet
+)
+```
+
+Then start the cluster:
+
```bash
cd docker
-HUGEGRAPH_VERSION=1.7.0 docker compose -f docker-compose-3pd-3store-3server.yml up -d
+docker compose -f docker-compose-3pd-3store-3server.yml up -d
-# To stop and remove all data volumes (clean restart)
+# To stop and remove all data volumes (clean restart).
+# The external hugegraph-net network is intentionally left in place.
+# If the Hubble add-on is running, see "Hubble for the 3-Node Cluster"
+# for the teardown that matches how it was started.
docker compose -f docker-compose-3pd-3store-3server.yml down -v
```
+Pin a release by setting `HUGEGRAPH_VERSION` in `docker/.env` — the
+cluster, the Hubble add-on, and the single-node quickstart file all read
+it from there, so those versions cannot drift apart
+(`docker-compose.dev.yml` builds PD/Store/Server from source and defaults
+Hubble to `hugegraph/hubble:latest`; set `HUBBLE_IMAGE` to pin it).
+Unpinned, the images default to `latest`; note the authenticated PD/Hubble
+integration requires a release newer than `1.7.x`. On an older image the
+Server ignores `PASSWORD` and `HG_SERVER_AUTH_TOKEN_SECRET`, which would
+leave you an unauthenticated cluster. Two defaults keep that from passing
+unnoticed: the cluster and add-on use `pull_policy: always`, so a stale
+cached `latest` is refreshed on every `up -d`, and the Server healthcheck
+requires an unauthenticated graph request to return `401` and an
+authenticated one to return `200`. An image that does not enforce
+authentication therefore never reports healthy, and `up -d --wait` fails
+instead of handing you a false green.
+
+Each Server registers its `HG_SERVER_REST_URL` with PD. The defaults
+(`server0`, `server1`, `server2`) are Docker-network names, which is correct
+for Hubble and anything else attached to `hugegraph-net`, but they do not
+resolve from an unrelated host network. A PD-aware client running outside
+Docker will receive names it cannot reach. For that case, set
+`HUGEGRAPH_SERVER0_REST_URL`, `HUGEGRAPH_SERVER1_REST_URL`, and
Review Comment:
This override cannot produce a working setup for server1 and server2.
`HG_SERVER_REST_URL` becomes `restserver.url`, which is the address Grizzly binds to
(`RestServer.java`, `GrizzlyHttpServerFactory.createHttpServer(uri, ...)`), not only the
address registered with PD. So an address that resolves from an external client is either
not local to the container (BindException, crash loop) or, with split-horizon DNS, makes
the Server listen on a port the published mapping does not forward, since the mapping
keeps container port 8080 hard-coded (`:8081:8080`). The healthcheck then probes the same
URL from inside the container, so that second case reports healthy behind a dead
published port.
The separation you need already exists in the code: `GraphManager.loadServices` uses
`server.urls_to_pd` instead of `restserver.url`, but only when `server.deploy_in_k8s` is
true. Ungating that is the real fix. Until then this paragraph promises something the
stack cannot do.
Same applies to `docker-compose-3x3.non-auth.yml:56`, which probes
`$${HG_SERVER_REST_URL}/versions`.
##########
docker/docker-compose-3pd-3store-3server.yml:
##########
@@ -58,21 +65,39 @@ x-store-common: &store-common
retries: 40
start_period: 120s
+# Shared Server environment; each server node adds its own
+# HG_SERVER_REST_URL on top of this map.
+x-server-env: &server-env
+ STORE_REST: store0:8520
+ HG_SERVER_BACKEND: hstore
+ HG_SERVER_PD_PEERS: pd0:8686,pd1:8686,pd2:8686
+ # Register every Server replica with PD under one application name so
+ # PD-aware clients such as Hubble discover the cluster as one logical
+ # Server; `hg` matches the Hubble configuration.
+ HG_SERVER_CLUSTER: hg
+ HG_SERVER_USE_PD: "true"
+ HG_SERVER_MIN_FREE_MEMORY: "0"
+ HG_SERVER_INIT_STORE_ENABLED: "false"
+ # All replicas must share one token secret so a token issued by any
+ # server validates on every other server.
+ HG_SERVER_AUTH_TOKEN_SECRET: ${HUGEGRAPH_AUTH_TOKEN_SECRET:?Set a shared auth token secret}
+ PASSWORD: ${HUGEGRAPH_ADMIN_PASSWORD:?Set a non-default admin password}
+
x-server-common: &server-common
image: hugegraph/server:${HUGEGRAPH_VERSION:-latest}
- pull_policy: missing
+ pull_policy: always
restart: unless-stopped
networks: [hg-net]
depends_on:
store0: { condition: service_healthy }
store1: { condition: service_healthy }
store2: { condition: service_healthy }
- environment:
- STORE_REST: store0:8520
- HG_SERVER_BACKEND: hstore
- HG_SERVER_PD_PEERS: pd0:8686,pd1:8686,pd2:8686
healthcheck:
- test: ["CMD-SHELL", "curl -fsS http://localhost:8080/versions >/dev/null || exit 1"]
+ # Prove the image actually enforces authentication. Older cached images
+ # may accept PASSWORD but ignore it while still returning healthy on
+ # /versions, so readiness requires both 401 without credentials and 200
+ # with the configured admin password.
+ test: ["CMD-SHELL", "base=$${HG_SERVER_REST_URL}; unauth=$$(curl -s -o /dev/null -w '%{http_code}' \"$${base}/graphs/hugegraph/schema/vertexlabels\"); auth=$$(curl -s -o /dev/null -w '%{http_code}' -u \"admin:$${PASSWORD}\" \"$${base}/graphs/hugegraph/schema/vertexlabels\"); test \"$${unauth}\" = 401 && test \"$${auth}\" = 200"]
Review Comment:
Two problems with this healthcheck, both stemming from running an authenticated probe
every 10 seconds forever.
1. It ties liveness to the *initial* admin password. `PASSWORD` only seeds the admin on
first init (`GraphManager.initAdminUserIfNeeded` skips an existing admin, and the
variable table below says the same). Once an operator rotates the admin password
through the API, the authenticated probe returns 401 and all three Servers go
permanently unhealthy on a cluster that is working fine. Any later `up -d --wait`
fails, and so does any health-gated `depends_on` a user adds. It also assumes the
`hugegraph` graph exists for the lifetime of the deployment.
2. It puts the admin password in process argv every 10 seconds in all three Server
containers, where `ps` on the host or `/proc/*/cmdline` in the container exposes it.
The README's own verification snippet refuses to do exactly this and says why: passing
the credential through `--config` "keeps it out of argv, where `ps` would expose it".
Piping a config into `curl -K -` instead of using `-u` addresses the second point, and
keeps the credential out of argv because `printf` is a builtin in the image's `/bin/sh`.
For the first, proving the image enforces auth is a one-time property: a startup gate, or
the smoke test this PR already adds, fits it better than a permanent probe.
##########
docker/README.md:
##########
@@ -196,11 +331,268 @@ curl http://localhost:8520/v1/health
# Check Server (Graph API)
curl http://localhost:8080/versions
-# List registered stores via PD
-curl http://localhost:8620/v1/stores
+# Every PD endpoint except /v1/health, /actuator/* and /v1/prom/targets/*
+# needs an Authorization header. PD only checks that the Basic-auth user is
+# one of its internal service names, so the empty password below is enough
+# — and it grants the full PD control plane, writes included, not just these
+# reads. That is exactly why the shared network must be treated as a trust
+# boundary. Without the header PD answers with an exception body, not data.
+pd_auth="Authorization: Basic $(printf 'hubble:' | base64)"
+
+# List registered stores via PD (expect three, each "state":"Up")
+curl -H "${pd_auth}" http://localhost:8620/v1/stores
# List partitions
-curl http://localhost:8620/v1/partitions
+curl -H "${pd_auth}" http://localhost:8620/v1/partitions
+```
+
+Confirm authentication actually engaged — `/versions` stays open by design,
+so it cannot tell you whether auth is on. A graph read without credentials
+must be rejected:
+
+```bash
+cd docker
+# Expect 401 on all three replicas
+for port in 8080 8081 8082; do
+ curl -s -o /dev/null -w "${port}: %{http_code}\n" \
+ "http://localhost:${port}/graphs/hugegraph/schema/vertexlabels"
+done
+
+# And a signed-in read must succeed. Parse the generated single-quoted
+# value without executing docker/.env as shell. Passing the credential
+# through --config keeps it out of argv, where `ps` would expose it.
+read_dotenv_value() {
+ key="$1"
+ key_pattern="^[[:space:]]*(export[[:space:]]+)?${key}="
+ count="$(grep -Ec "${key_pattern}" .env || true)"
+ [ "${count}" -eq 1 ] || return 1
+ sed -nE "s/${key_pattern}'([^']*)'[[:space:]]*$/\\2/p" .env
+}
+HUGEGRAPH_ADMIN_PASSWORD="$(read_dotenv_value HUGEGRAPH_ADMIN_PASSWORD)" || {
+ echo "HUGEGRAPH_ADMIN_PASSWORD must use the documented single-quoted format" >&2
+ exit 1
+}
+curl -s -o /dev/null -w '%{http_code}\n' \
+ --config <(printf 'user = "admin:%s"\n' "${HUGEGRAPH_ADMIN_PASSWORD}") \
+ http://localhost:8080/graphs/hugegraph/schema/vertexlabels
+unset HUGEGRAPH_ADMIN_PASSWORD
+```
+
+`200` from the second command with `401` from all three of the first means
+authentication is on and working. If the first command returns `200`, the
+running image ignored `PASSWORD` and the cluster is **unauthenticated** —
+the most likely cause is an image older than the release this integration
+needs (see the version note in the quickstart above).
+
+### Running without authentication
+
+The cluster above authenticates, and that is the right default: it is the
+configuration most people copy, and an open graph database on a reachable
+port is a bad surprise. For a throwaway local trial on a trusted network you
+can opt out with `docker-compose-3x3.non-auth.yml`, which drops the admin
+password and the JWT secret from all three Servers. Anyone who can reach the
+published ports then has full read and write access to every graph, so do not
+use it anywhere else.
+
+```bash
+cd docker
+
+# No docker/.env and no credential setup are needed for this mode. The two
Review Comment:
Two gaps make this mode unusable as written, despite "no credential setup are needed":
1. The external network is never created in this flow. The only place that creates it
before you need it is the credentialed setup block this line tells the reader to skip,
so the documented `up -d` fails on a fresh host with `network hugegraph-net declared as
external, but could not be found`. The troubleshooting entry covers it, but only after
the failure.
2. No teardown is documented, and the base file's `:?` guards fire on every subcommand,
`down` included. A later `down` in a fresh shell therefore demands credentials this
mode says it does not need, and the section explicitly forbids the obvious workaround
of putting them in `.env`.
Adding the `docker network create` line and a teardown command carrying the same
placeholders would close both.
##########
docker/README.md:
##########
@@ -196,11 +331,268 @@ curl http://localhost:8520/v1/health
# Check Server (Graph API)
curl http://localhost:8080/versions
-# List registered stores via PD
-curl http://localhost:8620/v1/stores
+# Every PD endpoint except /v1/health, /actuator/* and /v1/prom/targets/*
+# needs an Authorization header. PD only checks that the Basic-auth user is
+# one of its internal service names, so the empty password below is enough
+# — and it grants the full PD control plane, writes included, not just these
+# reads. That is exactly why the shared network must be treated as a trust
+# boundary. Without the header PD answers with an exception body, not data.
+pd_auth="Authorization: Basic $(printf 'hubble:' | base64)"
+
+# List registered stores via PD (expect three, each "state":"Up")
+curl -H "${pd_auth}" http://localhost:8620/v1/stores
# List partitions
-curl http://localhost:8620/v1/partitions
+curl -H "${pd_auth}" http://localhost:8620/v1/partitions
+```
+
+Confirm authentication actually engaged — `/versions` stays open by design,
+so it cannot tell you whether auth is on. A graph read without credentials
+must be rejected:
+
+```bash
+cd docker
+# Expect 401 on all three replicas
+for port in 8080 8081 8082; do
+ curl -s -o /dev/null -w "${port}: %{http_code}\n" \
+ "http://localhost:${port}/graphs/hugegraph/schema/vertexlabels"
+done
+
+# And a signed-in read must succeed. Parse the generated single-quoted
+# value without executing docker/.env as shell. Passing the credential
+# through --config keeps it out of argv, where `ps` would expose it.
+read_dotenv_value() {
+ key="$1"
+ key_pattern="^[[:space:]]*(export[[:space:]]+)?${key}="
+ count="$(grep -Ec "${key_pattern}" .env || true)"
+ [ "${count}" -eq 1 ] || return 1
+ sed -nE "s/${key_pattern}'([^']*)'[[:space:]]*$/\\2/p" .env
+}
+HUGEGRAPH_ADMIN_PASSWORD="$(read_dotenv_value HUGEGRAPH_ADMIN_PASSWORD)" || {
Review Comment:
This guard does not fire in the case it is written for. When the key is present but not
single-quoted, `grep -Ec` still counts 1 and `sed` prints nothing while exiting 0, so
`read_dotenv_value` succeeds with an empty value and the `||` branch never runs. I
reproduced this in bash, zsh and dash: the block then curls with `admin:` and an empty
password, gets 401, and the interpretation text below sends the operator off to blame the
image.
The setup block avoids this only through its separate non-empty check, whose message is
itself misleading (it reports "must be non-empty" for a value that is non-empty but
unquoted).
Making `read_dotenv_value` fail when `sed` produces nothing fixes both blocks.
##########
docker/README.md:
##########
@@ -196,11 +331,268 @@ curl http://localhost:8520/v1/health
# Check Server (Graph API)
curl http://localhost:8080/versions
-# List registered stores via PD
-curl http://localhost:8620/v1/stores
+# Every PD endpoint except /v1/health, /actuator/* and /v1/prom/targets/*
+# needs an Authorization header. PD only checks that the Basic-auth user is
+# one of its internal service names, so the empty password below is enough
+# — and it grants the full PD control plane, writes included, not just these
+# reads. That is exactly why the shared network must be treated as a trust
+# boundary. Without the header PD answers with an exception body, not data.
+pd_auth="Authorization: Basic $(printf 'hubble:' | base64)"
+
+# List registered stores via PD (expect three, each "state":"Up")
+curl -H "${pd_auth}" http://localhost:8620/v1/stores
# List partitions
-curl http://localhost:8620/v1/partitions
+curl -H "${pd_auth}" http://localhost:8620/v1/partitions
+```
+
+Confirm authentication actually engaged — `/versions` stays open by design,
+so it cannot tell you whether auth is on. A graph read without credentials
+must be rejected:
+
+```bash
+cd docker
Review Comment:
Unlike the setup block, this one is not wrapped in a subshell but still calls `exit 1`
below. Pasting it into an interactive shell with a malformed `.env` (missing or duplicated
key) terminates the operator's session. I reproduced this in bash and zsh, interactive and
non-interactive; wrapping the same lines in `( ... )` as the setup block does fixes it.
##########
docker/docker-compose-3pd-3store-3server.yml:
##########
@@ -31,8 +36,10 @@ volumes:
# ── Shared service defaults ──────────────────────────────────────────
x-pd-common: &pd-common
+ # Pin a release via HUGEGRAPH_VERSION in docker/.env; unset, the image
+ # tags default to latest. All Compose files here read the same variable.
image: hugegraph/pd:${HUGEGRAPH_VERSION:-latest}
- pull_policy: missing
+ pull_policy: always
Review Comment:
`pull_policy: always` here has no override variable, unlike the add-on's
`HUBBLE_PULL_POLICY` and the single-node file's `HUGEGRAPH_SERVER_PULL_POLICY`. That makes
the cluster unusable offline: I confirmed against an unreachable registry that `up -d`
exits non-zero and starts nothing even when every image is already cached, because the
pull error is only tolerated for services that have a `build:` section. It also silently
replaces a locally built tag.
```suggestion
pull_policy: ${HUGEGRAPH_PULL_POLICY:-always}
```
The same edit is needed on the store and server anchors (lines 54 and 88), or the offline
case still fails. The default stays `always`, which is what keeps a stale cached `latest`
from satisfying an auth-sensitive deployment; this only adds a deliberate opt-out.
##########
.github/workflows/server-ci.yml:
##########
@@ -138,6 +140,322 @@ jobs:
check_compose docker/docker-compose.yml always always
check_compose docker/docker-compose.dev.yml build missing
+ check_cluster_compose() {
+ local cluster="docker/docker-compose-3pd-3store-3server.yml"
+ local addon="docker/docker-compose-hubble.yml"
+ local rendered
+ local token_fixture=ci-test-token-secret-32-bytes-long
+ rendered="$(mktemp)"
+ if [ "${#token_fixture}" -lt 32 ]; then
+ echo "CI token fixture must be at least 32 bytes" >&2
+ return 1
+ fi
+
+ # RETURN only: an EXIT trap would fire after this function's
+ # `local rendered` has gone out of scope, which `set -u` turns
+ # into an "unbound variable" error. A hard errexit abort can
+ # therefore still leak one temp file, which is acceptable on an
+ # ephemeral runner.
+ trap 'rm -f "$rendered"' RETURN
+
+ # --env-file /dev/null on every invocation: Compose otherwise reads
+ # docker/.env automatically, and the quickstart tells operators to
+ # create one holding exactly the credentials these assertions
+ # control. Without it the checks pass in CI (which has no .env) but
+ # report false failures for anyone running them locally after
+ # following the quickstart, and the add-on render below — which
+ # deliberately unsets the variables to assert their defaults — would
+ # read whatever HUGEGRAPH_NETWORK/HUGEGRAPH_VERSION that .env holds.
+ # Pinning an empty env file makes both renders depend only on what
+ # each invocation sets explicitly.
+
+ # Both cluster credentials are required and may not be empty.
+ # Each case asserts the guard fired for the *intended* variable:
+ # a bare non-zero exit would also be produced by a YAML error, a
+ # renamed file, or a missing docker binary.
+ assert_guard() { # assert_guard <blamed-var> <description>
+ local var="$1" desc="$2" err
+ if err="$(docker compose --env-file /dev/null -f "$cluster" \
+ config -q 2>&1)"; then
+ echo "$cluster accepted $desc" >&2
+ return 1
+ fi
+ case "$err" in
+ *"$var"*) : ;;
+ *) echo "$cluster rejected $desc, but not because of $var: $err" >&2
+ return 1 ;;
+ esac
+ }
+
+ ( unset HUGEGRAPH_ADMIN_PASSWORD
+ export HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}"
+ assert_guard HUGEGRAPH_ADMIN_PASSWORD "an unset admin password" )
+ ( export HUGEGRAPH_ADMIN_PASSWORD=
+ export HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}"
+ assert_guard HUGEGRAPH_ADMIN_PASSWORD "an empty admin password" )
+ ( unset HUGEGRAPH_AUTH_TOKEN_SECRET
+ export HUGEGRAPH_ADMIN_PASSWORD=ci-test-password
+ assert_guard HUGEGRAPH_AUTH_TOKEN_SECRET "an unset token secret" )
+ ( export HUGEGRAPH_ADMIN_PASSWORD=ci-test-password
+ export HUGEGRAPH_AUTH_TOKEN_SECRET=
+ assert_guard HUGEGRAPH_AUTH_TOKEN_SECRET "an empty token secret" )
+ # The add-on alone must define Hubble and nothing else, join the
+ # shared external network with its default name, and need no
+ # credentials or overrides.
+ env -u HUGEGRAPH_ADMIN_PASSWORD -u HUGEGRAPH_AUTH_TOKEN_SECRET \
+ -u HUGEGRAPH_NETWORK -u HUGEGRAPH_VERSION \
+ -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY -u HUBBLE_PUBLISH_HOST \
+ -u HUBBLE_DB_VOLUME -u HUBBLE_UPLOAD_VOLUME \
+ docker compose --env-file /dev/null -f "$addon" config --format json > "$rendered"
+ jq -e '
+ (.services | keys) == ["hubble"] and
+ .networks."hg-net".external == true and
+ .networks."hg-net".name == "hugegraph-net" and
+ (.services.hubble.networks | has("hg-net")) and
+ (.services.hubble | has("depends_on") | not) and
+ .volumes."hg-hubble-db".name == "hugegraph-hubble-db" and
+ .volumes."hg-hubble-upload-files".name ==
+ "hugegraph-hubble-upload-files" and
+ .services.hubble.environment.SPRING_DATASOURCE_URL ==
+ "jdbc:h2:file:./db/hubble;DB_CLOSE_ON_EXIT=FALSE" and
+ any(.services.hubble.volumes[];
+ .target == "/hubble/conf/hugegraph-hubble.properties" and
+ (.source | endswith("hugegraph-hubble-3x3.properties")))
+ and any(.services.hubble.volumes[];
+ .source == "hg-hubble-db" and
+ .target == "/hubble/db")
+ and any(.services.hubble.volumes[];
+ .source == "hg-hubble-upload-files" and
+ .target == "/hubble/upload-files")
+ ' "$rendered" >/dev/null
+
+ # The cluster's own default network name must match the add-on's,
+ # or the attach flow and the cluster land on different networks.
+ # The combined render below pins an override, so it cannot catch a
+ # drifting default.
+ HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}" \
+ env -u HUGEGRAPH_NETWORK -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY \
+ -u HUBBLE_PUBLISH_HOST \
+ docker compose --env-file /dev/null -f "$cluster" \
+ config --format json > "$rendered"
+ jq -e '.networks."hg-net".name == "hugegraph-net"' \
+ "$rendered" >/dev/null
+
+ # The combined render carries the PD-registration and auth
+ # settings on every server replica and keeps Hubble on loopback.
+ # Rendered with non-default HUGEGRAPH_NETWORK/HUGEGRAPH_VERSION so
+ # CI fails if any file stops honoring the overrides (the add-on
+ # render above covers the defaults).
+ HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}" \
+ HUGEGRAPH_NETWORK=ci-test-net \
+ HUGEGRAPH_VERSION=ci-test-tag \
+ env -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY -u HUBBLE_PUBLISH_HOST \
Review Comment:
These renders assert defaults for variables they never unset, which reintroduces exactly
the local false-failure the comment above says the design prevents. `--env-file /dev/null`
stops Compose reading `docker/.env`, but an exported shell variable still wins.
Confirmed by rendering the real files: with `HUGEGRAPH_SERVER_PUBLISH_HOST=0.0.0.0`
exported, the `host_ip == "127.0.0.1"` assertion fails; same for
`HUGEGRAPH_CONTROL_PLANE_HOST`, for `HUGEGRAPH_SERVERn_REST_URL` against the
`http://serverN:8080` assertions, and for `HUBBLE_PROPERTIES` against the add-on render's
properties-path assertion. These are all variables the README tells operators to export.
Adding them to the two `env -u` lists is enough.
##########
docker/README.md:
##########
@@ -168,14 +175,142 @@ To validate local images without Compose replacing them with remote `latest`:
## 3-Node Cluster Quickstart
+The cluster and the Hubble add-on share one named Docker network so Hubble
+can attach to a running cluster without touching it. Treat that network as a
+trust boundary: only the Server layer performs real authentication. Store
+serves its control APIs with no credentials at all, and while PD's REST
+control APIs do require an `Authorization` header, PD only checks that the
+Basic-auth *user* is one of its internal service names and never validates
+the password — so any client on the network can read and drive them. Note
+that this stack depends on that behaviour: Hubble reaches PD as the service
+name `hubble` with an empty password, so tightening PD's credential check
+would also break Hubble's operations view. Because of that, PD and Store
+control-plane ports and the three Server REST ports bind to `127.0.0.1` by
+default, so an unauthenticated control plane is not reachable from outside the
+host. Publish them more widely only by setting `HUGEGRAPH_CONTROL_PLANE_HOST`
+or `HUGEGRAPH_SERVER_PUBLISH_HOST`, and only behind a network ACL or TLS
+terminator you control. Any container on the host can still join the network
+by declaring the well-known name. One-time setup:
+write the required credentials to a mode-600 `docker/.env` and create the
+network.
+The cluster file requires both credentials — the admin password enables
+authentication, and every Server replica must share one token secret so a
+token issued by any server validates on all of them. The `:?` guards fire
+on every Compose subcommand, including `down`.
+
+```bash
+(
+ set -eu
+ cd docker
+ command -v openssl >/dev/null 2>&1 || { echo "openssl not found" >&2; exit 1; }
+ [ -e .env ] || install -m 600 /dev/null .env
+ chmod 600 .env
+ # Keep appends on their own lines even if the file was hand-edited.
+ [ ! -s .env ] || [ -z "$(tail -c1 .env)" ] || printf '\n' >> .env
+ pat='^[[:space:]]*(export[[:space:]]+)?'
+ if ! grep -Eq "${pat}HUGEGRAPH_ADMIN_PASSWORD=" .env; then
+ admin_password="$(openssl rand -base64 12)"
+ printf "HUGEGRAPH_ADMIN_PASSWORD='%s'\n" "${admin_password}" >> .env
+ unset admin_password
+ fi
+ if ! grep -Eq "${pat}HUGEGRAPH_AUTH_TOKEN_SECRET=" .env; then
+ token_secret="$(openssl rand -hex 32)"
+ printf "HUGEGRAPH_AUTH_TOKEN_SECRET='%s'\n" "${token_secret}" >> .env
+ unset token_secret
+ fi
+ # Parse values as data. Do not source .env as shell: a value such as
+ # HUGEGRAPH_ADMIN_PASSWORD=$(...) would run on the operator host.
+ # The optional export group is capture 1; the quoted value is capture 2.
+ read_dotenv_value() {
+ key="$1"
+ key_pattern="^[[:space:]]*(export[[:space:]]+)?${key}="
+ count="$(grep -Ec "${key_pattern}" .env || true)"
+ [ "${count}" -eq 1 ] || return 1
+ sed -nE "s/${key_pattern}'([^']*)'[[:space:]]*$/\\2/p" .env
+ }
+ if ! admin_value="$(read_dotenv_value HUGEGRAPH_ADMIN_PASSWORD)"; then
+ echo "HUGEGRAPH_ADMIN_PASSWORD must use the documented single-quoted format" >&2
+ exit 1
+ fi
+ if ! token_value="$(read_dotenv_value HUGEGRAPH_AUTH_TOKEN_SECRET)"; then
+ echo "HUGEGRAPH_AUTH_TOKEN_SECRET must use the documented single-quoted format" >&2
+ exit 1
+ fi
+ [ -n "${admin_value}" ] || {
+ echo "HUGEGRAPH_ADMIN_PASSWORD must be non-empty in docker/.env" >&2
+ exit 1
+ }
+ [ "${#token_value}" -ge 32 ] || {
+ echo "HUGEGRAPH_AUTH_TOKEN_SECRET must be at least 32 bytes in docker/.env" >&2
+ exit 1
+ }
+ unset admin_value token_value
+ # The shared cluster network. To override the name, export
+ # HUGEGRAPH_NETWORK in this shell before running the block — a value
+ # in docker/.env is read by Compose, not by this script.
+ net="${HUGEGRAPH_NETWORK:-hugegraph-net}"
+ docker network inspect "${net}" >/dev/null 2>&1 ||
+ docker network create "${net}"
+ env -u HUGEGRAPH_ADMIN_PASSWORD -u HUGEGRAPH_AUTH_TOKEN_SECRET \
+ docker compose -f docker-compose-3pd-3store-3server.yml config --quiet
+)
+```
+
+Then start the cluster:
+
```bash
cd docker
-HUGEGRAPH_VERSION=1.7.0 docker compose -f docker-compose-3pd-3store-3server.yml up -d
+docker compose -f docker-compose-3pd-3store-3server.yml up -d
-# To stop and remove all data volumes (clean restart)
+# To stop and remove all data volumes (clean restart).
+# The external hugegraph-net network is intentionally left in place.
+# If the Hubble add-on is running, see "Hubble for the 3-Node Cluster"
+# for the teardown that matches how it was started.
docker compose -f docker-compose-3pd-3store-3server.yml down -v
```
+Pin a release by setting `HUGEGRAPH_VERSION` in `docker/.env` — the
+cluster, the Hubble add-on, and the single-node quickstart file all read
+it from there, so those versions cannot drift apart
+(`docker-compose.dev.yml` builds PD/Store/Server from source and defaults
+Hubble to `hugegraph/hubble:latest`; set `HUBBLE_IMAGE` to pin it).
+Unpinned, the images default to `latest`; note the authenticated PD/Hubble
+integration requires a release newer than `1.7.x`. On an older image the
+Server ignores `PASSWORD` and `HG_SERVER_AUTH_TOKEN_SECRET`, which would
+leave you an unauthenticated cluster. Two defaults keep that from passing
+unnoticed: the cluster and add-on use `pull_policy: always`, so a stale
+cached `latest` is refreshed on every `up -d`, and the Server healthcheck
+requires an unauthenticated graph request to return `401` and an
+authenticated one to return `200`. An image that does not enforce
+authentication therefore never reports healthy, and `up -d --wait` fails
+instead of handing you a false green.
+
+Each Server registers its `HG_SERVER_REST_URL` with PD. The defaults
+(`server0`, `server1`, `server2`) are Docker-network names, which is correct
+for Hubble and anything else attached to `hugegraph-net`, but they do not
+resolve from an unrelated host network. A PD-aware client running outside
+Docker will receive names it cannot reach. For that case, set
+`HUGEGRAPH_SERVER0_REST_URL`, `HUGEGRAPH_SERVER1_REST_URL`, and
+`HUGEGRAPH_SERVER2_REST_URL` to addresses that resolve from both the Server
+containers and that client, and publish the matching host ports with
+`HUGEGRAPH_SERVER_PUBLISH_HOST`.
+
+> [!NOTE]
+> Upgrading an existing 3-node deployment:
+> - Create `docker/.env` (block above) before running any Compose command
+> against an older stack, `down` included.
+> - The cluster now joins the pre-created `hugegraph-net` network instead of
+> a per-project bridge, so the first `up -d` recreates all nine containers.
+> Named data volumes are unchanged and survive the move; the orphaned
+> `hugegraph-3x3_hg-net` bridge can be removed with
+> `docker network rm hugegraph-3x3_hg-net`.
+> - Authentication is now enabled: previously unauthenticated clients of the
+> graph APIs on ports 8080–8082 will start receiving 401 responses and
Review Comment:
Remote clients will not see 401, they will see connection refused. This PR also moves every
published port from all interfaces to `127.0.0.1`, so for the deployments this note
addresses (another machine querying 8080-8082) the TCP connection fails before
authentication is reached.
The note is where an upgrader looks, and it does not mention
`HUGEGRAPH_SERVER_PUBLISH_HOST` or `HUGEGRAPH_CONTROL_PLANE_HOST` as the way to restore
reachability. Worth a line, especially given the advice just below to verify sign-in before
decommissioning an existing access path.
##########
docker/README.md:
##########
@@ -196,11 +331,268 @@ curl http://localhost:8520/v1/health
# Check Server (Graph API)
curl http://localhost:8080/versions
-# List registered stores via PD
-curl http://localhost:8620/v1/stores
+# Every PD endpoint except /v1/health, /actuator/* and /v1/prom/targets/*
+# needs an Authorization header. PD only checks that the Basic-auth user is
+# one of its internal service names, so the empty password below is enough
+# — and it grants the full PD control plane, writes included, not just these
+# reads. That is exactly why the shared network must be treated as a trust
+# boundary. Without the header PD answers with an exception body, not data.
+pd_auth="Authorization: Basic $(printf 'hubble:' | base64)"
+
+# List registered stores via PD (expect three, each "state":"Up")
+curl -H "${pd_auth}" http://localhost:8620/v1/stores
# List partitions
-curl http://localhost:8620/v1/partitions
+curl -H "${pd_auth}" http://localhost:8620/v1/partitions
+```
+
+Confirm authentication actually engaged — `/versions` stays open by design,
+so it cannot tell you whether auth is on. A graph read without credentials
+must be rejected:
+
+```bash
+cd docker
+# Expect 401 on all three replicas
+for port in 8080 8081 8082; do
+ curl -s -o /dev/null -w "${port}: %{http_code}\n" \
+ "http://localhost:${port}/graphs/hugegraph/schema/vertexlabels"
+done
+
+# And a signed-in read must succeed. Parse the generated single-quoted
+# value without executing docker/.env as shell. Passing the credential
+# through --config keeps it out of argv, where `ps` would expose it.
+read_dotenv_value() {
+ key="$1"
+ key_pattern="^[[:space:]]*(export[[:space:]]+)?${key}="
+ count="$(grep -Ec "${key_pattern}" .env || true)"
+ [ "${count}" -eq 1 ] || return 1
+ sed -nE "s/${key_pattern}'([^']*)'[[:space:]]*$/\\2/p" .env
+}
+HUGEGRAPH_ADMIN_PASSWORD="$(read_dotenv_value HUGEGRAPH_ADMIN_PASSWORD)" || {
+ echo "HUGEGRAPH_ADMIN_PASSWORD must use the documented single-quoted format" >&2
+ exit 1
+}
+curl -s -o /dev/null -w '%{http_code}\n' \
+ --config <(printf 'user = "admin:%s"\n' "${HUGEGRAPH_ADMIN_PASSWORD}") \
+ http://localhost:8080/graphs/hugegraph/schema/vertexlabels
+unset HUGEGRAPH_ADMIN_PASSWORD
+```
+
+`200` from the second command with `401` from all three of the first means
+authentication is on and working. If the first command returns `200`, the
+running image ignored `PASSWORD` and the cluster is **unauthenticated** —
+the most likely cause is an image older than the release this integration
+needs (see the version note in the quickstart above).
+
+### Running without authentication
+
+The cluster above authenticates, and that is the right default: it is the
+configuration most people copy, and an open graph database on a reachable
+port is a bad surprise. For a throwaway local trial on a trusted network you
+can opt out with `docker-compose-3x3.non-auth.yml`, which drops the admin
+password and the JWT secret from all three Servers. Anyone who can reach the
+published ports then has full read and write access to every graph, so do not
+use it anywhere else.
+
+```bash
+cd docker
+
+# No docker/.env and no credential setup are needed for this mode. The two
+# placeholders exist only because Compose evaluates the base file's
+# required-variable guards before applying the override, which then drops
+# both variables, so neither value reaches a container. Needs Compose v2.24+.
+#
+# Keep them inline on this command; do not write them into docker/.env. They
+# are published values rather than secrets, and the token placeholder is
+# deliberately shorter than the 32 bytes the Server requires, so if it ever
+# reaches an authenticated Server that Server refuses to start instead of
+# signing tokens with a key anyone can read here.
+HUGEGRAPH_ADMIN_PASSWORD=non-auth-placeholder \
+HUGEGRAPH_AUTH_TOKEN_SECRET=non-auth-placeholder \
+ docker compose -f docker-compose-3pd-3store-3server.yml \
+ -f docker-compose-3x3.non-auth.yml up -d
+
+# Every graph request now succeeds without credentials: expect 200, not 401.
+curl -s -o /dev/null -w '%{http_code}\n' \
+ http://localhost:8080/graphs/hugegraph/schema/vertexlabels
+```
+
+Hubble needs the matching configuration, because its own default is to demand
+a login. Point it at the non-auth properties file when you attach it:
+
+```bash
+HUBBLE_PROPERTIES=./hugegraph-hubble-3x3.non-auth.properties \
+ docker compose -p hugegraph-hubble -f docker-compose-hubble.yml up -d
+```
+
+<details>
+<summary>Prompt for an AI assistant</summary>
+
+Copy this to an assistant instead of transcribing the commands by hand.
+
+```markdown
+Start the Apache HugeGraph 3-node Docker cluster (3 PD, 3 Store, 3 Server)
+with Hubble, from the `docker/` directory of the hugegraph repository.
+
+Use the authenticated default unless I say otherwise:
+1. Follow "3-Node Cluster Quickstart" in docker/README.md to create
+ docker/.env with a generated admin password and JWT token secret, and to
+ create the external hugegraph-net network.
+2. Start the cluster with docker-compose-3pd-3store-3server.yml, then attach
+ Hubble with docker-compose-hubble.yml.
+3. Verify: an unauthenticated graph request returns 401, an authenticated one
+ returns 200, and Hubble answers on http://localhost:8088.
+4. Tell me the admin password so I can sign in to Hubble.
+
+Do not weaken authentication to work around an error. If a Server never
+becomes healthy, the image is probably older than the release this stack
+needs; report that instead of disabling the healthcheck. Only if I explicitly
+ask for an unauthenticated cluster, use the "Running without authentication"
+section instead.
+```
+
+</details>
+
+---
+
+## Hubble for the 3-Node Cluster
+
+`docker-compose-hubble.yml` defines only the Hubble service. It joins the
+cluster's external network (`hugegraph-net` by default, override with
+`HUGEGRAPH_NETWORK`) and has no `depends_on` on cluster services, so
+starting, stopping, or upgrading Hubble never recreates or restarts PD,
+Store, or Server containers. Hubble reads the cluster topology from
+`hugegraph-hubble-3x3.properties`; adjust that file when attaching to a
+cluster with different hostnames. Its `pd.server` is a single PD address
+with no failover, so if that PD is down the operations view goes blind even
+though the cluster still has quorum — repoint it at a surviving PD. The
+operations view also reports one `SERVER` node, not three: it describes the
+replica Hubble is currently talking to, not the whole Server tier. The add-on
+stores Hubble's H2 database and uploaded files in named volumes
+`hugegraph-hubble-db` (mounted at `/hubble/db`) and
+`hugegraph-hubble-upload-files`. Those names are explicit, so the attach
+and combined Compose projects share the same physical volumes. Hubble's
+default H2 URL would write `/hubble/db.mv.db` (outside that mount); the
+add-on sets `SPRING_DATASOURCE_URL` so the database file lives inside
+`/hubble/db`. Recreating the container does not discard that state.
+
+Sign in at `http://localhost:8088` as `admin` with the
+`HUGEGRAPH_ADMIN_PASSWORD` from `docker/.env`. Hubble binds to host
+loopback by default (`HUBBLE_PUBLISH_HOST`, same caveats as the
+single-node setup).
+
+Which flow to use: pick the attach flow when you do not have the cluster's
+`docker/.env` — the add-on carries no `:?` guards, so it is the only flow
+that runs without those credentials, which is what you want against a
+cluster someone else started. Otherwise use the combined flow, including to
+add Hubble to an already-running cluster (`up -d --no-deps hubble`).
+
+The two flows below create Hubble in different Compose projects, so manage
+Hubble with the same flags you started it with: the attach flow always uses
+`-p hugegraph-hubble -f docker-compose-hubble.yml`, the combined flow always
+uses both `-f` flags. The explicit `-p` keeps the attach project independent
+of the directory name and of other Compose projects.
+
+> [!IMPORTANT]
+> Do not drop the `-p` from the attach flow. Without it Compose names the
+> project after the current directory (`docker`), so Hubble starts in a
+> third project that none of the commands below manage: `down` reports
+> nothing to remove while `hg-hubble` keeps running, and every later `up`
+> in either flow fails on the container name. If that happens, find it with
+> `docker ps --filter name=hg-hubble` and remove it with
+> `docker rm -f hg-hubble`.
+
+Run one Hubble per host: the single-node stack and both add-on flows all
+publish `127.0.0.1:8088` and name their container `hg-hubble`. The two
+add-on flows are therefore mutually exclusive — starting one while the
+other's Hubble exists fails with a container-name conflict, so remove the
+Hubble of the flow you are leaving before switching. The two directions are
+not symmetric:
+
+```bash
+cd docker
+# leaving the attach flow (removes only Hubble)
+docker compose -p hugegraph-hubble -f docker-compose-hubble.yml down
+
+# leaving the combined flow — remove ONLY Hubble; a plain `down` with both
+# -f flags would stop all ten containers just to move one.
+docker compose -f docker-compose-3pd-3store-3server.yml \
+ -f docker-compose-hubble.yml rm -sf hubble
+```
+
+### Attach to a running cluster
+
+With the 3-node cluster already up:
+
+```bash
+cd docker
+docker compose -p hugegraph-hubble -f docker-compose-hubble.yml up -d
+```
+
+Lifecycle commands in this flow operate on Hubble alone and leave the
+cluster and the external network in place:
+
+```bash
+cd docker
+docker compose -p hugegraph-hubble -f docker-compose-hubble.yml ps
+docker compose -p hugegraph-hubble -f docker-compose-hubble.yml down
+```
+
+To remove everything in this flow, take down Hubble first, then the cluster:
+
+```bash
+cd docker
+docker compose -p hugegraph-hubble -f docker-compose-hubble.yml down
+docker compose -f docker-compose-3pd-3store-3server.yml down -v
Review Comment:
This command cannot run in the scenario the attach flow is prescribed for. The flow is
recommended above "when you do not have the cluster's `docker/.env`", but this line hits
the base file's `:?` guards and fails with `required variable
HUGEGRAPH_AUTH_TOKEN_SECRET is missing a value`.
Also, "everything" leaves both Hubble volumes behind: the attach `down` above has no `-v`,
and the cluster file does not declare them. Because the names are fixed, a later
deployment silently reattaches the stale H2 database.
##########
.github/workflows/server-ci.yml:
##########
@@ -138,6 +140,322 @@ jobs:
check_compose docker/docker-compose.yml always always
check_compose docker/docker-compose.dev.yml build missing
+ check_cluster_compose() {
+ local cluster="docker/docker-compose-3pd-3store-3server.yml"
+ local addon="docker/docker-compose-hubble.yml"
+ local rendered
+ local token_fixture=ci-test-token-secret-32-bytes-long
+ rendered="$(mktemp)"
+ if [ "${#token_fixture}" -lt 32 ]; then
+ echo "CI token fixture must be at least 32 bytes" >&2
+ return 1
+ fi
+
+ # RETURN only: an EXIT trap would fire after this function's
+ # `local rendered` has gone out of scope, which `set -u` turns
+ # into an "unbound variable" error. A hard errexit abort can
+ # therefore still leak one temp file, which is acceptable on an
+ # ephemeral runner.
+ trap 'rm -f "$rendered"' RETURN
+
+ # --env-file /dev/null on every invocation: Compose otherwise reads
+ # docker/.env automatically, and the quickstart tells operators to
+ # create one holding exactly the credentials these assertions
+ # control. Without it the checks pass in CI (which has no .env) but
+ # report false failures for anyone running them locally after
+ # following the quickstart, and the add-on render below — which
+ # deliberately unsets the variables to assert their defaults — would
+ # read whatever HUGEGRAPH_NETWORK/HUGEGRAPH_VERSION that .env holds.
+ # Pinning an empty env file makes both renders depend only on what
+ # each invocation sets explicitly.
+
+ # Both cluster credentials are required and may not be empty.
+ # Each case asserts the guard fired for the *intended* variable:
+ # a bare non-zero exit would also be produced by a YAML error, a
+ # renamed file, or a missing docker binary.
+ assert_guard() { # assert_guard <blamed-var> <description>
+ local var="$1" desc="$2" err
+ if err="$(docker compose --env-file /dev/null -f "$cluster" \
+ config -q 2>&1)"; then
+ echo "$cluster accepted $desc" >&2
+ return 1
+ fi
+ case "$err" in
+ *"$var"*) : ;;
+ *) echo "$cluster rejected $desc, but not because of $var: $err" >&2
+ return 1 ;;
+ esac
+ }
+
+ ( unset HUGEGRAPH_ADMIN_PASSWORD
+ export HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}"
+ assert_guard HUGEGRAPH_ADMIN_PASSWORD "an unset admin password" )
+ ( export HUGEGRAPH_ADMIN_PASSWORD=
+ export HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}"
+ assert_guard HUGEGRAPH_ADMIN_PASSWORD "an empty admin password" )
+ ( unset HUGEGRAPH_AUTH_TOKEN_SECRET
+ export HUGEGRAPH_ADMIN_PASSWORD=ci-test-password
+ assert_guard HUGEGRAPH_AUTH_TOKEN_SECRET "an unset token secret" )
+ ( export HUGEGRAPH_ADMIN_PASSWORD=ci-test-password
+ export HUGEGRAPH_AUTH_TOKEN_SECRET=
+ assert_guard HUGEGRAPH_AUTH_TOKEN_SECRET "an empty token secret" )
+ # The add-on alone must define Hubble and nothing else, join the
+ # shared external network with its default name, and need no
+ # credentials or overrides.
+ env -u HUGEGRAPH_ADMIN_PASSWORD -u HUGEGRAPH_AUTH_TOKEN_SECRET \
+ -u HUGEGRAPH_NETWORK -u HUGEGRAPH_VERSION \
+ -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY -u HUBBLE_PUBLISH_HOST \
+ -u HUBBLE_DB_VOLUME -u HUBBLE_UPLOAD_VOLUME \
+ docker compose --env-file /dev/null -f "$addon" config --format json > "$rendered"
+ jq -e '
+ (.services | keys) == ["hubble"] and
+ .networks."hg-net".external == true and
+ .networks."hg-net".name == "hugegraph-net" and
+ (.services.hubble.networks | has("hg-net")) and
+ (.services.hubble | has("depends_on") | not) and
+ .volumes."hg-hubble-db".name == "hugegraph-hubble-db" and
+ .volumes."hg-hubble-upload-files".name ==
+ "hugegraph-hubble-upload-files" and
+ .services.hubble.environment.SPRING_DATASOURCE_URL ==
+ "jdbc:h2:file:./db/hubble;DB_CLOSE_ON_EXIT=FALSE" and
+ any(.services.hubble.volumes[];
+ .target == "/hubble/conf/hugegraph-hubble.properties" and
+ (.source | endswith("hugegraph-hubble-3x3.properties")))
+ and any(.services.hubble.volumes[];
+ .source == "hg-hubble-db" and
+ .target == "/hubble/db")
+ and any(.services.hubble.volumes[];
+ .source == "hg-hubble-upload-files" and
+ .target == "/hubble/upload-files")
+ ' "$rendered" >/dev/null
+
+ # The cluster's own default network name must match the add-on's,
+ # or the attach flow and the cluster land on different networks.
+ # The combined render below pins an override, so it cannot catch a
+ # drifting default.
+ HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}" \
+ env -u HUGEGRAPH_NETWORK -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY \
+ -u HUBBLE_PUBLISH_HOST \
+ docker compose --env-file /dev/null -f "$cluster" \
+ config --format json > "$rendered"
+ jq -e '.networks."hg-net".name == "hugegraph-net"' \
+ "$rendered" >/dev/null
+
+ # The combined render carries the PD-registration and auth
+ # settings on every server replica and keeps Hubble on loopback.
+ # Rendered with non-default HUGEGRAPH_NETWORK/HUGEGRAPH_VERSION so
+ # CI fails if any file stops honoring the overrides (the add-on
+ # render above covers the defaults).
+ HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}" \
+ HUGEGRAPH_NETWORK=ci-test-net \
+ HUGEGRAPH_VERSION=ci-test-tag \
+ env -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY -u HUBBLE_PUBLISH_HOST \
+ -u HUBBLE_DB_VOLUME -u HUBBLE_UPLOAD_VOLUME \
+ docker compose --env-file /dev/null -f "$cluster" -f "$addon" \
+ config --format json > "$rendered"
+ jq -e '
+ . as $root |
+ .name == "hugegraph-3x3" and
+ (.services | keys | length) == 10 and
+ .networks."hg-net".external == true and
+ .networks."hg-net".name == "ci-test-net" and
+ all(.services[]; .networks | has("hg-net")) and
+ .services.pd0.image == "hugegraph/pd:ci-test-tag" and
+ .services.store0.image == "hugegraph/store:ci-test-tag" and
+ .services.server0.image == "hugegraph/server:ci-test-tag" and
+ .services.hubble.image == "hugegraph/hubble:ci-test-tag" and
+ all(["server0", "server1", "server2"][];
+ $root.services[.].environment.HG_SERVER_USE_PD == "true" and
+ $root.services[.].environment.HG_SERVER_CLUSTER == "hg" and
+ $root.services[.].environment.HG_SERVER_INIT_STORE_ENABLED ==
+ "false" and
+ $root.services[.].environment.PASSWORD ==
+ "ci-test-password" and
+ $root.services[.].environment.HG_SERVER_AUTH_TOKEN_SECRET ==
+ "ci-test-token-secret-32-bytes-long") and
+ .services.server0.environment.HG_SERVER_REST_URL ==
+ "http://server0:8080" and
+ .services.server1.environment.HG_SERVER_REST_URL ==
+ "http://server1:8080" and
+ .services.server2.environment.HG_SERVER_REST_URL ==
+ "http://server2:8080" and
+ (.services.hubble | has("depends_on") | not) and
+ .services.hubble.pull_policy == "always" and
+ .volumes."hg-hubble-db".name == "hugegraph-hubble-db" and
+ .volumes."hg-hubble-upload-files".name ==
+ "hugegraph-hubble-upload-files" and
+ .services.hubble.environment.SPRING_DATASOURCE_URL ==
+ "jdbc:h2:file:./db/hubble;DB_CLOSE_ON_EXIT=FALSE" and
+ (.services.hubble.healthcheck.test[1] |
+ contains("http://127.0.0.1:8088/about") and
+ contains("\"status\":200") and
+ contains("\"name\":\"hugegraph-hubble\"")) and
+ any(.services.hubble.ports[];
+ .target == 8088 and .published == "8088" and
+ .host_ip == "127.0.0.1") and
+ all(["pd0", "pd1", "pd2", "store0", "store1", "store2",
+ "server0", "server1", "server2"][];
+ all($root.services[.].ports[]; .host_ip == "127.0.0.1"))
+ ' "$rendered" >/dev/null
+
+ # Hubble's properties file is mounted, not rendered, so Compose
+ # validation alone cannot catch it drifting from the services it
+ # describes. Tie the two together: renaming a service, changing a
+ # container hostname, or moving a REST port must be reflected in
+ # both places or CI fails here. Values are derived from the
+ # rendered model (hostnames and ports included) so the assertions
+ # cannot silently agree with a stale file.
+ local props="docker/hugegraph-hubble-3x3.properties"
+ local pd_peers store_targets cluster_name pd_rest
+ assert_props() { # assert_props <exact-line> <what-it-must-match>
+ grep -Fqx "$1" "$props" ||
+ { echo "$props: expected line '$1' ($2)" >&2; return 1; }
+ }
+ cluster_name="$(jq -r '.services.server0.environment.HG_SERVER_CLUSTER' \
+ "$rendered")"
+ pd_peers="$(jq -r '.services.server0.environment.HG_SERVER_PD_PEERS' \
+ "$rendered")"
+ store_targets="[$(jq -r '[.services | to_entries[]
+ | select(.key | startswith("store"))
+ | "http://" + .value.hostname + ":"
+ + (.value.environment.HG_STORE_REST_PORT)]
+ | sort | join(",")' "$rendered")]"
+ pd_rest="$(jq -r --arg h "$(printf '%s' "${pd_peers}" | cut -d, -f1 | cut -d: -f1)" \
+ '.services | to_entries[]
+ | select(.value.hostname == $h)
+ | .value.hostname + ":" + .value.environment.HG_PD_REST_PORT' \
+ "$rendered")"
+ assert_props "cluster=${cluster_name}" "matches HG_SERVER_CLUSTER"
+ assert_props "pd.enabled=true" "keeps Hubble in PD mode"
+ assert_props "pd.peers=${pd_peers}" "matches HG_SERVER_PD_PEERS"
+ assert_props "pd.server=${pd_rest}" "names a real PD REST endpoint"
+ assert_props "operations.store.allowed_targets=${store_targets}" \
+ "lists every Store REST endpoint"
+
+ # The documented opt-out must actually opt out: no admin password,
+ # no shared token secret, and a healthcheck that does not demand
+ # the 401/200 pair the authenticated stack requires. A silent
+ # failure here would leave users on an unauthenticated cluster
+ # that still advertises itself as authenticated.
+ # Use the exact placeholders the README documents, including the
+ # deliberately short token: it must never reach a container, and
+ # the assertions below are what prove the override drops it.
+ local non_auth="docker/docker-compose-3x3.non-auth.yml"
+ HUGEGRAPH_ADMIN_PASSWORD=non-auth-placeholder \
+ HUGEGRAPH_AUTH_TOKEN_SECRET=non-auth-placeholder \
+ docker compose --env-file /dev/null -f "$cluster" -f "$non_auth" \
+ config --format json > "$rendered"
+ jq -e '
+ . as $root |
+ all(["server0", "server1", "server2"][];
+ ($root.services[.].environment | has("PASSWORD") | not) and
+ ($root.services[.].environment
+ | has("HG_SERVER_AUTH_TOKEN_SECRET") | not) and
+ ($root.services[.].healthcheck.test[1]
+ | contains("/versions") and (contains("401") | not)))
+ ' "$rendered" >/dev/null
+ grep -Fqx "auth.enabled=false" \
+ docker/hugegraph-hubble-3x3.non-auth.properties
+ }
+
+ check_cluster_compose
+
+ - name: Run distributed Compose auth and attach smoke test
+ if: ${{ env.BACKEND == 'rocksdb' }}
+ run: |
+ set -euo pipefail
+ command -v docker >/dev/null 2>&1
+ docker compose version >/dev/null
+ docker info >/dev/null
+
+ cluster="docker/docker-compose-3pd-3store-3server.yml"
+ addon="docker/docker-compose-hubble.yml"
+ run_id="${GITHUB_RUN_ID:-local}"
+ network="hugegraph-ci-${run_id}"
+ db_volume="hugegraph-ci-${run_id}-db"
+ upload_volume="hugegraph-ci-${run_id}-uploads"
+ admin_password='ci-smoke-admin-password'
+ token_secret='ci-smoke-token-secret-32-bytes-long'
+
+ # Every Compose call needs the same environment, and cleanup must run
+ # even when an assertion below fails.
+ export HUGEGRAPH_NETWORK="$network"
+ export HUGEGRAPH_ADMIN_PASSWORD="$admin_password"
+ export HUGEGRAPH_AUTH_TOKEN_SECRET="$token_secret"
+ export HUBBLE_DB_VOLUME="$db_volume"
+ export HUBBLE_UPLOAD_VOLUME="$upload_volume"
+
+ cleanup() {
+ docker compose --env-file /dev/null -f "$cluster" -f "$addon" \
+ down -v --remove-orphans >/dev/null 2>&1 || true
+ docker compose --env-file /dev/null -p "hugegraph-ci-attach-${run_id}" \
+ -f "$addon" down -v >/dev/null 2>&1 || true
+ docker network rm "$network" >/dev/null 2>&1 || true
+ docker volume rm "$db_volume" "$upload_volume" >/dev/null 2>&1 || true
+ }
+ trap cleanup EXIT
+ docker network create "$network"
+
+ # The cluster must come up healthy on its own. Its Server healthcheck
+ # already proves 401/200, so --wait failing here means the images do
+ # not enforce the authentication this stack configures.
+ docker compose --env-file /dev/null -f "$cluster" up -d --wait \
Review Comment:
This is the part of the CI addition I would most want changed. The smoke test boots
`hugegraph/{pd,store,server}:latest` from Docker Hub with `pull_policy: always` and then
asserts the 401/200 behavior. Nothing in the job builds the images from the PR under test,
so the step is really asserting a property of the published `latest` tag: it goes red on
unrelated PRs whenever `latest` lags master, the registry is slow, or the runner gets
rate-limited, and it would stay green on a PR that broke the entrypoint.
Worth noting the runner budget too: this is `ubuntu-22.04`, and the step boots ten JVM
containers with `--wait-timeout 420` twice plus 180, in the same job that already ran a
rocksdb backend and a Maven compile, against a README that asks for 12 GB for the nine
cluster JVMs alone. A separate job needing only checkout and Docker would parallelize it
and give the containers the full runner.
Two related coverage gaps while you are here: the Hubble-to-cluster path is never
exercised (PD and Store are probed from the runner, and Hubble only via its own `/about`,
which passes even if Hubble can reach nothing), and the non-auth flow is only ever
rendered, never booted.
##########
.github/workflows/server-ci.yml:
##########
@@ -138,6 +140,322 @@ jobs:
check_compose docker/docker-compose.yml always always
check_compose docker/docker-compose.dev.yml build missing
+ check_cluster_compose() {
+ local cluster="docker/docker-compose-3pd-3store-3server.yml"
+ local addon="docker/docker-compose-hubble.yml"
+ local rendered
+ local token_fixture=ci-test-token-secret-32-bytes-long
+ rendered="$(mktemp)"
+ if [ "${#token_fixture}" -lt 32 ]; then
+ echo "CI token fixture must be at least 32 bytes" >&2
+ return 1
+ fi
+
+ # RETURN only: an EXIT trap would fire after this function's
+ # `local rendered` has gone out of scope, which `set -u` turns
+ # into an "unbound variable" error. A hard errexit abort can
+ # therefore still leak one temp file, which is acceptable on an
+ # ephemeral runner.
+ trap 'rm -f "$rendered"' RETURN
+
+ # --env-file /dev/null on every invocation: Compose otherwise reads
+ # docker/.env automatically, and the quickstart tells operators to
+ # create one holding exactly the credentials these assertions
+ # control. Without it the checks pass in CI (which has no .env) but
+ # report false failures for anyone running them locally after
+ # following the quickstart, and the add-on render below — which
+ # deliberately unsets the variables to assert their defaults — would
+ # read whatever HUGEGRAPH_NETWORK/HUGEGRAPH_VERSION that .env holds.
+ # Pinning an empty env file makes both renders depend only on what
+ # each invocation sets explicitly.
+
+ # Both cluster credentials are required and may not be empty.
+ # Each case asserts the guard fired for the *intended* variable:
+ # a bare non-zero exit would also be produced by a YAML error, a
+ # renamed file, or a missing docker binary.
+ assert_guard() { # assert_guard <blamed-var> <description>
+ local var="$1" desc="$2" err
+ if err="$(docker compose --env-file /dev/null -f "$cluster" \
+ config -q 2>&1)"; then
+ echo "$cluster accepted $desc" >&2
+ return 1
+ fi
+ case "$err" in
+ *"$var"*) : ;;
+ *) echo "$cluster rejected $desc, but not because of $var: $err" >&2
+ return 1 ;;
+ esac
+ }
+
+ ( unset HUGEGRAPH_ADMIN_PASSWORD
+ export HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}"
+ assert_guard HUGEGRAPH_ADMIN_PASSWORD "an unset admin password" )
+ ( export HUGEGRAPH_ADMIN_PASSWORD=
+ export HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}"
+ assert_guard HUGEGRAPH_ADMIN_PASSWORD "an empty admin password" )
+ ( unset HUGEGRAPH_AUTH_TOKEN_SECRET
+ export HUGEGRAPH_ADMIN_PASSWORD=ci-test-password
+ assert_guard HUGEGRAPH_AUTH_TOKEN_SECRET "an unset token secret" )
+ ( export HUGEGRAPH_ADMIN_PASSWORD=ci-test-password
+ export HUGEGRAPH_AUTH_TOKEN_SECRET=
+ assert_guard HUGEGRAPH_AUTH_TOKEN_SECRET "an empty token secret" )
+ # The add-on alone must define Hubble and nothing else, join the
+ # shared external network with its default name, and need no
+ # credentials or overrides.
+ env -u HUGEGRAPH_ADMIN_PASSWORD -u HUGEGRAPH_AUTH_TOKEN_SECRET \
+ -u HUGEGRAPH_NETWORK -u HUGEGRAPH_VERSION \
+ -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY -u HUBBLE_PUBLISH_HOST \
+ -u HUBBLE_DB_VOLUME -u HUBBLE_UPLOAD_VOLUME \
+ docker compose --env-file /dev/null -f "$addon" config --format json > "$rendered"
+ jq -e '
+ (.services | keys) == ["hubble"] and
+ .networks."hg-net".external == true and
+ .networks."hg-net".name == "hugegraph-net" and
+ (.services.hubble.networks | has("hg-net")) and
+ (.services.hubble | has("depends_on") | not) and
+ .volumes."hg-hubble-db".name == "hugegraph-hubble-db" and
+ .volumes."hg-hubble-upload-files".name ==
+ "hugegraph-hubble-upload-files" and
+ .services.hubble.environment.SPRING_DATASOURCE_URL ==
+ "jdbc:h2:file:./db/hubble;DB_CLOSE_ON_EXIT=FALSE" and
+ any(.services.hubble.volumes[];
+ .target == "/hubble/conf/hugegraph-hubble.properties" and
+ (.source | endswith("hugegraph-hubble-3x3.properties")))
+ and any(.services.hubble.volumes[];
+ .source == "hg-hubble-db" and
+ .target == "/hubble/db")
+ and any(.services.hubble.volumes[];
+ .source == "hg-hubble-upload-files" and
+ .target == "/hubble/upload-files")
+ ' "$rendered" >/dev/null
+
+ # The cluster's own default network name must match the add-on's,
+ # or the attach flow and the cluster land on different networks.
+ # The combined render below pins an override, so it cannot catch a
+ # drifting default.
+ HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}" \
+ env -u HUGEGRAPH_NETWORK -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY \
+ -u HUBBLE_PUBLISH_HOST \
+ docker compose --env-file /dev/null -f "$cluster" \
+ config --format json > "$rendered"
+ jq -e '.networks."hg-net".name == "hugegraph-net"' \
+ "$rendered" >/dev/null
+
+ # The combined render carries the PD-registration and auth
+ # settings on every server replica and keeps Hubble on loopback.
+ # Rendered with non-default HUGEGRAPH_NETWORK/HUGEGRAPH_VERSION so
+ # CI fails if any file stops honoring the overrides (the add-on
+ # render above covers the defaults).
+ HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ HUGEGRAPH_AUTH_TOKEN_SECRET="${token_fixture}" \
+ HUGEGRAPH_NETWORK=ci-test-net \
+ HUGEGRAPH_VERSION=ci-test-tag \
+ env -u HUBBLE_IMAGE -u HUBBLE_PULL_POLICY -u HUBBLE_PUBLISH_HOST \
+ -u HUBBLE_DB_VOLUME -u HUBBLE_UPLOAD_VOLUME \
+ docker compose --env-file /dev/null -f "$cluster" -f "$addon" \
+ config --format json > "$rendered"
+ jq -e '
+ . as $root |
+ .name == "hugegraph-3x3" and
+ (.services | keys | length) == 10 and
+ .networks."hg-net".external == true and
+ .networks."hg-net".name == "ci-test-net" and
+ all(.services[]; .networks | has("hg-net")) and
+ .services.pd0.image == "hugegraph/pd:ci-test-tag" and
+ .services.store0.image == "hugegraph/store:ci-test-tag" and
+ .services.server0.image == "hugegraph/server:ci-test-tag" and
+ .services.hubble.image == "hugegraph/hubble:ci-test-tag" and
+ all(["server0", "server1", "server2"][];
+ $root.services[.].environment.HG_SERVER_USE_PD == "true" and
+ $root.services[.].environment.HG_SERVER_CLUSTER == "hg" and
+ $root.services[.].environment.HG_SERVER_INIT_STORE_ENABLED ==
+ "false" and
+ $root.services[.].environment.PASSWORD ==
+ "ci-test-password" and
+ $root.services[.].environment.HG_SERVER_AUTH_TOKEN_SECRET ==
+ "ci-test-token-secret-32-bytes-long") and
+ .services.server0.environment.HG_SERVER_REST_URL ==
+ "http://server0:8080" and
+ .services.server1.environment.HG_SERVER_REST_URL ==
+ "http://server1:8080" and
+ .services.server2.environment.HG_SERVER_REST_URL ==
+ "http://server2:8080" and
+ (.services.hubble | has("depends_on") | not) and
+ .services.hubble.pull_policy == "always" and
+ .volumes."hg-hubble-db".name == "hugegraph-hubble-db" and
+ .volumes."hg-hubble-upload-files".name ==
+ "hugegraph-hubble-upload-files" and
+ .services.hubble.environment.SPRING_DATASOURCE_URL ==
+ "jdbc:h2:file:./db/hubble;DB_CLOSE_ON_EXIT=FALSE" and
+ (.services.hubble.healthcheck.test[1] |
+ contains("http://127.0.0.1:8088/about") and
+ contains("\"status\":200") and
+ contains("\"name\":\"hugegraph-hubble\"")) and
+ any(.services.hubble.ports[];
+ .target == 8088 and .published == "8088" and
+ .host_ip == "127.0.0.1") and
+ all(["pd0", "pd1", "pd2", "store0", "store1", "store2",
+ "server0", "server1", "server2"][];
+ all($root.services[.].ports[]; .host_ip == "127.0.0.1"))
+ ' "$rendered" >/dev/null
+
+ # Hubble's properties file is mounted, not rendered, so Compose
+ # validation alone cannot catch it drifting from the services it
+ # describes. Tie the two together: renaming a service, changing a
+ # container hostname, or moving a REST port must be reflected in
+ # both places or CI fails here. Values are derived from the
+ # rendered model (hostnames and ports included) so the assertions
+ # cannot silently agree with a stale file.
+ local props="docker/hugegraph-hubble-3x3.properties"
+ local pd_peers store_targets cluster_name pd_rest
+ assert_props() { # assert_props <exact-line> <what-it-must-match>
+ grep -Fqx "$1" "$props" ||
+ { echo "$props: expected line '$1' ($2)" >&2; return 1; }
+ }
+ cluster_name="$(jq -r '.services.server0.environment.HG_SERVER_CLUSTER' \
+ "$rendered")"
+ pd_peers="$(jq -r '.services.server0.environment.HG_SERVER_PD_PEERS' \
+ "$rendered")"
+ store_targets="[$(jq -r '[.services | to_entries[]
+ | select(.key | startswith("store"))
+ | "http://" + .value.hostname + ":"
+ + (.value.environment.HG_STORE_REST_PORT)]
+ | sort | join(",")' "$rendered")]"
+ pd_rest="$(jq -r --arg h "$(printf '%s' "${pd_peers}" | cut -d, -f1 | cut -d: -f1)" \
+ '.services | to_entries[]
+ | select(.value.hostname == $h)
+ | .value.hostname + ":" + .value.environment.HG_PD_REST_PORT' \
+ "$rendered")"
+ assert_props "cluster=${cluster_name}" "matches HG_SERVER_CLUSTER"
+ assert_props "pd.enabled=true" "keeps Hubble in PD mode"
+ assert_props "pd.peers=${pd_peers}" "matches HG_SERVER_PD_PEERS"
+ assert_props "pd.server=${pd_rest}" "names a real PD REST endpoint"
+ assert_props "operations.store.allowed_targets=${store_targets}" \
+ "lists every Store REST endpoint"
+
+ # The documented opt-out must actually opt out: no admin password,
+ # no shared token secret, and a healthcheck that does not demand
+ # the 401/200 pair the authenticated stack requires. A silent
+ # failure here would leave users on an unauthenticated cluster
+ # that still advertises itself as authenticated.
+ # Use the exact placeholders the README documents, including the
+ # deliberately short token: it must never reach a container, and
+ # the assertions below are what prove the override drops it.
+ local non_auth="docker/docker-compose-3x3.non-auth.yml"
+ HUGEGRAPH_ADMIN_PASSWORD=non-auth-placeholder \
+ HUGEGRAPH_AUTH_TOKEN_SECRET=non-auth-placeholder \
+ docker compose --env-file /dev/null -f "$cluster" -f "$non_auth" \
+ config --format json > "$rendered"
+ jq -e '
+ . as $root |
+ all(["server0", "server1", "server2"][];
+ ($root.services[.].environment | has("PASSWORD") | not) and
+ ($root.services[.].environment
+ | has("HG_SERVER_AUTH_TOKEN_SECRET") | not) and
+ ($root.services[.].healthcheck.test[1]
+ | contains("/versions") and (contains("401") | not)))
+ ' "$rendered" >/dev/null
+ grep -Fqx "auth.enabled=false" \
Review Comment:
The non-auth properties file gets only this one grep, while `assert_props` ties the
authenticated file's topology to the rendered model. Its `pd.peers`, `pd.server`,
`operations.store.allowed_targets` and `cluster` can therefore drift from the compose files
with CI green, and since the non-auth flow is never booted, users are the detection
mechanism. All five expected lines are already present verbatim in that file, so
parameterising `assert_props` on the path and running it twice would cover it.
Related: `hugegraph-hubble-3x3.properties` never pins `auth.enabled=true`. The file's own
header warns that absent keys fall back to code defaults a future release can change, and
the non-auth twin pins its value with this grep behind it. Pinning the positive case too,
with a matching grep, makes that drift impossible.
##########
docker/README.md:
##########
@@ -196,11 +331,268 @@ curl http://localhost:8520/v1/health
# Check Server (Graph API)
curl http://localhost:8080/versions
-# List registered stores via PD
-curl http://localhost:8620/v1/stores
+# Every PD endpoint except /v1/health, /actuator/* and /v1/prom/targets/*
+# needs an Authorization header. PD only checks that the Basic-auth user is
+# one of its internal service names, so the empty password below is enough
+# — and it grants the full PD control plane, writes included, not just these
+# reads. That is exactly why the shared network must be treated as a trust
+# boundary. Without the header PD answers with an exception body, not data.
+pd_auth="Authorization: Basic $(printf 'hubble:' | base64)"
+
+# List registered stores via PD (expect three, each "state":"Up")
+curl -H "${pd_auth}" http://localhost:8620/v1/stores
# List partitions
-curl http://localhost:8620/v1/partitions
+curl -H "${pd_auth}" http://localhost:8620/v1/partitions
+```
+
+Confirm authentication actually engaged — `/versions` stays open by design,
+so it cannot tell you whether auth is on. A graph read without credentials
+must be rejected:
+
+```bash
+cd docker
+# Expect 401 on all three replicas
+for port in 8080 8081 8082; do
+ curl -s -o /dev/null -w "${port}: %{http_code}\n" \
+ "http://localhost:${port}/graphs/hugegraph/schema/vertexlabels"
+done
+
+# And a signed-in read must succeed. Parse the generated single-quoted
+# value without executing docker/.env as shell. Passing the credential
+# through --config keeps it out of argv, where `ps` would expose it.
+read_dotenv_value() {
+ key="$1"
+ key_pattern="^[[:space:]]*(export[[:space:]]+)?${key}="
+ count="$(grep -Ec "${key_pattern}" .env || true)"
+ [ "${count}" -eq 1 ] || return 1
+ sed -nE "s/${key_pattern}'([^']*)'[[:space:]]*$/\\2/p" .env
+}
+HUGEGRAPH_ADMIN_PASSWORD="$(read_dotenv_value HUGEGRAPH_ADMIN_PASSWORD)" || {
+ echo "HUGEGRAPH_ADMIN_PASSWORD must use the documented single-quoted format" >&2
+ exit 1
+}
+curl -s -o /dev/null -w '%{http_code}\n' \
+ --config <(printf 'user = "admin:%s"\n' "${HUGEGRAPH_ADMIN_PASSWORD}") \
+ http://localhost:8080/graphs/hugegraph/schema/vertexlabels
+unset HUGEGRAPH_ADMIN_PASSWORD
+```
+
+`200` from the second command with `401` from all three of the first means
+authentication is on and working. If the first command returns `200`, the
+running image ignored `PASSWORD` and the cluster is **unauthenticated** —
+the most likely cause is an image older than the release this integration
+needs (see the version note in the quickstart above).
+
+### Running without authentication
+
+The cluster above authenticates, and that is the right default: it is the
+configuration most people copy, and an open graph database on a reachable
+port is a bad surprise. For a throwaway local trial on a trusted network you
+can opt out with `docker-compose-3x3.non-auth.yml`, which drops the admin
+password and the JWT secret from all three Servers. Anyone who can reach the
+published ports then has full read and write access to every graph, so do not
+use it anywhere else.
+
+```bash
+cd docker
+
+# No docker/.env and no credential setup are needed for this mode. The two
+# placeholders exist only because Compose evaluates the base file's
+# required-variable guards before applying the override, which then drops
+# both variables, so neither value reaches a container. Needs Compose v2.24+.
+#
+# Keep them inline on this command; do not write them into docker/.env. They
+# are published values rather than secrets, and the token placeholder is
+# deliberately shorter than the 32 bytes the Server requires, so if it ever
+# reaches an authenticated Server that Server refuses to start instead of
+# signing tokens with a key anyone can read here.
+HUGEGRAPH_ADMIN_PASSWORD=non-auth-placeholder \
+HUGEGRAPH_AUTH_TOKEN_SECRET=non-auth-placeholder \
+ docker compose -f docker-compose-3pd-3store-3server.yml \
+ -f docker-compose-3x3.non-auth.yml up -d
+
+# Every graph request now succeeds without credentials: expect 200, not 401.
+curl -s -o /dev/null -w '%{http_code}\n' \
+ http://localhost:8080/graphs/hugegraph/schema/vertexlabels
+```
+
+Hubble needs the matching configuration, because its own default is to demand
+a login. Point it at the non-auth properties file when you attach it:
+
+```bash
+HUBBLE_PROPERTIES=./hugegraph-hubble-3x3.non-auth.properties \
+ docker compose -p hugegraph-hubble -f docker-compose-hubble.yml up -d
+```
+
+<details>
+<summary>Prompt for an AI assistant</summary>
+
+Copy this to an assistant instead of transcribing the commands by hand.
+
+```markdown
+Start the Apache HugeGraph 3-node Docker cluster (3 PD, 3 Store, 3 Server)
+with Hubble, from the `docker/` directory of the hugegraph repository.
+
+Use the authenticated default unless I say otherwise:
+1. Follow "3-Node Cluster Quickstart" in docker/README.md to create
+ docker/.env with a generated admin password and JWT token secret, and to
+ create the external hugegraph-net network.
+2. Start the cluster with docker-compose-3pd-3store-3server.yml, then attach
+ Hubble with docker-compose-hubble.yml.
+3. Verify: an unauthenticated graph request returns 401, an authenticated one
+ returns 200, and Hubble answers on http://localhost:8088.
+4. Tell me the admin password so I can sign in to Hubble.
+
+Do not weaken authentication to work around an error. If a Server never
+becomes healthy, the image is probably older than the release this stack
+needs; report that instead of disabling the healthcheck. Only if I explicitly
+ask for an unauthenticated cluster, use the "Running without authentication"
+section instead.
+```
+
+</details>
+
+---
+
+## Hubble for the 3-Node Cluster
+
+`docker-compose-hubble.yml` defines only the Hubble service. It joins the
+cluster's external network (`hugegraph-net` by default, override with
+`HUGEGRAPH_NETWORK`) and has no `depends_on` on cluster services, so
+starting, stopping, or upgrading Hubble never recreates or restarts PD,
+Store, or Server containers. Hubble reads the cluster topology from
+`hugegraph-hubble-3x3.properties`; adjust that file when attaching to a
+cluster with different hostnames. Its `pd.server` is a single PD address
+with no failover, so if that PD is down the operations view goes blind even
+though the cluster still has quorum — repoint it at a surviving PD. The
+operations view also reports one `SERVER` node, not three: it describes the
+replica Hubble is currently talking to, not the whole Server tier. The add-on
+stores Hubble's H2 database and uploaded files in named volumes
+`hugegraph-hubble-db` (mounted at `/hubble/db`) and
+`hugegraph-hubble-upload-files`. Those names are explicit, so the attach
+and combined Compose projects share the same physical volumes. Hubble's
+default H2 URL would write `/hubble/db.mv.db` (outside that mount); the
+add-on sets `SPRING_DATASOURCE_URL` so the database file lives inside
+`/hubble/db`. Recreating the container does not discard that state.
+
+Sign in at `http://localhost:8088` as `admin` with the
+`HUGEGRAPH_ADMIN_PASSWORD` from `docker/.env`. Hubble binds to host
+loopback by default (`HUBBLE_PUBLISH_HOST`, same caveats as the
+single-node setup).
+
+Which flow to use: pick the attach flow when you do not have the cluster's
+`docker/.env` — the add-on carries no `:?` guards, so it is the only flow
+that runs without those credentials, which is what you want against a
+cluster someone else started. Otherwise use the combined flow, including to
+add Hubble to an already-running cluster (`up -d --no-deps hubble`).
+
+The two flows below create Hubble in different Compose projects, so manage
+Hubble with the same flags you started it with: the attach flow always uses
+`-p hugegraph-hubble -f docker-compose-hubble.yml`, the combined flow always
+uses both `-f` flags. The explicit `-p` keeps the attach project independent
+of the directory name and of other Compose projects.
+
+> [!IMPORTANT]
+> Do not drop the `-p` from the attach flow. Without it Compose names the
+> project after the current directory (`docker`), so Hubble starts in a
+> third project that none of the commands below manage: `down` reports
+> nothing to remove while `hg-hubble` keeps running, and every later `up`
+> in either flow fails on the container name. If that happens, find it with
+> `docker ps --filter name=hg-hubble` and remove it with
+> `docker rm -f hg-hubble`.
+
+Run one Hubble per host: the single-node stack and both add-on flows all
+publish `127.0.0.1:8088` and name their container `hg-hubble`. The two
+add-on flows are therefore mutually exclusive — starting one while the
+other's Hubble exists fails with a container-name conflict, so remove the
+Hubble of the flow you are leaving before switching. The two directions are
+not symmetric:
+
+```bash
+cd docker
+# leaving the attach flow (removes only Hubble)
+docker compose -p hugegraph-hubble -f docker-compose-hubble.yml down
+
+# leaving the combined flow — remove ONLY Hubble; a plain `down` with both
+# -f flags would stop all ten containers just to move one.
+docker compose -f docker-compose-3pd-3store-3server.yml \
+ -f docker-compose-hubble.yml rm -sf hubble
+```
+
+### Attach to a running cluster
+
+With the 3-node cluster already up:
+
+```bash
+cd docker
+docker compose -p hugegraph-hubble -f docker-compose-hubble.yml up -d
+```
+
+Lifecycle commands in this flow operate on Hubble alone and leave the
+cluster and the external network in place:
+
+```bash
+cd docker
+docker compose -p hugegraph-hubble -f docker-compose-hubble.yml ps
+docker compose -p hugegraph-hubble -f docker-compose-hubble.yml down
+```
+
+To remove everything in this flow, take down Hubble first, then the cluster:
+
+```bash
+cd docker
+docker compose -p hugegraph-hubble -f docker-compose-hubble.yml down
+docker compose -f docker-compose-3pd-3store-3server.yml down -v
+```
+
+### Fresh cluster plus Hubble in one command
+
+After the one-time network and `docker/.env` setup from the quickstart:
+
+```bash
+cd docker
+docker compose -f docker-compose-3pd-3store-3server.yml \
+ -f docker-compose-hubble.yml up -d
+```
+
+Hubble has no startup dependency on the cluster, so it reports healthy while
+PD, Store, and Server are still forming the cluster; wait until every service
+shows healthy before signing in:
+
+```bash
+cd docker
+docker compose -f docker-compose-3pd-3store-3server.yml \
+ -f docker-compose-hubble.yml ps
+```
+
+In this flow Hubble belongs to the cluster project — use the same pair of
+`-f` flags for `ps`, `stop`, and `down`. The attach-flow `ps`/`stop`/`down`
+commands manage a different, empty project and do nothing here, and the
+cluster-only quickstart commands treat this Hubble as an orphan container
+(`--remove-orphans` would delete it) — always pass both `-f` flags.
+
+### Local Hubble image for development
+
+Build the Hubble image from `hugegraph-toolchain` source, then replace only
+the Hubble container. In the attach flow:
+
+```bash
+cd docker
+HUBBLE_IMAGE=local/hugegraph-hubble:dev \
+HUBBLE_PULL_POLICY=never \
+docker compose -p hugegraph-hubble -f docker-compose-hubble.yml up -d
Review Comment:
`HUBBLE_PROPERTIES` is not sticky, and this command drops it. Someone running the non-auth
stack attached Hubble with `HUBBLE_PROPERTIES=./hugegraph-hubble-3x3.non-auth.properties`;
running this dev-image command afterwards changes the bind source back to the default auth
properties, so Compose recreates the container and Hubble comes back demanding a login
against Servers that have no users configured. The same applies to any later plain `up -d`
in that flow.
Worth carrying `HUBBLE_PROPERTIES` through the commands in this section, or noting that it
has to be repeated every time.
##########
docker/README.md:
##########
@@ -168,14 +175,142 @@ To validate local images without Compose replacing them with remote `latest`:
## 3-Node Cluster Quickstart
+The cluster and the Hubble add-on share one named Docker network so Hubble
+can attach to a running cluster without touching it. Treat that network as a
+trust boundary: only the Server layer performs real authentication. Store
+serves its control APIs with no credentials at all, and while PD's REST
+control APIs do require an `Authorization` header, PD only checks that the
+Basic-auth *user* is one of its internal service names and never validates
+the password — so any client on the network can read and drive them. Note
+that this stack depends on that behaviour: Hubble reaches PD as the service
+name `hubble` with an empty password, so tightening PD's credential check
+would also break Hubble's operations view. Because of that, PD and Store
+control-plane ports and the three Server REST ports bind to `127.0.0.1` by
+default, so an unauthenticated control plane is not reachable from outside the
+host. Publish them more widely only by setting `HUGEGRAPH_CONTROL_PLANE_HOST`
+or `HUGEGRAPH_SERVER_PUBLISH_HOST`, and only behind a network ACL or TLS
+terminator you control. Any container on the host can still join the network
+by declaring the well-known name. One-time setup:
+write the required credentials to a mode-600 `docker/.env` and create the
+network.
+The cluster file requires both credentials — the admin password enables
+authentication, and every Server replica must share one token secret so a
+token issued by any server validates on all of them. The `:?` guards fire
+on every Compose subcommand, including `down`.
+
+```bash
+(
+ set -eu
+ cd docker
+ command -v openssl >/dev/null 2>&1 || { echo "openssl not found" >&2; exit 1; }
+ [ -e .env ] || install -m 600 /dev/null .env
+ chmod 600 .env
+ # Keep appends on their own lines even if the file was hand-edited.
+ [ ! -s .env ] || [ -z "$(tail -c1 .env)" ] || printf '\n' >> .env
+ pat='^[[:space:]]*(export[[:space:]]+)?'
+ if ! grep -Eq "${pat}HUGEGRAPH_ADMIN_PASSWORD=" .env; then
+ admin_password="$(openssl rand -base64 12)"
+ printf "HUGEGRAPH_ADMIN_PASSWORD='%s'\n" "${admin_password}" >> .env
+ unset admin_password
+ fi
+ if ! grep -Eq "${pat}HUGEGRAPH_AUTH_TOKEN_SECRET=" .env; then
+ token_secret="$(openssl rand -hex 32)"
+ printf "HUGEGRAPH_AUTH_TOKEN_SECRET='%s'\n" "${token_secret}" >> .env
+ unset token_secret
+ fi
+ # Parse values as data. Do not source .env as shell: a value such as
+ # HUGEGRAPH_ADMIN_PASSWORD=$(...) would run on the operator host.
+ # The optional export group is capture 1; the quoted value is capture 2.
+ read_dotenv_value() {
+ key="$1"
+ key_pattern="^[[:space:]]*(export[[:space:]]+)?${key}="
+ count="$(grep -Ec "${key_pattern}" .env || true)"
+ [ "${count}" -eq 1 ] || return 1
+ sed -nE "s/${key_pattern}'([^']*)'[[:space:]]*$/\\2/p" .env
+ }
+ if ! admin_value="$(read_dotenv_value HUGEGRAPH_ADMIN_PASSWORD)"; then
+ echo "HUGEGRAPH_ADMIN_PASSWORD must use the documented single-quoted format" >&2
+ exit 1
+ fi
+ if ! token_value="$(read_dotenv_value HUGEGRAPH_AUTH_TOKEN_SECRET)"; then
+ echo "HUGEGRAPH_AUTH_TOKEN_SECRET must use the documented single-quoted format" >&2
+ exit 1
+ fi
+ [ -n "${admin_value}" ] || {
+ echo "HUGEGRAPH_ADMIN_PASSWORD must be non-empty in docker/.env" >&2
+ exit 1
+ }
+ [ "${#token_value}" -ge 32 ] || {
+ echo "HUGEGRAPH_AUTH_TOKEN_SECRET must be at least 32 bytes in docker/.env" >&2
+ exit 1
+ }
+ unset admin_value token_value
+ # The shared cluster network. To override the name, export
+ # HUGEGRAPH_NETWORK in this shell before running the block — a value
+ # in docker/.env is read by Compose, not by this script.
+ net="${HUGEGRAPH_NETWORK:-hugegraph-net}"
+ docker network inspect "${net}" >/dev/null 2>&1 ||
+ docker network create "${net}"
+ env -u HUGEGRAPH_ADMIN_PASSWORD -u HUGEGRAPH_AUTH_TOKEN_SECRET \
+ docker compose -f docker-compose-3pd-3store-3server.yml config --quiet
+)
+```
+
+Then start the cluster:
+
```bash
cd docker
-HUGEGRAPH_VERSION=1.7.0 docker compose -f docker-compose-3pd-3store-3server.yml up -d
+docker compose -f docker-compose-3pd-3store-3server.yml up -d
-# To stop and remove all data volumes (clean restart)
+# To stop and remove all data volumes (clean restart).
+# The external hugegraph-net network is intentionally left in place.
+# If the Hubble add-on is running, see "Hubble for the 3-Node Cluster"
+# for the teardown that matches how it was started.
docker compose -f docker-compose-3pd-3store-3server.yml down -v
```
+Pin a release by setting `HUGEGRAPH_VERSION` in `docker/.env` — the
+cluster, the Hubble add-on, and the single-node quickstart file all read
+it from there, so those versions cannot drift apart
+(`docker-compose.dev.yml` builds PD/Store/Server from source and defaults
+Hubble to `hugegraph/hubble:latest`; set `HUBBLE_IMAGE` to pin it).
+Unpinned, the images default to `latest`; note the authenticated PD/Hubble
+integration requires a release newer than `1.7.x`. On an older image the
+Server ignores `PASSWORD` and `HG_SERVER_AUTH_TOKEN_SECRET`, which would
+leave you an unauthenticated cluster. Two defaults keep that from passing
+unnoticed: the cluster and add-on use `pull_policy: always`, so a stale
+cached `latest` is refreshed on every `up -d`, and the Server healthcheck
+requires an unauthenticated graph request to return `401` and an
+authenticated one to return `200`. An image that does not enforce
+authentication therefore never reports healthy, and `up -d --wait` fails
Review Comment:
The safety net promised here does not cover every file this paragraph claims it does. It
opens by saying the cluster, the add-on and the single-node quickstart all read
`HUGEGRAPH_VERSION` from `docker/.env`, then concludes that an image which does not
enforce authentication "never reports healthy".
Only the cluster file has the 401/200 check. The add-on ships no Server, so the sentence
does not apply to it; the single-node file is the gap. Its Server healthcheck is still
`curl -fsS http://server:8080/versions`, and `/versions` is open by design, as this README
says a few lines down. A reader who pins `HUGEGRAPH_VERSION` exactly as this paragraph
instructs gets, for the single-node file, the false green it promises to prevent.
Either scope the sentence to the cluster file or carry the same check into the single-node
file. Worth naming 1.8.0 here rather than "newer than `1.7.x`".
##########
docker/docker-compose-hubble.yml:
##########
@@ -0,0 +1,64 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+# Hubble add-on for the distributed cluster defined in
+# docker-compose-3pd-3store-3server.yml. Requires the pre-created cluster
+# network (docker network create hugegraph-net). See "Hubble for the
+# 3-Node Cluster" in docker/README.md for the attach and combined flows.
+
+networks:
+ hg-net:
+ external: true
+ name: ${HUGEGRAPH_NETWORK:-hugegraph-net}
+
+volumes:
+ hg-hubble-db:
+ # Explicit names so attach (-p hugegraph-hubble) and combined
+ # (project hugegraph-3x3) share the same physical volumes.
+ name: ${HUBBLE_DB_VOLUME:-hugegraph-hubble-db}
+ hg-hubble-upload-files:
+ name: ${HUBBLE_UPLOAD_VOLUME:-hugegraph-hubble-upload-files}
+
+services:
+ hubble:
+ # Pin a release via HUGEGRAPH_VERSION in docker/.env; unset, the image
+ # tag defaults to latest. Pull by default so an older cached image cannot
+ # silently satisfy an auth-sensitive deployment.
+ image: ${HUBBLE_IMAGE:-hugegraph/hubble:${HUGEGRAPH_VERSION:-latest}}
+ pull_policy: ${HUBBLE_PULL_POLICY:-always}
+ container_name: hg-hubble
+ hostname: hubble
+ restart: unless-stopped
+ networks: [hg-net]
+ ports:
+ - "${HUBBLE_PUBLISH_HOST:-127.0.0.1}:8088:8088"
+ environment:
+ # Image default jdbc:h2:file:./db writes /hubble/db.mv.db, outside
+ # the /hubble/db volume. Point H2 at a file inside the mount.
+ SPRING_DATASOURCE_URL: jdbc:h2:file:./db/hubble;DB_CLOSE_ON_EXIT=FALSE
Review Comment:
Nit: this fixes the H2 path with the same working-directory-relative mechanism that caused
the original problem. It lands inside the volume only while the container's working
directory stays `/hubble` (which it is today), so an image that changes WORKDIR silently
writes the database outside the mount again.
```suggestion
SPRING_DATASOURCE_URL: jdbc:h2:file:/hubble/db/hubble;DB_CLOSE_ON_EXIT=FALSE
```
Anchored to the mount target declared just below under `volumes:`. The two CI assertions
need the same string.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(docker): add attachable Hubble Compose add-on for the 3-node cluster [hugegraph]
Posted by "bitflicker64 (via GitHub)" <gi...@apache.org>.
bitflicker64 commented on code in PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149#discussion_r3852746430
##########
docker/docker-compose-3pd-3store-3server.yml:
##########
@@ -187,16 +207,25 @@ services:
<<: *server-common
container_name: hg-server0
hostname: server0
+ environment:
+ <<: *server-env
+ HG_SERVER_REST_URL: http://server0:8080
Review Comment:
Correction to my earlier reply on this thread: the fix I described does not work and has been removed in 96160375.
`HG_SERVER_REST_URL` becomes `restserver.url`, which is the address the REST server binds, not only the one registered with PD. Running the cluster with the documented override crash-looped that replica on `java.net.BindException: Cannot assign requested address` while the two default replicas stayed healthy.
Your original point stands: this needs a separate advertised-address setting in the Server. The README now documents the limitation and points external clients at the published ports instead of PD discovery.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(docker): add attachable Hubble Compose add-on for the 3-node cluster [hugegraph]
Posted by "bitflicker64 (via GitHub)" <gi...@apache.org>.
bitflicker64 commented on PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149#issuecomment-5431142330
@imbajin one open question before this is mergeable, on the PD-registered Server addresses. The inline thread is marked outdated (the lines it pointed at are gone), so raising it here where it is visible: https://github.com/apache/hugegraph/pull/3149#discussion_r3770576277
Your ask was a configurable advertised address, or an external client path that resolves. I could not do either from Compose: `HG_SERVER_REST_URL` becomes `restserver.url`, which is also the address the REST server binds, so an externally reachable value crash-loops the replica on `java.net.BindException: Cannot assign requested address`. I confirmed that by running it. Splitting bind from advertise needs a Server-side setting.
So, which way do you want it closed?
**A.** Accept it as a documented limitation. External clients use the published Server ports; PD discovery stays correct for anything on `hugegraph-net`, which is the Hubble case this add-on exists for.
**B.** I open a Server issue for a real advertised-address option and link it here.
I lean A for this PR, since B is a Server change that would hold up the add-on. Happy to file B regardless if you want it tracked.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(docker): add attachable Hubble Compose add-on for the 3-node cluster [hugegraph]
Posted by "bitflicker64 (via GitHub)" <gi...@apache.org>.
bitflicker64 commented on code in PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149#discussion_r3866735042
##########
docker/docker-compose-3pd-3store-3server.yml:
##########
@@ -187,16 +207,25 @@ services:
<<: *server-common
container_name: hg-server0
hostname: server0
+ environment:
+ <<: *server-env
+ HG_SERVER_REST_URL: http://server0:8080
Review Comment:
@imbajin which way do you want this one closed?
**A.** Accept it as a documented limitation and resolve the thread. External clients use the published Server ports; PD discovery stays correct for anything on `hugegraph-net`, which is what Hubble does.
**B.** I open a Server issue for a real advertised-address option, separate from `restserver.url` so bind and advertise can differ, and link it here.
I lean A for this PR, since B is a Server change and would hold up the add-on. Happy to file B either way if you want it tracked.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(docker): add attachable Hubble Compose add-on for the 3-node cluster [hugegraph]
Posted by "bitflicker64 (via GitHub)" <gi...@apache.org>.
bitflicker64 commented on code in PR #3149:
URL: https://github.com/apache/hugegraph/pull/3149#discussion_r3767823505
##########
docker/hugegraph-hubble-3x3.properties:
##########
@@ -0,0 +1,33 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+server.host=0.0.0.0
+server.port=8088
+
+cluster=hg
+idc=docker
+
+pd.enabled=true
+# Bootstrap target; the live topology comes from PD discovery. If this
+# replica is down when Hubble starts, point it at another server.
+server.direct_url=http://server0:8080
+pd.peers=pd0:8686,pd1:8686,pd2:8686
+# PD REST endpoint for the operations view (single address).
+pd.server=pd0:8620
Review Comment:
Addressed as an explicit operational limitation in . Hubble's current accepts one value for the operations HTTP calls; is used for discovery, not HTTP failover. The properties file and README now state that loss blinds the operations view and instruct operators to repoint it to a surviving PD. Automatic failover would require a separate Hubble code change.
##########
docker/hugegraph-hubble-3x3.properties:
##########
@@ -0,0 +1,33 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+server.host=0.0.0.0
+server.port=8088
+
+cluster=hg
+idc=docker
+
+pd.enabled=true
+# Bootstrap target; the live topology comes from PD discovery. If this
+# replica is down when Hubble starts, point it at another server.
+server.direct_url=http://server0:8080
Review Comment:
Addressed in . The comment now says it is unused when and applies only to standalone () mode.
##########
docker/docker-compose-hubble.yml:
##########
@@ -0,0 +1,47 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+# Hubble add-on for the distributed cluster defined in
+# docker-compose-3pd-3store-3server.yml. Requires the pre-created cluster
+# network (docker network create hugegraph-net). See "Hubble for the
+# 3-Node Cluster" in docker/README.md for the attach and combined flows.
+
+networks:
+ hg-net:
+ external: true
+ name: ${HUGEGRAPH_NETWORK:-hugegraph-net}
+
+services:
+ hubble:
+ # Pin a release via HUGEGRAPH_VERSION in docker/.env; unset, the image
+ # tag defaults to latest.
+ image: ${HUBBLE_IMAGE:-hugegraph/hubble:${HUGEGRAPH_VERSION:-latest}}
+ pull_policy: ${HUBBLE_PULL_POLICY:-missing}
+ container_name: hg-hubble
+ hostname: hubble
+ restart: unless-stopped
+ networks: [hg-net]
+ ports:
+ - "${HUBBLE_PUBLISH_HOST:-127.0.0.1}:8088:8088"
+ volumes:
+ - ./hugegraph-hubble-3x3.properties:/hubble/conf/hugegraph-hubble.properties:ro
Review Comment:
Addressed in . The Hubble add-on now mounts named volumes and ; CI asserts both mounts, and the README documents that H2 state and uploaded files survive Hubble recreation.
##########
docker/README.md:
##########
@@ -205,6 +283,110 @@ curl http://localhost:8620/v1/partitions
---
+## Hubble for the 3-Node Cluster
+
+`docker-compose-hubble.yml` defines only the Hubble service. It joins the
+cluster's external network (`hugegraph-net` by default, override with
+`HUGEGRAPH_NETWORK`) and has no `depends_on` on cluster services, so
+starting, stopping, or upgrading Hubble never recreates or restarts PD,
+Store, or Server containers. Hubble reads the cluster topology from
+`hugegraph-hubble-3x3.properties`; adjust that file when attaching to a
+cluster with different hostnames.
+
+Sign in at `http://localhost:8088` as `admin` with the
+`HUGEGRAPH_ADMIN_PASSWORD` from `docker/.env`. Hubble binds to host
+loopback by default (`HUBBLE_PUBLISH_HOST`, same caveats as the
+single-node setup).
+
+The two flows below create Hubble in different Compose projects, so manage
+Hubble with the same flags you started it with: the attach flow always uses
+`-p hugegraph-hubble -f docker-compose-hubble.yml`, the combined flow always
+uses both `-f` flags. The explicit `-p` keeps the attach project independent
+of the directory name and of other Compose projects.
+
+Run one Hubble per host: the single-node stack and both add-on flows all
+publish `127.0.0.1:8088` and name their container `hg-hubble`. The two
+add-on flows are therefore mutually exclusive — starting one while the
+other's Hubble exists fails with a container-name conflict, so `down` the
+flow you are leaving before switching.
+
+### Attach to a running cluster
+
+With the 3-node cluster already up:
+
+```bash
+cd docker
+docker compose -p hugegraph-hubble -f docker-compose-hubble.yml up -d
Review Comment:
Addressed in . The README now distinguishes attach mode from the combined flow, explains that attach requires the existing cluster to already expose the new PD-registration/auth settings, and documents the upgrade-first path for older clusters.
##########
.github/workflows/server-ci.yml:
##########
@@ -138,6 +138,107 @@ jobs:
check_compose docker/docker-compose.yml always always
check_compose docker/docker-compose.dev.yml build missing
+ check_cluster_compose() {
+ local cluster="docker/docker-compose-3pd-3store-3server.yml"
+ local addon="docker/docker-compose-hubble.yml"
+ local rendered
+ rendered="$(mktemp)"
+
+ # Both cluster credentials are required and may not be empty.
+ if HUGEGRAPH_AUTH_TOKEN_SECRET=ci-test-secret \
+ env -u HUGEGRAPH_ADMIN_PASSWORD \
+ docker compose -f "$cluster" config -q >/dev/null 2>&1; then
+ echo "$cluster accepted an unset admin password" >&2
+ return 1
+ fi
+ if HUGEGRAPH_ADMIN_PASSWORD= \
+ HUGEGRAPH_AUTH_TOKEN_SECRET=ci-test-secret \
+ docker compose -f "$cluster" config -q >/dev/null 2>&1; then
+ echo "$cluster accepted an empty admin password" >&2
+ return 1
+ fi
+ if HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ env -u HUGEGRAPH_AUTH_TOKEN_SECRET \
+ docker compose -f "$cluster" config -q >/dev/null 2>&1; then
+ echo "$cluster accepted an unset token secret" >&2
+ return 1
+ fi
+ if HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ HUGEGRAPH_AUTH_TOKEN_SECRET= \
+ docker compose -f "$cluster" config -q >/dev/null 2>&1; then
+ echo "$cluster accepted an empty token secret" >&2
+ return 1
+ fi
+
+ # The add-on alone must define Hubble and nothing else, join the
+ # shared external network with its default name, and need no
+ # credentials or overrides.
+ env -u HUGEGRAPH_ADMIN_PASSWORD -u HUGEGRAPH_AUTH_TOKEN_SECRET \
+ -u HUGEGRAPH_NETWORK -u HUGEGRAPH_VERSION \
+ docker compose -f "$addon" config --format json > "$rendered"
+ jq -e '
+ (.services | keys) == ["hubble"] and
+ .networks."hg-net".external == true and
+ .networks."hg-net".name == "hugegraph-net" and
+ (.services.hubble | has("depends_on") | not) and
+ any(.services.hubble.volumes[];
+ .target == "/hubble/conf/hugegraph-hubble.properties" and
+ (.source | endswith("hugegraph-hubble-3x3.properties")))
+ ' "$rendered" >/dev/null
+
+ # The combined render carries the PD-registration and auth
+ # settings on every server replica and keeps Hubble on loopback.
+ # Rendered with non-default HUGEGRAPH_NETWORK/HUGEGRAPH_VERSION so
+ # CI fails if any file stops honoring the overrides (the add-on
+ # render above covers the defaults).
+ HUGEGRAPH_ADMIN_PASSWORD=ci-test-password \
+ HUGEGRAPH_AUTH_TOKEN_SECRET=ci-test-secret \
Review Comment:
Addressed in commit . The CI fixture is now (>=32 bytes), and the cluster render uses it for every Server replica. The guard tests still run with ; Compose validation is complemented by the repository entrypoint test, so the fixture length is checked before rendering.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org