You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hugegraph.apache.org by "imbajin (via GitHub)" <gi...@apache.org> on 2026/10/03 22:41:43 UTC

[PR] docs: document ToplingDB development integration [hugegraph-doc]

imbajin opened a new pull request, #510:
URL: https://github.com/apache/hugegraph-doc/pull/510

   The 2025 ToplingDB articles describe the earlier direct-plugin interface. Retain those articles and their public routes, and link them to English/Chinese guides for the unreleased runtime integration in [hugegraph/hugegraph#264](https://github.com/hugegraph/hugegraph/pull/264).
   
   The guides explain which process owns the JNI, trusted external JAR input, isolated provider data, and startup/shutdown and interrupted snapshot-recovery boundaries. The existing route map registers the new guides only for latest; historical versions do not claim equivalent pages. No published image, data conversion, completed native acceptance, or atomic whole-graph restore is implied. Coordinate this documentation's merge with implementation delivery.
   
   Validation: links, bilingual command parity, source contracts, strict Hugo and latest artifact validation, existing search rankings, and guide/history/language-switch browser checks passed. Desktop/mobile renders were inspected. Fourteen unchanged Git worktree/output-safety fixtures were omitted under the gh-only workflow; a full source-suite pass is not claimed. JNI release-compliance materials remain a separately deferred release TODO.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


Re: [PR] docs: document ToplingDB development integration [hugegraph-doc]

Posted by "bitflicker64 (via GitHub)" <gi...@apache.org>.
bitflicker64 commented on code in PR #510:
URL: https://github.com/apache/hugegraph-doc/pull/510#discussion_r4175878829


##########
content/en/docs/guides/toplingdb.md:
##########
@@ -0,0 +1,154 @@
+---
+title: "ToplingDB development integration"

Review Comment:
   Minor: The new guide is not listed in `content/en/docs/SUMMARY.md` or `content/cn/docs/SUMMARY.md`. Both files list every page under the Guides section (Security, Toolchain Local Testing, FAQ), and the SeaTunnel guide in 3287b608 (#464) added its SUMMARY entry with the page. Please add `- [ToplingDB (development)](guides/toplingdb)` to the English Guides list and `- [ToplingDB(开发版)](guides/toplingdb)` to the Chinese one, or say in the PR that the page is left out of SUMMARY on purpose while the feature is unreleased.



##########
content/en/docs/guides/toplingdb.md:
##########
@@ -0,0 +1,154 @@
+---
+title: "ToplingDB development integration"
+linkTitle: "ToplingDB (development)"
+weight: 11
+description: "Build the optional ToplingDB runtime from development source, isolate its data, and understand startup and recovery boundaries."
+---
+
+> **Unreleased development interface.** This guide describes the proposed replacement for the 2025 ToplingDB integration.
+> It requires a complete source checkout containing the integration and its lifecycle and snapshot-recovery prerequisites.
+> These changes are under review; existing HugeGraph releases and registry tags do not establish support for this interface.
+> Follow the [integration proposal](https://github.com/hugegraph/hugegraph/pull/264) for its delivery status.
+
+## Choose the process that owns the database
+
+| Deployment | Process that loads Topling JNI | Provider setting |
+|---|---|---|
+| Standalone RocksDB Server | Server | `rocksdb.provider=topling` in `conf/graphs/hugegraph.properties` |
+| HStore PD | Each PD | `provider: topling` under `rocksdb` in `conf/application.yml` |
+| HStore Store | Each Store | `provider: topling` under `rocksdb` in `conf/application-pd.yml` |
+| HStore-backed Server | None | No local Topling setting |
+
+PD and Store select their providers independently. The HStore-backed Server is a remote client and does not need a Topling JAR or native library.
+Provider selection happens at process startup. All local RocksDB graphs within one Server process must agree on the provider.
+
+The historical `rocksdb.option_path` and `rocksdb.open_http` examples describe a different integration.
+Do not copy that configuration into this development interface. The component's launch scripts select the Easy Migrate YAML file.
+
+## Build an optional distribution
+
+Build on Linux x86_64 with Java 11+, Maven 3.5+, `rsync`, `unzip`, and `tar`.
+The Server mount preflight additionally requires util-linux 2.37+ `mountpoint` on `PATH`.
+Native macOS execution is outside this integration's support matrix; use a Linux x86_64 container when working on macOS.
+
+Obtain a compatible Topling Easy Migrate JNI JAR from its producer, together with its source revision, dependency licenses, notices, and trusted SHA-256.
+The HugeGraph source tree does not include this binary. A checksum identifies an artifact; it does not establish its origin, license, or runtime compatibility.
+Verify redistribution rights and retain the required license and notice files before distributing a derived package or image.
+
+From the complete source checkout containing this feature:
+
+```bash
+VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout)
+mvn clean package \
+  -pl hugegraph-server/hugegraph-dist,hugegraph-pd/hg-pd-dist,hugegraph-store/hg-store-dist \
+  -am -Dmaven.test.skip=true -Dmaven.javadoc.skip=true -ntp
+
+export TOPLING_JNI_JAR=/absolute/path/to/rocksdbjni-topling.jar
+export TOPLING_JNI_SHA256='<trusted 64-character SHA-256>'
+install-dist/scripts/build-topling-distribution.sh server "$VERSION"
+# Generate these only when running HStore:
+install-dist/scripts/build-topling-distribution.sh pd "$VERSION"
+install-dist/scripts/build-topling-distribution.sh store "$VERSION"
+```
+
+The Maven build creates standard packages. The generator creates separate `-topling` directories and archives beside them.
+It copies the external JAR into private staging, verifies that copy, and prepares the native library.
+Missing inputs, a checksum mismatch, or incompatible JAR structure stop generation; the generator does not search a Maven cache or download a substitute.
+Build success is not a native runtime acceptance test.
+
+Each Topling package contains `lib/topling/rocksdbjni-topling.jar`, `lib/topling/runtime.properties`,
+`library/librocksdbjni-linux64.so`, and the component's preparation and preload scripts.
+Run `bin/prepare-topling.sh` again after replacing the installed Topling JAR.
+
+For container builds, use the matching checkout's `docker/README.md` and `docker/bake.hcl` with the same external JNI inputs.
+Build locally before using the example `topling` tags and select `pull_policy: never` for that local deployment.
+For registry images, verify the source revision, JNI identity, and digest first. A tag or a healthy container alone does not prove which JNI is running.
+
+## Configure isolated data
+
+Changing the provider does not migrate or convert data. Give Topling a separate data directory and keep the standard RocksDB directory unchanged.
+Use these component-specific settings:
+
+| Component | Data setting | Example Topling root |
+|---|---|---|
+| Server | `rocksdb.data_path` | `/srv/hugegraph/topling/server` |
+| PD | `pd.data-path` | `/srv/hugegraph/topling/pd` |
+| Store | `app.data-path` | `/srv/hugegraph/topling/store` |
+
+For a standalone Server, edit the generated distribution's `conf/graphs/hugegraph.properties`:
+
+```properties
+backend=rocksdb
+rocksdb.provider=topling
+rocksdb.data_path=/srv/hugegraph/topling/server

Review Comment:
   Important: This example sets only `rocksdb.data_path`, so the Server WAL stays at the default `rocksdb.wal_path=rocksdb-data/wal`, which resolves inside the generated `-topling` distribution directory and not under `/srv/hugegraph/topling/server`.
   
   Evidence: `RocksDBOptions.WAL_PATH` defaults to `rocksdb-data/wal`. In hugegraph/hugegraph#264 at 9af9d14, `server_verify_graph_roots` in `bin/rocksdb-server-config.sh` treats both `rocksdb.data_path` and `rocksdb.wal_path` as provider-owned roots. `install-dist/scripts/build-topling-distribution.sh` moves an existing `-topling` directory to `$STAGING_ROOT/previous-directory` and its success cleanup runs `rm -rf "$STAGING_ROOT"`. Standalone (non-raft) Server writes keep the WAL enabled (`RocksDBStdSessions.StdSession` disables it only in raft mode) and HugeGraph does not flush memtables on close. A user who follows this page and later re-runs the generator command from the build section, or who keeps only the mounted data root as the mount paragraph says, loses every write not yet flushed to SST files, and the data directory still opens.
   
   Please add `rocksdb.wal_path` to the component table and to this properties block (for example `rocksdb.data_path=/srv/hugegraph/topling/server/data` and `rocksdb.wal_path=/srv/hugegraph/topling/server/wal`), state that both paths must sit under the mounted Topling root, and make the same change at line 83 of content/cn/docs/guides/toplingdb.md.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


Re: [PR] docs: document ToplingDB development integration [hugegraph-doc]

Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on code in PR #510:
URL: https://github.com/apache/hugegraph-doc/pull/510#discussion_r4176198151


##########
content/cn/docs/guides/toplingdb.md:
##########
@@ -0,0 +1,153 @@
+---
+title: "ToplingDB 开发版集成"
+linkTitle: "ToplingDB(开发版)"
+weight: 11
+description: "从开发源码构建可选 ToplingDB 运行时,隔离数据目录,并了解启动与恢复边界。"
+---
+
+> **尚未发布的开发接口。** 本文介绍用于替换 2025 年 ToplingDB 集成的候选实现。
+> 需要使用包含该集成及其生命周期、快照恢复前置修改的完整源码。
+> 这些修改仍在审查中;已有 HugeGraph 发行版和镜像标签不能证明其支持本文接口。
+> 交付状态见[集成提案](https://github.com/hugegraph/hugegraph/pull/264)。
+
+## 确认哪个进程持有数据库
+
+| 部署方式 | 加载 Topling JNI 的进程 | 运行时配置 |
+|---|---|---|
+| 单机 RocksDB Server | Server | `conf/graphs/hugegraph.properties` 中的 `rocksdb.provider=topling` |
+| HStore PD | 每个 PD | `conf/application.yml` 中 `rocksdb` 下的 `provider: topling` |
+| HStore Store | 每个 Store | `conf/application-pd.yml` 中 `rocksdb` 下的 `provider: topling` |
+| 使用 HStore 的 Server | 无 | 不配置本地 Topling 运行时 |
+
+PD 和 Store 分别选择自己的运行时。使用 HStore 的 Server 是远程客户端,不需要 Topling JAR 或原生库。
+运行时在进程启动时选定。同一 Server 进程中的所有本地 RocksDB 图必须使用相同的运行时。
+
+历史文档中的 `rocksdb.option_path` 和 `rocksdb.open_http` 示例属于另一套集成方式。
+不要将这些配置复制到本文的开发接口;组件启动脚本负责选择 Easy Migrate YAML 文件。
+
+## 构建可选发行包
+
+在 Linux x86_64 上构建,需要 Java 11+、Maven 3.5+、`rsync`、`unzip` 和 `tar`。
+Server 挂载预检查还要求 `PATH` 中有 util-linux 2.37+ 的 `mountpoint`。
+此集成不支持在 macOS 原生运行;macOS 用户应使用 Linux x86_64 容器。
+
+从生产方获取兼容的 Topling Easy Migrate JNI JAR,以及对应源码版本、依赖许可证、声明文件和可信 SHA-256。
+HugeGraph 源码不包含该二进制文件。校验和只能识别产物,不能证明来源、许可证或运行时兼容性。
+分发衍生包或镜像前,核查再分发权限并保留所需许可证和声明文件。
+
+在包含此功能的完整源码目录中执行:
+
+```bash
+VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout)
+mvn clean package \
+  -pl hugegraph-server/hugegraph-dist,hugegraph-pd/hg-pd-dist,hugegraph-store/hg-store-dist \
+  -am -Dmaven.test.skip=true -Dmaven.javadoc.skip=true -ntp
+
+export TOPLING_JNI_JAR=/absolute/path/to/rocksdbjni-topling.jar
+export TOPLING_JNI_SHA256='<trusted 64-character SHA-256>'
+install-dist/scripts/build-topling-distribution.sh server "$VERSION"
+# 仅在运行 HStore 时生成以下两个组件:
+install-dist/scripts/build-topling-distribution.sh pd "$VERSION"
+install-dist/scripts/build-topling-distribution.sh store "$VERSION"
+```
+
+构建命令先生成标准发行包。生成脚本在其旁边创建独立的 `-topling` 目录及压缩包。
+脚本将外部 JAR 复制到私有暂存目录,校验该副本后再准备原生库。
+缺少输入、校验和不一致或 JAR 结构不符合要求时,生成过程会停止;脚本不会搜索 Maven 缓存或下载替代文件。
+构建成功不等于原生运行时验收通过。
+
+每个 Topling 包包含 `lib/topling/rocksdbjni-topling.jar`、`lib/topling/runtime.properties`、
+`library/librocksdbjni-linux64.so` 以及组件的准备和预加载脚本。
+更换已安装的 Topling JAR 后,重新运行 `bin/prepare-topling.sh`。
+
+构建容器时,使用同一源码中的 `docker/README.md` 和 `docker/bake.hcl`,并传入相同的外部 JNI 参数。
+先在本地完成构建,再使用示例中的 `topling` 标签,本地部署选择 `pull_policy: never`。
+使用镜像仓库中的产物前,先核对源码版本、JNI 身份和镜像摘要。标签或容器健康状态本身不能证明实际运行的 JNI。
+
+## 配置独立数据目录
+
+切换运行时不会迁移或转换数据。为 Topling 设置独立目录,保留标准 RocksDB 目录不变。
+各组件使用以下配置:
+
+| 组件 | 数据配置 | Topling 根目录示例 |
+|---|---|---|
+| Server | `rocksdb.data_path` | `/srv/hugegraph/topling/server` |
+| PD | `pd.data-path` | `/srv/hugegraph/topling/pd` |
+| Store | `app.data-path` | `/srv/hugegraph/topling/store` |
+
+单机 Server 修改生成包中的 `conf/graphs/hugegraph.properties`:
+
+```properties
+backend=rocksdb
+rocksdb.provider=topling
+rocksdb.data_path=/srv/hugegraph/topling/server
+```
+
+启动前,挂载或创建实际的 Topling 根目录,不要使用包含符号链接的路径。
+挂载父数据根目录,不要单独挂载 `data/g` 等数据库目录,确保所有遵循恢复协议的进程都能看到同级恢复文件。
+单机 Server 启动脚本在 Java 启动前拒绝单独挂载的数据库目录,包括 Linux 上同一文件系统的绑定挂载。
+启动和恢复期间保持挂载布局不变。
+
+`.hugegraph-rocksdb-provider` 标记用于防止误用目录,不负责数据转换。
+标记冲突时停止启动。Topling 也会拒绝没有标记的非空数据目录。
+标准 RocksDB 为兼容历史数据而接受已有的无标记目录,但这不意味着可以打开经 Topling 修改的目录。

Review Comment:
   [P1] 限定目录保护实际覆盖的启动路径
   
   这段未限定组件或启动方式,下文又直接指导运行裸包 start-hugegraph-pd.sh / start-hugegraph-store.sh,会让读者认为这些路径也必定拒绝冲突 marker 和无 marker 的非空目录。但配套实现 9af9d14 的两个裸包 launcher 只 source preload-topling.sh,该 helper 仅在 Server 分支调用 server_verify_graph_roots;PD/Store 的 ownership 校验目前在 Docker entrypoint,Java 侧也没有补充检查。因此这里对裸包部署的保护承诺不成立,可能让读者在没有预期保护时打开已有数据目录。请与实现 PR 的既有 ownership 问题联动修复;在实现完成前,中英文都应明确实际覆盖的路径和裸包限制。现有实现线程:https://github.com/hugegraph/hugegraph/pull/264#discussion_r4175351859 。



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


Re: [PR] docs: explain ToplingDB runtime setup (2/4) [hugegraph-doc]

Posted by "bitflicker64 (via GitHub)" <gi...@apache.org>.
bitflicker64 commented on code in PR #510:
URL: https://github.com/apache/hugegraph-doc/pull/510#discussion_r4255562251


##########
content/en/docs/guides/toplingdb.md:
##########
@@ -0,0 +1,125 @@
+---
+title: "ToplingDB development integration"
+linkTitle: "ToplingDB (development)"
+weight: 11
+description: "Build the optional ToplingDB runtime from development source, isolate its data, and understand startup and recovery boundaries."
+---
+
+> **Unreleased development interface.** This guide follows the [three-component runtime integration](https://github.com/apache/hugegraph/pull/3275).
+> Use source containing that change. Existing releases and image tags do not establish support, and runtime acceptance remains a deployment prerequisite.
+
+## Choose the database owner
+
+HugeGraph defaults to standard RocksDB. Enable Topling separately for each process that owns local storage:
+
+| Component | Business provider setting | Start command |
+| --- | --- | --- |
+| Standalone RocksDB Server | `rocksdb.provider=topling` in each selected graph properties file | `bin/start-hugegraph.sh` |
+| PD | `provider: topling` under `rocksdb` in `conf/application.yml` | `bin/start-hugegraph-pd.sh` |
+| Store | `provider: topling` under `rocksdb` in `conf/application-pd.yml` | `bin/start-hugegraph-store.sh` |
+
+A standalone Server keeps `backend=rocksdb`; an HStore-backed Server uses `backend=hstore` and needs no local Topling runtime.
+All local RocksDB graphs in one Server process must use the same provider. The business setting must match the runtime selected by its launcher.
+
+## Build and prepare a standard distribution
+
+Build the normal distributions from the source repository root using Java 17 and Maven 3.6.3 or later:
+
+```bash
+mvn clean package -Dmaven.test.skip=true -Dmaven.javadoc.skip=true
+```
+
+Obtain a trusted Topling Easy Migrate JNI JAR and its SHA-256 through your artifact channel. Preparation requires a Java 17 JDK, Linux x86_64,
+`sha256sum`, `unzip`, `od`, `ldd`, and GNU `mv`. The selected JAR's native dependencies, including its glibc and libaio requirements, must be installed.
+
+Stop the component, unpack a fresh standard distribution, and run from that component directory:
+
+```bash
+export TOPLING_JNI_JAR=/absolute/path/to/rocksdbjni-topling.jar
+export TOPLING_JNI_SHA256='<trusted-64-character-sha256>'
+bash bin/prepare-topling.sh
+```
+
+The script verifies the copied JAR's hash, Topling Java marker, one Linux x86_64 JNI entry, ELF architecture and native dependencies.
+Before installation, a disposable database verifies that the selected JNI actually applies EasyMigrate configuration to its persisted write-buffer size.
+Unsupported builds are rejected even when their marker and hash match. The probe does not certify existing-data compatibility or every production setting.
+A checksum receipt binds this verification to the installed JAR and native library; launchers reject missing or changed receipts. Use a fresh distribution
+when upgrading from an older preparation script. Launch rejects inherited RocksDB JNI preloads in default/standard mode and competing JNI in Topling mode, while preserving unrelated preloads.
+It installs `topling/rocksdbjni.jar`, its native library and optional web resources outside `lib`. Preparation refuses an existing `topling` directory;
+replace a runtime by preparing another stopped, fresh distribution. Repeat preparation separately for Server, PD and Store as applicable.
+
+## Keep data outside the distribution
+
+Changing providers does not convert an existing database. Use separate, initially empty Topling directories and retain the standard-provider data.
+Create persistent directories writable by the service account, outside source checkouts, build outputs and unpacked distribution directories:
+
+| Component | Configuration file and key | Example persistent path |
+| --- | --- | --- |
+| Server data | Graph properties: `rocksdb.data_path` | `/srv/hugegraph/topling/server/data` |
+| Server WAL | Graph properties: `rocksdb.wal_path` | `/srv/hugegraph/topling/server/wal` |
+| PD | `conf/application.yml`: `pd.data-path` | `/srv/hugegraph/topling/pd` |
+| Store data | `conf/application.yml`: `app.data-path` | `/srv/hugegraph/topling/store/data` |
+| Store Raft | `conf/application.yml`: `app.raft-path` | `/srv/hugegraph/topling/store/raft` |
+
+For a standalone Server, edit every selected graph configuration, for example `conf/graphs/hugegraph.properties`:

Review Comment:
   Important: This paragraph tells readers to edit every selected graph configuration, but the block below gives one fixed `rocksdb.data_path` and `rocksdb.wal_path`. Copied into two graph files, both graphs point at the same RocksDB directories.
   
   Evidence, from apache/hugegraph#3275 at 96f99de: `RocksDBStore.wrapPath()` appends only the store name, and the store names are the fixed `m`, `g` and `s`, so both graphs resolve to `/srv/hugegraph/topling/server/data/g`, `/m` and `/s`. The `_<graph>` suffix is added only in `RocksDBStoreProvider.onCloneConfig()`, which does not run for files under `conf/graphs/`. The `dbs` map in `RocksDBStore` is per store instance, so a second open of the same directory has nothing to reuse and ends in `ConnectionException` (`Failed to open RocksDB '%s'`). From reading `InitStore`, the `bin/init-store.sh` step below hits this, because it initializes every non-HStore graph before closing any of them.
   
   Please say that each graph needs its own data and WAL directories and show graph-specific paths, such as `/srv/hugegraph/topling/server/hugegraph/data` and `.../hugegraph/wal`. The two Server rows in the path table above need the same change, and so do line 63 and the table in `content/cn/docs/guides/toplingdb.md`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


Re: [PR] docs: document ToplingDB development integration [hugegraph-doc]

Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on code in PR #510:
URL: https://github.com/apache/hugegraph-doc/pull/510#discussion_r4179655899


##########
content/cn/docs/guides/toplingdb.md:
##########
@@ -0,0 +1,153 @@
+---
+title: "ToplingDB 开发版集成"
+linkTitle: "ToplingDB(开发版)"
+weight: 11
+description: "从开发源码构建可选 ToplingDB 运行时,隔离数据目录,并了解启动与恢复边界。"
+---
+
+> **尚未发布的开发接口。** 本文介绍用于替换 2025 年 ToplingDB 集成的候选实现。
+> 需要使用包含该集成及其生命周期、快照恢复前置修改的完整源码。
+> 这些修改仍在审查中;已有 HugeGraph 发行版和镜像标签不能证明其支持本文接口。
+> 交付状态见[集成提案](https://github.com/hugegraph/hugegraph/pull/264)。
+
+## 确认哪个进程持有数据库
+
+| 部署方式 | 加载 Topling JNI 的进程 | 运行时配置 |
+|---|---|---|
+| 单机 RocksDB Server | Server | `conf/graphs/hugegraph.properties` 中的 `rocksdb.provider=topling` |
+| HStore PD | 每个 PD | `conf/application.yml` 中 `rocksdb` 下的 `provider: topling` |
+| HStore Store | 每个 Store | `conf/application-pd.yml` 中 `rocksdb` 下的 `provider: topling` |
+| 使用 HStore 的 Server | 无 | 不配置本地 Topling 运行时 |
+
+PD 和 Store 分别选择自己的运行时。使用 HStore 的 Server 是远程客户端,不需要 Topling JAR 或原生库。
+运行时在进程启动时选定。同一 Server 进程中的所有本地 RocksDB 图必须使用相同的运行时。
+
+历史文档中的 `rocksdb.option_path` 和 `rocksdb.open_http` 示例属于另一套集成方式。
+不要将这些配置复制到本文的开发接口;组件启动脚本负责选择 Easy Migrate YAML 文件。
+
+## 构建可选发行包
+
+在 Linux x86_64 上构建,需要 Java 11+、Maven 3.5+、`rsync`、`unzip` 和 `tar`。
+Server 挂载预检查还要求 `PATH` 中有 util-linux 2.37+ 的 `mountpoint`。
+此集成不支持在 macOS 原生运行;macOS 用户应使用 Linux x86_64 容器。
+
+从生产方获取兼容的 Topling Easy Migrate JNI JAR,以及对应源码版本、依赖许可证、声明文件和可信 SHA-256。
+HugeGraph 源码不包含该二进制文件。校验和只能识别产物,不能证明来源、许可证或运行时兼容性。
+分发衍生包或镜像前,核查再分发权限并保留所需许可证和声明文件。
+
+在包含此功能的完整源码目录中执行:
+
+```bash
+VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout)
+mvn clean package \
+  -pl hugegraph-server/hugegraph-dist,hugegraph-pd/hg-pd-dist,hugegraph-store/hg-store-dist \
+  -am -Dmaven.test.skip=true -Dmaven.javadoc.skip=true -ntp
+
+export TOPLING_JNI_JAR=/absolute/path/to/rocksdbjni-topling.jar
+export TOPLING_JNI_SHA256='<trusted 64-character SHA-256>'
+install-dist/scripts/build-topling-distribution.sh server "$VERSION"
+# 仅在运行 HStore 时生成以下两个组件:
+install-dist/scripts/build-topling-distribution.sh pd "$VERSION"
+install-dist/scripts/build-topling-distribution.sh store "$VERSION"
+```
+
+构建命令先生成标准发行包。生成脚本在其旁边创建独立的 `-topling` 目录及压缩包。
+脚本将外部 JAR 复制到私有暂存目录,校验该副本后再准备原生库。
+缺少输入、校验和不一致或 JAR 结构不符合要求时,生成过程会停止;脚本不会搜索 Maven 缓存或下载替代文件。
+构建成功不等于原生运行时验收通过。
+
+每个 Topling 包包含 `lib/topling/rocksdbjni-topling.jar`、`lib/topling/runtime.properties`、
+`library/librocksdbjni-linux64.so` 以及组件的准备和预加载脚本。
+更换已安装的 Topling JAR 后,重新运行 `bin/prepare-topling.sh`。
+
+构建容器时,使用同一源码中的 `docker/README.md` 和 `docker/bake.hcl`,并传入相同的外部 JNI 参数。
+先在本地完成构建,再使用示例中的 `topling` 标签,本地部署选择 `pull_policy: never`。
+使用镜像仓库中的产物前,先核对源码版本、JNI 身份和镜像摘要。标签或容器健康状态本身不能证明实际运行的 JNI。
+
+## 配置独立数据目录
+
+切换运行时不会迁移或转换数据。为 Topling 设置独立目录,保留标准 RocksDB 目录不变。
+各组件使用以下配置:
+
+| 组件 | 数据配置 | Topling 根目录示例 |
+|---|---|---|
+| Server | `rocksdb.data_path` | `/srv/hugegraph/topling/server` |
+| PD | `pd.data-path` | `/srv/hugegraph/topling/pd` |
+| Store | `app.data-path` | `/srv/hugegraph/topling/store` |
+
+单机 Server 修改生成包中的 `conf/graphs/hugegraph.properties`:
+
+```properties
+backend=rocksdb
+rocksdb.provider=topling
+rocksdb.data_path=/srv/hugegraph/topling/server
+```
+
+启动前,挂载或创建实际的 Topling 根目录,不要使用包含符号链接的路径。
+挂载父数据根目录,不要单独挂载 `data/g` 等数据库目录,确保所有遵循恢复协议的进程都能看到同级恢复文件。
+单机 Server 启动脚本在 Java 启动前拒绝单独挂载的数据库目录,包括 Linux 上同一文件系统的绑定挂载。
+启动和恢复期间保持挂载布局不变。
+
+`.hugegraph-rocksdb-provider` 标记用于防止误用目录,不负责数据转换。
+标记冲突时停止启动。Topling 也会拒绝没有标记的非空数据目录。
+标准 RocksDB 为兼容历史数据而接受已有的无标记目录,但这不意味着可以打开经 Topling 修改的目录。

Review Comment:
   已同步收窄中英文指南并配对新的 TP 核心 hugegraph/hugegraph#266:普通发行包使用显式 runtime 选择,启动时核对实际 JNI 与业务 provider;不再承诺 marker 认领、已有数据库归属识别或 Docker 目录保护。指南要求使用独立、初始为空的数据目录,并配置包外 data/WAL/Raft。目录归属强校验和 Docker 属于独立后续,当前文档没有把裸包路径说成已受这些保护。



##########
content/en/docs/guides/toplingdb.md:
##########
@@ -0,0 +1,163 @@
+---
+title: "ToplingDB development integration"
+linkTitle: "ToplingDB (development)"
+weight: 11
+description: "Build the optional ToplingDB runtime from development source, isolate its data, and understand startup and recovery boundaries."
+---
+
+> **Unreleased development interface.** This guide describes the proposed replacement for the 2025 ToplingDB integration.
+> It requires a complete source checkout containing the integration and its lifecycle and snapshot-recovery prerequisites.
+> These changes are under review; existing HugeGraph releases and registry tags do not establish support for this interface.
+> Follow the [integration proposal](https://github.com/hugegraph/hugegraph/pull/264) for its delivery status.
+
+## Choose the process that owns the database
+
+| Deployment | Process that loads Topling JNI | Provider setting |
+|---|---|---|
+| Standalone RocksDB Server | Server | `rocksdb.provider=topling` in `conf/graphs/hugegraph.properties` |
+| HStore PD | Each PD | `provider: topling` under `rocksdb` in `conf/application.yml` |
+| HStore Store | Each Store | `provider: topling` under `rocksdb` in `conf/application-pd.yml` |
+| HStore-backed Server | None | No local Topling setting |
+
+PD and Store select their providers independently. The HStore-backed Server is a remote client and does not need a Topling JAR or native library.
+Provider selection happens at process startup. All local RocksDB graphs within one Server process must agree on the provider.
+
+The historical `rocksdb.option_path` and `rocksdb.open_http` examples describe a different integration.
+Do not copy that configuration into this development interface. The component's launch scripts select the Easy Migrate YAML file.
+
+## Build an optional distribution
+
+Build on Linux x86_64 with Java 11+, Maven 3.5+, `rsync`, `unzip`, and `tar`.
+The Server mount preflight additionally requires util-linux 2.37+ `mountpoint` on `PATH`.
+Native macOS execution is outside this integration's support matrix; use a Linux x86_64 container when working on macOS.
+
+Obtain a compatible Topling Easy Migrate JNI JAR from its producer, together with its source revision, dependency licenses, notices, and trusted SHA-256.
+The HugeGraph source tree does not include this binary. A checksum identifies an artifact; it does not establish its origin, license, or runtime compatibility.
+Verify redistribution rights and retain the required license and notice files before distributing a derived package or image.
+
+From the complete source checkout containing this feature:
+
+```bash
+VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout)
+mvn clean package \
+  -pl hugegraph-server/hugegraph-dist,hugegraph-pd/hg-pd-dist,hugegraph-store/hg-store-dist \
+  -am -Dmaven.test.skip=true -Dmaven.javadoc.skip=true -ntp
+
+export TOPLING_JNI_JAR=/absolute/path/to/rocksdbjni-topling.jar
+export TOPLING_JNI_SHA256='<trusted 64-character SHA-256>'
+install-dist/scripts/build-topling-distribution.sh server "$VERSION"
+# Generate these only when running HStore:
+install-dist/scripts/build-topling-distribution.sh pd "$VERSION"
+install-dist/scripts/build-topling-distribution.sh store "$VERSION"
+```
+
+The Maven build creates standard packages. The generator creates separate `-topling` directories and archives beside them.
+It copies the external JAR into private staging, verifies that copy, and prepares the native library.
+Missing inputs, a checksum mismatch, or incompatible JAR structure stop generation; the generator does not search a Maven cache or download a substitute.
+Build success is not a native runtime acceptance test.
+
+Each Topling package contains `lib/topling/rocksdbjni-topling.jar`, `lib/topling/runtime.properties`,
+`library/librocksdbjni-linux64.so`, and the component's preparation and preload scripts.
+To change the JNI JAR, regenerate the component distribution with the new trusted JAR and SHA-256, then run preparation.
+`prepare-topling.sh` alone does not refresh the JAR checksum in `runtime.properties`.
+
+For container builds, use the matching checkout's `docker/README.md` and `docker/bake.hcl` with the same external JNI inputs.
+Build locally before using the example `topling` tags and select `pull_policy: never` for that local deployment.
+For registry images, verify the source revision, JNI identity, and digest first. A tag or a healthy container alone does not prove which JNI is running.
+
+## Configure isolated data
+
+Changing the provider does not migrate or convert data. Give Topling a separate data directory and keep the standard RocksDB directory unchanged.
+Use these component-specific settings:
+
+| Component | Data setting | Example Topling root |
+|---|---|---|
+| Server | `rocksdb.data_path` | `/srv/hugegraph/topling/server/data` |
+| Server WAL | `rocksdb.wal_path` | `/srv/hugegraph/topling/server/wal` |
+| PD | `pd.data-path` | `/srv/hugegraph/topling/pd` |
+| Store | `app.data-path` | `/srv/hugegraph/topling/store` |
+
+For a standalone Server, edit the generated distribution's `conf/graphs/hugegraph.properties`:
+
+```properties
+backend=rocksdb
+rocksdb.provider=topling
+rocksdb.data_path=/srv/hugegraph/topling/server/data
+rocksdb.wal_path=/srv/hugegraph/topling/server/wal
+```
+
+Both Server data and WAL paths must sit under the mounted `/srv/hugegraph/topling/server` root, outside the generated distribution. Do not leave WAL at its
+default path inside the distribution; regenerating the package replaces that directory.
+Mount or create the configured Topling root as a real directory before startup. Do not use symlinked path components.

Review Comment:
   Both guides now direct the service account to create persistent, initially empty directories outside the distribution before startup, and explicitly set both Server data and WAL paths. The example paths are /srv/hugegraph/topling/server/data and /srv/hugegraph/topling/server/wal. The guide now pairs with hugegraph/hugegraph#266, which omits the old shell ownership guards rather than claiming them for bare launchers. The actual standard/Topling Server init, write/read, clear/rewrite, normal stop/restart and dump-store acceptance has passed for that scoped implementation.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


Re: [PR] docs: document ToplingDB development integration [hugegraph-doc]

Posted by "bitflicker64 (via GitHub)" <gi...@apache.org>.
bitflicker64 commented on code in PR #510:
URL: https://github.com/apache/hugegraph-doc/pull/510#discussion_r4180479210


##########
content/en/docs/guides/toplingdb.md:
##########
@@ -0,0 +1,111 @@
+---
+title: "ToplingDB development integration"
+linkTitle: "ToplingDB (development)"
+weight: 11
+description: "Build the optional ToplingDB runtime from development source, isolate its data, and understand startup and recovery boundaries."
+---
+
+> **Unreleased development interface.** This guide follows the [three-component runtime integration](https://github.com/hugegraph/hugegraph/pull/266).
+> Use source containing that change. Existing releases and image tags do not establish support, and runtime acceptance remains a deployment prerequisite.
+
+## Choose the database owner
+
+HugeGraph defaults to standard RocksDB. Enable Topling separately for each process that owns local storage:
+
+| Component | Business provider setting | Start command |
+| --- | --- | --- |
+| Standalone RocksDB Server | `rocksdb.provider=topling` in each selected graph properties file | `bin/start-hugegraph.sh` |
+| PD | `provider: topling` under `rocksdb` in `conf/application.yml` | `bin/start-hugegraph-pd.sh` |
+| Store | `provider: topling` under `rocksdb` in `conf/application-pd.yml` | `bin/start-hugegraph-store.sh` |
+
+A standalone Server keeps `backend=rocksdb`; an HStore-backed Server uses `backend=hstore` and needs no local Topling runtime.
+All local RocksDB graphs in one Server process must use the same provider. The business setting must match the runtime selected by its launcher.
+
+## Build and prepare a standard distribution
+
+Build the normal distributions from the source repository root using its supported JDK and Maven:
+
+```bash
+mvn clean package -Dmaven.test.skip=true -Dmaven.javadoc.skip=true
+```
+
+Obtain a trusted Topling Easy Migrate JNI JAR and its SHA-256 through your artifact channel. Preparation requires Linux x86_64,
+`sha256sum`, `unzip`, `od`, `ldd`, and GNU `mv`. The selected JAR's native dependencies, including its glibc and libaio requirements, must be installed.
+
+Stop the component, unpack a fresh standard distribution, and run from that component directory:
+
+```bash
+export TOPLING_JNI_JAR=/absolute/path/to/rocksdbjni-topling.jar
+export TOPLING_JNI_SHA256='<trusted-64-character-sha256>'
+bash bin/prepare-topling.sh
+```
+
+The script verifies the copied JAR's hash, Topling Java marker, one Linux x86_64 JNI entry, ELF architecture and native dependencies.
+It installs `topling/rocksdbjni.jar`, its native library and optional web resources outside `lib`. Preparation refuses an existing `topling` directory;
+replace a runtime by preparing another stopped, fresh distribution. Repeat preparation separately for Server, PD and Store as applicable.
+
+## Keep data outside the distribution
+
+Changing providers does not convert an existing database. Use separate, initially empty Topling directories and retain the standard-provider data.
+Create persistent directories writable by the service account, outside source checkouts, build outputs and unpacked distribution directories:
+
+| Component | Configuration file and key | Example persistent path |
+| --- | --- | --- |
+| Server data | Graph properties: `rocksdb.data_path` | `/srv/hugegraph/topling/server/data` |
+| Server WAL | Graph properties: `rocksdb.wal_path` | `/srv/hugegraph/topling/server/wal` |
+| PD | `conf/application.yml`: `pd.data-path` | `/srv/hugegraph/topling/pd` |
+| Store data | `conf/application.yml`: `app.data-path` | `/srv/hugegraph/topling/store/data` |
+| Store Raft | `conf/application.yml`: `app.raft-path` | `/srv/hugegraph/topling/store/raft` |
+
+For a standalone Server, edit every selected graph configuration, for example `conf/graphs/hugegraph.properties`:
+
+```properties
+backend=rocksdb
+rocksdb.provider=topling
+rocksdb.data_path=/srv/hugegraph/topling/server/data
+rocksdb.wal_path=/srv/hugegraph/topling/server/wal
+```
+
+Set **both** Server data and WAL paths. Leaving WAL at its package-local default risks losing it when a build or package replacement removes that directory.
+Keep the external data, WAL and Raft roots across runtime/package changes.
+
+For PD and Store, edit the existing `rocksdb` mapping in the provider configuration file listed above, retaining its other options:
+
+```yaml
+rocksdb:
+  provider: topling
+```
+
+Set their persistent paths separately in `conf/application.yml`; keep the normal network addresses and cluster configuration.
+
+## Select, start and verify
+
+In the startup shell of each component that owns a Topling database, explicitly select the runtime:
+
+```bash
+export TOPLINGDB_ROCKSDB_PROVIDER=topling
+# Run this component's start command from the first table.
+```
+
+The launcher uses that component's `conf/toplingdb.yaml`. Set `TOPLINGDB_EASY_MIGRATE_CONF=/absolute/path/config.yaml` for an explicit override.
+The shipped native profile disables its HTTP server and keeps `memtable_as_log_index=false`, which the Java write path requires.
+Tune that profile for the host; it is separate from the business provider and data-path configuration.
+
+For a standalone Server, `bin/init-store.sh` and `bin/dump-store.sh` use the same explicit selection as `bin/start-hugegraph.sh`.
+For HStore, start PD, then Store, then the HStore-backed Server with its default runtime environment; do not enable local Topling on that Server.

Review Comment:
   Minor: "with its default runtime environment" does not tell readers to clear the variable they exported a few lines earlier. Readers who start PD, Store and Server from one shell will still have `TOPLINGDB_ROCKSDB_PROVIDER=topling` set when they start the Server.
   
   Evidence: at hugegraph/hugegraph#266 head 3ec2871, `bin/hugegraph-server.sh` and `bin/init-store.sh` source `bin/preload-topling.sh`. That script reads only `TOPLINGDB_ROCKSDB_PROVIDER`, not `backend`. On an unprepared HStore Server distribution it exits with `Error: run .../bin/prepare-topling.sh before enabling Topling`. That message tells the reader to prepare Topling on the Server, which this guide says not to do.
   
   Please say so explicitly, for example: "Start the HStore-backed Server from a shell where `TOPLINGDB_ROCKSDB_PROVIDER` is unset (or `rocksdb`)". Make the same change at line 95 of content/cn/docs/guides/toplingdb.md.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


Re: [PR] docs: explain ToplingDB runtime setup (2/4) [hugegraph-doc]

Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on code in PR #510:
URL: https://github.com/apache/hugegraph-doc/pull/510#discussion_r4187683995


##########
content/en/docs/guides/toplingdb.md:
##########
@@ -0,0 +1,111 @@
+---
+title: "ToplingDB development integration"
+linkTitle: "ToplingDB (development)"
+weight: 11
+description: "Build the optional ToplingDB runtime from development source, isolate its data, and understand startup and recovery boundaries."
+---
+
+> **Unreleased development interface.** This guide follows the [three-component runtime integration](https://github.com/hugegraph/hugegraph/pull/266).
+> Use source containing that change. Existing releases and image tags do not establish support, and runtime acceptance remains a deployment prerequisite.
+
+## Choose the database owner
+
+HugeGraph defaults to standard RocksDB. Enable Topling separately for each process that owns local storage:
+
+| Component | Business provider setting | Start command |
+| --- | --- | --- |
+| Standalone RocksDB Server | `rocksdb.provider=topling` in each selected graph properties file | `bin/start-hugegraph.sh` |
+| PD | `provider: topling` under `rocksdb` in `conf/application.yml` | `bin/start-hugegraph-pd.sh` |
+| Store | `provider: topling` under `rocksdb` in `conf/application-pd.yml` | `bin/start-hugegraph-store.sh` |
+
+A standalone Server keeps `backend=rocksdb`; an HStore-backed Server uses `backend=hstore` and needs no local Topling runtime.
+All local RocksDB graphs in one Server process must use the same provider. The business setting must match the runtime selected by its launcher.
+
+## Build and prepare a standard distribution
+
+Build the normal distributions from the source repository root using its supported JDK and Maven:
+
+```bash
+mvn clean package -Dmaven.test.skip=true -Dmaven.javadoc.skip=true
+```
+
+Obtain a trusted Topling Easy Migrate JNI JAR and its SHA-256 through your artifact channel. Preparation requires Linux x86_64,
+`sha256sum`, `unzip`, `od`, `ldd`, and GNU `mv`. The selected JAR's native dependencies, including its glibc and libaio requirements, must be installed.
+
+Stop the component, unpack a fresh standard distribution, and run from that component directory:
+
+```bash
+export TOPLING_JNI_JAR=/absolute/path/to/rocksdbjni-topling.jar
+export TOPLING_JNI_SHA256='<trusted-64-character-sha256>'
+bash bin/prepare-topling.sh
+```
+
+The script verifies the copied JAR's hash, Topling Java marker, one Linux x86_64 JNI entry, ELF architecture and native dependencies.
+It installs `topling/rocksdbjni.jar`, its native library and optional web resources outside `lib`. Preparation refuses an existing `topling` directory;
+replace a runtime by preparing another stopped, fresh distribution. Repeat preparation separately for Server, PD and Store as applicable.
+
+## Keep data outside the distribution
+
+Changing providers does not convert an existing database. Use separate, initially empty Topling directories and retain the standard-provider data.
+Create persistent directories writable by the service account, outside source checkouts, build outputs and unpacked distribution directories:
+
+| Component | Configuration file and key | Example persistent path |
+| --- | --- | --- |
+| Server data | Graph properties: `rocksdb.data_path` | `/srv/hugegraph/topling/server/data` |
+| Server WAL | Graph properties: `rocksdb.wal_path` | `/srv/hugegraph/topling/server/wal` |
+| PD | `conf/application.yml`: `pd.data-path` | `/srv/hugegraph/topling/pd` |
+| Store data | `conf/application.yml`: `app.data-path` | `/srv/hugegraph/topling/store/data` |
+| Store Raft | `conf/application.yml`: `app.raft-path` | `/srv/hugegraph/topling/store/raft` |
+
+For a standalone Server, edit every selected graph configuration, for example `conf/graphs/hugegraph.properties`:
+
+```properties
+backend=rocksdb
+rocksdb.provider=topling
+rocksdb.data_path=/srv/hugegraph/topling/server/data
+rocksdb.wal_path=/srv/hugegraph/topling/server/wal
+```
+
+Set **both** Server data and WAL paths. Leaving WAL at its package-local default risks losing it when a build or package replacement removes that directory.
+Keep the external data, WAL and Raft roots across runtime/package changes.
+
+For PD and Store, edit the existing `rocksdb` mapping in the provider configuration file listed above, retaining its other options:
+
+```yaml
+rocksdb:
+  provider: topling
+```
+
+Set their persistent paths separately in `conf/application.yml`; keep the normal network addresses and cluster configuration.
+
+## Select, start and verify
+
+In the startup shell of each component that owns a Topling database, explicitly select the runtime:
+
+```bash
+export TOPLINGDB_ROCKSDB_PROVIDER=topling
+# Run this component's start command from the first table.
+```
+
+The launcher uses that component's `conf/toplingdb.yaml`. Set `TOPLINGDB_EASY_MIGRATE_CONF=/absolute/path/config.yaml` for an explicit override.
+The shipped native profile disables its HTTP server and keeps `memtable_as_log_index=false`, which the Java write path requires.
+Tune that profile for the host; it is separate from the business provider and data-path configuration.
+
+For a standalone Server, `bin/init-store.sh` and `bin/dump-store.sh` use the same explicit selection as `bin/start-hugegraph.sh`.
+For HStore, start PD, then Store, then the HStore-backed Server with its default runtime environment; do not enable local Topling on that Server.

Review Comment:
   Updated both guides in f29822cb: before starting the HStore-backed Server from the same shell, explicitly run `unset TOPLINGDB_ROCKSDB_PROVIDER` (or set it to `rocksdb`). The PD/Store Topling opt-in no longer reads as the Server default. Focused link and bilingual consistency checks passed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


Re: [PR] docs: explain ToplingDB runtime setup (2/4) [hugegraph-doc]

Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on code in PR #510:
URL: https://github.com/apache/hugegraph-doc/pull/510#discussion_r4207998311


##########
content/en/docs/guides/toplingdb.md:
##########
@@ -0,0 +1,118 @@
+---
+title: "ToplingDB development integration"
+linkTitle: "ToplingDB (development)"
+weight: 11
+description: "Build the optional ToplingDB runtime from development source, isolate its data, and understand startup and recovery boundaries."
+---
+
+> **Unreleased development interface.** This guide follows the [three-component runtime integration](https://github.com/hugegraph/hugegraph/pull/266).
+> Use source containing that change. Existing releases and image tags do not establish support, and runtime acceptance remains a deployment prerequisite.
+
+## Choose the database owner
+
+HugeGraph defaults to standard RocksDB. Enable Topling separately for each process that owns local storage:
+
+| Component | Business provider setting | Start command |
+| --- | --- | --- |
+| Standalone RocksDB Server | `rocksdb.provider=topling` in each selected graph properties file | `bin/start-hugegraph.sh` |
+| PD | `provider: topling` under `rocksdb` in `conf/application.yml` | `bin/start-hugegraph-pd.sh` |
+| Store | `provider: topling` under `rocksdb` in `conf/application-pd.yml` | `bin/start-hugegraph-store.sh` |
+
+A standalone Server keeps `backend=rocksdb`; an HStore-backed Server uses `backend=hstore` and needs no local Topling runtime.
+All local RocksDB graphs in one Server process must use the same provider. The business setting must match the runtime selected by its launcher.
+
+## Build and prepare a standard distribution
+
+Build the normal distributions from the source repository root using Java 17 and Maven 3.6.3 or later:
+
+```bash
+mvn clean package -Dmaven.test.skip=true -Dmaven.javadoc.skip=true
+```
+
+Obtain a trusted Topling Easy Migrate JNI JAR and its SHA-256 through your artifact channel. Preparation requires a Java 17 JDK, Linux x86_64,
+`sha256sum`, `unzip`, `od`, `ldd`, and GNU `mv`. The selected JAR's native dependencies, including its glibc and libaio requirements, must be installed.
+
+Stop the component, unpack a fresh standard distribution, and run from that component directory:
+
+```bash
+export TOPLING_JNI_JAR=/absolute/path/to/rocksdbjni-topling.jar
+export TOPLING_JNI_SHA256='<trusted-64-character-sha256>'
+bash bin/prepare-topling.sh
+```
+
+The script verifies the copied JAR's hash, Topling Java marker, one Linux x86_64 JNI entry, ELF architecture and native dependencies.
+Before installation, a disposable database verifies that the selected JNI actually applies EasyMigrate configuration to its persisted write-buffer size.
+Unsupported builds are rejected even when their marker and hash match. The probe does not certify existing-data compatibility or every production setting.
+A checksum receipt binds this verification to the installed JAR and native library; launchers reject missing or changed receipts. Use a fresh distribution
+when upgrading from an older preparation script. Launch rejects inherited RocksDB JNI preloads in default/standard mode and competing JNI in Topling mode, while preserving unrelated preloads.
+It installs `topling/rocksdbjni.jar`, its native library and optional web resources outside `lib`. Preparation refuses an existing `topling` directory;
+replace a runtime by preparing another stopped, fresh distribution. Repeat preparation separately for Server, PD and Store as applicable.
+
+## Keep data outside the distribution
+
+Changing providers does not convert an existing database. Use separate, initially empty Topling directories and retain the standard-provider data.
+Create persistent directories writable by the service account, outside source checkouts, build outputs and unpacked distribution directories:
+
+| Component | Configuration file and key | Example persistent path |
+| --- | --- | --- |
+| Server data | Graph properties: `rocksdb.data_path` | `/srv/hugegraph/topling/server/data` |
+| Server WAL | Graph properties: `rocksdb.wal_path` | `/srv/hugegraph/topling/server/wal` |
+| PD | `conf/application.yml`: `pd.data-path` | `/srv/hugegraph/topling/pd` |
+| Store data | `conf/application.yml`: `app.data-path` | `/srv/hugegraph/topling/store/data` |
+| Store Raft | `conf/application.yml`: `app.raft-path` | `/srv/hugegraph/topling/store/raft` |
+
+For a standalone Server, edit every selected graph configuration, for example `conf/graphs/hugegraph.properties`:
+
+```properties
+backend=rocksdb
+rocksdb.provider=topling
+rocksdb.data_path=/srv/hugegraph/topling/server/data
+rocksdb.wal_path=/srv/hugegraph/topling/server/wal
+```
+
+Set **both** Server data and WAL paths. Leaving WAL at its package-local default risks losing it when a build or package replacement removes that directory.
+Keep the external data, WAL and Raft roots across runtime/package changes.
+
+For PD and Store, edit the existing `rocksdb` mapping in the provider configuration file listed above, retaining its other options:
+
+```yaml
+rocksdb:
+  provider: topling
+```
+
+Set their persistent paths separately in `conf/application.yml`; keep the normal network addresses and cluster configuration.
+
+The published runtime directories use 0755 and regular files use 0644, so a service account different from the preparer can read the JNI. The service account also needs traversal permission on the component's parent directories; preparation does not change those parents.
+
+## Select, start and verify
+
+In the startup shell of each component that owns a Topling database, explicitly select the runtime:
+
+```bash
+export TOPLINGDB_ROCKSDB_PROVIDER=topling
+# Run this component's start command from the first table.
+```
+
+The launcher uses that component's `conf/toplingdb.yaml`. Set `TOPLINGDB_EASY_MIGRATE_CONF=/absolute/path/config.yaml` for an explicit override.
+The shipped native profile disables its HTTP server and keeps `memtable_as_log_index=false`, which the Java write path requires.
+Tune that profile for the host; it is separate from the business provider and data-path configuration.
+
+For a standalone Server, `bin/init-store.sh` and `bin/dump-store.sh` use the same explicit selection as `bin/start-hugegraph.sh`.

Review Comment:
   Updated both English and Chinese standalone Server instructions to run `init-store.sh` once before the first `start-hugegraph.sh` for a fresh graph. Published in [5363a1](https://github.com/apache/hugegraph-doc/commit/5363a1); bilingual link validation passed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


Re: [PR] docs: explain ToplingDB runtime setup (2/4) [hugegraph-doc]

Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on code in PR #510:
URL: https://github.com/apache/hugegraph-doc/pull/510#discussion_r4203255397


##########
content/en/docs/guides/toplingdb.md:
##########
@@ -0,0 +1,118 @@
+---
+title: "ToplingDB development integration"
+linkTitle: "ToplingDB (development)"
+weight: 11
+description: "Build the optional ToplingDB runtime from development source, isolate its data, and understand startup and recovery boundaries."
+---
+
+> **Unreleased development interface.** This guide follows the [three-component runtime integration](https://github.com/hugegraph/hugegraph/pull/266).
+> Use source containing that change. Existing releases and image tags do not establish support, and runtime acceptance remains a deployment prerequisite.
+
+## Choose the database owner
+
+HugeGraph defaults to standard RocksDB. Enable Topling separately for each process that owns local storage:
+
+| Component | Business provider setting | Start command |
+| --- | --- | --- |
+| Standalone RocksDB Server | `rocksdb.provider=topling` in each selected graph properties file | `bin/start-hugegraph.sh` |
+| PD | `provider: topling` under `rocksdb` in `conf/application.yml` | `bin/start-hugegraph-pd.sh` |
+| Store | `provider: topling` under `rocksdb` in `conf/application-pd.yml` | `bin/start-hugegraph-store.sh` |
+
+A standalone Server keeps `backend=rocksdb`; an HStore-backed Server uses `backend=hstore` and needs no local Topling runtime.
+All local RocksDB graphs in one Server process must use the same provider. The business setting must match the runtime selected by its launcher.
+
+## Build and prepare a standard distribution
+
+Build the normal distributions from the source repository root using Java 17 and Maven 3.6.3 or later:
+
+```bash
+mvn clean package -Dmaven.test.skip=true -Dmaven.javadoc.skip=true
+```
+
+Obtain a trusted Topling Easy Migrate JNI JAR and its SHA-256 through your artifact channel. Preparation requires a Java 17 JDK, Linux x86_64,
+`sha256sum`, `unzip`, `od`, `ldd`, and GNU `mv`. The selected JAR's native dependencies, including its glibc and libaio requirements, must be installed.
+
+Stop the component, unpack a fresh standard distribution, and run from that component directory:
+
+```bash
+export TOPLING_JNI_JAR=/absolute/path/to/rocksdbjni-topling.jar
+export TOPLING_JNI_SHA256='<trusted-64-character-sha256>'
+bash bin/prepare-topling.sh
+```
+
+The script verifies the copied JAR's hash, Topling Java marker, one Linux x86_64 JNI entry, ELF architecture and native dependencies.
+Before installation, a disposable database verifies that the selected JNI actually applies EasyMigrate configuration to its persisted write-buffer size.
+Unsupported builds are rejected even when their marker and hash match. The probe does not certify existing-data compatibility or every production setting.
+A checksum receipt binds this verification to the installed JAR and native library; launchers reject missing or changed receipts. Use a fresh distribution
+when upgrading from an older preparation script. Launch rejects inherited RocksDB JNI preloads in default/standard mode and competing JNI in Topling mode, while preserving unrelated preloads.
+It installs `topling/rocksdbjni.jar`, its native library and optional web resources outside `lib`. Preparation refuses an existing `topling` directory;
+replace a runtime by preparing another stopped, fresh distribution. Repeat preparation separately for Server, PD and Store as applicable.
+
+## Keep data outside the distribution
+
+Changing providers does not convert an existing database. Use separate, initially empty Topling directories and retain the standard-provider data.
+Create persistent directories writable by the service account, outside source checkouts, build outputs and unpacked distribution directories:
+
+| Component | Configuration file and key | Example persistent path |
+| --- | --- | --- |
+| Server data | Graph properties: `rocksdb.data_path` | `/srv/hugegraph/topling/server/data` |
+| Server WAL | Graph properties: `rocksdb.wal_path` | `/srv/hugegraph/topling/server/wal` |
+| PD | `conf/application.yml`: `pd.data-path` | `/srv/hugegraph/topling/pd` |
+| Store data | `conf/application.yml`: `app.data-path` | `/srv/hugegraph/topling/store/data` |
+| Store Raft | `conf/application.yml`: `app.raft-path` | `/srv/hugegraph/topling/store/raft` |
+
+For a standalone Server, edit every selected graph configuration, for example `conf/graphs/hugegraph.properties`:
+
+```properties
+backend=rocksdb
+rocksdb.provider=topling
+rocksdb.data_path=/srv/hugegraph/topling/server/data
+rocksdb.wal_path=/srv/hugegraph/topling/server/wal
+```
+
+Set **both** Server data and WAL paths. Leaving WAL at its package-local default risks losing it when a build or package replacement removes that directory.
+Keep the external data, WAL and Raft roots across runtime/package changes.
+
+For PD and Store, edit the existing `rocksdb` mapping in the provider configuration file listed above, retaining its other options:
+
+```yaml
+rocksdb:
+  provider: topling
+```
+
+Set their persistent paths separately in `conf/application.yml`; keep the normal network addresses and cluster configuration.
+
+The published runtime directories use 0755 and regular files use 0644, so a service account different from the preparer can read the JNI. The service account also needs traversal permission on the component's parent directories; preparation does not change those parents.
+
+## Select, start and verify
+
+In the startup shell of each component that owns a Topling database, explicitly select the runtime:
+
+```bash
+export TOPLINGDB_ROCKSDB_PROVIDER=topling
+# Run this component's start command from the first table.
+```
+
+The launcher uses that component's `conf/toplingdb.yaml`. Set `TOPLINGDB_EASY_MIGRATE_CONF=/absolute/path/config.yaml` for an explicit override.
+The shipped native profile disables its HTTP server and keeps `memtable_as_log_index=false`, which the Java write path requires.
+Tune that profile for the host; it is separate from the business provider and data-path configuration.
+
+For a standalone Server, `bin/init-store.sh` and `bin/dump-store.sh` use the same explicit selection as `bin/start-hugegraph.sh`.

Review Comment:
   ⚠️ Important: This section defines initially empty Server data and WAL directories, but the standalone startup flow never instructs readers to initialize the fresh backend before starting it. The current [Server quickstart](https://github.com/apache/hugegraph-doc/blob/da23367301e625c6eefceac69c03cb399e10e92c/content/en/docs/quickstart/hugegraph/hugegraph-server.md#L438) says to run `bin/init-store.sh` before first start, and paired [Topling core PR #266](https://github.com/hugegraph/hugegraph/pull/266) gives the same order. Please add that command and order here and mirror it in Chinese.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


Re: [PR] docs: document ToplingDB development integration [hugegraph-doc]

Posted by "bitflicker64 (via GitHub)" <gi...@apache.org>.
bitflicker64 commented on code in PR #510:
URL: https://github.com/apache/hugegraph-doc/pull/510#discussion_r4178842090


##########
content/en/docs/guides/toplingdb.md:
##########
@@ -0,0 +1,163 @@
+---
+title: "ToplingDB development integration"
+linkTitle: "ToplingDB (development)"
+weight: 11
+description: "Build the optional ToplingDB runtime from development source, isolate its data, and understand startup and recovery boundaries."
+---
+
+> **Unreleased development interface.** This guide describes the proposed replacement for the 2025 ToplingDB integration.
+> It requires a complete source checkout containing the integration and its lifecycle and snapshot-recovery prerequisites.
+> These changes are under review; existing HugeGraph releases and registry tags do not establish support for this interface.
+> Follow the [integration proposal](https://github.com/hugegraph/hugegraph/pull/264) for its delivery status.
+
+## Choose the process that owns the database
+
+| Deployment | Process that loads Topling JNI | Provider setting |
+|---|---|---|
+| Standalone RocksDB Server | Server | `rocksdb.provider=topling` in `conf/graphs/hugegraph.properties` |
+| HStore PD | Each PD | `provider: topling` under `rocksdb` in `conf/application.yml` |
+| HStore Store | Each Store | `provider: topling` under `rocksdb` in `conf/application-pd.yml` |
+| HStore-backed Server | None | No local Topling setting |
+
+PD and Store select their providers independently. The HStore-backed Server is a remote client and does not need a Topling JAR or native library.
+Provider selection happens at process startup. All local RocksDB graphs within one Server process must agree on the provider.
+
+The historical `rocksdb.option_path` and `rocksdb.open_http` examples describe a different integration.
+Do not copy that configuration into this development interface. The component's launch scripts select the Easy Migrate YAML file.
+
+## Build an optional distribution
+
+Build on Linux x86_64 with Java 11+, Maven 3.5+, `rsync`, `unzip`, and `tar`.
+The Server mount preflight additionally requires util-linux 2.37+ `mountpoint` on `PATH`.
+Native macOS execution is outside this integration's support matrix; use a Linux x86_64 container when working on macOS.
+
+Obtain a compatible Topling Easy Migrate JNI JAR from its producer, together with its source revision, dependency licenses, notices, and trusted SHA-256.
+The HugeGraph source tree does not include this binary. A checksum identifies an artifact; it does not establish its origin, license, or runtime compatibility.
+Verify redistribution rights and retain the required license and notice files before distributing a derived package or image.
+
+From the complete source checkout containing this feature:
+
+```bash
+VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout)
+mvn clean package \
+  -pl hugegraph-server/hugegraph-dist,hugegraph-pd/hg-pd-dist,hugegraph-store/hg-store-dist \
+  -am -Dmaven.test.skip=true -Dmaven.javadoc.skip=true -ntp
+
+export TOPLING_JNI_JAR=/absolute/path/to/rocksdbjni-topling.jar
+export TOPLING_JNI_SHA256='<trusted 64-character SHA-256>'
+install-dist/scripts/build-topling-distribution.sh server "$VERSION"
+# Generate these only when running HStore:
+install-dist/scripts/build-topling-distribution.sh pd "$VERSION"
+install-dist/scripts/build-topling-distribution.sh store "$VERSION"
+```
+
+The Maven build creates standard packages. The generator creates separate `-topling` directories and archives beside them.
+It copies the external JAR into private staging, verifies that copy, and prepares the native library.
+Missing inputs, a checksum mismatch, or incompatible JAR structure stop generation; the generator does not search a Maven cache or download a substitute.
+Build success is not a native runtime acceptance test.
+
+Each Topling package contains `lib/topling/rocksdbjni-topling.jar`, `lib/topling/runtime.properties`,
+`library/librocksdbjni-linux64.so`, and the component's preparation and preload scripts.
+To change the JNI JAR, regenerate the component distribution with the new trusted JAR and SHA-256, then run preparation.
+`prepare-topling.sh` alone does not refresh the JAR checksum in `runtime.properties`.
+
+For container builds, use the matching checkout's `docker/README.md` and `docker/bake.hcl` with the same external JNI inputs.
+Build locally before using the example `topling` tags and select `pull_policy: never` for that local deployment.
+For registry images, verify the source revision, JNI identity, and digest first. A tag or a healthy container alone does not prove which JNI is running.
+
+## Configure isolated data
+
+Changing the provider does not migrate or convert data. Give Topling a separate data directory and keep the standard RocksDB directory unchanged.
+Use these component-specific settings:
+
+| Component | Data setting | Example Topling root |
+|---|---|---|
+| Server | `rocksdb.data_path` | `/srv/hugegraph/topling/server/data` |
+| Server WAL | `rocksdb.wal_path` | `/srv/hugegraph/topling/server/wal` |
+| PD | `pd.data-path` | `/srv/hugegraph/topling/pd` |
+| Store | `app.data-path` | `/srv/hugegraph/topling/store` |
+
+For a standalone Server, edit the generated distribution's `conf/graphs/hugegraph.properties`:
+
+```properties
+backend=rocksdb
+rocksdb.provider=topling
+rocksdb.data_path=/srv/hugegraph/topling/server/data
+rocksdb.wal_path=/srv/hugegraph/topling/server/wal
+```
+
+Both Server data and WAL paths must sit under the mounted `/srv/hugegraph/topling/server` root, outside the generated distribution. Do not leave WAL at its
+default path inside the distribution; regenerating the package replaces that directory.
+Mount or create the configured Topling root as a real directory before startup. Do not use symlinked path components.

Review Comment:
   Important: Creating only the mounted root is not enough for the standalone Server. With `rocksdb.provider=topling`, `bin/init-store.sh` and `bin/start-hugegraph.sh` run `server_verify_graph_roots` before Java starts. For an unmarked path it calls `verify-rocksdb-provider.sh` on the configured `data_path` and `wal_path`, which fails with `RocksDB data path must be an existing directory` when either is missing. Only the standard-provider branch runs `mkdir -p`.
   
   Evidence: hugegraph/hugegraph#264 at 36058b0, `bin/rocksdb-server-config.sh` and `bin/verify-rocksdb-provider.sh`. This follows up the WAL thread (r4175878816).
   
   Please tell readers to run `mkdir -p /srv/hugegraph/topling/server/data /srv/hugegraph/topling/server/wal` before the first `bin/init-store.sh`; on the first run both must be empty real directories. Mirror this at line 90 of content/cn/docs/guides/toplingdb.md.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


Re: [PR] docs: document ToplingDB development integration [hugegraph-doc]

Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on code in PR #510:
URL: https://github.com/apache/hugegraph-doc/pull/510#discussion_r4176320680


##########
content/en/docs/guides/toplingdb.md:
##########
@@ -0,0 +1,154 @@
+---
+title: "ToplingDB development integration"
+linkTitle: "ToplingDB (development)"
+weight: 11
+description: "Build the optional ToplingDB runtime from development source, isolate its data, and understand startup and recovery boundaries."
+---
+
+> **Unreleased development interface.** This guide describes the proposed replacement for the 2025 ToplingDB integration.
+> It requires a complete source checkout containing the integration and its lifecycle and snapshot-recovery prerequisites.
+> These changes are under review; existing HugeGraph releases and registry tags do not establish support for this interface.
+> Follow the [integration proposal](https://github.com/hugegraph/hugegraph/pull/264) for its delivery status.
+
+## Choose the process that owns the database
+
+| Deployment | Process that loads Topling JNI | Provider setting |
+|---|---|---|
+| Standalone RocksDB Server | Server | `rocksdb.provider=topling` in `conf/graphs/hugegraph.properties` |
+| HStore PD | Each PD | `provider: topling` under `rocksdb` in `conf/application.yml` |
+| HStore Store | Each Store | `provider: topling` under `rocksdb` in `conf/application-pd.yml` |
+| HStore-backed Server | None | No local Topling setting |
+
+PD and Store select their providers independently. The HStore-backed Server is a remote client and does not need a Topling JAR or native library.
+Provider selection happens at process startup. All local RocksDB graphs within one Server process must agree on the provider.
+
+The historical `rocksdb.option_path` and `rocksdb.open_http` examples describe a different integration.
+Do not copy that configuration into this development interface. The component's launch scripts select the Easy Migrate YAML file.
+
+## Build an optional distribution
+
+Build on Linux x86_64 with Java 11+, Maven 3.5+, `rsync`, `unzip`, and `tar`.
+The Server mount preflight additionally requires util-linux 2.37+ `mountpoint` on `PATH`.
+Native macOS execution is outside this integration's support matrix; use a Linux x86_64 container when working on macOS.
+
+Obtain a compatible Topling Easy Migrate JNI JAR from its producer, together with its source revision, dependency licenses, notices, and trusted SHA-256.
+The HugeGraph source tree does not include this binary. A checksum identifies an artifact; it does not establish its origin, license, or runtime compatibility.
+Verify redistribution rights and retain the required license and notice files before distributing a derived package or image.
+
+From the complete source checkout containing this feature:
+
+```bash
+VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout)
+mvn clean package \
+  -pl hugegraph-server/hugegraph-dist,hugegraph-pd/hg-pd-dist,hugegraph-store/hg-store-dist \
+  -am -Dmaven.test.skip=true -Dmaven.javadoc.skip=true -ntp
+
+export TOPLING_JNI_JAR=/absolute/path/to/rocksdbjni-topling.jar
+export TOPLING_JNI_SHA256='<trusted 64-character SHA-256>'
+install-dist/scripts/build-topling-distribution.sh server "$VERSION"
+# Generate these only when running HStore:
+install-dist/scripts/build-topling-distribution.sh pd "$VERSION"
+install-dist/scripts/build-topling-distribution.sh store "$VERSION"
+```
+
+The Maven build creates standard packages. The generator creates separate `-topling` directories and archives beside them.
+It copies the external JAR into private staging, verifies that copy, and prepares the native library.
+Missing inputs, a checksum mismatch, or incompatible JAR structure stop generation; the generator does not search a Maven cache or download a substitute.
+Build success is not a native runtime acceptance test.
+
+Each Topling package contains `lib/topling/rocksdbjni-topling.jar`, `lib/topling/runtime.properties`,
+`library/librocksdbjni-linux64.so`, and the component's preparation and preload scripts.
+Run `bin/prepare-topling.sh` again after replacing the installed Topling JAR.

Review Comment:
   ⚠️ Important: The generator writes the JAR SHA-256 into `lib/topling/runtime.properties`, but `bin/prepare-topling.sh` only prepares the native library and does not refresh that manifest. Replacing the installed JAR and running only this script leaves the recorded JAR checksum stale, contradicting the verification guidance below. Please instruct readers to regenerate the component distribution with the new trusted JAR and SHA-256, then run preparation; sync Chinese.



##########
content/en/docs/guides/toplingdb.md:
##########
@@ -0,0 +1,154 @@
+---
+title: "ToplingDB development integration"
+linkTitle: "ToplingDB (development)"

Review Comment:
   ⚠️ Important: This page has a version route, but the authored active sidebar tree in `data/docs_nav.json` has no `/docs/guides/toplingdb` entry. `materialize_docs_navigation()` builds the Docs sidebar and active-path map from that tree, so the EN and CN guides have direct URLs but are absent from normal Docs navigation. Please add both localized routes to the appropriate navigation group and update the generated navigation metadata.



##########
content/en/docs/guides/toplingdb.md:
##########
@@ -0,0 +1,154 @@
+---
+title: "ToplingDB development integration"
+linkTitle: "ToplingDB (development)"
+weight: 11
+description: "Build the optional ToplingDB runtime from development source, isolate its data, and understand startup and recovery boundaries."
+---
+
+> **Unreleased development interface.** This guide describes the proposed replacement for the 2025 ToplingDB integration.
+> It requires a complete source checkout containing the integration and its lifecycle and snapshot-recovery prerequisites.
+> These changes are under review; existing HugeGraph releases and registry tags do not establish support for this interface.
+> Follow the [integration proposal](https://github.com/hugegraph/hugegraph/pull/264) for its delivery status.
+
+## Choose the process that owns the database
+
+| Deployment | Process that loads Topling JNI | Provider setting |
+|---|---|---|
+| Standalone RocksDB Server | Server | `rocksdb.provider=topling` in `conf/graphs/hugegraph.properties` |
+| HStore PD | Each PD | `provider: topling` under `rocksdb` in `conf/application.yml` |
+| HStore Store | Each Store | `provider: topling` under `rocksdb` in `conf/application-pd.yml` |
+| HStore-backed Server | None | No local Topling setting |
+
+PD and Store select their providers independently. The HStore-backed Server is a remote client and does not need a Topling JAR or native library.
+Provider selection happens at process startup. All local RocksDB graphs within one Server process must agree on the provider.
+
+The historical `rocksdb.option_path` and `rocksdb.open_http` examples describe a different integration.
+Do not copy that configuration into this development interface. The component's launch scripts select the Easy Migrate YAML file.
+
+## Build an optional distribution
+
+Build on Linux x86_64 with Java 11+, Maven 3.5+, `rsync`, `unzip`, and `tar`.
+The Server mount preflight additionally requires util-linux 2.37+ `mountpoint` on `PATH`.
+Native macOS execution is outside this integration's support matrix; use a Linux x86_64 container when working on macOS.
+
+Obtain a compatible Topling Easy Migrate JNI JAR from its producer, together with its source revision, dependency licenses, notices, and trusted SHA-256.
+The HugeGraph source tree does not include this binary. A checksum identifies an artifact; it does not establish its origin, license, or runtime compatibility.
+Verify redistribution rights and retain the required license and notice files before distributing a derived package or image.
+
+From the complete source checkout containing this feature:
+
+```bash
+VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout)
+mvn clean package \
+  -pl hugegraph-server/hugegraph-dist,hugegraph-pd/hg-pd-dist,hugegraph-store/hg-store-dist \
+  -am -Dmaven.test.skip=true -Dmaven.javadoc.skip=true -ntp
+
+export TOPLING_JNI_JAR=/absolute/path/to/rocksdbjni-topling.jar
+export TOPLING_JNI_SHA256='<trusted 64-character SHA-256>'
+install-dist/scripts/build-topling-distribution.sh server "$VERSION"
+# Generate these only when running HStore:
+install-dist/scripts/build-topling-distribution.sh pd "$VERSION"
+install-dist/scripts/build-topling-distribution.sh store "$VERSION"
+```
+
+The Maven build creates standard packages. The generator creates separate `-topling` directories and archives beside them.
+It copies the external JAR into private staging, verifies that copy, and prepares the native library.
+Missing inputs, a checksum mismatch, or incompatible JAR structure stop generation; the generator does not search a Maven cache or download a substitute.
+Build success is not a native runtime acceptance test.
+
+Each Topling package contains `lib/topling/rocksdbjni-topling.jar`, `lib/topling/runtime.properties`,
+`library/librocksdbjni-linux64.so`, and the component's preparation and preload scripts.
+Run `bin/prepare-topling.sh` again after replacing the installed Topling JAR.
+
+For container builds, use the matching checkout's `docker/README.md` and `docker/bake.hcl` with the same external JNI inputs.
+Build locally before using the example `topling` tags and select `pull_policy: never` for that local deployment.
+For registry images, verify the source revision, JNI identity, and digest first. A tag or a healthy container alone does not prove which JNI is running.
+
+## Configure isolated data
+
+Changing the provider does not migrate or convert data. Give Topling a separate data directory and keep the standard RocksDB directory unchanged.
+Use these component-specific settings:
+
+| Component | Data setting | Example Topling root |
+|---|---|---|
+| Server | `rocksdb.data_path` | `/srv/hugegraph/topling/server` |
+| PD | `pd.data-path` | `/srv/hugegraph/topling/pd` |
+| Store | `app.data-path` | `/srv/hugegraph/topling/store` |
+
+For a standalone Server, edit the generated distribution's `conf/graphs/hugegraph.properties`:
+
+```properties
+backend=rocksdb
+rocksdb.provider=topling
+rocksdb.data_path=/srv/hugegraph/topling/server
+```
+
+Mount or create the configured Topling root as a real directory before startup. Do not use symlinked path components.
+Mount the parent data root, not an individual store directory such as `data/g`; sibling recovery files must remain visible to all cooperating processes.
+The standalone Server launcher rejects individual store mounts before starting Java, including same-filesystem bind mounts on Linux.
+Keep the mount layout unchanged throughout startup and recovery.
+
+The `.hugegraph-rocksdb-provider` marker guards against accidental directory reuse; it is not a converter.
+A conflicting marker stops startup. Topling also rejects an unmarked, non-empty data directory.
+Standard RocksDB retains compatibility with existing unmarked data, but that is not permission to open a directory modified by Topling.
+
+## Start, verify, and stop
+
+Run from the standalone Topling distribution:
+
+```bash
+bin/init-store.sh
+bin/start-hugegraph.sh
+curl --fail http://127.0.0.1:8080/versions
+# Stop cleanly when finished:
+bin/stop-hugegraph.sh
+```
+
+For HStore, configure the normal network addresses and start PD, then Store, then the HStore-backed Server.
+Use `bin/start-hugegraph-pd.sh`, `bin/start-hugegraph-store.sh`, and `bin/start-hugegraph.sh` in their respective distributions.

Review Comment:
   🧹 Minor: The HStore steps name the PD, Store, and Server start scripts but give no matching orderly shutdown sequence, although the section is titled “Start, verify, and stop” and the following guidance requires a clean shutdown. Please list the component stop scripts in reverse startup order and mirror them in Chinese.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


Re: [PR] docs: explain ToplingDB runtime setup (2/4) [hugegraph-doc]

Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on code in PR #510:
URL: https://github.com/apache/hugegraph-doc/pull/510#discussion_r4221977811


##########
content/en/docs/guides/toplingdb.md:
##########
@@ -0,0 +1,125 @@
+---
+title: "ToplingDB development integration"
+linkTitle: "ToplingDB (development)"
+weight: 11
+description: "Build the optional ToplingDB runtime from development source, isolate its data, and understand startup and recovery boundaries."
+---
+
+> **Unreleased development interface.** This guide follows the [three-component runtime integration](https://github.com/apache/hugegraph/pull/3275).
+> Use source containing that change. Existing releases and image tags do not establish support, and runtime acceptance remains a deployment prerequisite.
+
+## Choose the database owner
+
+HugeGraph defaults to standard RocksDB. Enable Topling separately for each process that owns local storage:
+
+| Component | Business provider setting | Start command |
+| --- | --- | --- |
+| Standalone RocksDB Server | `rocksdb.provider=topling` in each selected graph properties file | `bin/start-hugegraph.sh` |
+| PD | `provider: topling` under `rocksdb` in `conf/application.yml` | `bin/start-hugegraph-pd.sh` |
+| Store | `provider: topling` under `rocksdb` in `conf/application-pd.yml` | `bin/start-hugegraph-store.sh` |
+
+A standalone Server keeps `backend=rocksdb`; an HStore-backed Server uses `backend=hstore` and needs no local Topling runtime.
+All local RocksDB graphs in one Server process must use the same provider. The business setting must match the runtime selected by its launcher.
+
+## Build and prepare a standard distribution
+
+Build the normal distributions from the source repository root using Java 17 and Maven 3.6.3 or later:
+
+```bash
+mvn clean package -Dmaven.test.skip=true -Dmaven.javadoc.skip=true
+```
+
+Obtain a trusted Topling Easy Migrate JNI JAR and its SHA-256 through your artifact channel. Preparation requires a Java 17 JDK, Linux x86_64,
+`sha256sum`, `unzip`, `od`, `ldd`, and GNU `mv`. The selected JAR's native dependencies, including its glibc and libaio requirements, must be installed.
+
+Stop the component, unpack a fresh standard distribution, and run from that component directory:
+
+```bash
+export TOPLING_JNI_JAR=/absolute/path/to/rocksdbjni-topling.jar
+export TOPLING_JNI_SHA256='<trusted-64-character-sha256>'
+bash bin/prepare-topling.sh
+```
+
+The script verifies the copied JAR's hash, Topling Java marker, one Linux x86_64 JNI entry, ELF architecture and native dependencies.
+Before installation, a disposable database verifies that the selected JNI actually applies EasyMigrate configuration to its persisted write-buffer size.
+Unsupported builds are rejected even when their marker and hash match. The probe does not certify existing-data compatibility or every production setting.
+A checksum receipt binds this verification to the installed JAR and native library; launchers reject missing or changed receipts. Use a fresh distribution
+when upgrading from an older preparation script. Launch rejects inherited RocksDB JNI preloads in default/standard mode and competing JNI in Topling mode, while preserving unrelated preloads.
+It installs `topling/rocksdbjni.jar`, its native library and optional web resources outside `lib`. Preparation refuses an existing `topling` directory;
+replace a runtime by preparing another stopped, fresh distribution. Repeat preparation separately for Server, PD and Store as applicable.
+
+## Keep data outside the distribution
+
+Changing providers does not convert an existing database. Use separate, initially empty Topling directories and retain the standard-provider data.
+Create persistent directories writable by the service account, outside source checkouts, build outputs and unpacked distribution directories:
+
+| Component | Configuration file and key | Example persistent path |
+| --- | --- | --- |
+| Server data | Graph properties: `rocksdb.data_path` | `/srv/hugegraph/topling/server/data` |
+| Server WAL | Graph properties: `rocksdb.wal_path` | `/srv/hugegraph/topling/server/wal` |
+| PD | `conf/application.yml`: `pd.data-path` | `/srv/hugegraph/topling/pd` |
+| Store data | `conf/application.yml`: `app.data-path` | `/srv/hugegraph/topling/store/data` |
+| Store Raft | `conf/application.yml`: `app.raft-path` | `/srv/hugegraph/topling/store/raft` |
+
+For a standalone Server, edit every selected graph configuration, for example `conf/graphs/hugegraph.properties`:

Review Comment:
   ⚠️ Important: This tells readers to edit every selected graph config but gives the same data and WAL roots for all of them. In paired Server PR #3275, `RocksDBStore.wrapPath()` appends only the store name, while `RocksDBStoreProvider.onCloneConfig()` adds a graph suffix only when cloning; the local `conf/graphs/*.properties` files are loaded independently. Reusing this example across two graphs therefore opens the same RocksDB roots and the second graph cannot open them. Please state that each graph needs unique data and WAL paths, show graph-specific directories, and sync the Chinese guide. Evidence: [config scan](https://github.com/apache/hugegraph/blob/96f99de73820af6a4b41f09df1423da6302ab58a/hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/util/ConfigUtil.java#L90-L108), [local graph loading](https://github.com/apache/hugegraph/blob/96f99de73820af6a4b41f09df1423da6302ab58a/hugegraph-server/hugegraph-api/src/main/java/org/apache/hugegraph/core/GraphManager
 .java#L1263-L1273), [path wrapping](https://github.com/apache/hugegraph/blob/96f99de73820af6a4b41f09df1423da6302ab58a/hugegraph-server/hugegraph-rocksdb/src/main/java/org/apache/hugegraph/backend/store/rocksdb/RocksDBStore.java#L387-L395), [clone suffix](https://github.com/apache/hugegraph/blob/96f99de73820af6a4b41f09df1423da6302ab58a/hugegraph-server/hugegraph-rocksdb/src/main/java/org/apache/hugegraph/backend/store/rocksdb/RocksDBStoreProvider.java#L44-L53).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


Re: [PR] docs: explain ToplingDB runtime setup [hugegraph-doc]

Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin merged PR #510:
URL: https://github.com/apache/hugegraph-doc/pull/510


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


Re: [PR] docs: explain ToplingDB runtime setup (2/4) [hugegraph-doc]

Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on code in PR #510:
URL: https://github.com/apache/hugegraph-doc/pull/510#discussion_r4263581862


##########
content/en/docs/guides/toplingdb.md:
##########
@@ -0,0 +1,125 @@
+---
+title: "ToplingDB development integration"
+linkTitle: "ToplingDB (development)"
+weight: 11
+description: "Build the optional ToplingDB runtime from development source, isolate its data, and understand startup and recovery boundaries."
+---
+
+> **Unreleased development interface.** This guide follows the [three-component runtime integration](https://github.com/apache/hugegraph/pull/3275).
+> Use source containing that change. Existing releases and image tags do not establish support, and runtime acceptance remains a deployment prerequisite.
+
+## Choose the database owner
+
+HugeGraph defaults to standard RocksDB. Enable Topling separately for each process that owns local storage:
+
+| Component | Business provider setting | Start command |
+| --- | --- | --- |
+| Standalone RocksDB Server | `rocksdb.provider=topling` in each selected graph properties file | `bin/start-hugegraph.sh` |
+| PD | `provider: topling` under `rocksdb` in `conf/application.yml` | `bin/start-hugegraph-pd.sh` |
+| Store | `provider: topling` under `rocksdb` in `conf/application-pd.yml` | `bin/start-hugegraph-store.sh` |
+
+A standalone Server keeps `backend=rocksdb`; an HStore-backed Server uses `backend=hstore` and needs no local Topling runtime.
+All local RocksDB graphs in one Server process must use the same provider. The business setting must match the runtime selected by its launcher.
+
+## Build and prepare a standard distribution
+
+Build the normal distributions from the source repository root using Java 17 and Maven 3.6.3 or later:
+
+```bash
+mvn clean package -Dmaven.test.skip=true -Dmaven.javadoc.skip=true
+```
+
+Obtain a trusted Topling Easy Migrate JNI JAR and its SHA-256 through your artifact channel. Preparation requires a Java 17 JDK, Linux x86_64,
+`sha256sum`, `unzip`, `od`, `ldd`, and GNU `mv`. The selected JAR's native dependencies, including its glibc and libaio requirements, must be installed.
+
+Stop the component, unpack a fresh standard distribution, and run from that component directory:
+
+```bash
+export TOPLING_JNI_JAR=/absolute/path/to/rocksdbjni-topling.jar
+export TOPLING_JNI_SHA256='<trusted-64-character-sha256>'
+bash bin/prepare-topling.sh
+```
+
+The script verifies the copied JAR's hash, Topling Java marker, one Linux x86_64 JNI entry, ELF architecture and native dependencies.
+Before installation, a disposable database verifies that the selected JNI actually applies EasyMigrate configuration to its persisted write-buffer size.
+Unsupported builds are rejected even when their marker and hash match. The probe does not certify existing-data compatibility or every production setting.
+A checksum receipt binds this verification to the installed JAR and native library; launchers reject missing or changed receipts. Use a fresh distribution
+when upgrading from an older preparation script. Launch rejects inherited RocksDB JNI preloads in default/standard mode and competing JNI in Topling mode, while preserving unrelated preloads.
+It installs `topling/rocksdbjni.jar`, its native library and optional web resources outside `lib`. Preparation refuses an existing `topling` directory;
+replace a runtime by preparing another stopped, fresh distribution. Repeat preparation separately for Server, PD and Store as applicable.
+
+## Keep data outside the distribution
+
+Changing providers does not convert an existing database. Use separate, initially empty Topling directories and retain the standard-provider data.
+Create persistent directories writable by the service account, outside source checkouts, build outputs and unpacked distribution directories:
+
+| Component | Configuration file and key | Example persistent path |
+| --- | --- | --- |
+| Server data | Graph properties: `rocksdb.data_path` | `/srv/hugegraph/topling/server/data` |
+| Server WAL | Graph properties: `rocksdb.wal_path` | `/srv/hugegraph/topling/server/wal` |
+| PD | `conf/application.yml`: `pd.data-path` | `/srv/hugegraph/topling/pd` |
+| Store data | `conf/application.yml`: `app.data-path` | `/srv/hugegraph/topling/store/data` |
+| Store Raft | `conf/application.yml`: `app.raft-path` | `/srv/hugegraph/topling/store/raft` |
+
+For a standalone Server, edit every selected graph configuration, for example `conf/graphs/hugegraph.properties`:

Review Comment:
   Addressed in f1bbb9a83: explicitly documented that each graph requires dedicated data and WAL directories, showed graph-specific example paths (/srv/hugegraph/topling/server/<graph>/data and .../wal), and synchronized both Chinese and English guides.



##########
content/en/docs/guides/toplingdb.md:
##########
@@ -0,0 +1,125 @@
+---
+title: "ToplingDB development integration"
+linkTitle: "ToplingDB (development)"
+weight: 11
+description: "Build the optional ToplingDB runtime from development source, isolate its data, and understand startup and recovery boundaries."
+---
+
+> **Unreleased development interface.** This guide follows the [three-component runtime integration](https://github.com/apache/hugegraph/pull/3275).
+> Use source containing that change. Existing releases and image tags do not establish support, and runtime acceptance remains a deployment prerequisite.
+
+## Choose the database owner
+
+HugeGraph defaults to standard RocksDB. Enable Topling separately for each process that owns local storage:
+
+| Component | Business provider setting | Start command |
+| --- | --- | --- |
+| Standalone RocksDB Server | `rocksdb.provider=topling` in each selected graph properties file | `bin/start-hugegraph.sh` |
+| PD | `provider: topling` under `rocksdb` in `conf/application.yml` | `bin/start-hugegraph-pd.sh` |
+| Store | `provider: topling` under `rocksdb` in `conf/application-pd.yml` | `bin/start-hugegraph-store.sh` |
+
+A standalone Server keeps `backend=rocksdb`; an HStore-backed Server uses `backend=hstore` and needs no local Topling runtime.
+All local RocksDB graphs in one Server process must use the same provider. The business setting must match the runtime selected by its launcher.
+
+## Build and prepare a standard distribution
+
+Build the normal distributions from the source repository root using Java 17 and Maven 3.6.3 or later:
+
+```bash
+mvn clean package -Dmaven.test.skip=true -Dmaven.javadoc.skip=true
+```
+
+Obtain a trusted Topling Easy Migrate JNI JAR and its SHA-256 through your artifact channel. Preparation requires a Java 17 JDK, Linux x86_64,
+`sha256sum`, `unzip`, `od`, `ldd`, and GNU `mv`. The selected JAR's native dependencies, including its glibc and libaio requirements, must be installed.
+
+Stop the component, unpack a fresh standard distribution, and run from that component directory:
+
+```bash
+export TOPLING_JNI_JAR=/absolute/path/to/rocksdbjni-topling.jar
+export TOPLING_JNI_SHA256='<trusted-64-character-sha256>'
+bash bin/prepare-topling.sh
+```
+
+The script verifies the copied JAR's hash, Topling Java marker, one Linux x86_64 JNI entry, ELF architecture and native dependencies.
+Before installation, a disposable database verifies that the selected JNI actually applies EasyMigrate configuration to its persisted write-buffer size.
+Unsupported builds are rejected even when their marker and hash match. The probe does not certify existing-data compatibility or every production setting.
+A checksum receipt binds this verification to the installed JAR and native library; launchers reject missing or changed receipts. Use a fresh distribution
+when upgrading from an older preparation script. Launch rejects inherited RocksDB JNI preloads in default/standard mode and competing JNI in Topling mode, while preserving unrelated preloads.
+It installs `topling/rocksdbjni.jar`, its native library and optional web resources outside `lib`. Preparation refuses an existing `topling` directory;
+replace a runtime by preparing another stopped, fresh distribution. Repeat preparation separately for Server, PD and Store as applicable.
+
+## Keep data outside the distribution
+
+Changing providers does not convert an existing database. Use separate, initially empty Topling directories and retain the standard-provider data.
+Create persistent directories writable by the service account, outside source checkouts, build outputs and unpacked distribution directories:
+
+| Component | Configuration file and key | Example persistent path |
+| --- | --- | --- |
+| Server data | Graph properties: `rocksdb.data_path` | `/srv/hugegraph/topling/server/data` |
+| Server WAL | Graph properties: `rocksdb.wal_path` | `/srv/hugegraph/topling/server/wal` |
+| PD | `conf/application.yml`: `pd.data-path` | `/srv/hugegraph/topling/pd` |
+| Store data | `conf/application.yml`: `app.data-path` | `/srv/hugegraph/topling/store/data` |
+| Store Raft | `conf/application.yml`: `app.raft-path` | `/srv/hugegraph/topling/store/raft` |
+
+For a standalone Server, edit every selected graph configuration, for example `conf/graphs/hugegraph.properties`:

Review Comment:
   Addressed in f1bbb9a83: updated the path table and graph properties example with distinct graph-specific directories to avoid conflicts when initializing or loading multiple graphs.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org