You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hugegraph.apache.org by ji...@apache.org on 2026/10/10 08:23:58 UTC
(hugegraph) branch master updated: refactor(server): complete request transaction cleanup (#3302)
This is an automated email from the ASF dual-hosted git repository.
imbajin pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hugegraph.git
The following commit(s) were added to refs/heads/master by this push:
new 0f86e2a03 refactor(server): complete request transaction cleanup (#3302)
0f86e2a03 is described below
commit 0f86e2a0354d64685b1edb8edb7ccea025f07b9b
Author: imbajin <ji...@apache.org>
AuthorDate: Sat Oct 10 16:23:52 2026 +0800
refactor(server): complete request transaction cleanup (#3302)
Request and task cleanup releases owned graph transactions while preserving failures and worker reuse. Backend Store open and final session close share their synchronization boundary; HStore also rejects borrowers after its shared session closes.
Graph shutdown continues safe cleanup after an owner error. Ordinary task-worker failures are collected, and rollback failures retain later owner-close errors as suppressed. Pending local tasks keep their task database transaction until completion; an incomplete distributed drain no longer queues another unbounded wait on the same worker. The configured tx.close policy is preserved.
Regression tests cover multiple graphs, worker cleanup failures, rollback plus owner-close failure, closed HStore borrowing, task completion persistence and drain retry. Independent review and focused local tests passed; backend integration results are reported by the PR checks. Logical graph closure does not prove an unsuccessful native release succeeded.
---
README.md | 4 +-
docs/BUILDING.md | 2 +-
docs/ci.md | 32 +-
docs/cypher-compatibility.md | 96 -----
docs/query-semantics.md | 41 --
docs/server-runtime.md | 440 +++++++++++++++++++++
docs/upgrade-tinkerpop-3.8.md | 261 ------------
hugegraph-server/README.md | 2 +-
.../hugegraph/auth/HugeFactoryAuthProxy.java | 11 +-
.../apache/hugegraph/auth/HugeGraphAuthProxy.java | 8 +-
.../java/org/apache/hugegraph/HugeFactory.java | 6 +
.../org/apache/hugegraph/StandardHugeGraph.java | 148 ++++---
.../backend/store/BackendSessionPool.java | 23 +-
.../hugegraph/backend/tx/GraphTransaction.java | 2 +-
.../hugegraph/task/DistributedTaskScheduler.java | 70 +++-
.../hugegraph/task/StandardTaskScheduler.java | 61 ++-
.../org/apache/hugegraph/task/TaskManager.java | 104 ++++-
.../traversal/algorithm/OltpTraverser.java | 2 +-
.../hugegraph/backend/store/hbase/HbaseStore.java | 2 +-
.../backend/store/hstore/HstoreSessionsImpl.java | 2 +-
.../backend/store/hstore/HstoreStore.java | 2 +-
.../backend/store/rocksdb/RocksDBStore.java | 2 +-
.../java/org/apache/hugegraph/HugeFactoryTest.java | 400 +++++++++++++++++++
.../org/apache/hugegraph/auth/ContextTaskTest.java | 85 ++++
.../hugegraph/auth/TransactionLifecycleTest.java | 148 +++++++
.../org/apache/hugegraph/core/CoreTestSuite.java | 4 +
.../task/StandardTaskSchedulerTxTest.java | 65 +++
.../org/apache/hugegraph/unit/UnitTestSuite.java | 6 +
.../unit/core/BackendSessionPoolTest.java | 329 +++++++++++++++
.../hugegraph/unit/core/HstoreSessionsTest.java | 37 ++
30 files changed, 1857 insertions(+), 538 deletions(-)
diff --git a/README.md b/README.md
index 00cacdb56..515b5cffc 100644
--- a/README.md
+++ b/README.md
@@ -16,7 +16,7 @@
---
-**Quick Navigation:** [Architecture](#architecture) • [Quick Start](#quick-start) • [Module Map](#module-map) • [Ecosystem](#ecosystem) • [For Contributors](#for-contributors) • [Community](#community)
+**Quick Navigation:** [Architecture](#architecture) • [Quick Start](#quick-start) • [Module Map](#module-map) • [Ecosystem](#ecosystem) • [For Contributors](#contributing) • [Community](#community)
---
@@ -190,7 +190,7 @@ The launch scripts reject Java versions older than 17. That minimum-version
check does not qualify later Java releases; use Java 17 unless another release
is explicitly listed as supported.
-For this upgrade, read the [TinkerPop 3.8.1 migration guide](docs/upgrade-tinkerpop-3.8.md)
+Read the [Server runtime guide](docs/server-runtime.md#runtime-and-client-configuration)
for client configuration and query compatibility checks.
### Option 1: Docker (Fastest)
diff --git a/docs/BUILDING.md b/docs/BUILDING.md
index 3501f174d..d111820fd 100644
--- a/docs/BUILDING.md
+++ b/docs/BUILDING.md
@@ -15,7 +15,7 @@ bytecode for downstream compatibility. Their build and tests still run on
Java 17; this library bytecode target does not lower HugeGraph's runtime
requirement.
-See the [TinkerPop 3.8.1 migration guide](upgrade-tinkerpop-3.8.md) for the
+See the [Server runtime guide](server-runtime.md#runtime-and-client-configuration) for the
upgraded runtime, client configuration and compatibility checks.
To build without executing tests: `mvn clean package -Dmaven.test.skip=true`
diff --git a/docs/ci.md b/docs/ci.md
index 74b562ee6..e20d338eb 100644
--- a/docs/ci.md
+++ b/docs/ci.md
@@ -52,13 +52,11 @@ remain. This changes CI preparation, not product support or backend deprecation
TinkerPop suites run only when the source, target or push/manual branch starts with
`release-`, `test-` or `tinkerpop-`. PRs run no TP suites when neither source nor target
branch matches these prefixes, including `upgrade/1.8.0` targeting `master`.
-Historical task-branch exceptions and the temporary TP skip are removed. Executed TP suites
-still require non-empty reports with actual executed tests.
+Executed TP suites require non-empty reports with actual executed tests.
-PD, Store and Commons have total job budgets of 30, 45 and 20 minutes, respectively,
-based on six recent successful master runs across Org and ASF. Cluster has 120 minutes,
-preserving its two existing 45-minute test windows and allowing build/diagnostic time.
-Server and HStore budgets remain unchanged pending complete TP execution history.
+PD, Store and Commons have total job budgets of 30, 45 and 20 minutes, respectively.
+Cluster has 120 minutes for its two 45-minute test windows and build/diagnostic time.
+Server and HStore budgets are defined in their workflow job settings.
## Documentation and freshness
@@ -80,9 +78,8 @@ Plans and actual results are diagnostics, not execution credentials for later ru
`.asf.yaml` requests only `check-license` and `Server memory tests`. Keep these required check
names stable. The Memory workflow runs real unit, core and API tests on the project runtime
-and reports their results through `Server memory tests`; the former Java 11 placeholder is
-removed. The former `build-commons (11)` placeholder is also removed; real Commons tests
-remain advisory and install only the Commons reactor and its upstream modules.
+and reports their results through `Server memory tests`. Commons tests remain advisory
+and install only the Commons reactor and its upstream modules.
`affected-module-tests` and CodeQL remain advisory. Verify live branch protection
when changing required checks; do not forge a successful result or leave retired contexts required.
@@ -90,20 +87,3 @@ Only failed push runs of License Checker and Server Memory CI automatically retr
twice. The trusted checker verifies the workflow path, attempt, repository and unchanged branch
head before and after the delay. PRs and advisory workflows never automatically retry.
Use a manual rerun when appropriate; a rerun retains its original commit and event.
-
-## Follow-up work
-
-These changes need separate CI policy decisions or broader validation:
-
-- Split compatibility selection by backend, native code, JDK and shared startup/core impact,
- retaining Server/PD/Store/Cluster coverage and periodic complete compatibility checks.
-- Narrow PD/Store/HStore/Cluster reactors only after verifying every required distribution;
- evaluate sharing artifacts between jobs separately.
-- Choose an explicit RocksDB comparison baseline for the first push to a new release-/test-
- branch, where `before` is all zeros. Do not silently compare a commit with itself.
-- Revisit RISC-V `continue-on-error`; it remains advisory and is not a required check.
-- Consider merging runtime resolution into planning and extracting long diagnostic scripts.
-- Replace broad push failure reruns with bounded retries at known transient operations;
- preserve workflow, repository, attempt and unchanged-head checks.
-- Set total timeouts only where complete test history supports a budget, including diagnostics.
- Do not size HStore or Server TP budgets from runs that skipped TP suites.
diff --git a/docs/cypher-compatibility.md b/docs/cypher-compatibility.md
deleted file mode 100644
index ac5f312a1..000000000
--- a/docs/cypher-compatibility.md
+++ /dev/null
@@ -1,96 +0,0 @@
-# Cypher compatibility
-
-This note records the Cypher behavior verified for this change. It does not claim full openCypher or Neo4j
-compatibility. The runtime used Java 17.0.20.1, TinkerPop 3.8.1, `org.opencypher.gremlin:translation:1.0.4`,
-and RocksDB, rebased onto the Apache master that includes the Java 17 / TinkerPop 3.8.1 upgrade.
-
-## Request forms
-
-The endpoint is `/graphspaces/{graphspace}/graphs/{graph}/cypher`.
-
-| Method and content type | Request | Verified behavior |
-| --- | --- | --- |
-| GET | `?cypher=<URL-encoded statement>` | Query-string form works: `testGet` |
-| POST `application/json` | Raw Cypher text | Legacy raw-body form works: `testPost` |
-| POST `text/plain` | Any body | Rejected with HTTP 415: `testRejectPlainTextPost` |
-| POST `application/json` | JSON object below | Separate bindings work: `testParameters` |
-
-```json
-{
- "cypher": "MATCH (n:person) WHERE n.name = $name RETURN n.name",
- "parameters": {"name": "marko"}
-}
-```
-
-POST bodies containing JSON arrays, strings, numbers, booleans, or null are rejected with HTTP 400.
-Malformed JSON objects and trailing tokens are also request errors and do not fall back to raw Cypher.
-
-For the JSON-object form, `cypher` must be a nonblank string and `parameters`, when present, must be an
-object. Omitting `parameters` means an empty map. Missing bindings produce an execution error; an explicit
-null value is accepted. Tests also cover bindings named `id` and `label`, and the fixed limit of 16
-top-level parameters. This Cypher processor does not support a `maxParameters` configuration setting.
-Values containing quotes and newlines remain bound data rather than changing query text.
-
-A Map counts as one top-level parameter. Group related values into one Map and reference its fields
-explicitly; the vertex label and required properties must match the graph's existing schema:
-
-```json
-{
- "cypher": "CREATE (n:person {name:$props.name, age:$props.age, city:$props.city}) RETURN n.name",
- "parameters": {"props": {"name": "new-person", "age": 20, "city": "Beijing"}}
-}
-```
-
-Do not use the shorthand `CREATE (n:person $props)`: the pinned translator does not assign the bound
-Map's properties in that path. Live REST verification confirmed that one Map containing 17 entries
-works with explicit property references, with all 17 persisted values verified through native REST.
-A 17-entry returned Map also succeeds, while 17 top-level bindings produce an execution error.
-
-The translator's reserved null-marker string `" cypher.null"` (two leading spaces) is rejected in binding
-values, including nested maps and lists, to avoid silently converting a user string to null. Actual JSON
-null remains accepted. The same marker in query literals or stored properties is an existing translator
-limitation and is outside this compatibility target.
-Check the HTTP status and body `status.code`: execution failures retain HTTP 200 with `status.code` 400
-and null result data.
-
-Traversal iteration failures, including Java errors, attempt rollback on the execution thread and return
-a terminal failure response. A rollback exception is attached to the original failure.
-
-## Verified behavior
-
-The API fixture uses a strong schema with a `SECONDARY` index on `city` and a `RANGE` index on `age`.
-Acceptance tests verified:
-
-| Area | Verified cases and test methods |
-| --- | --- |
-| Reads | Label scans, equality and range predicates, computed regex full-string matching, boolean combinations, directed one- and two-hop patterns, empty results: `testGet`, `testExactReadsAndPredicates`, `testComputedRegexExecutesExtensionPredicate`, `testComputedRegexRequiresWholeStringMatch`, `testRelationQuery` |
-| Results | Aliases, scalar and node values, nested maps/lists, relationship ids, path shape, and null/missing-property semantics: `testReturnNodeIdAsPrimitiveValue`, `testReturnNodeDoesNotLeakInternalIdTypes`, `testReturnNestedIdDoesNotLeakInternalIdTypes`, `testReturnRelationIdDoesNotLeakInternalIdTypes`, `testReturnPathShape`, `testNullAndMissingProperty` |
-| Aggregation and pagination | `DISTINCT`, `count`/`sum`/`min`/`max`/`avg`, `ORDER BY`, `SKIP`, and `LIMIT`: `testDuplicatesDistinctAndPagination`, `testAggregates` |
-| Parameters | String, number, boolean, null, empty and missing bindings; quoting/newline safety: `testParameters`, `testRequestPreservesQueryAndParameterValues`, `testRequestAcceptsEmptyParameters`, `testAcceptCypherParameterNamesAndNullValues`, `testEnforceParameterCountBoundary` |
-| Writes | Vertex and edge `CREATE`, property `SET`, edge and vertex `DELETE`; state checked through native REST reads: `testCreate`, `testCreateSetAndDeleteWithNativeReadback` |
-| Failures and routing | Invalid syntax, request shape, binding keys, schema values, authentication, and graph routing: `testInvalidRequests`, `testRejectInvalidQuery`, `testRejectInvalidBindingShapeAndKeys`, `testAuthenticationAndGraphRouting`, `testSpecifiedGraphRouting` |
-| Failed write | A rejected multi-vertex create left no residue in native reads or in 32 later query/native-read checks: `testFailedWriteDoesNotLeakIntoLaterRequests` |
-
-Write verification describes how acceptance was checked; clients are not required to issue native REST reads
-after each write. A successful Cypher response alone was not used as proof of persisted state.
-
-## Verification record
-
-The final verification covers the rebased tree, including non-object JSON request rejection, the
-unsupported `text/plain` media type, computed regex fixture cases, and fatal-error rollback and response.
-`CypherApiTest` passed all 23 cases, `CypherClientTest` all seven, and `CypherOpProcessorTest` all eight,
-with no Cypher skips. The related predicate and Gremlin HTTP ownership/context tests also passed.
-`AbstractRestClientTest` passed all nine cases, including charset serialization and an ASCII-default JVM.
-Gremlin API tests passed ten cases with one existing non-shared-backend skip; Login API tests passed all
-three. EditorConfig formatting, the root clean compile, and reactor installation passed on Java 17.
-The live RocksDB server reported Gremlin 3.8.1; its packaged API JAR and compiled target JAR were
-byte-identical before the API run.
-
-Test sources: [`CypherApiTest`](../hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/api/CypherApiTest.java),
-[`CypherClientTest`](../hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/api/cypher/CypherClientTest.java),
-and [`CypherOpProcessorTest`](../hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/opencypher/CypherOpProcessorTest.java).
-
-## Outside this compatibility target
-
-Advanced constructs including `MERGE`, `OPTIONAL MATCH`, `WITH`, `UNWIND`, `UNION`, and variable-length paths,
-as well as HStore, Bolt, cross-request transactions, and performance, were not verified by this change.
diff --git a/docs/query-semantics.md b/docs/query-semantics.md
deleted file mode 100644
index 5cfb410ce..000000000
--- a/docs/query-semantics.md
+++ /dev/null
@@ -1,41 +0,0 @@
-# Query count and filter behavior
-
-Counts made inside a transaction include that transaction's uncommitted vertex
-and edge changes. Only `count()` supports this fallback; other aggregates with
-uncommitted changes are rejected. Queries that combine uncommitted changes with
-pagination, a limit, or an offset remain unsupported.
-
-Queries and counts select the current transaction version before applying filters.
-When an element is removed and re-added with the same ID, the new element shadows
-the old updated record, even if only the old record matches the query's label.
-
-A count query that enters an index query path remains unsupported when that index
-transaction has uncommitted changes. It reports `Can't do index query when there
-are changes in transaction`. This includes indexed-property conditions and
-label-only queries on backends that need a label index. Graph-wide and explicit-ID
-counts can still use transaction merging without entering that index path.
-
-Unsupported text predicates remain traversal filters. For ordinary GraphStep and
-VertexStep extraction, if any condition in a `HasStep` cannot be converted, the
-whole step remains local, including its sibling label, ID and property conditions.
-Count optimization preserves that filter step. Existing special handling around
-`match()` and connective label filters is inherited from master.
-
-New generalized selective pushdown, predicate-specific local ID/SEARCH matching and
-candidate-index coverage are outside this upgrade series. They are coordinated
-through [PR #2994](https://github.com/apache/hugegraph/pull/2994),
-[issue #3201](https://github.com/apache/hugegraph/issues/3201) and
-[PR #3243](https://github.com/apache/hugegraph/pull/3243).
-
-A self-loop contributes two occurrences to a vertex's `bothE()` traversal and
-one to each of `outE()`, `inE()` and the graph-wide `E()` traversal. Counts retain
-these multiplicities before and after transaction commit, including edge updates.
-
-Count optimization preserves steps that can filter candidates. Resetting an
-optimized count traversal permits it to execute again. Query-step equality does
-not depend on the result iterator from a previous execution.
-
-A scan in the middle of a traversal is counted once per incoming traverser,
-including its bulk. It is not replaced by a single backend count. For example,
-with three vertices, `g.V().V().count()` returns `9L`, while `g.V().count()`
-returns `3L` and remains eligible for count optimization.
diff --git a/docs/server-runtime.md b/docs/server-runtime.md
new file mode 100644
index 000000000..84103875f
--- /dev/null
+++ b/docs/server-runtime.md
@@ -0,0 +1,440 @@
+# Server runtime and query contracts
+
+This guide covers Server configuration, wire formats, query semantics and transaction
+ownership. Use it when upgrading applications or changing the
+Server runtime. Deployment instructions remain in the [build guide](BUILDING.md)
+and [Server module guidance](../hugegraph-server/AGENTS.md).
+
+## Runtime and client configuration
+
+The runtime uses TinkerPop 3.8.1 and Groovy 4.0.25. Build and run every module with Java
+17. Applications migrating from TinkerPop 3.5.1 must review the full version interval,
+including the configuration changes below.
+
+### Serializer configuration
+
+Use the Gremlin configuration files shipped with the upgraded Server. Serializer classes
+move from `org.apache.tinkerpop.gremlin.driver.ser` to
+`org.apache.tinkerpop.gremlin.util.ser`. The supplied remote configurations include the
+corresponding HugeGraph registries.
+
+The Java Driver sample `gremlin-driver-settings.yaml` and Gremlin Console's
+`remote.yaml` now default to GraphBinary V1. The serializer entry changes as follows:
+
+```diff
+- className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0
++ className: org.apache.tinkerpop.gremlin.util.ser.GraphBinaryMessageSerializerV1
+```
+
+Keep the HugeGraph builder and registry in both client configurations:
+
+```yaml
+serializer:
+ className: org.apache.tinkerpop.gremlin.util.ser.GraphBinaryMessageSerializerV1
+ config:
+ serializeResultToString: false
+ builder: org.apache.hugegraph.io.HugeGraphTypeSerializerRegistryBuilder
+ ioRegistries: [org.apache.hugegraph.io.HugeGraphIoRegistry]
+```
+
+GraphBinary preserves graph elements and UUID IDs for Java Driver accessors such as
+`Result.getVertex()` and `Result.get(UUID.class)`. Untyped GraphSON V1 decodes these
+results as maps and strings.
+
+`remote-objects.yaml` serves the Cypher HTTP JSON path and continues to use
+`org.apache.tinkerpop.gremlin.util.ser.GraphSONUntypedMessageSerializerV1` with
+`HugeGraphIoRegistry`. It retains its duplicate registry entry as a TinkerPop driver
+compatibility workaround.
+
+For the server, keep the GraphBinary builder and registry. The serializer order is
+GraphBinary, untyped GraphSON V1/V2/V3, then typed GraphSON V2/V3. The GraphSON entries
+keep the HugeGraph registry. The first entries look like this:
+
+```yaml
+serializers:
+ - className: org.apache.tinkerpop.gremlin.util.ser.GraphBinaryMessageSerializerV1
+ config:
+ serializeResultToString: false
+ builder: org.apache.hugegraph.io.HugeGraphTypeSerializerRegistryBuilder
+ ioRegistries: [org.apache.hugegraph.io.HugeGraphIoRegistry]
+ - className: org.apache.tinkerpop.gremlin.util.ser.GraphSONUntypedMessageSerializerV1
+ config:
+ serializeResultToString: false
+ ioRegistries: [org.apache.hugegraph.io.HugeGraphIoRegistry,
+ org.apache.hugegraph.io.HugeGraphSONV1MessageIoRegistry]
+```
+
+The full ordering is in `gremlin-server.yaml`. `application/json` selects untyped V1 in
+the supplied server configuration variants.
+
+Typed GraphSON is available for the supported simple values and IDs, not as a
+replacement for untyped serialization of every HugeGraph schema or graph object. Typed
+V1 is not registered: its legacy `@class` deserialization can construct Java objects
+before request authentication. V2/V3 use `@type`/`@value`.
+
+GraphBinary uses `HugeGraphTypeSerializerRegistryBuilder` to write HugeGraph IDs as
+their underlying values. Schema results use standard maps and Blob values use standard
+binary values (`ByteBuffer` in the Java Driver), including when nested in returned
+vertices, edges or collections. `Optional` results carry their value or `null` when
+empty, including results from `graph.variables().get()`. `File` results retain the map
+shape `{"file": "name"}` using the file name, including inside nested results. Untyped
+GraphSON V1 keeps the legacy Tree array of `key`/`value` entries and accepts scalar as
+well as element keys through the additional `HugeGraphSONV1MessageIoRegistry` in that
+serializer's configuration. Keep this registry limited to untyped V1 messages. Standard
+graph-file IO and typed GraphSON serializers keep `HugeGraphIoRegistry` alone and use
+the TinkerPop Tree format for their negotiated version.
+
+HugeGraph enums such as `DataType.TEXT` and `Directions.OUT` return their names over
+GraphBinary, including in maps and lists. `graph.schema()` returns a map with
+`propertykeys`, `vertexlabels`, `edgelabels` and `indexlabels` lists, using the same
+schema conversion. TinkerPop enums retain their native wire types. Backend `Shard`
+values, including `graph.metadata(HugeType.VERTEX, 'splits', size)`, return standard
+maps with `start`, `end` and an exact 64-bit `length`, including inside lists and maps.
+The default Driver does not need a private Shard serializer.
+
+Local edge ID filters accept both serialized strings and native edge IDs in mixed
+collections: `hasId(not(without(ids)))` and `hasId(within(ids))` agree on membership and
+count. Numeric and string vertex IDs keep their distinct types.
+
+Groovy interpolated strings (`GString`, including subclasses) return as standard strings
+over GraphBinary, including inside lists and maps. For example, `def value = 42;
+"value:${value}"` returns `"value:42"`. The untyped GraphSON path retains the
+legacy GString bean map with its `values` and `strings` fields.
+
+Cypher extension predicates use the TinkerPop predicate constructor through the
+translator extension point. Computed-expression regex filters retain whole-string
+matching (`String.matches`), so `"marko"` matches `"mar.*"` and does not match `"ark"`.
+The public `cypherRegex`, `cypherIsString`, `cypherIsNode` and `cypherIsRelationship`
+factories imported by the bundled Gremlin configuration use that same adapter, so the
+Gremlin translation returned by `EXPLAIN` can be replayed.
+
+Cypher results normalize HugeGraph IDs, including values nested in collections, maps and
+paths. Cyclic results and nesting deeper than 32 levels are rejected.
+
+The default script engine is Gremlin Groovy. Validate application scripts against the
+full TinkerPop 3.5.1 to 3.8.1 upgrade interval before deploying.
+
+## YAML configuration safety
+
+Jackson JSON and YAML modules share the root BOM version. Fabric8 kubeconfig extensions
+can contain integer and decimal values; the K8s configuration regression verifies auto-
+configuration without contacting a cluster.
+
+The TinkerPop 3.8.1 runtime uses SnakeYAML 2.2, Spring Boot 2.5.15 and Spring Framework
+5.3.27. The Boot patch keeps PD and Store's YAML startup loader compatible with
+SnakeYAML 2.x. These versions belong to the upgraded runtime; do not apply the SnakeYAML
+override to a TinkerPop 3.5.1 deployment. The runtime also manages Jackson YAML 2.15.2
+so Fabric8 can read kubeconfig with SnakeYAML 2.2. Store's Log4j SLF4J binding and CLI
+logging use 2.18.0; Store Node retains its Boot-managed Log4j core version.
+
+Quota templates and PD-delivered Store configuration accept standard YAML maps, lists
+and scalars. Custom Java object tags such as `!!com.example.Type` are rejected before
+object construction. Keep these files as configuration data; Java class names used as
+ordinary string values remain supported.
+
+## List property compatibility
+
+The public TinkerPop `Graph.Features` contract changes:
+`features().vertex().properties().supportsUniformListValues()` and
+`features().edge().properties().supportsUniformListValues()` now return `false` instead
+of `true`. Clients and compatibility tests that inspect these flags may stop enabling
+uniform-list operations. The corresponding graph-variable flag remains `true`.
+
+HugeGraph still supports vertex and edge properties whose schema declares `LIST`
+cardinality with `valueList()` and a supported element type, for example
+`schema.propertyKey("tags").asText().valueList().create()`. Values must match the
+schema's element type. This schema capability does not provide arbitrary Java `List`
+values for `SINGLE` properties. The feature-reporting change itself does not remove
+stored lists or require rewriting existing list data.
+
+When upgrading, review client feature-detection branches. Applications using HugeGraph
+list properties should retain their explicit element type and `LIST` schema, and verify
+list writes and reads with that schema. Generic TinkerPop clients should honor the
+reported feature flags; do not treat a skipped uniform-list compatibility test as
+validation of HugeGraph's schema list path.
+
+## Query semantics
+
+### Predicate and value compatibility
+
+Version-specific handling retains negated predicates and filtering barriers where
+pushdown would change their meaning. For UUID IDs, local negated filters such as
+`hasId(P.not(P.neq(uuid)))` retain the same matches as ID equality. This also applies
+when an ID filter stays local beside a property filter that cannot be pushed down. The
+complete `HasStep` remains local; string and numeric ID comparisons keep their existing
+semantics. `P.typeOf()` runs as a local filter for system and schema properties,
+including inside negated or connective predicates. Its type operands are preserved
+without schema value conversion. Missing properties do not match, including when the
+type predicate is negated. Type filters do not use backend indexes on their own. The new
+TinkerPop step APIs are adapted without discarding property metadata.
+
+Named `GValue` predicates retain their bindings when reused or cloned, and
+`updateVariable()` applies when the predicate builds subsequent traversals. Explicit
+named `limit()`/`range()` bounds have the same count behavior as literal bounds. Local
+String ID comparisons retain their leaf semantics inside `not`, `and` and `or`
+predicates. Mixed ID collections retain the typed matches of strings, numbers and UUIDs
+when a filter stays local; all-string collections keep standard string-ID comparison
+behavior. Fractional numeric IDs are not truncated to integer IDs.
+
+Count filtering is only truncated when every predicate branch has a supported bound that
+preserves the filter result. Unsupported negated collection or custom predicate branches
+keep the complete count, including inside `and` or `or` conditions.
+
+DATE schema properties accept TinkerPop `OffsetDateTime` values for writes and indexed
+equality or range queries. Values normalize by instant to the existing `java.util.Date`
+millisecond representation; offsets do not change the stored instant, and sub-
+millisecond precision is truncated. Existing DATE data needs no migration.
+
+### Counts and transaction visibility
+
+Counts made inside a transaction include that transaction's uncommitted vertex and edge
+changes. Only `count()` supports this fallback; other aggregates with uncommitted
+changes are rejected. Queries that combine uncommitted changes with pagination, a limit,
+or an offset remain unsupported.
+
+Queries and counts select the current transaction version before applying filters. When
+an element is removed and re-added with the same ID, the new element shadows the old
+updated record, even if only the old record matches the query's label.
+
+A count query that enters an index query path remains unsupported when that index
+transaction has uncommitted changes. It reports `Can't do index query when there are
+changes in transaction`. This includes indexed-property conditions and label-only
+queries on backends that need a label index. Graph-wide and explicit-ID counts can still
+use transaction merging without entering that index path.
+
+Unsupported text predicates remain traversal filters. For ordinary GraphStep and
+VertexStep extraction, if any condition in a `HasStep` cannot be converted, the whole
+step remains local, including its sibling label, ID and property conditions. Count
+optimization preserves that filter step. Existing special handling around `match()` and
+connective label filters remains part of traversal optimization.
+
+A self-loop contributes two occurrences to a vertex's `bothE()` traversal and one to
+each of `outE()`, `inE()` and the graph-wide `E()` traversal. Counts retain these
+multiplicities before and after transaction commit, including edge updates.
+
+Count optimization preserves steps that can filter candidates. Resetting an optimized
+count traversal permits it to execute again. Query-step equality does not depend on the
+result iterator from a previous execution.
+
+A scan in the middle of a traversal is counted once per incoming traverser, including
+its bulk. It is not replaced by a single backend count. For example, with three
+vertices, `g.V().V().count()` returns `9L`, while `g.V().count()` returns `3L` and
+remains eligible for count optimization.
+
+### Read-only application checks
+
+Run these against an existing graph, replacing the label, keys and values with ones
+present in that graph. The queries do not write graph or schema data. Use property keys
+with the query indexes required by your backend. The `age` example assumes an integer
+property; the path example needs an existing outgoing edge.
+
+```groovy
+g.V().hasLabel('person').has('name', 'marko').count()
+g.V().hasLabel('person').has('age', P.gt(0)).order().by('age').range(0, 2).values('age').toList()
+g.V().has('name', 'marko').limit(1).out().limit(1).path().toList()
+g.V().has('name', 'marko').limit(1).id().toList()
+```
+
+The first result is a numeric count. The second is a list of at most two numeric values
+in ascending order. The third is a list of paths; untyped GraphSON represents each path
+with `labels` and `objects`. The fourth is a list of graph IDs encoded as scalar values
+by GraphSON or GraphBinary; the scalar type follows the graph's ID type.
+
+For Cypher, send this read-only query to `POST
+/graphspaces/{graphspace}/graphs/{graph}/cypher`, substituting existing graph data:
+
+```cypher
+MATCH (n:person) WHERE n.name = 'marko' RETURN id(n) AS nodeId
+```
+
+The response has rows under `result.data`; each row is a map and `nodeId` is a JSON
+scalar, not a HugeGraph internal ID class. The Cypher API test covers this shape, nested
+IDs and paths.
+
+## HTTP request errors
+
+The supplied WebSocket/HTTP channelizer releases each HTTP request buffer once,
+including when a traversal alias is unknown. Such a request returns one HTTP 400, and a
+later valid request on the same keep-alive connection receives its own response. This
+avoids TinkerPop 3.8.1's duplicate release and second HTTP 500, which could be mistaken
+for the next request's response by a connection pool. The correction applies when the
+configured channelizer is TinkerPop's standard `WsAndHttpChannelizer`; HugeGraph selects
+its compatibility implementation at startup and retains TinkerPop's WebSocket handling.
+Custom channelizer classes keep their configured behavior. Unknown aliases still require
+the caller to use a graph or traversal-source name exposed by the server.
+
+## Cypher HTTP contracts
+
+### Request forms
+
+The endpoint is `/graphspaces/{graphspace}/graphs/{graph}/cypher`.
+
+| Method and content type | Request | Behavior |
+| --- | --- | --- |
+| GET | `?cypher=<URL-encoded statement>` | Query-string form works |
+| POST `application/json` | Raw Cypher text | Legacy raw-body form works |
+| POST `text/plain` | Any body | Rejected with HTTP 415 |
+| POST `application/json` | JSON object below | Separate bindings work |
+
+```json
+{
+ "cypher": "MATCH (n:person) WHERE n.name = $name RETURN n.name",
+ "parameters": {"name": "marko"}
+}
+```
+
+POST bodies containing JSON arrays, strings, numbers, booleans, or null are rejected
+with HTTP 400. Malformed JSON objects and trailing tokens are also request errors and do
+not fall back to raw Cypher.
+
+For the JSON-object form, `cypher` must be a nonblank string and `parameters`, when
+present, must be an object. Omitting `parameters` means an empty map. Missing bindings
+produce an execution error; an explicit null value is accepted. Bindings may be named
+`id` and `label`. There is a fixed limit of 16 top-level parameters. This Cypher
+processor does not support a `maxParameters` configuration setting. Values containing
+quotes and newlines remain bound data rather than changing query text.
+
+A Map counts as one top-level parameter. Group related values into one Map and reference
+its fields explicitly; the vertex label and required properties must match the graph's
+existing schema:
+
+```json
+{
+ "cypher": "CREATE (n:person {name:$props.name, age:$props.age, city:$props.city}) RETURN n.name",
+ "parameters": {"props": {"name": "new-person", "age": 20, "city": "Beijing"}}
+}
+```
+
+Do not use the shorthand `CREATE (n:person $props)`: the pinned translator does not
+assign the bound Map's properties in that path. The parameter limit counts top-level
+bindings, not the entries inside a bound Map or a returned Map. A Map containing 17
+entries can be bound through one parameter with explicit property references; 17 top-level
+bindings are rejected.
+
+The translator's reserved null-marker string `" cypher.null"` (two leading spaces) is
+rejected in binding values, including nested maps and lists, to avoid silently
+converting a user string to null. Actual JSON null remains accepted. The same marker in
+query literals or stored properties is an existing translator limitation and is outside
+this compatibility target. Check the HTTP status and body `status.code`: execution
+failures retain HTTP 200 with `status.code` 400 and null result data.
+
+Traversal iteration failures, including Java errors, attempt rollback on the execution
+thread and return a terminal failure response. A rollback exception is attached to the
+original failure.
+
+### Compatibility coverage
+
+The API fixture uses a strong schema with a `SECONDARY` index on `city` and a `RANGE`
+index on `age`. The regression suite covers:
+
+| Area | Covered behavior |
+|---|---|
+| Reads | Label scans, equality and range predicates, computed regex whole-string matching, boolean combinations, directed one- and two-hop patterns, empty results. |
+| Results | Aliases, scalar and node values, nested maps/lists, relationship IDs, path shape, null and missing-property semantics. |
+| Aggregation and pagination | `DISTINCT`, `count`/`sum`/`min`/`max`/`avg`, `ORDER BY`, `SKIP`, and `LIMIT`. |
+| Parameters | String, number, boolean, null, empty and missing bindings; quote/newline preservation. |
+| Writes | Vertex and edge `CREATE`, property `SET`, edge and vertex `DELETE`, with native REST readback. |
+| Failures and routing | Invalid syntax, request shape, binding keys, schema values, authentication, and graph routing. |
+| Failed writes | A rejected multi-vertex create leaves no residue in later Cypher and native reads. |
+
+Write regressions check persisted state through native REST reads. Applications are not
+required to issue native reads after each successful Cypher write.
+
+### Scope and regression sources
+
+Advanced constructs including `MERGE`, `OPTIONAL MATCH`, `WITH`, `UNWIND`, `UNION`, and
+variable-length paths are not established by the compatibility coverage described here.
+HStore-specific behavior, Bolt, cross-request transactions, external client versions and
+performance also require separate validation. This guide does not claim full openCypher
+or Neo4j compatibility.
+
+The pinned translator is `org.opencypher.gremlin:translation:1.0.4`. Its translation
+behavior defines the limitations documented above.
+
+Long-term regression sources:
+
+- [`CypherApiTest`](../hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/api/CypherApiTest.java)
+ covers request forms, schema-aware reads and writes, result shapes,
+ authentication and graph routing, and failed-write isolation.
+- [`CypherClientTest`](../hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/api/cypher/CypherClientTest.java)
+ covers result normalization and HTTP client behavior.
+- [`CypherOpProcessorTest`](../hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/opencypher/CypherOpProcessorTest.java)
+ covers binding validation and execution failure handling.
+
+## Transaction and graph ownership
+
+Server releases the current thread's graph transactions when a REST request finishes and
+when an authenticated context task exits. Cleanup visits every registered graph even
+when one graph fails to close, and reports failures. Cleanup explicitly rolls back
+unfinished writes, including when `onClose(COMMIT)` was configured, then releases
+backend transactions and resets thread-local transaction behavior and listeners.
+Applications must commit successful writes before leaving the request/task boundary.
+Closing an OLTP traverser preserves the caller's transaction so that the caller can
+still commit or roll it back.
+
+The cleanup also applies when schema caches have become cold. Deleting an auth project
+or user must remove its associated access or belong edges while preserving unrelated
+relationships. The special OLAP vertex path remains separate; negative schema IDs alone
+do not identify edges that can be skipped.
+
+Shared schema and element caches retain their invalidation listeners until the graph
+closes. Request cleanup releases backend leases while preserving those graph caches and
+their schema identity.
+
+Task database workers release their owned task and graph transactions on the worker
+thread. A failure while closing one owner must not skip cleanup of other owners or
+server-info resources. Cleanup retains the first exception and attaches later cleanup
+failures as suppressed exceptions.
+
+Graph close attempts auth, scheduler and caller-transaction cleanup separately. If a
+scheduler remains registered or the graph reports active transactions, the graph remains
+open for a later close attempt. Pending local tasks retain their task database transaction
+so they can persist completion before the next close attempt. An incomplete scheduler
+drain does not queue transaction cleanup behind that scheduler's blocked worker. Stopping
+distributed task dispatch is distinct from completing drain: a timeout does not make
+the next close return success.
+
+Once these logical checks pass, graph close releases shared cache listeners, invokes
+`provider.close()` and destroys graph locks even if an earlier cleanup step reported an error. The error
+remains visible to the caller. Logical graph closure does not prove that a failed
+backend native-resource release succeeded; callers must inspect the reported failure
+rather than infer physical release from graph state.
+
+Regression sources:
+
+- [`TransactionLifecycleTest`](../hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/auth/TransactionLifecycleTest.java)
+ exercises request and authenticated-task ownership and rollback.
+- [`StandardTaskSchedulerTxTest`](../hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/task/StandardTaskSchedulerTxTest.java)
+ exercises task worker transaction cleanup.
+- [`HugeFactoryTest`](../hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/HugeFactoryTest.java)
+ exercises graph close failure continuation and distributed drain retry.
+
+## Compatibility evidence and deployment checks
+
+Checked-in regression tests define a coverage boundary, not a supported matrix for every
+external client, authentication method or backend deployment.
+
+| Path | Regression coverage | Deployment checks still required |
+|---|---|---|
+| GraphSON | Serializer configuration, MIME selection and wire fixtures. | External client versions, network and authentication combinations. |
+| GraphBinary | Shipped Driver configuration, typed results and in-process ID, predicate and element round trips. | External client, network and authentication combinations. |
+| Cypher HTTP | Local API harness, result IDs and paths, Basic-auth test client. | Bearer authentication and external REST clients. |
+
+[`GremlinConfigCompatibilityTest`](../hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/unit/config/GremlinConfigCompatibilityTest.java)
+contains serializer and configuration regressions. Use the Cypher sources linked above
+for API behavior. Keep runtime dependency versions aligned with the root POM and shipped
+configuration rather than copying old configuration fragments into a different runtime.
+
+Structure and Process suites exercise TinkerPop behavior on Memory, RocksDB and HStore.
+HStore has separate standard-process and feature-test selections.
+[`HugeGraphProviderLifecycleTest`](../hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/tinkerpop/HugeGraphProviderLifecycleTest.java)
+checks backend cleanup and graph reopening. The test selections and setup are
+maintained in the [test POM](../hugegraph-server/hugegraph-test/pom.xml),
+[Server CI](../.github/workflows/server-ci.yml) and
+[PD/Store CI](../.github/workflows/pd-store-ci.yml).
+
+Inspect executed tests for the backend and protocols used by the deployment. Selected
+Structure or Process suites must execute tests; an all-skipped report is not
+compatibility evidence. Run the read-only application examples against representative
+existing data and validate the serializers, client versions and authentication
+combinations actually used by the application.
diff --git a/docs/upgrade-tinkerpop-3.8.md b/docs/upgrade-tinkerpop-3.8.md
deleted file mode 100644
index d1eaf2ecb..000000000
--- a/docs/upgrade-tinkerpop-3.8.md
+++ /dev/null
@@ -1,261 +0,0 @@
-# TinkerPop 3.8.1 and Groovy 4 upgrade
-
-This upgrade moves Server and the distributed modules from TinkerPop 3.5.1 to
-3.8.1 and uses Groovy 4.0.25. Build and run every module with Java 17.
-
-## Configuration and clients
-
-Use the Gremlin configuration files shipped with the upgraded Server. Serializer
-classes move from `org.apache.tinkerpop.gremlin.driver.ser` to
-`org.apache.tinkerpop.gremlin.util.ser`. The supplied remote configurations
-include the corresponding HugeGraph registries.
-
-The Java Driver sample `gremlin-driver-settings.yaml` and Gremlin Console's
-`remote.yaml` now default to GraphBinary V1. The serializer entry changes as
-follows:
-
-```diff
-- className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0
-+ className: org.apache.tinkerpop.gremlin.util.ser.GraphBinaryMessageSerializerV1
-```
-
-Keep the HugeGraph builder and registry in both client configurations:
-
-```yaml
-serializer:
- className: org.apache.tinkerpop.gremlin.util.ser.GraphBinaryMessageSerializerV1
- config:
- serializeResultToString: false
- builder: org.apache.hugegraph.io.HugeGraphTypeSerializerRegistryBuilder
- ioRegistries: [org.apache.hugegraph.io.HugeGraphIoRegistry]
-```
-
-GraphBinary preserves graph elements and UUID IDs for Java Driver accessors such
-as `Result.getVertex()` and `Result.get(UUID.class)`. Untyped GraphSON V1 decodes
-these results as maps and strings.
-
-`remote-objects.yaml` serves the Cypher HTTP JSON path and continues to use
-`org.apache.tinkerpop.gremlin.util.ser.GraphSONUntypedMessageSerializerV1` with
-`HugeGraphIoRegistry`. It retains its duplicate registry entry as a TinkerPop
-driver compatibility workaround.
-
-For the server, keep the GraphBinary builder and registry. The serializer order
-is GraphBinary, untyped GraphSON V1/V2/V3, then typed GraphSON V2/V3. The
-GraphSON entries keep the HugeGraph registry. The first entries look like this:
-
-```yaml
-serializers:
- - className: org.apache.tinkerpop.gremlin.util.ser.GraphBinaryMessageSerializerV1
- config:
- serializeResultToString: false
- builder: org.apache.hugegraph.io.HugeGraphTypeSerializerRegistryBuilder
- ioRegistries: [org.apache.hugegraph.io.HugeGraphIoRegistry]
- - className: org.apache.tinkerpop.gremlin.util.ser.GraphSONUntypedMessageSerializerV1
- config:
- serializeResultToString: false
- ioRegistries: [org.apache.hugegraph.io.HugeGraphIoRegistry,
- org.apache.hugegraph.io.HugeGraphSONV1MessageIoRegistry]
-```
-
-The full ordering is in `gremlin-server.yaml`; the config test checks that
-`application/json` selects untyped V1 in each server config variant.
-
-Typed GraphSON is available for the supported simple values and IDs, not as a
-replacement for untyped serialization of every HugeGraph schema or graph
-object. Typed V1 is not registered: its legacy `@class` deserialization can
-construct Java objects before request authentication. V2/V3 use `@type`/`@value`.
-
-GraphBinary uses `HugeGraphTypeSerializerRegistryBuilder` to write HugeGraph IDs
-as their underlying values. Schema results use standard maps and Blob values use
-standard binary values (`ByteBuffer` in the Java Driver), including when nested
-in returned vertices, edges or collections. `Optional` results carry their value
-or `null` when empty, including results from `graph.variables().get()`. `File`
-results retain the map shape `{"file": "name"}` using the file name, including
-inside nested results. Untyped GraphSON V1 keeps the legacy Tree array of `key`/`value`
-entries and accepts scalar as well as element keys through the additional
-`HugeGraphSONV1MessageIoRegistry` in that serializer's configuration. Keep this
-registry limited to untyped V1 messages. Standard graph-file IO and typed
-GraphSON serializers keep `HugeGraphIoRegistry` alone and use the TinkerPop
-Tree format for their negotiated version.
-
-HugeGraph enums such as `DataType.TEXT` and `Directions.OUT` return their names
-over GraphBinary, including in maps and lists. `graph.schema()` returns a map
-with `propertykeys`, `vertexlabels`, `edgelabels` and `indexlabels` lists, using
-the same schema conversion. TinkerPop enums retain their native wire types.
-Backend `Shard` values, including `graph.metadata(HugeType.VERTEX, 'splits', size)`,
-return standard maps with `start`, `end` and an exact 64-bit `length`, including
-inside lists and maps. The default Driver does not need a private Shard serializer.
-
-Local edge ID filters accept both serialized strings and native edge IDs in
-mixed collections: `hasId(not(without(ids)))` and `hasId(within(ids))` agree on
-membership and count. Numeric and string vertex IDs keep their distinct types.
-
-Groovy interpolated strings (`GString`, including subclasses) return as standard
-strings over GraphBinary, including inside lists and maps. For example,
-`def value = 42; "r3-value:${value}"` returns `"r3-value:42"`. The untyped GraphSON
-path retains the legacy GString bean map with its `values` and `strings` fields.
-
-Cypher extension predicates use the current TinkerPop predicate constructor
-through the existing translator extension point. Computed-expression regex
-filters retain whole-string matching (`String.matches`), so `"marko"` matches
-`"mar.*"` and does not match `"ark"`.
-The public `cypherRegex`, `cypherIsString`, `cypherIsNode` and `cypherIsRelationship`
-factories imported by the bundled Gremlin configuration use that same adapter,
-so the Gremlin translation returned by `EXPLAIN` can be replayed.
-
-Cypher results normalize HugeGraph IDs, including
-values nested in collections, maps and paths. Cyclic results and nesting deeper
-than 32 levels are rejected.
-
-The default script engine remains Gremlin Groovy. This upgrade does not switch
-remote requests to GremlinLang. Validate application scripts against the full
-TinkerPop 3.5.1 to 3.8.1 upgrade interval before deploying.
-
-## YAML configuration safety
-
-Jackson JSON and YAML modules share the root BOM version. Fabric8 kubeconfig
-extensions can contain integer and decimal values; the K8s configuration
-regression verifies auto-configuration without contacting a cluster.
-
-This TinkerPop 3.8.1 upgrade uses SnakeYAML 2.2, Spring Boot 2.5.15 and
-Spring Framework 5.3.27. The Boot patch keeps PD and Store's YAML startup
-loader compatible with SnakeYAML 2.x. These versions belong to the upgraded
-runtime; do not apply the SnakeYAML override to a TinkerPop 3.5.1 deployment.
-The runtime also manages Jackson YAML 2.15.2 so Fabric8 can read kubeconfig
-with SnakeYAML 2.2. Store's Log4j SLF4J binding and CLI logging use 2.18.0;
-Store Node retains its Boot-managed Log4j core version.
-
-Quota templates and PD-delivered Store configuration accept standard YAML
-maps, lists and scalars. Custom Java object tags such as `!!com.example.Type`
-are rejected before object construction. Keep these files as configuration
-data; Java class names used as ordinary string values remain supported.
-
-## List property compatibility
-
-The public TinkerPop `Graph.Features` contract changes:
-`features().vertex().properties().supportsUniformListValues()` and
-`features().edge().properties().supportsUniformListValues()` now return `false`
-instead of `true`. Clients and compatibility tests that inspect these flags may
-stop enabling uniform-list operations. The corresponding graph-variable flag
-remains `true`.
-
-HugeGraph still supports vertex and edge properties whose schema declares
-`LIST` cardinality with `valueList()` and a supported element type, for example
-`schema.propertyKey("tags").asText().valueList().create()`. Values must match the
-schema's element type. This schema capability does not provide arbitrary Java
-`List` values for `SINGLE` properties. The feature-reporting change itself does
-not remove stored lists or require rewriting existing list data.
-
-When upgrading, review client feature-detection branches. Applications using
-HugeGraph list properties should retain their explicit element type and `LIST`
-schema, and verify list writes and reads with that schema. Generic TinkerPop
-clients should honor the reported feature flags; do not treat a skipped
-uniform-list compatibility test as validation of HugeGraph's schema list path.
-
-## Queries
-
-See [query count and filter behavior](query-semantics.md) for transaction counts
-and local predicate filtering. Version-specific handling retains negated
-predicates and filtering barriers where pushdown would change their meaning.
-For UUID IDs, local negated filters such as `hasId(P.not(P.neq(uuid)))` retain
-the same matches as ID equality. This also applies when an ID filter stays
-local beside a property filter that cannot be pushed down. The complete
-`HasStep` remains local;
-string and numeric ID comparisons keep their existing semantics.
-`P.typeOf()` runs as a local filter for system and schema properties, including
-inside negated or connective predicates. Its type operands are preserved without
-schema value conversion. Missing properties do not match, including when the
-type predicate is negated. Type filters do not use backend indexes on their own.
-The new TinkerPop step APIs are adapted without discarding property metadata.
-
-Named `GValue` predicates retain their bindings when reused or cloned, and
-`updateVariable()` applies when the predicate builds subsequent traversals. Explicit named
-`limit()`/`range()` bounds have the same count behavior as literal bounds.
-Local String ID comparisons retain their leaf semantics inside `not`, `and`
-and `or` predicates. Mixed ID collections retain the typed matches of
-strings, numbers and UUIDs when a filter stays local; all-string collections
-keep standard string-ID comparison behavior. Fractional numeric IDs are not
-truncated to integer IDs.
-
-Count filtering is only truncated when every predicate branch has a supported
-bound that preserves the filter result. Unsupported negated collection or
-custom predicate branches keep the complete count, including inside `and` or
-`or` conditions.
-
-DATE schema properties accept TinkerPop `OffsetDateTime` values for writes and
-indexed equality or range queries. Values normalize by instant to the existing
-`java.util.Date` millisecond representation; offsets do not change the stored
-instant, and sub-millisecond precision is truncated. Existing DATE data needs
-no migration.
-
-## Read-only upgrade smoke
-
-Run these against an existing graph, replacing the label, keys and values with
-ones present in that graph. The queries do not write graph or schema data.
-Use property keys with the query indexes required by your backend.
-The `age` example assumes an integer property; the path example needs an
-existing outgoing edge.
-
-```groovy
-g.V().hasLabel('person').has('name', 'marko').count()
-g.V().hasLabel('person').has('age', P.gt(0)).order().by('age').range(0, 2).values('age').toList()
-g.V().has('name', 'marko').limit(1).out().limit(1).path().toList()
-g.V().has('name', 'marko').limit(1).id().toList()
-```
-
-The first result is a numeric count. The second is a list of at most two numeric
-values in ascending order. The third is a list of paths; untyped GraphSON
-represents each path with `labels` and `objects`. The fourth is a list of graph
-IDs encoded as scalar values by GraphSON or GraphBinary; the scalar type follows
-the graph's ID type.
-
-For Cypher, send this read-only query to
-`POST /graphspaces/{graphspace}/graphs/{graph}/cypher`, substituting existing
-graph data:
-
-```cypher
-MATCH (n:person) WHERE n.name = 'marko' RETURN id(n) AS nodeId
-```
-
-The response has rows under `result.data`; each row is a map and `nodeId` is a
-JSON scalar, not a HugeGraph internal ID class. The Cypher API test covers this
-shape, nested IDs and paths.
-
-## HTTP request errors
-
-The supplied WebSocket/HTTP channelizer releases each HTTP request buffer once,
-including when a traversal alias is unknown. Such a request returns one HTTP 400,
-and a later valid request on the same keep-alive connection receives its own
-response. This avoids TinkerPop 3.8.1's duplicate release and second HTTP 500,
-which could be mistaken for the next request's response by a connection pool.
-The correction applies when the configured channelizer is TinkerPop's standard
-`WsAndHttpChannelizer`; HugeGraph selects its compatibility implementation at
-startup and retains TinkerPop's WebSocket handling. Custom channelizer classes
-keep their configured behavior. Unknown aliases still require the caller to
-use a graph or traversal-source name exposed by the server.
-
-## Compatibility verification
-
-The repository's checked-in tests establish the following boundaries:
-
-| Path | Evidence | Not established by these tests |
-|---|---|---|
-| GraphSON | Config, MIME and serializer fixtures. | External client/version, network and auth matrix. |
-| GraphBinary | Shipped Driver config, typed Driver result access, and in-process ID, predicate and element round trips. | External client, network and auth. |
-| Cypher HTTP | Local API test checks IDs and paths with Basic auth. | Bearer or external REST client/version matrix. |
-
-Config, GraphSON and GraphBinary evidence comes from
-`GremlinConfigCompatibilityTest`. The Cypher evidence comes from
-`CypherApiTest`, which uses the local API harness and its Basic-auth test client.
-
-The upgrade CI explicitly selects Structure and Process tests for Memory,
-RocksDB and HStore. HStore has separate standard-process and feature-test steps.
-Provider lifecycle checks run against the selected backend, including reopening
-the graph after cleanup and after closing the provider context. Each selected
-Structure or Process suite must report at least one executed test; a report
-containing only skipped tests fails the gate.
-A green build with those steps skipped is not compatibility evidence. Review
-reports from the actual PR revision before deploying. The serializer tests use
-the project's TinkerPop 3.8.1 dependency; they do not define a supported external
-client matrix for version, GraphSON or GraphBinary, and authentication. Verify
-the combinations used by your deployment.
diff --git a/hugegraph-server/README.md b/hugegraph-server/README.md
index c83e4a305..ea22a03d3 100644
--- a/hugegraph-server/README.md
+++ b/hugegraph-server/README.md
@@ -6,7 +6,7 @@ HugeGraph Server consists of two layers of functionality: the graph engine layer
- Graph Engine Layer:
- REST Server: Provides a RESTful API for querying graph/schema information, supports the [Gremlin](https://tinkerpop.apache.org/gremlin.html) and [Cypher](https://en.wikipedia.org/wiki/Cypher) query languages, and offers APIs for service monitoring and operations.
- - See [Cypher compatibility notes](../docs/cypher-compatibility.md) for request forms, verified behavior, and
+ - See [Cypher HTTP contracts](../docs/server-runtime.md#cypher-http-contracts) for request forms, verified behavior, and
unverified scope.
- Graph Engine: Supports both OLTP and OLAP graph computation types, with OLTP implementing the [Apache TinkerPop3](https://tinkerpop.apache.org) framework.
- Backend Interface: Implements the storage of graph data to the backend.
diff --git a/hugegraph-server/hugegraph-api/src/main/java/org/apache/hugegraph/auth/HugeFactoryAuthProxy.java b/hugegraph-server/hugegraph-api/src/main/java/org/apache/hugegraph/auth/HugeFactoryAuthProxy.java
index 78ef3d72d..98f9ea8bc 100644
--- a/hugegraph-server/hugegraph-api/src/main/java/org/apache/hugegraph/auth/HugeFactoryAuthProxy.java
+++ b/hugegraph-server/hugegraph-api/src/main/java/org/apache/hugegraph/auth/HugeFactoryAuthProxy.java
@@ -185,7 +185,7 @@ public final class HugeFactoryAuthProxy {
"this$0");
Reflection.registerFieldsToFilter(HugeGraphAuthProxy.Context.class, "ADMIN", "user");
Reflection.registerFieldsToFilter(HugeGraphAuthProxy.ContextTask.class, "runner",
- "context");
+ "cleanup", "context");
Reflection.registerFieldsToFilter(StandardHugeGraph.class, "LOG", "started", "closed",
"mode", "variables", "name", "params", "configuration",
"schemaEventHub", "graphEventHub", "indexEventHub",
@@ -203,7 +203,8 @@ public final class HugeFactoryAuthProxy {
"access$14", "access$15", "access$16", "access$17",
"access$18", "serializer", "loadSchemaStore",
"loadSystemStore", "loadGraphStore", "closeTx",
- "analyzer", "serverInfoManager", "reloadRamtable",
+ "closeCurrentThreadTransaction", "analyzer",
+ "serverInfoManager", "reloadRamtable",
"reloadRamtable", "access$19", "access$20", "access$21");
Reflection.registerFieldsToFilter(
loadClass("org.apache.hugegraph.StandardHugeGraph$StandardHugeGraphParams"),
@@ -298,8 +299,10 @@ public final class HugeFactoryAuthProxy {
"autoCommit", "beforeRead", "afterWrite", "afterRead",
"commitMutation2Backend", "checkOwnerThread", "doAction",
"store", "reset");
- Reflection.registerFieldsToFilter(HugeFactory.class, "LOG", "NAME_REGEX", "graphs");
- Reflection.registerMethodsToFilter(HugeFactory.class, "lambda$0");
+ Reflection.registerFieldsToFilter(HugeFactory.class, "LOG", "NAME_REGEX", "graphs",
+ "GRAPHS");
+ Reflection.registerMethodsToFilter(HugeFactory.class, "lambda$0",
+ "closeCurrentThreadTransactions");
Reflection.registerFieldsToFilter(SchemaElement.class, "graph", "id", "name", "userdata",
"status");
Reflection.registerFieldsToFilter(HugeVertex.class, "EMPTY_SET", "id", "label", "edges",
diff --git a/hugegraph-server/hugegraph-api/src/main/java/org/apache/hugegraph/auth/HugeGraphAuthProxy.java b/hugegraph-server/hugegraph-api/src/main/java/org/apache/hugegraph/auth/HugeGraphAuthProxy.java
index 9d23e8255..2ae3d7312 100644
--- a/hugegraph-server/hugegraph-api/src/main/java/org/apache/hugegraph/auth/HugeGraphAuthProxy.java
+++ b/hugegraph-server/hugegraph-api/src/main/java/org/apache/hugegraph/auth/HugeGraphAuthProxy.java
@@ -1291,11 +1291,17 @@ public final class HugeGraphAuthProxy implements HugeGraph {
static class ContextTask implements Runnable {
private final Runnable runner;
+ private final Runnable cleanup;
private final Context context;
public ContextTask(Runnable runner) {
+ this(runner, HugeFactory::closeCurrentThreadTransactions);
+ }
+
+ ContextTask(Runnable runner, Runnable cleanup) {
this.context = getContext();
this.runner = runner;
+ this.cleanup = cleanup;
}
@Override
@@ -1305,7 +1311,7 @@ public final class HugeGraphAuthProxy implements HugeGraph {
this.runner.run();
} finally {
try {
- HugeFactory.closeCurrentThreadTransactions();
+ this.cleanup.run();
} catch (Throwable e) {
LOG.error("Failed to close Gremlin worker transactions", e);
} finally {
diff --git a/hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/HugeFactory.java b/hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/HugeFactory.java
index 451455ba0..78a6de78f 100644
--- a/hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/HugeFactory.java
+++ b/hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/HugeFactory.java
@@ -22,6 +22,7 @@ import org.apache.hugegraph.exception.HugeException;
import java.io.File;
import java.net.URL;
import java.util.ArrayList;
+import java.util.Collection;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
@@ -118,6 +119,11 @@ public class HugeFactory {
synchronized (HugeFactory.class) {
graphs = new ArrayList<>(GRAPHS.values());
}
+ closeCurrentThreadTransactions(graphs);
+ }
+
+ static void closeCurrentThreadTransactions(
+ Collection<? extends StandardHugeGraph> graphs) {
Throwable failure = null;
for (StandardHugeGraph graph : graphs) {
try {
diff --git a/hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/StandardHugeGraph.java b/hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/StandardHugeGraph.java
index d9b838e89..3191fa91e 100644
--- a/hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/StandardHugeGraph.java
+++ b/hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/StandardHugeGraph.java
@@ -386,25 +386,51 @@ public class StandardHugeGraph implements HugeGraph {
}
void closeCurrentThreadTransaction() {
+ Throwable failure = null;
try {
if (this.tx.isOpen()) {
// Request/task cleanup must never commit unfinished writes.
this.tx.rollback();
}
+ } catch (RuntimeException | Error error) {
+ failure = error;
+ throw error;
} finally {
this.tx.clearTransactionListeners();
this.tx.resetState();
- this.tx.destroyTransaction();
+ try {
+ this.tx.destroyTransaction();
+ } catch (RuntimeException | Error error) {
+ if (failure == null) {
+ throw error;
+ }
+ if (failure != error) {
+ failure.addSuppressed(error);
+ }
+ }
}
}
private void closeTx() {
+ Throwable failure = null;
try {
if (this.tx.isOpen()) {
this.tx.close();
}
+ } catch (RuntimeException | Error error) {
+ failure = error;
+ throw error;
} finally {
- this.tx.destroyTransaction();
+ try {
+ this.tx.destroyTransaction();
+ } catch (RuntimeException | Error error) {
+ if (failure == null) {
+ throw error;
+ }
+ if (failure != error) {
+ failure.addSuppressed(error);
+ }
+ }
}
}
@@ -1106,43 +1132,64 @@ public class StandardHugeGraph implements HugeGraph {
}
LOG.info("Close graph {}", this);
- if (StandardAuthManager.isLocal(this.authManager)) {
- this.authManager.close();
- }
- this.taskManager.closeScheduler(this.params);
- try {
- this.closeTx();
- } finally {
- this.closed = true;
+ Throwable failure = null;
+ for (Runnable close : new Runnable[]{() -> {
+ if (StandardAuthManager.isLocal(this.authManager)) {
+ this.authManager.close();
+ }
+ }, () -> this.taskManager.closeScheduler(this.params), this::closeTx}) {
try {
- CachedGraphTransaction.closeGraph(this.params);
- } finally {
- try {
- try {
- CachedSchemaTransaction.closeGraph(this.params);
- } finally {
- CachedSchemaTransactionV2.closeGraph(this.params);
- }
- } finally {
- try {
- this.storeProvider.close();
- } finally {
- LockUtil.destroy(this.spaceGraphName());
- }
+ close.run();
+ } catch (RuntimeException | Error error) {
+ if (failure == null) {
+ failure = error;
+ } else if (failure != error) {
+ failure.addSuppressed(error);
}
}
}
- // Make sure that all transactions are closed in all threads
- if (!this.tx.closed()) {
+ // A failed task drain retains its scheduler and remains retryable.
+ boolean drained = this.tx.closed() && this.taskManager.getScheduler(this.params) == null;
+ if (!drained) {
for (String key : this.tx.openedThreads) {
LOG.warn("thread [{}] did not close transaction", key);
}
+ RuntimeException active = new IllegalStateException(String.format(
+ "Ensure scheduler and tx closed in all threads when closing graph '%s'",
+ this.spaceGraphName()));
+ if (failure == null) {
+ failure = active;
+ } else {
+ failure.addSuppressed(active);
+ }
+ }
+ if (drained) {
+ // Plain provider close only notifies listeners; the Raft wrapper also
+ // stops its context. Owner-close errors do not imply native release.
+ this.closed = true;
+ for (Runnable close : new Runnable[]{
+ () -> CachedGraphTransaction.closeGraph(this.params),
+ () -> CachedSchemaTransaction.closeGraph(this.params),
+ () -> CachedSchemaTransactionV2.closeGraph(this.params),
+ this.storeProvider::close, () -> LockUtil.destroy(this.spaceGraphName())}) {
+ try {
+ close.run();
+ } catch (RuntimeException | Error error) {
+ if (failure == null) {
+ failure = error;
+ } else if (failure != error) {
+ failure.addSuppressed(error);
+ }
+ }
+ }
+ }
+ if (failure instanceof Error) {
+ throw (Error) failure;
+ }
+ if (failure != null) {
+ throw (RuntimeException) failure;
}
- E.checkState(this.tx.closed(),
- "Ensure tx closed in all threads when closing graph '%s'",
- this.spaceGraphName());
-
}
@Override
@@ -1361,22 +1408,25 @@ public class StandardHugeGraph implements HugeGraph {
}
public void close() {
- try {
- this.graphTx.close();
- } catch (Exception e) {
- LOG.error("Failed to close GraphTransaction", e);
+ Throwable failure = null;
+ for (Runnable close : new Runnable[]{this.graphTx::close,
+ this.systemTx::close,
+ this.schemaTx::close}) {
+ try {
+ close.run();
+ } catch (RuntimeException | Error error) {
+ if (failure == null) {
+ failure = error;
+ } else if (failure != error) {
+ failure.addSuppressed(error);
+ }
+ }
}
-
- try {
- this.systemTx.close();
- } catch (Exception e) {
- LOG.error("Failed to close SystemTransaction", e);
+ if (failure instanceof Error) {
+ throw (Error) failure;
}
-
- try {
- this.schemaTx.close();
- } catch (Exception e) {
- LOG.error("Failed to close SchemaTransaction", e);
+ if (failure != null) {
+ throw (RuntimeException) failure;
}
}
@@ -1852,10 +1902,14 @@ public class StandardHugeGraph implements HugeGraph {
// Do close if needed, then remove the reference
Txs txs = this.transactions.get();
- if (txs != null) {
- txs.close();
+ try {
+ if (txs != null) {
+ txs.close();
+ }
+ } finally {
+ // A reused request/worker must never inherit partially closed owners.
+ this.transactions.remove();
}
- this.transactions.remove();
}
}
}
diff --git a/hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/backend/store/BackendSessionPool.java b/hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/backend/store/BackendSessionPool.java
index 521f44556..518247159 100644
--- a/hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/backend/store/BackendSessionPool.java
+++ b/hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/backend/store/BackendSessionPool.java
@@ -25,6 +25,7 @@ import java.util.concurrent.atomic.AtomicInteger;
import org.apache.commons.lang3.tuple.Pair;
import org.apache.hugegraph.config.CoreOptions;
import org.apache.hugegraph.config.HugeConfig;
+import org.apache.hugegraph.util.E;
import org.apache.hugegraph.util.Log;
import org.slf4j.Logger;
@@ -56,14 +57,18 @@ public abstract class BackendSessionPool {
public final BackendSession getOrNewSession() {
BackendSession session = this.threadLocalSession.get();
if (session == null) {
- session = this.newSession();
- assert session != null;
- this.threadLocalSession.set(session);
- assert !this.sessions.containsKey(Thread.currentThread().getId());
- this.sessions.put(Thread.currentThread().getId(), session);
- int sessionCount = this.sessionCount.incrementAndGet();
- LOG.debug("Now(after connect({})) session count is: {}",
- this, sessionCount);
+ // Serialize new borrowers with the last-session native close.
+ synchronized (this) {
+ E.checkState(this.opened(), "Backend session pool is closed");
+ session = this.newSession();
+ assert session != null;
+ this.threadLocalSession.set(session);
+ assert !this.sessions.containsKey(Thread.currentThread().getId());
+ this.sessions.put(Thread.currentThread().getId(), session);
+ int sessionCount = this.sessionCount.incrementAndGet();
+ LOG.debug("Now(after connect({})) session count is: {}",
+ this, sessionCount);
+ }
} else {
this.detectSession(session);
}
@@ -131,7 +136,7 @@ public abstract class BackendSessionPool {
}
}
- public boolean close() {
+ public synchronized boolean close() {
Pair<Integer, Integer> result = Pair.of(-1, -1);
try {
result = this.closeSession();
diff --git a/hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/backend/tx/GraphTransaction.java b/hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/backend/tx/GraphTransaction.java
index 70487f159..e1b5c37d2 100644
--- a/hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/backend/tx/GraphTransaction.java
+++ b/hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/backend/tx/GraphTransaction.java
@@ -765,7 +765,7 @@ public class GraphTransaction extends IndexableTransaction {
}
public Iterator<Vertex> queryTaskInfos(Object... vertexIds) {
- if (this.graph().backendStoreFeatures().supportsTaskAndServerVertex()) {
+ if (this.storeFeatures().supportsTaskAndServerVertex()) {
return this.queryVerticesByIds(vertexIds, false, false,
HugeType.TASK);
}
diff --git a/hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/task/DistributedTaskScheduler.java b/hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/task/DistributedTaskScheduler.java
index 802049eb0..d89a32040 100644
--- a/hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/task/DistributedTaskScheduler.java
+++ b/hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/task/DistributedTaskScheduler.java
@@ -25,6 +25,7 @@ import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
+import java.util.concurrent.RejectedExecutionException;
import java.util.concurrent.ScheduledFuture;
import java.util.concurrent.ScheduledThreadPoolExecutor;
import java.util.concurrent.ThreadPoolExecutor;
@@ -65,6 +66,7 @@ public class DistributedTaskScheduler extends TaskAndResultScheduler {
* the status of scheduler
*/
private final AtomicBoolean closed = new AtomicBoolean(true);
+ private volatile boolean closeCompleted;
private final ConcurrentHashMap<Id, HugeTask<?>> runningTasks = new ConcurrentHashMap<>();
private final Set<Id> deletingTasks = ConcurrentHashMap.newKeySet();
@@ -525,7 +527,8 @@ public class DistributedTaskScheduler extends TaskAndResultScheduler {
@Override
public boolean close() {
- if (this.closed.get()) {
+ // Dispatch stopped does not imply running jobs and owners have drained.
+ if (this.closeCompleted) {
return true;
}
@@ -545,11 +548,19 @@ public class DistributedTaskScheduler extends TaskAndResultScheduler {
barrier.get(schedulePeriod + 5, TimeUnit.SECONDS);
} catch (TimeoutException e) {
LOG.warn("Cron task did not complete in time when closing scheduler");
+ return false;
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
LOG.warn("Interrupted while waiting for cron task to complete", e);
+ return false;
} catch (ExecutionException e) {
LOG.warn("Exception while waiting for cron task to complete", e);
+ return false;
+ } catch (RejectedExecutionException e) {
+ if (!this.schedulerExecutor.isTerminated()) {
+ LOG.warn("Scheduler executor has not drained when closing scheduler", e);
+ return false;
+ }
}
// cancel all running tasks
@@ -566,19 +577,54 @@ public class DistributedTaskScheduler extends TaskAndResultScheduler {
return false;
}
- if (!this.taskDbExecutor.isShutdown()) {
- this.call(() -> {
- try {
- this.tx().close();
- } catch (ConnectionException ignored) {
- // ConnectionException means no connection established
+ Throwable failure = null;
+ boolean closed = false;
+ try {
+ if (!this.taskDbExecutor.isShutdown()) {
+ this.call(() -> {
+ Throwable workerFailure = null;
+ for (Runnable close : new Runnable[]{() -> {
+ try {
+ this.tx().close();
+ } catch (ConnectionException ignored) {
+ // ConnectionException means no connection established
+ }
+ }, this.graph::closeTx}) {
+ try {
+ close.run();
+ } catch (RuntimeException | Error error) {
+ if (workerFailure == null) {
+ workerFailure = error;
+ } else if (workerFailure != error) {
+ workerFailure.addSuppressed(error);
+ }
+ }
+ }
+ if (workerFailure instanceof Error) {
+ throw (Error) workerFailure;
+ }
+ if (workerFailure != null) {
+ throw (RuntimeException) workerFailure;
+ }
+ });
+ }
+ } catch (RuntimeException | Error error) {
+ failure = error;
+ throw error;
+ } finally {
+ try {
+ closed = this.serverManager().close();
+ } catch (RuntimeException | Error error) {
+ if (failure == null) {
+ throw error;
}
- this.graph.closeTx();
- });
+ if (failure != error) {
+ failure.addSuppressed(error);
+ }
+ }
}
-
- // TODO: serverInfoManager section should be removed in the future.
- return this.serverManager().close();
+ this.closeCompleted = closed;
+ return closed;
}
@Override
diff --git a/hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/task/StandardTaskScheduler.java b/hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/task/StandardTaskScheduler.java
index 9c0d3e9d1..15249067b 100644
--- a/hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/task/StandardTaskScheduler.java
+++ b/hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/task/StandardTaskScheduler.java
@@ -332,17 +332,58 @@ public class StandardTaskScheduler implements TaskScheduler {
@Override
public boolean close() {
- if (!this.taskDbExecutor.isShutdown()) {
- this.call(() -> {
- try {
- this.tx().close();
- } catch (ConnectionException ignored) {
- // ConnectionException means no connection established
+ // Running tasks still need the task DB transaction to persist done().
+ // Retain the scheduler and its owners until a later close attempt.
+ if (this.pendingTasks() != 0) {
+ return false;
+ }
+ Throwable failure = null;
+ boolean closed = false;
+ try {
+ if (!this.taskDbExecutor.isShutdown()) {
+ this.call(() -> {
+ Throwable workerFailure = null;
+ for (Runnable close : new Runnable[]{() -> {
+ try {
+ this.tx().close();
+ } catch (ConnectionException ignored) {
+ // ConnectionException means no connection established
+ }
+ }, this.graph::closeTx}) {
+ try {
+ close.run();
+ } catch (RuntimeException | Error error) {
+ if (workerFailure == null) {
+ workerFailure = error;
+ } else if (workerFailure != error) {
+ workerFailure.addSuppressed(error);
+ }
+ }
+ }
+ if (workerFailure instanceof Error) {
+ throw (Error) workerFailure;
+ }
+ if (workerFailure != null) {
+ throw (RuntimeException) workerFailure;
+ }
+ });
+ }
+ } catch (RuntimeException | Error error) {
+ failure = error;
+ throw error;
+ } finally {
+ try {
+ closed = this.serverManager.close();
+ } catch (RuntimeException | Error error) {
+ if (failure == null) {
+ throw error;
}
- this.graph.closeTx();
- });
+ if (failure != error) {
+ failure.addSuppressed(error);
+ }
+ }
}
- return this.serverManager.close();
+ return closed;
}
@Override
@@ -596,7 +637,7 @@ public class StandardTaskScheduler implements TaskScheduler {
boolean withResult) {
return this.call(() -> {
ConditionQuery query;
- if (this.graph.backendStoreFeatures().supportsTaskAndServerVertex()) {
+ if (this.tx().storeFeatures().supportsTaskAndServerVertex()) {
query = new ConditionQuery(HugeType.TASK);
} else {
query = new ConditionQuery(HugeType.VERTEX);
diff --git a/hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/task/TaskManager.java b/hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/task/TaskManager.java
index e529897a7..0342eb63b 100644
--- a/hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/task/TaskManager.java
+++ b/hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/task/TaskManager.java
@@ -18,8 +18,10 @@
package org.apache.hugegraph.task;
import java.util.Map;
+import java.util.Queue;
import java.util.concurrent.Callable;
import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
@@ -122,31 +124,65 @@ public final class TaskManager {
public void closeScheduler(HugeGraphParams graph) {
TaskScheduler scheduler = this.schedulers.get(graph);
+ Throwable failure = null;
+ boolean drained = scheduler == null;
if (scheduler != null) {
/*
- * Synch close+remove scheduler and iterate scheduler, details:
- * 'closeScheduler' should sync with 'scheduleOrExecuteJob'.
- * Because 'closeScheduler' will be called by 'graph.close()' in
- * main thread and there is gap between 'scheduler.close()'
- * (will close graph tx) and 'this.schedulers.remove(graph)'.
- * In this gap 'scheduleOrExecuteJob' may be run in
- * scheduler-db-thread and 'scheduleOrExecuteJob' will reopen
- * graph tx. As a result, graph tx will mistakenly not be closed
- * after 'graph.close()'.
+ * Keep close+remove exclusive with scheduler iteration: in their gap
+ * a scheduler DB worker could otherwise reopen the graph transaction.
*/
synchronized (scheduler) {
- if (scheduler.close()) {
- this.schedulers.remove(graph);
+ boolean stopped = false;
+ try {
+ stopped = scheduler.close();
+ } catch (RuntimeException | Error error) {
+ failure = error;
+ // Standard closes its server manager in finally; Distributed
+ // sets its dispatch-stop flag before cancellation/drain begins.
+ stopped = true;
+ }
+ // A timeout leaves running jobs registered for the next drain attempt.
+ if (stopped && scheduler.pendingTasks() == 0) {
+ this.schedulers.remove(graph, scheduler);
+ drained = true;
}
}
}
-
- if (!this.taskExecutor.isTerminated()) {
- this.closeTaskTx(graph);
+ if (!drained) {
+ // A running cron/job must finish before owner callbacks are queued
+ // on its executor. In particular, do not rejoin a timed-out cron.
+ if (failure instanceof Error) {
+ throw (Error) failure;
+ }
+ if (failure != null) {
+ throw (RuntimeException) failure;
+ }
+ return;
}
-
- if (!this.distributedSchedulerExecutor.isTerminated()) {
- this.closeDistributedSchedulerTx(graph);
+ for (Runnable close : new Runnable[]{() -> {
+ if (!this.taskExecutor.isTerminated()) {
+ this.closeTaskTx(graph);
+ }
+ }, () -> {
+ if (!this.distributedSchedulerExecutor.isTerminated()) {
+ this.closeDistributedSchedulerTx(graph);
+ }
+ }}) {
+ try {
+ close.run();
+ } catch (RuntimeException | Error error) {
+ if (failure == null) {
+ failure = error;
+ } else if (failure != error) {
+ failure.addSuppressed(error);
+ }
+ }
+ }
+ if (failure instanceof Error) {
+ throw (Error) failure;
+ }
+ if (failure != null) {
+ throw (RuntimeException) failure;
}
}
@@ -158,16 +194,42 @@ public final class TaskManager {
final boolean selfIsTaskWorker = Thread.currentThread().getName()
.startsWith(TASK_WORKER_PREFIX);
final int totalThreads = selfIsTaskWorker ? THREADS - 1 : THREADS;
+ Queue<Throwable> failures = new ConcurrentLinkedQueue<>();
+ Runnable close = () -> {
+ try {
+ graph.closeTx();
+ } catch (RuntimeException | Error error) {
+ // invokeAll() does not inspect worker Futures. Capture failures
+ // here so each worker still completes its execution accounting.
+ failures.add(error);
+ LOG.error("Failed to close task tx in thread '{}'", Thread.currentThread().getName(), error);
+ }
+ };
try {
if (selfIsTaskWorker) {
// Call closeTx directly if myself is task thread(ignore others)
- graph.closeTx();
+ close.run();
} else {
Consumers.executeOncePerThread(this.taskExecutor, totalThreads,
- graph::closeTx, TX_CLOSE_TIMEOUT);
+ close, TX_CLOSE_TIMEOUT);
}
- } catch (Exception e) {
- throw new HugeException("Exception when closing task tx", e);
+ } catch (Exception error) {
+ failures.add(error);
+ if (error instanceof InterruptedException) {
+ Thread.currentThread().interrupt();
+ }
+ }
+ Throwable failure = failures.poll();
+ if (failure != null) {
+ for (Throwable error : failures) {
+ if (error != failure) {
+ failure.addSuppressed(error);
+ }
+ }
+ if (failure instanceof Error) {
+ throw (Error) failure;
+ }
+ throw new HugeException("Exception when closing task tx", failure);
}
}
diff --git a/hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/traversal/algorithm/OltpTraverser.java b/hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/traversal/algorithm/OltpTraverser.java
index 6e9bae539..9d12b0ee9 100644
--- a/hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/traversal/algorithm/OltpTraverser.java
+++ b/hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/traversal/algorithm/OltpTraverser.java
@@ -73,7 +73,7 @@ public abstract class OltpTraverser extends HugeTraverser
@Override
public void close() {
- // pass
+ // The graph's thread-local transaction belongs to the caller.
}
public static void destroy() {
diff --git a/hugegraph-server/hugegraph-hbase/src/main/java/org/apache/hugegraph/backend/store/hbase/HbaseStore.java b/hugegraph-server/hugegraph-hbase/src/main/java/org/apache/hugegraph/backend/store/hbase/HbaseStore.java
index b539fc542..be5995881 100644
--- a/hugegraph-server/hugegraph-hbase/src/main/java/org/apache/hugegraph/backend/store/hbase/HbaseStore.java
+++ b/hugegraph-server/hugegraph-hbase/src/main/java/org/apache/hugegraph/backend/store/hbase/HbaseStore.java
@@ -177,7 +177,7 @@ public abstract class HbaseStore extends AbstractBackendStore<HbaseSessions.Sess
}
@Override
- public void close() {
+ public synchronized void close() {
this.checkOpened();
this.sessions.close();
diff --git a/hugegraph-server/hugegraph-hstore/src/main/java/org/apache/hugegraph/backend/store/hstore/HstoreSessionsImpl.java b/hugegraph-server/hugegraph-hstore/src/main/java/org/apache/hugegraph/backend/store/hstore/HstoreSessionsImpl.java
index 7c3ab3dc6..9fe2dfacd 100755
--- a/hugegraph-server/hugegraph-hstore/src/main/java/org/apache/hugegraph/backend/store/hstore/HstoreSessionsImpl.java
+++ b/hugegraph-server/hugegraph-hstore/src/main/java/org/apache/hugegraph/backend/store/hstore/HstoreSessionsImpl.java
@@ -143,7 +143,7 @@ public class HstoreSessionsImpl extends HstoreSessions {
@Override
protected boolean opened() {
- return this.session != null;
+ return this.session.opened();
}
@Override
diff --git a/hugegraph-server/hugegraph-hstore/src/main/java/org/apache/hugegraph/backend/store/hstore/HstoreStore.java b/hugegraph-server/hugegraph-hstore/src/main/java/org/apache/hugegraph/backend/store/hstore/HstoreStore.java
index 22cc02676..c8d8e3577 100644
--- a/hugegraph-server/hugegraph-hstore/src/main/java/org/apache/hugegraph/backend/store/hstore/HstoreStore.java
+++ b/hugegraph-server/hugegraph-hstore/src/main/java/org/apache/hugegraph/backend/store/hstore/HstoreStore.java
@@ -232,7 +232,7 @@ public abstract class HstoreStore extends AbstractBackendStore<Session> {
}
@Override
- public void close() {
+ public synchronized void close() {
this.checkOpened();
this.sessions.close();
diff --git a/hugegraph-server/hugegraph-rocksdb/src/main/java/org/apache/hugegraph/backend/store/rocksdb/RocksDBStore.java b/hugegraph-server/hugegraph-rocksdb/src/main/java/org/apache/hugegraph/backend/store/rocksdb/RocksDBStore.java
index a160955e4..d28030b36 100644
--- a/hugegraph-server/hugegraph-rocksdb/src/main/java/org/apache/hugegraph/backend/store/rocksdb/RocksDBStore.java
+++ b/hugegraph-server/hugegraph-rocksdb/src/main/java/org/apache/hugegraph/backend/store/rocksdb/RocksDBStore.java
@@ -419,7 +419,7 @@ public abstract class RocksDBStore extends AbstractBackendStore<RocksDBSessions.
}
@Override
- public void close() {
+ public synchronized void close() {
LOG.debug("Store close: {}", this.store);
// A failed restore may close the native owner while sessions still need
// releasing. Native availability is not a prerequisite for lease cleanup.
diff --git a/hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/HugeFactoryTest.java b/hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/HugeFactoryTest.java
new file mode 100644
index 000000000..ba34359d4
--- /dev/null
+++ b/hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/HugeFactoryTest.java
@@ -0,0 +1,400 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hugegraph;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Map;
+import java.util.Queue;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ConcurrentLinkedQueue;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Future;
+import java.util.concurrent.ScheduledFuture;
+import java.util.concurrent.ScheduledThreadPoolExecutor;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.hugegraph.backend.store.BackendStoreProvider;
+import org.apache.hugegraph.backend.tx.GraphTransaction;
+import org.apache.hugegraph.config.HugeConfig;
+import org.apache.hugegraph.dist.RegisterUtil;
+import org.apache.hugegraph.exception.HugeException;
+import org.apache.hugegraph.masterelection.GlobalMasterInfo;
+import org.apache.hugegraph.id.Id;
+import org.apache.hugegraph.id.IdGenerator;
+import org.apache.hugegraph.task.DistributedTaskScheduler;
+import org.apache.hugegraph.task.HugeTask;
+import org.apache.hugegraph.task.TaskAndResultSchedulerTest;
+import org.apache.hugegraph.task.TaskStatus;
+import org.apache.hugegraph.task.TaskManager;
+import org.apache.hugegraph.task.TaskScheduler;
+import org.apache.hugegraph.testutil.Assert;
+import org.apache.hugegraph.testutil.Whitebox;
+import org.apache.hugegraph.unit.FakeObjects;
+import org.apache.hugegraph.util.Consumers;
+import org.junit.Test;
+import org.mockito.AdditionalAnswers;
+import org.mockito.Mockito;
+
+public class HugeFactoryTest {
+
+ @Test
+ public void testCleanupContinuesAfterGraphFailure() {
+ StandardHugeGraph failed = Mockito.mock(StandardHugeGraph.class);
+ StandardHugeGraph cleaned = Mockito.mock(StandardHugeGraph.class);
+ Mockito.doThrow(new HugeException("test"))
+ .when(failed).closeCurrentThreadTransaction();
+
+ Assert.assertThrows(HugeException.class, () -> {
+ HugeFactory.closeCurrentThreadTransactions(
+ Arrays.asList(failed, cleaned));
+ });
+
+ Mockito.verify(failed).closeCurrentThreadTransaction();
+ Mockito.verify(cleaned).closeCurrentThreadTransaction();
+ }
+
+ @Test
+ public void testGraphCloseContinuesAfterReleasedWorkerOwnerFails() throws Exception {
+ this.checkGraphCloseAfterOwnerFailure(false);
+ }
+
+ @Test
+ public void testGraphCloseContinuesAfterWorkerOwnerFailsBeforeRelease() throws Exception {
+ this.checkGraphCloseAfterOwnerFailure(true);
+ }
+
+ private void checkGraphCloseAfterOwnerFailure(boolean beforeRelease) throws Exception {
+ RegisterUtil.registerBackends();
+ HugeConfig config = FakeObjects.newConfig();
+ config.setProperty("backend", "memory");
+ config.setProperty("serializer", "text");
+ config.setProperty("store", "graph_close_owner_failure_" + (beforeRelease ? "before" : "after"));
+ HugeGraph graph = HugeFactory.open(config);
+ TaskScheduler scheduler = graph.taskScheduler();
+ AtomicReference<GraphTransaction> workerOwner = new AtomicReference<>();
+ try {
+ graph.initBackend();
+ graph.serverStarted(GlobalMasterInfo.master("close-owner-test"));
+ HugeGraphParams params = Whitebox.getInternalState(graph, "params");
+ TaskManager manager = TaskManager.instance();
+ ExecutorService workers = Whitebox.getInternalState(manager, "taskExecutor");
+ ThreadLocal<?> owners = Whitebox.getInternalState(graph.tx(), "transactions");
+ RuntimeException first = new IllegalStateException("released task DB owner failed");
+ RuntimeException second = new IllegalArgumentException("released caller owner failed");
+ AtomicInteger workerClosed = new AtomicInteger();
+ AtomicInteger callerClosed = new AtomicInteger();
+ AtomicInteger otherWorkerClosed = new AtomicInteger();
+ BackendStoreProvider provider = Mockito.spy(graph.storeProvider());
+ Whitebox.setInternalState(graph, "storeProvider", provider);
+
+ scheduler.call(() -> {
+ workerOwner.set(failOwnerClose(graph, params, first, workerClosed, beforeRelease));
+ return null;
+ });
+ workers.submit(() -> {
+ GraphTransaction realOwner = params.graphTransaction();
+ GraphTransaction owner = Mockito.mock(GraphTransaction.class,
+ AdditionalAnswers.delegatesTo(realOwner));
+ Mockito.doAnswer(invocation -> {
+ realOwner.close();
+ otherWorkerClosed.incrementAndGet();
+ return null;
+ }).when(owner).close();
+ Whitebox.setInternalState(owners.get(), "graphTx", owner);
+ }).get(10L, TimeUnit.SECONDS);
+ failOwnerClose(graph, params, second, callerClosed, false);
+
+ Throwable failure = Assert.assertThrows(HugeException.class, graph::close);
+ Assert.assertSame(first, HugeException.rootCause(failure));
+ Assert.assertTrue(Arrays.asList(failure.getSuppressed()).contains(second));
+ Assert.assertTrue(graph.closed());
+ Assert.assertNull(manager.getScheduler(params));
+ Assert.assertTrue((boolean) Whitebox.getInternalState(scheduler.serverManager(), "closed"));
+ Assert.assertNull(scheduler.call(owners::get));
+ Assert.assertNull(owners.get());
+ Assert.assertEquals(1, workerClosed.get());
+ Assert.assertEquals(1, callerClosed.get());
+ Assert.assertEquals(1, otherWorkerClosed.get());
+ Mockito.verify(provider).close();
+ } finally {
+ try {
+ // Only the memory fixture owns this retained failed transaction.
+ // Logical graph closure deliberately does not claim it was released.
+ GraphTransaction retained = workerOwner.get();
+ if (beforeRelease && retained != null) {
+ scheduler.call(() -> {
+ retained.close();
+ return null;
+ });
+ }
+ if (!graph.closed()) {
+ graph.close();
+ }
+ } finally {
+ HugeFactory.remove(graph);
+ }
+ }
+ }
+
+ private static GraphTransaction failOwnerClose(HugeGraph graph, HugeGraphParams params,
+ RuntimeException failure, AtomicInteger closed,
+ boolean beforeRelease) {
+ GraphTransaction realOwner = params.graphTransaction();
+ GraphTransaction owner = Mockito.mock(GraphTransaction.class,
+ AdditionalAnswers.delegatesTo(realOwner));
+ Mockito.doAnswer(invocation -> {
+ if (!beforeRelease) {
+ realOwner.close();
+ }
+ closed.incrementAndGet();
+ throw failure;
+ }).when(owner).close();
+ ThreadLocal<?> owners = Whitebox.getInternalState(graph.tx(), "transactions");
+ Whitebox.setInternalState(owners.get(), "graphTx", owner);
+ return realOwner;
+ }
+
+ @Test
+ public void testGraphCloseReportsEveryTaskWorkerOwnerFailure() throws Exception {
+ RegisterUtil.registerBackends();
+ HugeConfig config = FakeObjects.newConfig();
+ config.setProperty("backend", "memory");
+ config.setProperty("serializer", "text");
+ config.setProperty("store", "graph_close_task_worker_failures");
+ HugeGraph graph = HugeFactory.open(config);
+ CountDownLatch ready = new CountDownLatch(4);
+ CountDownLatch release = new CountDownLatch(1);
+ List<Future<?>> initialized = new ArrayList<>();
+ try {
+ graph.initBackend();
+ graph.serverStarted(GlobalMasterInfo.master("close-task-worker-test"));
+ HugeGraphParams params = Whitebox.getInternalState(graph, "params");
+ TaskManager manager = TaskManager.instance();
+ ExecutorService workers = Whitebox.getInternalState(manager, "taskExecutor");
+ ThreadLocal<?> owners = Whitebox.getInternalState(graph.tx(), "transactions");
+ Queue<RuntimeException> failures = new ConcurrentLinkedQueue<>();
+ AtomicInteger closed = new AtomicInteger();
+ for (int i = 0; i < 4; i++) {
+ initialized.add(workers.submit(() -> {
+ try {
+ RuntimeException failure = new IllegalStateException(
+ "Task owner failed: " + Thread.currentThread().getName());
+ failures.add(failure);
+ failOwnerClose(graph, params, failure, closed, false);
+ } finally {
+ ready.countDown();
+ }
+ Assert.assertTrue(release.await(10L, TimeUnit.SECONDS));
+ return null;
+ }));
+ }
+ Assert.assertTrue(ready.await(10L, TimeUnit.SECONDS));
+ release.countDown();
+ for (Future<?> future : initialized) {
+ future.get(10L, TimeUnit.SECONDS);
+ }
+
+ Throwable failure = Assert.assertThrows(HugeException.class, graph::close);
+ Throwable first = HugeException.rootCause(failure);
+ Assert.assertTrue(failures.contains(first));
+ Assert.assertEquals(3, first.getSuppressed().length);
+ for (RuntimeException workerFailure : failures) {
+ Assert.assertTrue(workerFailure == first ||
+ Arrays.asList(first.getSuppressed()).contains(workerFailure));
+ }
+ Assert.assertEquals(4, closed.get());
+ Assert.assertTrue(graph.closed());
+ Assert.assertNull(manager.getScheduler(params));
+ AtomicInteger cleared = new AtomicInteger();
+ Consumers.executeOncePerThread(workers, 4, () -> {
+ if (owners.get() == null) {
+ cleared.incrementAndGet();
+ }
+ }, 10L);
+ Assert.assertEquals(4, cleared.get());
+ } finally {
+ release.countDown();
+ try {
+ if (!graph.closed()) {
+ graph.close();
+ }
+ } finally {
+ HugeFactory.remove(graph);
+ }
+ }
+ }
+
+ @Test
+ public void testGraphClosePreservesRollbackFailureWhenOwnerCloseFails() throws Exception {
+ RegisterUtil.registerBackends();
+ HugeConfig config = FakeObjects.newConfig();
+ config.setProperty("backend", "memory");
+ config.setProperty("serializer", "text");
+ config.setProperty("store", "graph_close_rollback_and_owner_failure");
+ HugeGraph graph = HugeFactory.open(config);
+ try {
+ graph.initBackend();
+ graph.serverStarted(GlobalMasterInfo.master("close-rollback-test"));
+ HugeGraphParams params = Whitebox.getInternalState(graph, "params");
+ graph.tx().open();
+ GraphTransaction realOwner = params.graphTransaction();
+ GraphTransaction owner = Mockito.mock(GraphTransaction.class,
+ AdditionalAnswers.delegatesTo(realOwner));
+ RuntimeException first = new IllegalStateException("Rollback failed");
+ RuntimeException second = new IllegalArgumentException("Owner close failed");
+ Mockito.doThrow(first).when(owner).rollback();
+ Mockito.doAnswer(invocation -> {
+ realOwner.close();
+ throw second;
+ }).when(owner).close();
+ ThreadLocal<?> owners = Whitebox.getInternalState(graph.tx(), "transactions");
+ Whitebox.setInternalState(owners.get(), "graphTx", owner);
+
+ Throwable failure = Assert.assertThrows(IllegalStateException.class, graph::close);
+ Assert.assertSame(first, failure);
+ Assert.assertTrue(Arrays.asList(first.getSuppressed()).contains(second));
+ Assert.assertTrue(graph.closed());
+ Assert.assertNull(owners.get());
+ Assert.assertNull(TaskManager.instance().getScheduler(params));
+ Mockito.verify(owner).rollback();
+ Mockito.verify(owner).close();
+ } finally {
+ try {
+ if (!graph.closed()) {
+ graph.close();
+ }
+ } finally {
+ HugeFactory.remove(graph);
+ }
+ }
+ }
+
+ @Test
+ public void testLocalCloseKeepsPendingTaskPersistenceUsable() throws Exception {
+ RegisterUtil.registerBackends();
+ HugeConfig config = FakeObjects.newConfig();
+ config.setProperty("backend", "memory");
+ config.setProperty("serializer", "text");
+ config.setProperty("store", "graph_close_pending_local_task");
+ HugeGraph graph = HugeFactory.open(config);
+ Map<Id, HugeTask<?>> pending = null;
+ HugeTask<Object> task = new HugeTask<>(IdGenerator.of(9999998L), null,
+ new TaskAndResultSchedulerTest.EmptyCallable());
+ task.type("test");
+ task.name("pending-task-close");
+ try {
+ graph.initBackend();
+ graph.serverStarted(GlobalMasterInfo.master("pending-task-close-test"));
+ TaskScheduler scheduler = graph.taskScheduler();
+ pending = Whitebox.getInternalState(scheduler, "tasks");
+ task.overwriteStatus(TaskStatus.RUNNING);
+ scheduler.save(task);
+ pending.put(task.id(), task);
+
+ Assert.assertThrows(IllegalStateException.class, graph::close);
+ Assert.assertFalse(graph.closed());
+ HugeGraphParams params = Whitebox.getInternalState(graph, "params");
+ Assert.assertSame(scheduler, TaskManager.instance().getScheduler(params));
+ Assert.assertFalse((boolean) Whitebox.getInternalState(scheduler.serverManager(), "closed"));
+
+ // This is the same persistence path a running job uses from done().
+ task.overwriteStatus(TaskStatus.SUCCESS);
+ scheduler.save(task);
+ Assert.assertEquals(TaskStatus.SUCCESS, scheduler.task(task.id()).status());
+ pending.remove(task.id());
+ graph.close();
+ Assert.assertTrue(graph.closed());
+ Assert.assertNull(TaskManager.instance().getScheduler(params));
+ } finally {
+ if (pending != null) {
+ pending.remove(task.id());
+ }
+ try {
+ if (!graph.closed()) {
+ graph.close();
+ }
+ } finally {
+ HugeFactory.remove(graph);
+ }
+ }
+ }
+
+ @Test
+ public void testIncompleteSchedulerCloseDoesNotQueueOwnerCleanup() {
+ HugeGraphParams params = Mockito.mock(HugeGraphParams.class);
+ TaskScheduler scheduler = Mockito.mock(TaskScheduler.class);
+ TaskManager manager = TaskManager.instance();
+ Map<HugeGraphParams, TaskScheduler> schedulers = Whitebox.getInternalState(manager, "schedulers");
+ schedulers.put(params, scheduler);
+ try {
+ Mockito.when(scheduler.close()).thenReturn(false);
+ manager.closeScheduler(params);
+ Assert.assertSame(scheduler, manager.getScheduler(params));
+ Mockito.verify(params, Mockito.never()).closeTx();
+
+ Mockito.when(scheduler.close()).thenReturn(true);
+ manager.closeScheduler(params);
+ Assert.assertNull(manager.getScheduler(params));
+ Mockito.verify(params, Mockito.atLeastOnce()).closeTx();
+ } finally {
+ schedulers.remove(params);
+ }
+ }
+
+ @Test
+ public void testDistributedCloseRetriesDrainAfterTimeout() throws Exception {
+ HugeGraphParams params = Mockito.mock(HugeGraphParams.class);
+ Mockito.when(params.graph()).thenReturn(Mockito.mock(HugeGraph.class));
+ Mockito.when(params.configuration()).thenReturn(FakeObjects.newConfig());
+ ScheduledThreadPoolExecutor cron = Mockito.mock(ScheduledThreadPoolExecutor.class);
+ Mockito.doReturn(Mockito.mock(ScheduledFuture.class)).when(cron).scheduleWithFixedDelay(
+ Mockito.any(Runnable.class), Mockito.anyLong(), Mockito.anyLong(), Mockito.eq(TimeUnit.SECONDS));
+ Mockito.doReturn(CompletableFuture.completedFuture(null)).when(cron).submit(Mockito.any(Runnable.class));
+ ExecutorService executor = Mockito.mock(ExecutorService.class);
+ Mockito.when(executor.isShutdown()).thenReturn(true);
+ DistributedTaskScheduler scheduler = Mockito.spy(new DistributedTaskScheduler(
+ params, cron, executor, executor, executor, executor, executor));
+ AtomicBoolean active = new AtomicBoolean(true);
+ AtomicInteger drainAttempts = new AtomicInteger();
+ Mockito.doAnswer(invocation -> {
+ drainAttempts.incrementAndGet();
+ if (active.get()) {
+ throw new TimeoutException("Task still running");
+ }
+ return null;
+ }).when(scheduler).waitUntilAllTasksCompleted(Mockito.anyLong());
+
+ Assert.assertFalse(scheduler.close());
+ Assert.assertFalse(scheduler.close());
+ Assert.assertEquals(2, drainAttempts.get());
+ Assert.assertFalse((boolean) Whitebox.getInternalState(scheduler.serverManager(), "closed"));
+ active.set(false);
+ Assert.assertTrue(scheduler.close());
+ Assert.assertEquals(3, drainAttempts.get());
+ Assert.assertTrue((boolean) Whitebox.getInternalState(scheduler.serverManager(), "closed"));
+ Assert.assertTrue(scheduler.close());
+ Assert.assertEquals(3, drainAttempts.get());
+ }
+}
diff --git a/hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/auth/ContextTaskTest.java b/hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/auth/ContextTaskTest.java
new file mode 100644
index 000000000..a5ef4ad8d
--- /dev/null
+++ b/hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/auth/ContextTaskTest.java
@@ -0,0 +1,85 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hugegraph.auth;
+
+import java.util.concurrent.atomic.AtomicBoolean;
+
+import org.apache.hugegraph.auth.HugeGraphAuthProxy.Context;
+import org.apache.hugegraph.auth.HugeGraphAuthProxy.ContextTask;
+import org.apache.hugegraph.testutil.Assert;
+import org.junit.After;
+import org.junit.Test;
+
+public class ContextTaskTest {
+
+ @After
+ public void teardown() {
+ HugeGraphAuthProxy.resetContext();
+ }
+
+ @Test
+ public void testCleanupRunsBeforeContextReset() {
+ Context context = Context.admin();
+ AtomicBoolean runnerCalled = new AtomicBoolean(false);
+ AtomicBoolean cleanupCalled = new AtomicBoolean(false);
+ HugeGraphAuthProxy.setContext(context);
+
+ ContextTask task = new ContextTask(() -> {
+ Assert.assertSame(context, HugeGraphAuthProxy.getContext());
+ runnerCalled.set(true);
+ }, () -> {
+ Assert.assertSame(context, HugeGraphAuthProxy.getContext());
+ cleanupCalled.set(true);
+ });
+ HugeGraphAuthProxy.resetContext();
+
+ task.run();
+
+ Assert.assertTrue(runnerCalled.get());
+ Assert.assertTrue(cleanupCalled.get());
+ Assert.assertNull(HugeGraphAuthProxy.getContext());
+ }
+
+ @Test
+ public void testCleanupRunsWhenTaskFails() {
+ AtomicBoolean cleanupCalled = new AtomicBoolean(false);
+ ContextTask task = new ContextTask(() -> {
+ throw new IllegalStateException("test");
+ }, () -> cleanupCalled.set(true));
+
+ Assert.assertThrows(IllegalStateException.class, task::run);
+ Assert.assertTrue(cleanupCalled.get());
+ Assert.assertNull(HugeGraphAuthProxy.getContext());
+ }
+
+ @Test
+ public void testContextResetRunsWhenCleanupFails() {
+ Context context = Context.admin();
+ HugeGraphAuthProxy.setContext(context);
+ ContextTask task = new ContextTask(() -> {
+ Assert.assertSame(context, HugeGraphAuthProxy.getContext());
+ }, () -> {
+ throw new IllegalStateException("test");
+ });
+ HugeGraphAuthProxy.resetContext();
+
+ task.run();
+
+ Assert.assertNull(HugeGraphAuthProxy.getContext());
+ }
+}
diff --git a/hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/auth/TransactionLifecycleTest.java b/hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/auth/TransactionLifecycleTest.java
new file mode 100644
index 000000000..bf4864b62
--- /dev/null
+++ b/hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/auth/TransactionLifecycleTest.java
@@ -0,0 +1,148 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hugegraph.auth;
+
+import java.lang.reflect.Constructor;
+
+import org.apache.hugegraph.HugeFactory;
+import org.apache.hugegraph.HugeGraphParams;
+import org.apache.hugegraph.backend.store.BackendStore;
+import org.apache.hugegraph.backend.tx.GraphTransaction;
+import org.apache.hugegraph.backend.tx.ISchemaTransaction;
+import org.apache.hugegraph.exception.HugeException;
+import org.apache.hugegraph.core.BaseCoreTest;
+import org.apache.hugegraph.id.IdGenerator;
+import org.apache.hugegraph.testutil.Assert;
+import org.apache.hugegraph.testutil.Whitebox;
+import org.apache.hugegraph.traversal.algorithm.KneighborTraverser;
+import org.apache.hugegraph.type.define.Directions;
+import org.apache.tinkerpop.gremlin.structure.T;
+import org.apache.tinkerpop.gremlin.structure.Transaction;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Test;
+import org.mockito.Mockito;
+
+public class TransactionLifecycleTest extends BaseCoreTest {
+
+ @Before
+ public void initSchema() {
+ graph().schema().vertexLabel("lifecycle").useCustomizeNumberId().create();
+ HugeFactory.closeCurrentThreadTransactions();
+ }
+
+ @After
+ public void cleanup() {
+ try {
+ HugeFactory.closeCurrentThreadTransactions();
+ } finally {
+ HugeGraphAuthProxy.resetContext();
+ }
+ }
+
+ @Test
+ public void testTraverserClosePreservesCallerWrites() {
+ graph().addVertex(T.label, "lifecycle", T.id, 1L);
+ graph().tx().commit();
+
+ try (KneighborTraverser traverser = new KneighborTraverser(graph())) {
+ Assert.assertTrue(traverser.kneighbor(IdGenerator.of(1L), Directions.BOTH,
+ null, 1, 100L, 100L).isEmpty());
+ graph().addVertex(T.label, "lifecycle", T.id, 2L);
+ }
+
+ Assert.assertTrue(graph().tx().isOpen());
+ graph().tx().commit();
+ HugeFactory.closeCurrentThreadTransactions();
+ Assert.assertEquals(2L, graph().traversal().V().count().next().longValue());
+ }
+
+ @Test
+ public void testFailedCleanupReleasesOwnersBeforeWorkerReuse() throws Exception {
+ this.checkFailedCleanupReleasesOwnersBeforeWorkerReuse(false);
+ }
+
+ @Test
+ public void testFailedRollbackPreservesCauseAndReleasesOwnersBeforeWorkerReuse() throws Exception {
+ this.checkFailedCleanupReleasesOwnersBeforeWorkerReuse(true);
+ }
+
+ private void checkFailedCleanupReleasesOwnersBeforeWorkerReuse(boolean failRollback) throws Exception {
+ Class<?> holderType = Class.forName("org.apache.hugegraph.StandardHugeGraph$Txs");
+ Class<?> systemType = Class.forName("org.apache.hugegraph.StandardHugeGraph$SysTransaction");
+ GraphTransaction graphTx = Mockito.mock(GraphTransaction.class);
+ BackendStore systemStore = Mockito.mock(BackendStore.class);
+ Constructor<?> systemConstructor = systemType.getDeclaredConstructor(
+ HugeGraphParams.class, BackendStore.class);
+ systemConstructor.setAccessible(true);
+ GraphTransaction systemTx =
+ (GraphTransaction) systemConstructor.newInstance(this.params(), systemStore);
+ ISchemaTransaction schemaTx = Mockito.mock(ISchemaTransaction.class);
+ RuntimeException first = new IllegalStateException("graph close failed");
+ RuntimeException second = new IllegalArgumentException("system close failed");
+ RuntimeException rollbackFailure = new IllegalStateException("rollback failed");
+ if (failRollback) {
+ Mockito.doThrow(rollbackFailure).when(graphTx).rollback();
+ }
+ Mockito.doThrow(first).when(graphTx).close();
+ Mockito.doThrow(second).when(systemStore).close();
+ Constructor<?> constructor = holderType.getDeclaredConstructor(
+ ISchemaTransaction.class, systemType, GraphTransaction.class);
+ constructor.setAccessible(true);
+ Object failedOwners = constructor.newInstance(schemaTx, systemTx, graphTx);
+ ThreadLocal<Object> owners = Whitebox.getInternalState(graph().tx(), "transactions");
+ Assert.assertNull(owners.get());
+ owners.set(failedOwners);
+
+ // Dirty request behavior must also be reset when backend close throws.
+ graph().tx().onReadWrite(Transaction.READ_WRITE_BEHAVIOR.MANUAL);
+ graph().tx().onClose(Transaction.CLOSE_BEHAVIOR.COMMIT);
+ if (failRollback) {
+ graph().tx().open();
+ }
+ Throwable failure = Assert.assertThrows(
+ HugeException.class, HugeFactory::closeCurrentThreadTransactions);
+ if (failRollback) {
+ Assert.assertSame(rollbackFailure, failure.getCause());
+ Assert.assertEquals(1, rollbackFailure.getSuppressed().length);
+ Assert.assertSame(first, rollbackFailure.getSuppressed()[0]);
+ } else {
+ Assert.assertSame(first, failure.getCause());
+ }
+ Assert.assertEquals(1, first.getSuppressed().length);
+ Assert.assertSame(second, first.getSuppressed()[0]);
+ Mockito.verify(graphTx).close();
+ // SysTransaction owns both its primary transaction and index transaction.
+ Mockito.verify(systemStore, Mockito.times(2)).close();
+ Mockito.verify(schemaTx).close();
+ Assert.assertNull(owners.get());
+ Assert.assertFalse(graph().tx().isOpen());
+
+ // The same thread must acquire fresh owners and retain default AUTO/ROLLBACK.
+ graph().addVertex(T.label, "lifecycle", T.id, 1L);
+ Assert.assertNotNull(owners.get());
+ Assert.assertNotSame(failedOwners, owners.get());
+ graph().tx().close();
+ Assert.assertEquals(0L, graph().traversal().V().count().next().longValue());
+ graph().addVertex(T.label, "lifecycle", T.id, 2L);
+ graph().tx().commit();
+ HugeFactory.closeCurrentThreadTransactions();
+ Assert.assertNull(owners.get());
+ Assert.assertEquals(1L, graph().traversal().V().count().next().longValue());
+ }
+}
diff --git a/hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/core/CoreTestSuite.java b/hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/core/CoreTestSuite.java
index 9b45f40a6..3d6d3149f 100644
--- a/hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/core/CoreTestSuite.java
+++ b/hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/core/CoreTestSuite.java
@@ -19,11 +19,13 @@ package org.apache.hugegraph.core;
import org.apache.hugegraph.HugeGraph;
import org.apache.hugegraph.auth.BackendLeaseCleanupTest;
+import org.apache.hugegraph.auth.TransactionLifecycleTest;
import org.apache.hugegraph.constant.ServiceConstant;
import org.apache.hugegraph.dist.RegisterUtil;
import org.apache.hugegraph.masterelection.GlobalMasterInfo;
import org.apache.hugegraph.meta.MetaManager;
import org.apache.hugegraph.meta.PdMetaDriver;
+import org.apache.hugegraph.task.StandardTaskSchedulerTxTest;
import org.apache.hugegraph.task.TaskAndResultSchedulerTest;
import org.apache.hugegraph.testutil.Utils;
import org.apache.hugegraph.util.Log;
@@ -52,7 +54,9 @@ import org.slf4j.Logger;
RestoreCoreTest.class,
TaskCoreTest.class,
TaskAndResultSchedulerTest.class,
+ StandardTaskSchedulerTxTest.class,
AuthTest.class,
+ TransactionLifecycleTest.class,
MultiGraphsTest.class,
RamTableTest.class
})
diff --git a/hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/task/StandardTaskSchedulerTxTest.java b/hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/task/StandardTaskSchedulerTxTest.java
new file mode 100644
index 000000000..6e741e28a
--- /dev/null
+++ b/hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/task/StandardTaskSchedulerTxTest.java
@@ -0,0 +1,65 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hugegraph.task;
+
+import java.util.Collections;
+
+import org.apache.hugegraph.HugeGraph;
+import org.apache.hugegraph.id.IdGenerator;
+import org.apache.hugegraph.core.BaseCoreTest;
+import org.junit.Assert;
+import org.junit.Test;
+
+public class StandardTaskSchedulerTxTest extends BaseCoreTest {
+
+ @Test
+ public void testTaskQueryDoesNotOpenUpperGraphTransaction() {
+ HugeGraph graph = this.graph();
+ TaskScheduler scheduler = graph.taskScheduler();
+ HugeTask<Object> task = new HugeTask<>(IdGenerator.of(9999999L), null,
+ new TaskAndResultSchedulerTest.EmptyCallable());
+ task.type("test");
+ task.name("transaction-lifecycle-query");
+ boolean saved = false;
+ try {
+ // The task worker must also be quiescent after graph startup.
+ Assert.assertFalse(scheduler.call(() -> graph.tx().isOpen()));
+
+ scheduler.save(task);
+ saved = true;
+ scheduler.tasks(TaskStatus.NEW, 1L, null).hasNext();
+ Assert.assertFalse(scheduler.call(() -> graph.tx().isOpen()));
+
+ Assert.assertTrue(scheduler.tasks(Collections.singletonList(task.id())).hasNext());
+ Assert.assertFalse(scheduler.call(() -> graph.tx().isOpen()));
+ } finally {
+ try {
+ if (saved) {
+ scheduler.delete(task.id(), true);
+ }
+ } finally {
+ scheduler.call(() -> {
+ if (graph.tx().isOpen()) {
+ graph.tx().rollback();
+ }
+ return null;
+ });
+ }
+ }
+ }
+}
diff --git a/hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/unit/UnitTestSuite.java b/hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/unit/UnitTestSuite.java
index 3346f8ad6..dd456d0e5 100644
--- a/hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/unit/UnitTestSuite.java
+++ b/hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/unit/UnitTestSuite.java
@@ -17,11 +17,13 @@
package org.apache.hugegraph.unit;
+import org.apache.hugegraph.HugeFactoryTest;
import org.apache.hugegraph.api.auth.GraphSpaceAuthPayloadTest;
import org.apache.hugegraph.api.auth.GraphSpaceGroupAPITest;
import org.apache.hugegraph.api.cypher.CypherClientTest;
import org.apache.hugegraph.auth.ContextGremlinServerHttpTest;
import org.apache.hugegraph.auth.HttpGremlinRequestHandlerTest;
+import org.apache.hugegraph.auth.ContextTaskTest;
import org.apache.hugegraph.auth.StandardAuthManagerV2Test;
import org.apache.hugegraph.auth.WsAndHttpBasicAuthHandlerTest;
import org.apache.hugegraph.backend.page.QueryListTest;
@@ -54,6 +56,7 @@ import org.apache.hugegraph.unit.config.GremlinConfigCompatibilityTest;
import org.apache.hugegraph.unit.config.GraphBinarySchemaCompatibilityTest;
import org.apache.hugegraph.unit.config.K8sResourceQuotaYamlTest;
import org.apache.hugegraph.unit.core.AnalyzerTest;
+import org.apache.hugegraph.unit.core.BackendSessionPoolTest;
import org.apache.hugegraph.unit.core.BackendMutationTest;
import org.apache.hugegraph.unit.core.BackendProviderFactoryTest;
import org.apache.hugegraph.unit.core.BackendStoreInfoTest;
@@ -142,6 +145,8 @@ import org.junit.runners.Suite;
GraphSpaceGroupAPITest.class,
GraphSpaceAuthPayloadTest.class,
StandardAuthManagerV2Test.class,
+ ContextTaskTest.class,
+ HugeFactoryTest.class,
AuthMetaManagerTest.class,
/* api space */
@@ -179,6 +184,7 @@ import org.junit.runners.Suite;
RowLockTest.class,
AnalyzerTest.class,
BackendMutationTest.class,
+ BackendSessionPoolTest.class,
BackendProviderFactoryTest.class,
ConditionTest.class,
GroovyScriptEngineCompatibilityTest.class,
diff --git a/hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/unit/core/BackendSessionPoolTest.java b/hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/unit/core/BackendSessionPoolTest.java
new file mode 100644
index 000000000..80970b8e5
--- /dev/null
+++ b/hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/unit/core/BackendSessionPoolTest.java
@@ -0,0 +1,329 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hugegraph.unit.core;
+
+import java.lang.management.ManagementFactory;
+import java.lang.management.ThreadInfo;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.FutureTask;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.concurrent.locks.LockSupport;
+
+import org.apache.hugegraph.backend.store.BackendSession;
+import org.apache.hugegraph.backend.store.BackendSessionPool;
+import org.apache.hugegraph.backend.store.BackendStoreProvider;
+import org.apache.hugegraph.backend.store.rocksdb.RocksDBSessions;
+import org.apache.hugegraph.backend.store.rocksdb.RocksDBStore.RocksDBGraphStore;
+import org.apache.hugegraph.testutil.Assert;
+import org.apache.hugegraph.testutil.Whitebox;
+import org.apache.hugegraph.unit.FakeObjects;
+import org.junit.Test;
+import org.mockito.Mockito;
+
+public class BackendSessionPoolTest {
+
+ private static final long TIMEOUT_SECONDS = 10L;
+ private static final long CLOSE_HOLD_SECONDS = 30L;
+
+ @Test
+ public void testConcurrentAcquireDoesNotLeaveSessionOnClosedBackend()
+ throws Exception {
+ TestSessionPool pool = new TestSessionPool();
+ pool.open();
+
+ AtomicReference<Throwable> closeFailure = new AtomicReference<>();
+ CountDownLatch acquireAttempted = new CountDownLatch(1);
+ FutureTask<BackendSession> acquisition =
+ new FutureTask<>(() -> {
+ acquireAttempted.countDown();
+ return pool.getOrNewSession();
+ });
+
+ Thread closer = new Thread(() -> {
+ try {
+ pool.getOrNewSession();
+ pool.close();
+ } catch (Throwable e) {
+ closeFailure.set(e);
+ }
+ }, "backend-session-closer");
+ Thread acquirer = new Thread(acquisition, "backend-session-acquirer");
+
+ closer.start();
+ try {
+ Assert.assertTrue("last-session close did not enter doClose",
+ pool.closeEntered.await(TIMEOUT_SECONDS,
+ TimeUnit.SECONDS));
+
+ acquirer.start();
+ Assert.assertTrue("acquirer did not start",
+ acquireAttempted.await(TIMEOUT_SECONDS, TimeUnit.SECONDS));
+ boolean acquiredWhileClosePaused =
+ awaitBorrowerInCloseWindow(pool, closer, acquirer);
+ BackendSession activeSession = acquiredWhileClosePaused ?
+ acquisition.get(TIMEOUT_SECONDS,
+ TimeUnit.SECONDS) : null;
+ pool.allowClose.countDown();
+
+ closer.join(TimeUnit.SECONDS.toMillis(TIMEOUT_SECONDS));
+ acquirer.join(TimeUnit.SECONDS.toMillis(TIMEOUT_SECONDS));
+ Assert.assertFalse("closer did not finish", closer.isAlive());
+ Assert.assertFalse("acquirer did not finish", acquirer.isAlive());
+ Assert.assertNull(closeFailure.get());
+ if (acquiredWhileClosePaused) {
+ Assert.assertNotNull(activeSession);
+ Assert.assertTrue("new session should still be open",
+ activeSession.opened());
+ Assert.assertFalse("pool should still count the new active session",
+ pool.closed());
+
+ Assert.assertTrue("doClose closed the backend while a newly " +
+ "acquired session remained active",
+ pool.opened());
+ } else {
+ try {
+ acquisition.get(TIMEOUT_SECONDS, TimeUnit.SECONDS);
+ Assert.fail("late acquire should be rejected after backend close");
+ } catch (ExecutionException e) {
+ Assert.assertTrue("late acquire must fail because backend is closed",
+ e.getCause() instanceof IllegalStateException);
+ Assert.assertEquals("Backend session pool is closed", e.getCause().getMessage());
+ Assert.assertFalse("backend should be closed before rejecting late acquire",
+ pool.opened());
+ Assert.assertTrue("pool should count no active session after rejection",
+ pool.closed());
+ }
+ }
+ } finally {
+ pool.allowClose.countDown();
+ closer.join(TimeUnit.SECONDS.toMillis(TIMEOUT_SECONDS));
+ if (acquirer.getState() != Thread.State.NEW) {
+ acquirer.join(TimeUnit.SECONDS.toMillis(TIMEOUT_SECONDS));
+ }
+ }
+ }
+
+ @Test
+ public void testAcquireAfterLastSessionCloseIsRejectedByPool() {
+ TestSessionPool pool = new TestSessionPool();
+ pool.open();
+ BackendSession original = pool.getOrNewSession();
+ pool.allowClose.countDown();
+ Assert.assertTrue(pool.close());
+ Assert.assertFalse(pool.opened());
+
+ Throwable failure = Assert.assertThrows(IllegalStateException.class, pool::getOrNewSession);
+ Assert.assertEquals("Backend session pool is closed", failure.getMessage());
+ Assert.assertTrue(pool.closed());
+ Assert.assertSame("rejected acquire must never call newSession", original, pool.session());
+ }
+
+ @Test
+ public void testStoreOpenKeepsExistingPoolAliveUntilBorrowCompletes() throws Exception {
+ RocksDBGraphStore store = new RocksDBGraphStore(
+ Mockito.mock(BackendStoreProvider.class), "db", "test");
+ RocksDBSessions pool = Mockito.mock(RocksDBSessions.class);
+ AtomicInteger leases = new AtomicInteger(1);
+ AtomicBoolean nativeOpened = new AtomicBoolean(true);
+ CountDownLatch borrowing = new CountDownLatch(1);
+ CountDownLatch finishBorrow = new CountDownLatch(1);
+ CountDownLatch closing = new CountDownLatch(1);
+ CountDownLatch closeFinished = new CountDownLatch(1);
+ Mockito.doAnswer(invocation -> leases.get() == 0).when(pool).closed();
+ Mockito.doAnswer(invocation -> {
+ borrowing.countDown();
+ Assert.assertTrue(finishBorrow.await(TIMEOUT_SECONDS, TimeUnit.SECONDS));
+ Assert.assertTrue("last close released the backend before the new store borrow",
+ nativeOpened.get());
+ leases.incrementAndGet();
+ return new TestSession();
+ }).when(pool).useSession();
+ Mockito.doAnswer(invocation -> {
+ boolean last = leases.decrementAndGet() == 0;
+ if (last) {
+ nativeOpened.set(false);
+ }
+ return last;
+ }).when(pool).close();
+ Whitebox.setInternalState(store, "sessions", pool);
+ ConcurrentHashMap<String, RocksDBSessions> databases = new ConcurrentHashMap<>();
+ databases.put("db", pool);
+ Whitebox.setInternalState(store, "dbs", databases);
+
+ AtomicReference<Throwable> failure = new AtomicReference<>();
+ Thread opener = new Thread(() -> {
+ try {
+ store.open(FakeObjects.newConfig());
+ } catch (Throwable error) {
+ failure.compareAndSet(null, error);
+ }
+ }, "store-session-opener");
+ Thread closer = new Thread(() -> {
+ closing.countDown();
+ try {
+ store.close();
+ } catch (Throwable error) {
+ failure.compareAndSet(null, error);
+ } finally {
+ closeFinished.countDown();
+ }
+ }, "store-session-closer");
+ opener.start();
+ try {
+ Assert.assertTrue(borrowing.await(TIMEOUT_SECONDS, TimeUnit.SECONDS));
+ closer.start();
+ Assert.assertTrue(closing.await(TIMEOUT_SECONDS, TimeUnit.SECONDS));
+ long deadline = System.nanoTime() + TimeUnit.SECONDS.toNanos(TIMEOUT_SECONDS);
+ // Wait for close to either reach the held store monitor or finish.
+ while (closeFinished.getCount() != 0L) {
+ ThreadInfo info = ManagementFactory.getThreadMXBean().getThreadInfo(closer.getId());
+ if (info != null && info.getThreadState() == Thread.State.BLOCKED &&
+ info.getLockOwnerId() == opener.getId()) {
+ break;
+ }
+ Assert.assertTrue("close did not reach the store open window", System.nanoTime() < deadline);
+ LockSupport.parkNanos(TimeUnit.MILLISECONDS.toNanos(1L));
+ }
+ } finally {
+ finishBorrow.countDown();
+ opener.join(TimeUnit.SECONDS.toMillis(TIMEOUT_SECONDS));
+ if (closer.getState() != Thread.State.NEW) {
+ closer.join(TimeUnit.SECONDS.toMillis(TIMEOUT_SECONDS));
+ }
+ }
+ Assert.assertFalse(opener.isAlive());
+ Assert.assertFalse(closer.isAlive());
+ Assert.assertNull(failure.get());
+ Assert.assertTrue(nativeOpened.get());
+ Assert.assertEquals(1, leases.get());
+ Mockito.verify(pool).useSession();
+ Mockito.verify(pool).close();
+ }
+
+ private static boolean awaitBorrowerInCloseWindow(TestSessionPool pool,
+ Thread closer,
+ Thread acquirer) {
+ long deadline = System.nanoTime() + TimeUnit.SECONDS.toNanos(TIMEOUT_SECONDS);
+ while (System.nanoTime() < deadline) {
+ if (pool.acquirerEnteredNewSession.getCount() == 0L) {
+ return true;
+ }
+ ThreadInfo info = ManagementFactory.getThreadMXBean()
+ .getThreadInfo(acquirer.getId());
+ if (info != null && info.getThreadState() == Thread.State.BLOCKED &&
+ info.getLockOwnerId() == closer.getId() &&
+ info.getLockInfo() != null &&
+ info.getLockInfo().getClassName().equals(TestSessionPool.class.getName())) {
+ return false;
+ }
+ LockSupport.parkNanos(TimeUnit.MILLISECONDS.toNanos(1L));
+ }
+ throw new AssertionError("acquirer neither entered newSession nor blocked on close");
+ }
+
+ private static final class TestSessionPool extends BackendSessionPool {
+
+ private final AtomicBoolean opened;
+ private final AtomicReference<BackendSession> session;
+ private final CountDownLatch closeEntered;
+ private final CountDownLatch allowClose;
+ private final CountDownLatch acquirerEnteredNewSession;
+
+ private TestSessionPool() {
+ super(FakeObjects.newConfig(), "test");
+ this.opened = new AtomicBoolean();
+ this.session = new AtomicReference<>();
+ this.closeEntered = new CountDownLatch(1);
+ this.allowClose = new CountDownLatch(1);
+ this.acquirerEnteredNewSession = new CountDownLatch(1);
+ }
+
+ @Override
+ public void open() {
+ this.opened.set(true);
+ }
+
+ @Override
+ protected boolean opened() {
+ return this.opened.get();
+ }
+
+ @Override
+ public BackendSession session() {
+ return this.session.get();
+ }
+
+ @Override
+ protected BackendSession newSession() {
+ if (Thread.currentThread().getName().equals("backend-session-acquirer")) {
+ this.acquirerEnteredNewSession.countDown();
+ }
+ BackendSession newSession = new TestSession();
+ this.session.set(newSession);
+ return newSession;
+ }
+
+ @Override
+ protected void doClose() {
+ this.closeEntered.countDown();
+ try {
+ if (!this.allowClose.await(CLOSE_HOLD_SECONDS, TimeUnit.SECONDS)) {
+ throw new AssertionError("timed out waiting to finish doClose");
+ }
+ } catch (InterruptedException e) {
+ Thread.currentThread().interrupt();
+ throw new AssertionError("interrupted while waiting to finish doClose", e);
+ }
+ this.opened.set(false);
+ }
+ }
+
+ private static final class TestSession extends BackendSession.AbstractBackendSession {
+
+ @Override
+ public void open() {
+ this.opened = true;
+ }
+
+ @Override
+ public void close() {
+ this.opened = false;
+ }
+
+ @Override
+ public Object commit() {
+ return null;
+ }
+
+ @Override
+ public void rollback() {
+ // No transaction state in this test session.
+ }
+
+ @Override
+ public boolean hasChanges() {
+ return false;
+ }
+ }
+
+}
diff --git a/hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/unit/core/HstoreSessionsTest.java b/hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/unit/core/HstoreSessionsTest.java
index c7879beb9..5d05476a6 100644
--- a/hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/unit/core/HstoreSessionsTest.java
+++ b/hugegraph-server/hugegraph-test/src/main/java/org/apache/hugegraph/unit/core/HstoreSessionsTest.java
@@ -21,11 +21,48 @@ import java.lang.reflect.Method;
import java.lang.reflect.Modifier;
import org.apache.hugegraph.backend.store.hstore.HstoreSessions;
+import org.apache.hugegraph.backend.store.hstore.HstoreSessionsImpl;
+import org.apache.hugegraph.store.HgStoreSession;
+import org.apache.hugegraph.store.HgStoreClient;
+import org.apache.hugegraph.store.HgSessionProvider;
+import org.apache.hugegraph.testutil.Whitebox;
+import org.apache.hugegraph.unit.FakeObjects;
+import org.mockito.Mockito;
import org.junit.Assert;
import org.junit.Test;
public class HstoreSessionsTest {
+ @Test
+ public void testPoolRejectsBorrowAfterLastHstoreSessionCloses() {
+ Boolean initialized = Whitebox.getInternalState(HstoreSessionsImpl.class, "initializedNode");
+ HgStoreClient previous = Whitebox.getInternalState(HstoreSessionsImpl.class, "hgStoreClient");
+ HgStoreClient client = new HgStoreClient();
+ HgSessionProvider provider = Mockito.mock(HgSessionProvider.class);
+ Mockito.when(provider.createSession(Mockito.anyString(), Mockito.isNull()))
+ .thenReturn(Mockito.mock(HgStoreSession.class));
+ Whitebox.setInternalState(client, "sessionProvider", provider);
+ Whitebox.setInternalState(HstoreSessionsImpl.class, "initializedNode", Boolean.TRUE);
+ Whitebox.setInternalState(HstoreSessionsImpl.class, "hgStoreClient", client);
+ try {
+ HstoreSessionsImpl pool = new HstoreSessionsImpl(FakeObjects.newConfig(), "test", "closed");
+ pool.getOrNewSession();
+ Assert.assertTrue(pool.close());
+ Assert.assertTrue(pool.closed());
+ try {
+ pool.getOrNewSession();
+ Assert.fail("closed HStore pool accepted a new borrower");
+ } catch (IllegalStateException expected) {
+ Assert.assertEquals("Backend session pool is closed", expected.getMessage());
+ }
+ Assert.assertTrue(pool.closed());
+ Mockito.verify(provider, Mockito.times(2)).createSession(Mockito.anyString(), Mockito.isNull());
+ } finally {
+ Whitebox.setInternalState(HstoreSessionsImpl.class, "hgStoreClient", previous);
+ Whitebox.setInternalState(HstoreSessionsImpl.class, "initializedNode", initialized);
+ }
+ }
+
@Test
public void testOrderedScanDoesNotAddAbstractSubclassRequirement()
throws Exception {