You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hugegraph.apache.org by "bitflicker64 (via GitHub)" <gi...@apache.org> on 2026/10/10 20:08:09 UTC

[PR] test: improve coverage across toolchain modules before 1.8.0 [hugegraph-toolchain]

bitflicker64 opened a new pull request, #805:
URL: https://github.com/apache/hugegraph-toolchain/pull/805

   ## Purpose of the PR
   
   More test coverage across the toolchain modules before the 1.8.0 release, as requested for the Java 17 migration. This PR only adds or registers tests. It changes no production code, dependencies, CI workflows or documentation.
   
   Each module is a separate commit, so the PR can be reviewed one commit at a time.
   
   ## Main Changes
   
   **hugegraph-client**
   - Register tests that compiled but never ran: `HugeClientBuilderTest` and `GremlinManagerTest` (unit), `CypherApiTest` and `AdamicAdarAPITest` (API) and `GraphModeTest` (functional). All pass against the locally built Server 1.8; the Server 1.7 baseline is exercised by CI only. `SameNeighborsBatchApiTest` is left to #673.
   - `ServerExceptionTest`: error bodies with the standard keys, the alternative keys, non-JSON bodies, empty bodies and empty causes.
   - `HugeClientFakeServerTest`: the client against a local `HttpServer` that returns 5xx HTML, non-JSON bodies or an unsupported API version, answers slowly or drops the connection. It also checks that requests to a legacy server leave out the `graphspaces/DEFAULT` prefix.
   - `GraphElementTest` and `JsonUtilTest`: property value validation and serialization error paths.
   
   **hugegraph-tools** (all registered in `FuncTestSuite`)
   - `BackupFailureTest`: backup against a fake server. A 5xx on the shard query is retried and then fails with the original `ServerException`; non-JSON shards and schema errors fail the backup; an invalid split size or label/type combination is rejected.
   - `LocalDirectoryTest`: plain and zip streams, append and override, corrupt, empty and missing archives, directory checks.
   - `RetryManagerTest`: retry success, exhaustion and the default of no retries.
   - `JsonGraphTest`: the dump graph, `JsonFormatter` and formatter loading.
   
   **hugegraph-loader** (all registered in the `unit` profile)
   - `DataTypeUtilTest`: conversion of numbers, booleans, UUIDs, text, dates (extra formats, time zones, timestamps, JDBC temporal types) and list/set values, each with invalid inputs.
   - `LineParserTest`: text, CSV (quotes, escaped quotes) and JSON line parsing, missing and extra columns.
   - `LoadMappingTest`: valid and invalid struct files (version, empty or duplicate structs, a missing input type, an unknown source type).
   - `BatchInsertRetryTest`: `BatchInsertTask` against a fake server. A 5xx is retried until it succeeds or the retry limit is reached. `IllegalArgumentException` and the unacceptable messages fail at once.
   
   **hugegraph-spark-connector**
   - `DateUtilsTest` and more `DataTypeUtilsTest` cases: invalid numbers, text conversion, string dates, null values, unsupported list/set cardinality.
   
   **hugegraph-hubble**
   - hubble-be: `EntityUtilTest` covers `@MergeProperty` update semantics. `HubbleUtilTest` covers collection equality, date helpers, digests, the host pattern and the size and duration serializers.
   - hubble-fe: Jest tests for storage and duration formatting, `removeNilKeys` (algorithm request payloads) and `convertStringToJSON`.
   
   **hugegraph-client-go** (`httptest` only, no live server)
   - `hgtransport`: URL prefix, basic auth from config or URL, explicit `Authorization`, user agent, content type, 5xx pass-through, dropped connections, timeouts, logger body handling.
   - `api.Response` helpers, `NewCommonClient` config validation, `/versions` with non-JSON, JSON error and dropped-connection responses.
   - Vertex create, batch create and property update, with and without a graphspace. The package's only test, `TestVertex`, has its whole body commented out, so the vertex package showed 0.0% coverage with the CI test flags; it now shows 85.5%.
   
   ## Verifying these changes
   
   - [ ] Trivial rework / code cleanup without any test coverage. (No Need)
   - [ ] Already covered by existing tests, such as *(please modify tests here)*.
   - [x] Need tests and can be verified as follows:
       - Environment: Toolchain at 039153cd, Java 17.0.20, Maven 3.9.14, Node 18.20.8, Go 1.22.2. The Server 1.8 SDK and server were built from the CI-pinned commit 68855199 with `install-candidate-sdk.sh` into a separate Maven repository. Authentication was on, with the batch limits that CI sets.
       - Each new class was also run on its own twice, and a timeout check was confirmed to fail when its condition is removed.
   
   | Command | Before | After |
   | --- | --- | --- |
   | client `mvn test -Dtest=UnitTestSuite` | 77 passed | 118 passed |
   | client `mvn test -Dtest=ApiTestSuite` | 376 run, 3 skipped | 382 run, 3 skipped, 0 failed |
   | client `mvn test -Dtest=FuncTestSuite` | 94 run, 1 error (`HugeClientHttpsTest`) | 104 run, 6 errors, all in `HugeClientHttpsTest` |
   | tools `mvn test -Dtest=FuncTestSuite -Dtools.test.disposable=true` | 30 passed | 55 passed |
   | loader `mvn test -P unit` | 100 passed | 134 passed |
   | loader `mvn test -P file` (unchanged) | | 99 run, 2 errors, both HTTPS tests |
   | spark-connector `mvn test` | 59 passed | 67 passed |
   | hubble-be `mvn test -P unit-test` | 614 passed | 623 passed |
   | hubble-fe `yarn lint`, `yarn i18n:check`, `yarn test` | 162 suites, 1032 tests | 165 suites, 1079 tests, lint clean |
   | client-go `go test -race -count=1 ./...` | all packages pass | all packages pass |
   
   Notes:
   1. Port 8443 on the test host was held by an unrelated process, so the HTTPS server could not start. `HugeClientHttpsTest` and the loader HTTPS tests fail for that reason in both runs. CI starts its own HTTPS server.
   2. Not run: the loader `hdfs`, `jdbc` and `kafka` profiles (no HDFS, MySQL or Kafka started), the Hubble packaged API/browser test, and the opt-in `HugeGraphIdentityIntegrationTest`, which needs a server without authentication.
   
   ### Production issues found (tests not included)
   
   These tests fail on the current code, so they are not part of this PR:
   
   1. **A malformed JSON line aborts the whole load.** `JsonLineParser.parse` catches `SerializeException`, but `JsonUtil.convertMap` throws the loader's `DeserializeException`, so no `ReadException` is raised and `--max-read-errors` has no effect. Reproduced with `FileLoadTest` resources (`reload_json_failure_files`): one valid edge line plus `{"source_name": `, with `--max-read-errors 10`. The load threw `DeserializeException`, and the valid edge was not loaded either.
   2. **Batch inserts never retry network failures.** `BatchInsertTask` retries on `ClientException`, but the client now throws a raw `IOException` for timeouts and dropped connections. With `retryTimes=3`, a fake server that answers after 2.5 s (client timeout 1 s) got one request before `SocketTimeoutException` reached the caller. A dropped connection got two requests: the first try and one OkHttp retry, none from the loader. 5xx responses are retried correctly; `BatchInsertRetryTest` covers that case.
   3. **LONG values above `Long.MAX_VALUE` wrap around.** `DataTypeUtil.parseLong` uses `Long.parseUnsignedLong` for non-negative input, so `"9223372036854775808"` becomes `-9223372036854775808` and `"18446744073709551615"` becomes `-1` instead of being rejected. The same helper parses numeric vertex IDs. The Spark connector's `DataTypeUtils.parseLong` has the same code (checked by reading, not run).
   4. **BLOB values become null.** `DataTypeUtil.parseBlob` has its body commented out and returns `null` for any input, for example `"0x01"` or `"AQI="`.
   5. **Unimplemented Go vertex methods panic.** `client.Vertex.GetByID()` and the other unassigned function fields cause a nil pointer panic instead of returning an error.
   6. **A latent race in the Spark connector `DateUtils`.** It shares one `SafeDateFormat` per pattern and calls `setTimeZone` before `parse`, the code the loader fixed in #734. A port of the loader's concurrent test returned 1 wrong instant out of 1600. The connector currently always passes GMT+8, so the sink path does not hit it.
   
   ## Does this PR potentially affect the following parts?
   
   - [x]  Nope
   - [ ]  Dependencies (add/update license info)
   - [ ]  Modify configurations
   - [ ]  The public API
   - [ ]  Other affects (typed here)
   
   ## Documentation Status
   
   - [ ]  `Doc - TODO`
   - [ ]  `Doc - Done`
   - [x]  `Doc - No Need`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


Re: [PR] test: improve coverage across toolchain modules before 1.8.0 [hugegraph-toolchain]

Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on code in PR #805:
URL: https://github.com/apache/hugegraph-toolchain/pull/805#discussion_r4274244554


##########
hugegraph-client/src/test/java/org/apache/hugegraph/unit/JsonUtilTest.java:
##########
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hugegraph.unit;
+
+import java.util.Map;
+
+import org.apache.hugegraph.rest.SerializeException;
+import org.apache.hugegraph.structure.graph.Vertex;
+import org.apache.hugegraph.testutil.Assert;
+import org.apache.hugegraph.util.JsonUtil;
+import org.junit.Test;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.node.JsonNodeFactory;
+import com.fasterxml.jackson.databind.node.ObjectNode;
+
+public class JsonUtilTest extends BaseUnitTest {

Review Comment:
   Addressed in the current PR update. The existing six JsonUtilTest cases from master, including all three fromJson2List cases, are preserved; the five added cases have distinct method names, including testConvertValueWithVertexAndMismatchedType, and UnitTestSuite registers JsonUtilTest once. GitHub now reports MERGEABLE. Both Client validation lanes (candidate and release-1.7) passed in [the current-head run](https://github.com/apache/hugegraph-toolchain/actions/runs/38366145675). Resolving the earlier conflict finding.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


Re: [PR] test: improve coverage across toolchain modules before 1.8.0 [hugegraph-toolchain]

Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin merged PR #805:
URL: https://github.com/apache/hugegraph-toolchain/pull/805


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


Re: [PR] test: improve coverage across toolchain modules before 1.8.0 [hugegraph-toolchain]

Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #805:
URL: https://github.com/apache/hugegraph-toolchain/pull/805#issuecomment-6101784440

   ## [Codecov](https://app.codecov.io/gh/apache/hugegraph-toolchain/pull/805?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :white_check_mark: All modified and coverable lines are covered by tests.
   :white_check_mark: Project coverage is 58.08%. Comparing base ([`039153c`](https://app.codecov.io/gh/apache/hugegraph-toolchain/commit/039153cdbfe26798fdc4fb2628ef016948917f23?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`1b96264`](https://app.codecov.io/gh/apache/hugegraph-toolchain/commit/1b96264146dc5b9948fd9476332f88cb6f3a6267?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@              Coverage Diff              @@
   ##             master     #805       +/-   ##
   =============================================
   + Coverage     44.74%   58.08%   +13.34%     
   + Complexity     3323     1191     -2132     
   =============================================
     Files           377      111      -266     
     Lines         22486     6256    -16230     
     Branches       3006      840     -2166     
   =============================================
   - Hits          10062     3634     -6428     
   + Misses        11352     2365     -8987     
   + Partials       1072      257      -815     
   ```
   </details>
   
   [:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/hugegraph-toolchain/pull/805?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).   
   :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   - :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


Re: [PR] test: improve coverage across toolchain modules before 1.8.0 [hugegraph-toolchain]

Posted by "bitflicker64 (via GitHub)" <gi...@apache.org>.
bitflicker64 commented on code in PR #805:
URL: https://github.com/apache/hugegraph-toolchain/pull/805#discussion_r4255561644


##########
hugegraph-client/src/test/java/org/apache/hugegraph/unit/JsonUtilTest.java:
##########
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hugegraph.unit;
+
+import java.util.Map;
+
+import org.apache.hugegraph.rest.SerializeException;
+import org.apache.hugegraph.structure.graph.Vertex;
+import org.apache.hugegraph.testutil.Assert;
+import org.apache.hugegraph.util.JsonUtil;
+import org.junit.Test;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.node.JsonNodeFactory;
+import com.fasterxml.jackson.databind.node.ObjectNode;
+
+public class JsonUtilTest extends BaseUnitTest {

Review Comment:
   Important: this head does not merge into `master`. #577 landed on master as 8e6ed847 and added its own `hugegraph-client/src/test/java/org/apache/hugegraph/unit/JsonUtilTest.java` and its own `JsonUtilTest.class` entry in `UnitTestSuite`.
   
   Evidence: `gh pr view 805 --json mergeable` reports `CONFLICTING`, and `git merge-tree --write-tree origin/master 1b962641` (master at 8e6ed847) reports `CONFLICT (add/add)` for `JsonUtilTest.java` and `CONFLICT (content)` for `UnitTestSuite.java`. No other path conflicts. The two classes also differ in shape: master's is a plain class on `org.junit.Assert` with `testToJson`, `testFromJson`, `testConvertValue` and three `testFromJson2List*` cases, while this one extends `BaseUnitTest`, uses `org.apache.hugegraph.testutil.Assert`, and also declares `testConvertValue()` (line 81), so the two bodies cannot be concatenated as they are.
   
   Requested change: rebase onto current master, fold these five cases into master's `JsonUtilTest` (rename this `testConvertValue`, for example to `testConvertValueErrors`, and keep the `fromJson2List` cases), and keep a single `JsonUtilTest.class` entry in `UnitTestSuite`. The green checks on this head predate 8e6ed847, so `client-ci` needs to run again on the rebased head.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


Re: [PR] test: improve coverage across toolchain modules before 1.8.0 [hugegraph-toolchain]

Posted by "bitflicker64 (via GitHub)" <gi...@apache.org>.
bitflicker64 commented on code in PR #805:
URL: https://github.com/apache/hugegraph-toolchain/pull/805#discussion_r4278148873


##########
hugegraph-loader/src/test/java/org/apache/hugegraph/loader/test/unit/BatchInsertRetryTest.java:
##########
@@ -0,0 +1,258 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hugegraph.loader.test.unit;
+
+import java.io.IOException;
+import java.io.OutputStream;
+import java.lang.reflect.Field;
+import java.lang.reflect.Method;
+import java.net.InetSocketAddress;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import org.apache.hugegraph.driver.HugeClient;
+import org.apache.hugegraph.exception.ServerException;
+import org.apache.hugegraph.loader.builder.Record;
+import org.apache.hugegraph.loader.executor.LoadContext;
+import org.apache.hugegraph.loader.executor.LoadOptions;
+import org.apache.hugegraph.loader.mapping.EdgeMapping;
+import org.apache.hugegraph.loader.mapping.InputStruct;
+import org.apache.hugegraph.loader.metrics.LoadMetrics;
+import org.apache.hugegraph.loader.metrics.LoadSummary;
+import org.apache.hugegraph.loader.progress.LoadProgress;
+import org.apache.hugegraph.loader.source.file.FileSource;
+import org.apache.hugegraph.loader.task.BatchInsertTask;
+import org.apache.hugegraph.structure.graph.Edge;
+import org.apache.hugegraph.testutil.Assert;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.rules.TemporaryFolder;
+
+import com.sun.net.httpserver.HttpExchange;
+import com.sun.net.httpserver.HttpServer;
+
+/**
+ * Batch insert retries against a local fake server
+ */
+public class BatchInsertRetryTest {
+
+    private static final String VERSIONS =
+            "{\"versions\":{\"version\":\"v1\",\"core\":\"1.8.0\"," +
+            "\"gremlin\":\"3.8.1\",\"api\":\"0.72.0.0\"}}";
+
+    @Rule
+    public TemporaryFolder folder = new TemporaryFolder();
+
+    private HttpServer server;
+    private HugeClient client;
+    private final AtomicInteger batchRequests = new AtomicInteger();
+    private final List<Handler> batchHandlers = Collections.synchronizedList(new ArrayList<>());
+
+    @Before
+    public void setup() throws IOException {
+        this.server = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
+        this.server.createContext("/", exchange -> {
+            String path = exchange.getRequestURI().getPath();
+            if (path.endsWith("/versions")) {
+                reply(exchange, 200, VERSIONS);
+            } else if (path.endsWith("/graph/edges/batch")) {
+                int index = this.batchRequests.getAndIncrement();
+                Handler handler = index < this.batchHandlers.size() ?
+                                  this.batchHandlers.get(index) :
+                                  e -> reply(e, 201, "[\"e1\"]");
+                handler.handle(exchange);
+            } else {
+                reply(exchange, 404, "{\"exception\":\"NotFoundException\"," +
+                                     "\"message\":\"" + path + "\"}");
+            }
+        });
+        this.server.start();
+        this.client = HugeClient.builder("http://127.0.0.1:" +
+                                         this.server.getAddress().getPort(),
+                                         "DEFAULT", "hugegraph")
+                                .configTimeout(1)
+                                .build();
+    }
+
+    @After
+    public void teardown() {
+        if (this.client != null) {
+            this.client.close();
+        }
+        this.server.stop(0);
+    }
+
+    @Test
+    public void testServerErrorIsRetriedUntilSuccess() throws Exception {
+        this.batchHandlers.add(e -> reply(e, 500, serverError("java.lang.IllegalStateException",
+                                                              "backend busy")));
+        this.batchHandlers.add(e -> reply(e, 503, "<html>Service Unavailable</html>"));
+        LoadContext context = this.context(3);
+        InputStruct struct = edgeStruct(context);
+        EdgeMapping mapping = struct.edges().get(0);
+
+        new BatchInsertTask(context, struct, mapping, batch()).execute();
+
+        Assert.assertEquals(3, this.batchRequests.get());
+        Assert.assertEquals(1L, context.summary().metrics(struct).insertSuccess(mapping));
+    }
+
+    @Test
+    public void testServerErrorFailsAfterRetries() throws Exception {
+        for (int i = 0; i < 3; i++) {
+            this.batchHandlers.add(e -> reply(e, 500, serverError(
+                                   "java.lang.IllegalStateException", "backend busy")));
+        }
+        LoadContext context = this.context(2);
+        InputStruct struct = edgeStruct(context);
+        EdgeMapping mapping = struct.edges().get(0);
+
+        Assert.assertThrows(ServerException.class, () -> {
+            new BatchInsertTask(context, struct, mapping, batch()).execute();
+        }, e -> {
+            Assert.assertEquals(500, ((ServerException) e).status());
+        });
+        // The first request and two retries
+        Assert.assertEquals(3, this.batchRequests.get());
+        Assert.assertEquals(0L, context.summary().metrics(struct).insertSuccess(mapping));
+    }
+
+    @Test
+    public void testIllegalArgumentIsNotRetried() throws Exception {
+        this.batchHandlers.add(e -> reply(e, 400, serverError(
+                               "class java.lang.IllegalArgumentException",
+                               "Invalid edge label 'knows'")));
+        LoadContext context = this.context(3);
+        InputStruct struct = edgeStruct(context);
+        EdgeMapping mapping = struct.edges().get(0);
+
+        Assert.assertThrows(ServerException.class, () -> {
+            new BatchInsertTask(context, struct, mapping, batch()).execute();
+        }, e -> {
+            Assert.assertEquals("Invalid edge label 'knows'", e.getMessage());
+        });
+        Assert.assertEquals(1, this.batchRequests.get());
+    }
+
+    @Test
+    public void testUnacceptableMessageIsNotRetried() throws Exception {
+        this.batchHandlers.add(e -> reply(e, 500, serverError(
+                               "java.lang.IllegalStateException",
+                               "The server is being shutting down")));
+        LoadContext context = this.context(3);
+        InputStruct struct = edgeStruct(context);
+        EdgeMapping mapping = struct.edges().get(0);
+
+        Assert.assertThrows(ServerException.class, () -> {
+            new BatchInsertTask(context, struct, mapping, batch()).execute();
+        });
+        Assert.assertEquals(1, this.batchRequests.get());
+    }
+
+    private LoadContext context(int retryTimes) throws Exception {
+        LoadOptions options = new LoadOptions();
+        options.retryTimes = retryTimes;
+        // No sleep between retries
+        options.retryInterval = 0;
+        options.file = this.folder.newFile("mapping.json").getAbsolutePath();
+        LoadContext context = (LoadContext) allocateInstance(LoadContext.class);

Review Comment:
   Minor: `context()` builds `LoadContext` with `sun.misc.Unsafe.allocateInstance` and then sets 13 private fields by name, so the test bypasses the constructor and silently diverges from a real context. `LoadContext` has 15 instance fields; `loadingModeSet` and `failure` are left at defaults, and `schemaCache` and `parseGroup` are set to `null` although the constructor always makes them non-null. A later change that renames a field fails here with `NoSuchFieldException`, and one that adds a constructor-initialised field or reads `context.schemaCache()` on the insert path fails with an NPE that has nothing to do with retries. None of this is needed: the public `LoadContext(LoadOptions)` constructor only calls `HugeClientHolder.create(options)`, which builds the client and requests `/versions`, the one endpoint the fake server already serves besides `/graph/edges/batch`. Please set `options.host = "127.0.0.1"`, `options.port` to the fake server's port and `options.timeout = 1`, use `n
 ew LoadContext(options)`, close `context.client()` in `teardown()`, and drop `allocateInstance` and `setField`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org


Re: [PR] test: improve coverage across toolchain modules before 1.8.0 [hugegraph-toolchain]

Posted by "bitflicker64 (via GitHub)" <gi...@apache.org>.
bitflicker64 commented on code in PR #805:
URL: https://github.com/apache/hugegraph-toolchain/pull/805#discussion_r4286344352


##########
hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/unit/HubbleUtilTest.java:
##########
@@ -0,0 +1,125 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.hugegraph.unit;
+
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Date;
+import java.util.TimeZone;
+
+import org.apache.hugegraph.testutil.Assert;
+import org.apache.hugegraph.util.HubbleUtil;
+import org.apache.hugegraph.util.SerializeUtil;
+import org.junit.Test;
+
+import com.fasterxml.jackson.annotation.JsonProperty;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.annotation.JsonSerialize;
+
+public class HubbleUtilTest {
+
+    @Test
+    public void testEqualCollection() {
+        Assert.assertTrue(HubbleUtil.equalCollection(null, null));
+        Assert.assertTrue(HubbleUtil.equalCollection(Arrays.asList("a", "b"),
+                                                     Arrays.asList("b", "a")));
+        Assert.assertTrue(HubbleUtil.equalCollection(Collections.emptyList(),
+                                                     Collections.emptySet()));
+        Assert.assertFalse(HubbleUtil.equalCollection(Arrays.asList("a"), null));
+        Assert.assertFalse(HubbleUtil.equalCollection(null, Arrays.asList("a")));
+        Assert.assertFalse(HubbleUtil.equalCollection(Arrays.asList("a", "a"),
+                                                      Arrays.asList("a")));
+        Assert.assertFalse(HubbleUtil.equalCollection(Arrays.asList("a"),
+                                                      Arrays.asList("b")));
+    }
+
+    @Test
+    public void testTimestampsBefore24Hours() {
+        long[] timestamps = HubbleUtil.getTimestampsBefore24Hours("20240115 12:00:00");
+
+        // 2024-01-15 12:00:00 is parsed in the default time zone
+        long offset = TimeZone.getDefault().getOffset(1705320000000L) / 1000L;
+        long expected = 1705320000L - offset;
+        Assert.assertArrayEquals(new long[]{expected - 86400L, expected}, timestamps);
+
+        long[] now = HubbleUtil.getTimestampsBefore24Hours();
+        Assert.assertEquals(86400L, now[1] - now[0]);
+
+        Assert.assertThrows(RuntimeException.class, () -> {
+            HubbleUtil.getTimestampsBefore24Hours("2024-01-15 12:00:00");
+        }, e -> {
+            Assert.assertTrue(e.getCause() instanceof java.text.ParseException);
+        });
+    }
+
+    @Test
+    public void testDateFormatDay() {
+        Assert.assertEquals("2024-01-15", HubbleUtil.dateFormatDay("20240115"));
+        Assert.assertEquals("20240115", HubbleUtil.dateFormatDay(new Date(1705291200000L)));

Review Comment:
   Important: this assertion only passes in time zones at UTC-4 or further east. `HubbleUtil.dateFormatDay(Date)` is `new SimpleDateFormat("yyyyMMdd").format(date)`, so it formats in the JVM default zone, and `1705291200000L` is 2024-01-15T04:00:00Z. That instant is still 14 January in New York, Los Angeles or Honolulu, so `mvn test -P unit-test` in hubble-be fails for a contributor there. CI passes only because the GitHub runners use UTC, and the hubble-be surefire config does not pin `user.timezone`.
   
   Evidence: the same call, run with `java -Duser.timezone=<zone>`:
   
   ```
   UTC                 day=20240115 month=202401
   Asia/Shanghai       day=20240115 month=202401
   America/Sao_Paulo   day=20240115 month=202401
   America/New_York    day=20240114 month=202401
   America/Los_Angeles day=20240114 month=202401
   Pacific/Honolulu    day=20240114 month=202401
   ```
   
   `testTimestampsBefore24Hours` in the same class already handles this by computing the offset from `TimeZone.getDefault()`.
   
   Requested change: build the date from local calendar fields so it is 15 January in every zone, for example `Date day = new GregorianCalendar(2024, Calendar.JANUARY, 15, 12, 0).getTime();`, and use it for both the `dateFormatDay` and `dateFormatMonth` assertions on lines 73 and 74.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org