You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hugegraph.apache.org by ji...@apache.org on 2026/10/23 04:19:10 UTC
(hugegraph-toolchain) branch master updated: refactor(hubble): simplify Java 17 maintenance (#839)
This is an automated email from the ASF dual-hosted git repository.
imbajin pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hugegraph-toolchain.git
The following commit(s) were added to refs/heads/master by this push:
new 45a5c0c7 refactor(hubble): simplify Java 17 maintenance (#839)
45a5c0c7 is described below
commit 45a5c0c761df4008495fc78560da625d02732309
Author: imbajin <ji...@apache.org>
AuthorDate: Fri Oct 23 12:19:05 2026 +0800
refactor(hubble): simplify Java 17 maintenance (#839)
Hubble already targets Java 17, but legacy date handling, duplicated file previews and mutable empty query results still cause avoidable failures. Long imports can also occupy every core worker and delay progress updates and cleanup. This cleanup fixes those paths while retaining the existing Server compatibility boundaries and metadata format.
---
docs/java17-migration.md | 6 +
hugegraph-dist/release-docs/NOTICE | 2 +-
.../scripts/dependency/known-dependencies.txt | 5 +-
hugegraph-hubble/hubble-be/pom.xml | 26 --
.../org/apache/hugegraph/config/AsyncConfig.java | 66 +++--
.../org/apache/hugegraph/config/JacksonConfig.java | 14 -
.../hugegraph/controller/AboutController.java | 17 --
.../controller/ingest/IngestController.java | 97 +------
.../hugegraph/controller/op/LogController.java | 14 +-
.../controller/saas/GraphMetricsController.java | 107 ++++----
.../saas/GraphSpaceMetricsController.java | 9 +-
.../apache/hugegraph/entity/query/GraphView.java | 1 -
.../apache/hugegraph/entity/query/TableView.java | 1 -
.../hugegraph/handler/CustomApplicationRunner.java | 51 ----
.../hugegraph/handler/CustomInterceptor.java | 27 +-
.../hugegraph/handler/HubbleDisposableBean.java | 2 +-
.../org/apache/hugegraph/license/ExtraParam.java | 132 ----------
.../hugegraph/license/LicenseVerifyParam.java | 61 -----
.../org/apache/hugegraph/license/MachineInfo.java | 127 ---------
.../org/apache/hugegraph/license/ServerInfo.java | 38 ---
.../apache/hugegraph/options/HubbleOptions.java | 8 -
.../apache/hugegraph/service/auth/UserService.java | 78 +++---
.../hugegraph/service/load/FileMappingService.java | 66 +----
.../hugegraph/service/load/FilePreviewReader.java | 125 +++++++++
.../hugegraph/service/load/LoadTaskService.java | 4 +-
.../service/query/ExecuteHistoryService.java | 4 +-
.../hugegraph/service/query/QueryService.java | 150 ++++-------
.../hugegraph/service/space/GraphSpaceService.java | 21 +-
.../java/org/apache/hugegraph/util/ESUtil.java | 20 +-
.../java/org/apache/hugegraph/util/HubbleUtil.java | 165 +++---------
.../src/main/resources/i18n/messages.properties | 15 --
.../main/resources/i18n/messages_zh_CN.properties | 15 --
.../controller/ingest/IngestControllerTest.java | 27 ++
.../hugegraph/controller/op/LogControllerTest.java | 67 +++++
.../service/load/FilePreviewReaderTest.java | 218 ++++++++++++++++
.../load/IngestTransactionIntegrationTest.java | 6 +-
.../service/load/JobAdmissionIntegrationTest.java | 6 +-
.../service/op/OperationsHttpClientTest.java | 25 +-
.../service/space/GraphSpaceServiceTest.java | 42 +++
.../hugegraph/unit/AsyncConfigLifecycleTest.java | 36 ++-
.../hugegraph/unit/GraphMetricsControllerTest.java | 43 +++
.../org/apache/hugegraph/unit/HubbleUtilTest.java | 16 +-
.../apache/hugegraph/unit/JacksonConfigTest.java | 124 +++++++++
.../hugegraph/unit/MaintenanceSchedulingTest.java | 288 +++++++++++++++++++++
.../apache/hugegraph/unit/QueryServiceTest.java | 125 +++++++++
.../org/apache/hugegraph/unit/UnitTestSuite.java | 6 +
.../apache/hugegraph/unit/UserCsvImportTest.java | 128 +++++++++
.../hugegraph/unit/UserImportSafetyTest.java | 2 +-
.../apache/hugegraph/util/HubbleDateTimeTest.java | 110 ++++++++
hugegraph-hubble/pom.xml | 23 --
50 files changed, 1626 insertions(+), 1140 deletions(-)
diff --git a/docs/java17-migration.md b/docs/java17-migration.md
index 426dc1cd..f1fc5d10 100644
--- a/docs/java17-migration.md
+++ b/docs/java17-migration.md
@@ -91,6 +91,12 @@ Hubble uses Spring Boot 3, Jakarta APIs and H2 metadata. Start with a new metada
Retain the new database and configuration across restarts. Local H2 file URL shorthand and encrypted files are supported; `INIT` settings are rejected before opening the database. The full initial schema includes the release marker. Packaging excludes runtime data from archives without deleting local data. See the [Hubble guide](../hugegraph-hubble/README.md) for configuration and UI validation evidence.
+Import progress and periodic cleanup run separately from Loader workers and streaming heartbeats, so long imports do not queue maintenance behind them. Each maintenance method runs at most once at a time. Shutdown pauses imports and waits for running work to finish before closing H2, including when the closing thread is interrupted.
+
+Hubble's standalone entry point retains UTC+08:00 for existing metadata. Date helpers use immutable `java.time` formatters with that explicit offset; loading a helper no longer changes the JVM timezone. Date-based task counts include the complete day, and invalid dates are rejected instead of silently rolling into another month.
+
+File previews use the configured charset and the Loader's CSV/TEXT parsers, including quoted CSV cells and skipped lines. Graph import previews retain the Loader's one-record-per-physical-line constraint. Account CSV imports support quoted multiline values and validate all row widths and headers before requesting batch creation.
+
## Compatibility and peripheral engines
The compatibility scope below distinguishes supported operations from validation that must be repeated for each release. Use Java 17 for new Toolchain applications; released servers and original clients retain their supported JVMs.
diff --git a/hugegraph-dist/release-docs/NOTICE b/hugegraph-dist/release-docs/NOTICE
index a54f97fd..266ffa23 100644
--- a/hugegraph-dist/release-docs/NOTICE
+++ b/hugegraph-dist/release-docs/NOTICE
@@ -537,7 +537,7 @@ For source see 'http://stephenc.github.com/findbugs-annotations/findbugs-annotat
--
This product includes error_prone_annotations
-incorporated from com.google.errorprone:error_prone_annotations:jar:2.3.3
+incorporated from com.google.errorprone:error_prone_annotations:jar:2.49.0
For source see 'http://nexus.sonatype.org/oss-repository-hosting.html/error_prone_parent/error_prone_annotations/error_prone_annotations'.
--
diff --git a/hugegraph-dist/scripts/dependency/known-dependencies.txt b/hugegraph-dist/scripts/dependency/known-dependencies.txt
index b9ccd62d..91cdd50f 100644
--- a/hugegraph-dist/scripts/dependency/known-dependencies.txt
+++ b/hugegraph-dist/scripts/dependency/known-dependencies.txt
@@ -42,11 +42,10 @@ avro-1.10.2.jar
avro-1.7.7.jar
bolt-1.6.4.jar
cache-api-1.1.1.jar
-caffeine-2.8.0.jar
+caffeine-3.2.4.jar
calcite-core-1.16.0.jar
calcite-druid-1.16.0.jar
calcite-linq4j-1.16.0.jar
-checker-qual-2.10.0.jar
checker-qual-3.5.0.jar
ci-environment-10.0.1.jar
commons-beanutils-1.9.4.jar
@@ -112,8 +111,8 @@ eclipse-collections-api-11.1.0.jar
ehcache-3.10.9.jar
elasticsearch-java-7.16.1.jar
elasticsearch-rest-client-7.16.1.jar
-error_prone_annotations-2.3.3.jar
error_prone_annotations-2.3.4.jar
+error_prone_annotations-2.49.0.jar
esri-geometry-api-2.2.0.jar
exp4j-0.4.8.jar
failsafe-2.4.1.jar
diff --git a/hugegraph-hubble/hubble-be/pom.xml b/hugegraph-hubble/hubble-be/pom.xml
index 6a7cd296..0b0039a0 100644
--- a/hugegraph-hubble/hubble-be/pom.xml
+++ b/hugegraph-hubble/hubble-be/pom.xml
@@ -29,10 +29,8 @@
<artifactId>hubble-be</artifactId>
<properties>
- <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<release.name>hugegraph-hubble</release.name>
<final.name>${release.name}-${project.version}</final.name>
- <java.version>17</java.version>
<skip.dump>true</skip.dump>
<java.inputs.script>
${project.basedir}/../hubble-dist/assembly/travis/check-java-build-inputs.py
@@ -92,12 +90,6 @@
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
- <!--
- <dependency>
- <groupId>org.springframework.boot</groupId>
- <artifactId>spring-boot-starter-actuator</artifactId>
- </dependency>
- -->
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-client</artifactId>
@@ -310,7 +302,6 @@
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
- <version>3.17.3</version>
</dependency>
<!-- Use 'core' classifier to avoid embedded protobuf-java 2.5.0 conflict
with hg-pd-grpc which requires protobuf-java 3.17.x -->
@@ -358,11 +349,9 @@
<groupId>org.apache.calcite.avatica</groupId>
<artifactId>avatica-core</artifactId>
</dependency>
- <!-- https://mvnrepository.com/artifact/com.squareup.okhttp3/okhttp -->
<dependency>
<groupId>com.squareup.okhttp3</groupId>
<artifactId>okhttp</artifactId>
- <version>4.12.0</version>
</dependency>
</dependencies>
@@ -423,21 +412,6 @@
<failOnViolation>true</failOnViolation>
</configuration>
</plugin>
- <!--
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-jar-plugin</artifactId>
- <configuration>
- <archive>
- <manifest>
- <addClasspath>true</addClasspath>
- <classpathPrefix>lib/</classpathPrefix>
- <mainClass>org.apache.hugegraph.HugeGraphHubble</mainClass>
- </manifest>
- </archive>
- </configuration>
- </plugin>
- -->
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
diff --git a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/config/AsyncConfig.java b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/config/AsyncConfig.java
index 104de102..bafc3926 100644
--- a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/config/AsyncConfig.java
+++ b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/config/AsyncConfig.java
@@ -19,6 +19,7 @@
package org.apache.hugegraph.config;
import java.util.concurrent.Executor;
+import java.util.concurrent.ExecutorService;
import java.util.concurrent.TimeUnit;
import org.springframework.context.annotation.Bean;
@@ -27,6 +28,7 @@ import org.springframework.context.annotation.DependsOn;
import org.springframework.scheduling.annotation.AsyncConfigurer;
import org.springframework.scheduling.annotation.EnableAsync;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;
+import org.springframework.scheduling.concurrent.ThreadPoolTaskScheduler;
@Configuration
@EnableAsync
@@ -42,24 +44,7 @@ public class AsyncConfig implements AsyncConfigurer {
try {
super.destroy();
} finally {
- // A native shutdown may throw after initiating shutdown.
- // Drain in that case too; never await a pool still accepting.
- if (this.getThreadPoolExecutor().isShutdown()) {
- boolean interrupted = false;
- try {
- while (!this.getThreadPoolExecutor().isTerminated()) {
- try {
- this.getThreadPoolExecutor().awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS);
- } catch (InterruptedException e) {
- interrupted = true;
- }
- }
- } finally {
- if (interrupted) {
- Thread.currentThread().interrupt();
- }
- }
- }
+ awaitTermination(this.getThreadPoolExecutor());
}
}
};
@@ -69,4 +54,49 @@ public class AsyncConfig implements AsyncConfigurer {
taskExecutor.setWaitForTasksToCompleteOnShutdown(true);
return taskExecutor;
}
+
+ @Bean
+ @DependsOn("dataSource")
+ public ThreadPoolTaskScheduler maintenanceScheduler() {
+ ThreadPoolTaskScheduler scheduler = new ThreadPoolTaskScheduler() {
+ @Override
+ public void destroy() {
+ try {
+ super.destroy();
+ } finally {
+ awaitTermination(this.getScheduledExecutor());
+ }
+ }
+ };
+ // One slot for each periodic method keeps slow cleanup from blocking
+ // progress updates. Native periodic tasks cannot overlap themselves.
+ scheduler.setPoolSize(4);
+ scheduler.setThreadNamePrefix("hubble-maintenance-");
+ scheduler.setRemoveOnCancelPolicy(true);
+ scheduler.setExecuteExistingDelayedTasksAfterShutdownPolicy(false);
+ scheduler.setWaitForTasksToCompleteOnShutdown(true);
+ return scheduler;
+ }
+
+ private static void awaitTermination(ExecutorService executor) {
+ // A native shutdown may throw after initiating shutdown.
+ // Drain in that case too; never await a pool still accepting.
+ if (!executor.isShutdown()) {
+ return;
+ }
+ boolean interrupted = false;
+ try {
+ while (!executor.isTerminated()) {
+ try {
+ executor.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS);
+ } catch (InterruptedException e) {
+ interrupted = true;
+ }
+ }
+ } finally {
+ if (interrupted) {
+ Thread.currentThread().interrupt();
+ }
+ }
+ }
}
diff --git a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/config/JacksonConfig.java b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/config/JacksonConfig.java
index c8ea6603..5b1f87f2 100644
--- a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/config/JacksonConfig.java
+++ b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/config/JacksonConfig.java
@@ -20,16 +20,12 @@ package org.apache.hugegraph.config;
import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.databind.JsonSerializer;
-import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializerProvider;
-import com.fasterxml.jackson.databind.module.SimpleModule;
import org.apache.hugegraph.common.Response;
import org.apache.hugegraph.entity.graph.VertexQueryEntity;
import org.apache.hugegraph.structure.graph.Edge;
import org.apache.hugegraph.structure.graph.Vertex;
import org.springframework.boot.jackson.JsonComponent;
-import org.springframework.context.annotation.Bean;
-import org.springframework.http.converter.json.Jackson2ObjectMapperBuilder;
import java.io.IOException;
import java.util.Map;
@@ -37,16 +33,6 @@ import java.util.Map;
@JsonComponent
public class JacksonConfig {
- @Bean
- public ObjectMapper jacksonObjectMapper(Jackson2ObjectMapperBuilder builder) {
- ObjectMapper mapper = builder.createXmlMapper(false).build();
-
- SimpleModule module = new SimpleModule();
- // Add global serializer here
- mapper.registerModule(module);
- return mapper;
- }
-
public static class ResponseSerailizer extends JsonSerializer<Response> {
@Override
diff --git a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/controller/AboutController.java b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/controller/AboutController.java
index 57418ed0..b26d22e6 100644
--- a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/controller/AboutController.java
+++ b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/controller/AboutController.java
@@ -19,39 +19,22 @@
package org.apache.hugegraph.controller;
import org.apache.hugegraph.common.Constant;
-import org.apache.hugegraph.handler.MessageSourceHandler;
-import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import java.util.LinkedHashMap;
import java.util.Map;
-//import org.apache.hugegraph.license.LicenseVerifier; // TODO C Remove Licence
@RestController
@RequestMapping("about")
public class AboutController extends BaseController {
- @Autowired
- private MessageSourceHandler messageHandler;
-
@GetMapping
public Map<String, Object> about() {
- //LicenseVerifier verifier = LicenseVerifier.instance();// TODO C Remove Licence
Map<String, Object> about = new LinkedHashMap<>();
about.put("name", Constant.SERVER_NAME);
about.put("version", "3.0.0");
- //about.put("edition", verifier.edition());// TODO C Remove Licence
- //about.put("allowed_graphs", verifier.allowedGraphs());
- //long datasize = verifier.allowedDataSize();
- //if (datasize == -1) {
- // about.put("allowed_datasize",
- // this.messageHandler.getMessage("license.datasize.no-limit"));
- //} else {
- // about.put("allowed_datasize",
- // FileUtils.byteCountToDisplaySize(datasize * Bytes.MB));
- //}
return about;
}
}
diff --git a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/controller/ingest/IngestController.java b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/controller/ingest/IngestController.java
index dc1bd779..6723a17c 100644
--- a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/controller/ingest/IngestController.java
+++ b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/controller/ingest/IngestController.java
@@ -19,9 +19,6 @@ package org.apache.hugegraph.controller.ingest;
import com.baomidou.mybatisplus.core.metadata.IPage;
import com.fasterxml.jackson.annotation.JsonProperty;
-import com.fasterxml.jackson.core.JsonParser;
-import com.fasterxml.jackson.core.type.TypeReference;
-import com.fasterxml.jackson.databind.ObjectMapper;
import lombok.Data;
import lombok.extern.log4j.Log4j2;
import org.apache.commons.io.FileUtils;
@@ -52,6 +49,7 @@ import org.apache.hugegraph.loader.source.file.FileFormat;
import org.apache.hugegraph.options.HubbleOptions;
import org.apache.hugegraph.service.load.DatasourceService;
import org.apache.hugegraph.service.load.FileMappingService;
+import org.apache.hugegraph.service.load.FilePreviewReader;
import org.apache.hugegraph.service.load.JobManagerService;
import org.apache.hugegraph.service.load.LoadTaskService;
import org.apache.hugegraph.util.Ex;
@@ -70,17 +68,14 @@ import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.multipart.MultipartFile;
-import java.io.BufferedReader;
import java.io.File;
import java.io.IOException;
-import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Date;
import java.util.HashMap;
-import java.util.LinkedHashMap;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Locale;
@@ -94,10 +89,6 @@ import java.util.stream.Collectors;
@RequestMapping(Constant.API_VERSION + "ingest")
public class IngestController extends BaseController {
- private static final ObjectMapper JSON_MAPPER =
- new ObjectMapper().disable(
- JsonParser.Feature.INCLUDE_SOURCE_IN_LOCATION);
-
@Autowired
private JobManagerService jobManagerService;
@Autowired
@@ -207,10 +198,10 @@ public class IngestController extends BaseController {
boolean hasPhysicalHeader = this.hasPhysicalHeader(config);
FileSetting setting = this.buildFileSetting(config, Collections.emptyList(),
hasPhysicalHeader);
- ColumnInfo columns = this.readColumns(this.requireUploadFile(config),
- setting, hasPhysicalHeader);
+ FilePreviewReader.Columns columns = FilePreviewReader.read(
+ this.requireUploadFile(config).toPath(), setting);
return Response.builder().status(Constant.STATUS_OK)
- .data(columns.names).build();
+ .data(columns.names()).build();
}
@PostMapping("/jdbc/check")
@@ -251,10 +242,9 @@ public class IngestController extends BaseController {
hasPhysicalHeader);
if (setting.getColumnNames() == null ||
setting.getColumnNames().isEmpty()) {
- ColumnInfo columns = this.readColumns(sourceFile, setting,
- hasPhysicalHeader);
- setting.setColumnNames(columns.names);
- setting.setColumnValues(columns.values);
+ FilePreviewReader.Columns columns = FilePreviewReader.read(sourceFile.toPath(), setting);
+ setting.setColumnNames(columns.names());
+ setting.setColumnValues(columns.values());
}
Set<VertexMapping> vertexMappings = this.vertexMappings(struct.vertices);
@@ -676,70 +666,6 @@ public class IngestController extends BaseController {
this.stringList(config.get("header")).isEmpty();
}
- private ColumnInfo readColumns(File file, FileSetting setting,
- boolean hasPhysicalHeader) {
- try (BufferedReader reader = Files.newBufferedReader(file.toPath())) {
- String line;
- while ((line = reader.readLine()) != null) {
- if (!line.matches(setting.getSkippedLine())) {
- break;
- }
- }
- Ex.check(line != null, "The file has no data line can treat as header");
-
- if (FileFormat.JSON.name().equals(setting.getFormat())) {
- return this.readJsonColumns(line, file);
- }
-
- List<String> firstLine = this.splitLine(line, setting.getDelimiter());
- if (hasPhysicalHeader) {
- String sample = reader.readLine();
- List<String> values = sample == null ? Collections.emptyList() :
- this.splitLine(sample,
- setting.getDelimiter());
- return new ColumnInfo(firstLine, values);
- }
-
- List<String> names = new ArrayList<>();
- for (int i = 1; i <= firstLine.size(); i++) {
- names.add("col-" + i);
- }
- return new ColumnInfo(names, firstLine);
- } catch (IOException e) {
- throw new InternalException("Failed to read columns from file %s",
- e, file);
- }
- }
-
- private ColumnInfo readJsonColumns(String line, File file) {
- try {
- Map<String, Object> fields = JSON_MAPPER.readValue(
- line, new TypeReference<LinkedHashMap<String, Object>>() {
- });
- List<String> names = new ArrayList<>(fields.keySet());
- List<String> values = names.stream()
- .map(fields::get)
- .map(String::valueOf)
- .collect(Collectors.toList());
- return new ColumnInfo(names, values);
- } catch (IOException ignored) {
- throw new InternalException(
- "Failed to read JSON fields from file %s", file);
- }
- }
-
- private List<String> splitLine(String line, String delimiter) {
- if (line == null) {
- return Collections.emptyList();
- }
- String[] values = StringUtils.split(line, delimiter);
- List<String> list = new ArrayList<>();
- if (values != null) {
- Collections.addAll(list, values);
- }
- return list;
- }
-
private Set<VertexMapping> vertexMappings(List<Map<String, Object>> raw) {
Set<VertexMapping> mappings = new LinkedHashSet<>();
if (raw == null) {
@@ -968,13 +894,4 @@ public class IngestController extends BaseController {
@JsonProperty("edges") List<Map<String, Object>> edges;
}
- static class ColumnInfo {
- private final List<String> names;
- private final List<String> values;
-
- ColumnInfo(List<String> names, List<String> values) {
- this.names = names;
- this.values = values;
- }
- }
}
diff --git a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/controller/op/LogController.java b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/controller/op/LogController.java
index 991527e0..a9ddc4ab 100644
--- a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/controller/op/LogController.java
+++ b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/controller/op/LogController.java
@@ -26,6 +26,7 @@ import org.apache.hugegraph.controller.BaseController;
import org.apache.hugegraph.entity.op.LogEntity;
import org.apache.hugegraph.exception.InternalException;
import org.apache.hugegraph.service.op.LogService;
+import org.apache.hugegraph.util.HubbleUtil;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PostMapping;
@@ -37,13 +38,18 @@ import jakarta.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.io.OutputStream;
import java.nio.charset.StandardCharsets;
-import java.text.SimpleDateFormat;
+import java.time.format.DateTimeFormatter;
import java.util.Arrays;
import java.util.List;
+import java.util.Locale;
@RestController
@RequestMapping(Constant.API_VERSION + "logs")
public class LogController extends BaseController {
+
+ private static final DateTimeFormatter EXPORT_DATE_FORMAT =
+ DateTimeFormatter.ofPattern("uuuuMMddHHmmss", Locale.ROOT).withZone(HubbleUtil.TIME_ZONE);
+
@Autowired
LogService logService;
@@ -80,12 +86,10 @@ public class LogController extends BaseController {
@PostMapping("export")
public void export(HttpServletResponse response,
@RequestBody LogService.LogReq logReq) {
- SimpleDateFormat dateFormat =
- new SimpleDateFormat("yyyyMMddHHmmss");
String start = logReq.startDatetime == null ?
- "begin" : dateFormat.format(logReq.startDatetime);
+ "begin" : EXPORT_DATE_FORMAT.format(logReq.startDatetime.toInstant());
String end = logReq.endDatetime == null ?
- "end" : dateFormat.format(logReq.endDatetime);
+ "end" : EXPORT_DATE_FORMAT.format(logReq.endDatetime.toInstant());
String fileName = String.format("log_%s_%s.txt", start, end);
response.setCharacterEncoding("UTF-8");
diff --git a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/controller/saas/GraphMetricsController.java b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/controller/saas/GraphMetricsController.java
index 9ec1a6a9..5f4f4b50 100644
--- a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/controller/saas/GraphMetricsController.java
+++ b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/controller/saas/GraphMetricsController.java
@@ -25,14 +25,11 @@ import lombok.Data;
import lombok.NoArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.apache.hugegraph.api.graph.GraphMetricsAPI;
-// TODO fix import
-//import org.apache.hugegraph.client.api.graph.GraphMetricsAPI;
import org.apache.hugegraph.common.Constant;
import org.apache.hugegraph.controller.BaseController;
-import org.apache.hugegraph.date.SafeDateFormat;
import org.apache.hugegraph.driver.HugeClient;
import org.apache.hugegraph.exception.HugeException;
-import org.apache.hugegraph.util.DateUtil;
+import org.apache.hugegraph.util.HubbleUtil;
import org.apache.hugegraph.util.E;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -43,9 +40,12 @@ import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
-import java.text.DecimalFormat;
+import java.math.BigDecimal;
+import java.math.RoundingMode;
+import java.time.LocalDate;
+import java.time.format.DateTimeFormatter;
import java.util.ArrayList;
-import java.util.Calendar;
+import java.util.Date;
import java.util.Collections;
import java.util.List;
import java.util.Map;
@@ -58,14 +58,12 @@ public class GraphMetricsController extends BaseController {
private static final Logger LOG = LoggerFactory.getLogger(
GraphMetricsController.class);
- public static final SafeDateFormat SAFE_DATE_FORMAT = new SafeDateFormat("yyyyMMdd");
-
@GetMapping("schema")
public Object schemaMetrics(@PathVariable("graphspace") String graphSpace,
@PathVariable("graph") String graph) {
HugeClient client = this.authClient(graphSpace, graph);
- String today = SAFE_DATE_FORMAT.format(DateUtil.now());
+ String today = HubbleUtil.dateFormatDay(new Date());
// 获取当前日期前14天的记录
List<String> dateRange = getRangeDate(today, -13);
@@ -163,7 +161,7 @@ public class GraphMetricsController extends BaseController {
private PerEvEntity calGrowthRates(HugeClient client, String type,
String schema) {
- String today = SAFE_DATE_FORMAT.format(DateUtil.now());
+ String today = HubbleUtil.dateFormatDay(new Date());
List<String> last4Days = getRangeDate(today, -3);
GraphMetricsAPI.TypeCounts last3TypeCount =
client.graph().getTypeCounts(last4Days.get(0), last4Days.get(2));
@@ -264,39 +262,38 @@ public class GraphMetricsController extends BaseController {
map = Collections.emptyMap();
}
- SchemaCount<Long> s1 = new SchemaCount<>(0L, 0L, 0L, 0L);
- SchemaCount<Long> s2 = new SchemaCount<>(0L, 0L, 0L, 0L);
- // 统计近一周的前一周
- for (int i = 0; i < 7; i++) {
- GraphMetricsAPI.TypeCount typeCount = map.get(dateRange.get(i));
- if (typeCount == null) {
- continue;
- }
- s1.pk = s1.pk + typeCount.getSchemas().get("pk_count");
- s1.vl = s1.vl + typeCount.getSchemas().get("vl_count");
- s1.el = s1.el + typeCount.getSchemas().get("el_count");
- s1.il = s1.il + typeCount.getSchemas().get("il_count");
- }
+ SchemaCount<Long> previous = sumSchemaCounts(map, dateRange.subList(0, 7));
+ SchemaCount<Long> current = sumSchemaCounts(map, dateRange.subList(7, 13));
+ current.pk += todaySchema.pk;
+ current.vl += todaySchema.vl;
+ current.el += todaySchema.el;
+ current.il += todaySchema.il;
+ return new SchemaCount<>(growthRate(previous.pk, current.pk),
+ growthRate(previous.vl, current.vl),
+ growthRate(previous.el, current.el),
+ growthRate(previous.il, current.il));
+ }
- // 近一周：取最近6天的数据 + 当天数据用实时数据
- s2.pk = s2.pk + todaySchema.pk;
- s2.vl = s2.vl + todaySchema.vl;
- s2.el = s2.el + todaySchema.el;
- s2.il = s2.il + todaySchema.il;
- for (int i = 7; i < 13; i++) {
- GraphMetricsAPI.TypeCount typeCount = map.get(dateRange.get(i));
- if (typeCount == null) {
+ private static SchemaCount<Long> sumSchemaCounts(
+ Map<String, GraphMetricsAPI.TypeCount> counts, List<String> dates) {
+ SchemaCount<Long> total = new SchemaCount<>(0L, 0L, 0L, 0L);
+ for (String date : dates) {
+ GraphMetricsAPI.TypeCount count = counts.get(date);
+ if (count == null || count.getSchemas() == null) {
continue;
}
- s2.pk = s2.pk + typeCount.getSchemas().get("pk_count");
- s2.vl = s2.vl + typeCount.getSchemas().get("vl_count");
- s2.el = s2.el + typeCount.getSchemas().get("el_count");
- s2.il = s2.il + typeCount.getSchemas().get("il_count");
+ Map<String, Long> schemas = count.getSchemas();
+ total.pk += schemaCount(schemas, "pk_count");
+ total.vl += schemaCount(schemas, "vl_count");
+ total.el += schemaCount(schemas, "el_count");
+ total.il += schemaCount(schemas, "il_count");
}
- return new SchemaCount<>(growthRate(s1.pk, s2.pk),
- growthRate(s1.vl, s2.vl),
- growthRate(s1.el, s2.el),
- growthRate(s1.il, s2.il));
+ return total;
+ }
+
+ private static long schemaCount(Map<String, Long> schemas, String key) {
+ Long count = schemas.get(key);
+ return count == null ? 0L : count;
}
/**
@@ -317,32 +314,18 @@ public class GraphMetricsController extends BaseController {
if (num2 == 0) {
return -1.0;
}
- // 保留两位小数
- DecimalFormat decimalFormat = new DecimalFormat("#.##");
- String formattedResult =
- decimalFormat.format((num2 - num1) / ((double) num1));
- return Double.parseDouble(formattedResult);
+ return BigDecimal.valueOf(num2).subtract(BigDecimal.valueOf(num1))
+ .divide(BigDecimal.valueOf(num1), 2, RoundingMode.HALF_EVEN)
+ .doubleValue();
}
public static List<String> getRangeDate(String start, int days) {
- List<String> dateList = new ArrayList<>();
- try {
- Calendar calendar = Calendar.getInstance();
- calendar.setTime(SAFE_DATE_FORMAT.parse(start));
-
- for (int i = 0; i <= Math.abs(days); i++) {
- dateList.add(SAFE_DATE_FORMAT.format(calendar.getTime()));
- if (days > 0) {
- calendar.add(Calendar.DAY_OF_MONTH, 1);
- } else {
- calendar.add(Calendar.DAY_OF_MONTH, -1);
- }
- }
-
- } catch (Exception e) {
- LOG.error("Failed to parse graph metrics date range", e);
- }
- return dateList.stream().sorted().collect(Collectors.toList());
+ LocalDate date = LocalDate.parse(start, DateTimeFormatter.BASIC_ISO_DATE);
+ LocalDate first = days < 0 ? date.plusDays(days) : date;
+ LocalDate last = days > 0 ? date.plusDays(days) : date;
+ return first.datesUntil(last.plusDays(1))
+ .map(DateTimeFormatter.BASIC_ISO_DATE::format)
+ .collect(Collectors.toList());
}
@Data
diff --git a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/controller/saas/GraphSpaceMetricsController.java b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/controller/saas/GraphSpaceMetricsController.java
index 86215255..71174c42 100644
--- a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/controller/saas/GraphSpaceMetricsController.java
+++ b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/controller/saas/GraphSpaceMetricsController.java
@@ -19,12 +19,13 @@
package org.apache.hugegraph.controller.saas;
import java.util.ArrayList;
+import java.util.Date;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
-import org.apache.hugegraph.util.DateUtil;
+import org.apache.hugegraph.util.HubbleUtil;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.format.annotation.DateTimeFormat;
import org.springframework.web.bind.annotation.GetMapping;
@@ -34,8 +35,6 @@ import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import org.apache.hugegraph.api.graph.GraphMetricsAPI;
-// TODO fix import
-//import org.apache.hugegraph.client.api.graph.GraphMetricsAPI;
import org.apache.hugegraph.common.Constant;
import org.apache.hugegraph.controller.BaseController;
import org.apache.hugegraph.driver.HugeClient;
@@ -59,8 +58,6 @@ public class GraphSpaceMetricsController extends BaseController {
HugeClient client = this.authClient(graphSpace, null);
Set<String> graphs = g.listGraphNames(client, graphSpace, "");
- List<GraphMetricsController.EvCountEntity> result = new ArrayList<>();
-
Map<String, GraphMetricsAPI.TypeCount> gsTypeCounts = new HashMap<>();
for (String graph : graphs) {
@@ -85,7 +82,7 @@ public class GraphSpaceMetricsController extends BaseController {
HugeClient client = this.authClient(graphSpace, null);
Set<String> graphs = g.listGraphNames(client, graphSpace, "");
- String today = GraphMetricsController.SAFE_DATE_FORMAT.format(DateUtil.now());
+ String today = HubbleUtil.dateFormatDay(new Date());
// 获取当前日期前14天的记录
List<String> dateRange = GraphMetricsController.getRangeDate(today, -13);
diff --git a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/entity/query/GraphView.java b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/entity/query/GraphView.java
index 802064db..7e359f25 100644
--- a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/entity/query/GraphView.java
+++ b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/entity/query/GraphView.java
@@ -45,7 +45,6 @@ import lombok.NoArgsConstructor;
@Builder
public class GraphView {
- public static final GraphView EMPTY = new GraphView();
private static final int STATISTICS_ROWS = 10;
private static final int TOP_DEGREE_NUMBER = 10;
diff --git a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/entity/query/TableView.java b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/entity/query/TableView.java
index f898c316..f468a91e 100644
--- a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/entity/query/TableView.java
+++ b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/entity/query/TableView.java
@@ -34,7 +34,6 @@ import lombok.NoArgsConstructor;
@Builder
public class TableView {
- public static final TableView EMPTY = new TableView();
public static final List<String> GENERAL_HEADER = ImmutableList.of(
"result"
diff --git a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/handler/CustomApplicationRunner.java b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/handler/CustomApplicationRunner.java
deleted file mode 100644
index d9882293..00000000
--- a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/handler/CustomApplicationRunner.java
+++ /dev/null
@@ -1,51 +0,0 @@
-/*
- *
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with this
- * work for additional information regarding copyright ownership. The ASF
- * licenses this file to You under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
- * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
- * License for the specific language governing permissions and limitations
- * under the License.
- */
-
-package org.apache.hugegraph.handler;
-
-//import org.apache.hugegraph.license.LicenseVerifier; // TODO C Remove Licence
-
-import lombok.extern.log4j.Log4j2;
-import org.apache.hugegraph.config.HugeConfig;
-import org.apache.hugegraph.license.ServerInfo;
-import org.apache.hugegraph.options.HubbleOptions;
-import org.springframework.beans.factory.annotation.Autowired;
-import org.springframework.boot.ApplicationArguments;
-import org.springframework.boot.ApplicationRunner;
-import org.springframework.stereotype.Component;
-
-@Log4j2
-@Component
-public class CustomApplicationRunner implements ApplicationRunner {
-
- @Autowired
- private HugeConfig config;
-
- @Override
- public void run(ApplicationArguments args) throws Exception {
- String serverId = this.config.get(HubbleOptions.SERVER_ID);
- ServerInfo serverInfo = new ServerInfo(serverId);
- log.info("The server info has been inited");
- // this.installLicense(serverInfo,
- // "9662b261c388fc5923ace0ebe2a34b02");
- }
-
- private void installLicense(ServerInfo serverInfo, String md5) {
- //LicenseVerifier.instance().install(serverInfo, md5);// TODO C Remove Licence
- }
-}
diff --git a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/handler/CustomInterceptor.java b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/handler/CustomInterceptor.java
index 4a938e0a..ca4106e5 100644
--- a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/handler/CustomInterceptor.java
+++ b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/handler/CustomInterceptor.java
@@ -18,7 +18,6 @@
package org.apache.hugegraph.handler;
-import java.util.regex.Pattern;
import java.net.URLDecoder;
import java.nio.charset.StandardCharsets;
@@ -26,11 +25,9 @@ import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;
import jakarta.servlet.http.HttpSession;
-//import org.apache.hugegraph.license.LicenseVerifier; // TODO C Remove Licence
import org.apache.hugegraph.service.HugeClientPoolService;
import org.apache.hugegraph.service.auth.AuthModeService;
import org.apache.hugegraph.service.space.GraphSpaceService;
-//import org.apache.hugegraph.service.license.LicenseService;// TODO C Remove Licence
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
import org.springframework.util.StringUtils;
@@ -49,8 +46,6 @@ import lombok.extern.log4j.Log4j2;
@Component
public class CustomInterceptor implements HandlerInterceptor {
- //@Autowired
- //private LicenseService licenseService;// TODO C Remove Licence
@Autowired
protected HugeClientPoolService hugeClientPoolService;
@Autowired
@@ -60,9 +55,6 @@ public class CustomInterceptor implements HandlerInterceptor {
@Autowired
protected GraphSpaceService graphSpaceService;
- private static final Pattern CHECK_API_PATTERN =
- Pattern.compile(".*/graph-connections/\\d+/.+");
-
@Override
public boolean preHandle(HttpServletRequest request,
HttpServletResponse response,
@@ -73,21 +65,6 @@ public class CustomInterceptor implements HandlerInterceptor {
if (url.endsWith("/config")) {
return true;
}
- if (!CHECK_API_PATTERN.matcher(url).matches()) {
- setHugeClientToRequest(request);
- return true;
- }
-
- // String connIdValue = StringUtils.substringBetween(
- // url, "/graph-connections/", "/");
- // if (StringUtils.isEmpty(connIdValue)) {
- // throw new InternalException("Not found conn id in url");
- // }
-
- // int connId = Integer.parseInt(connIdValue);
- // Check graph connection valid
- // this.licenseService.checkGraphStatus(connId);
- //LicenseVerifier.instance().verifyIfNeeded(); // TODO C Remove Licence
setHugeClientToRequest(request);
return true;
}
@@ -116,8 +93,8 @@ public class CustomInterceptor implements HandlerInterceptor {
HttpServletResponse response,
Object handler, Exception exception) {
Object value = request.getAttribute("hugeClient");
- if (value instanceof HugeClient) {
- ((HugeClient) value).close();
+ if (value instanceof HugeClient client) {
+ client.close();
request.removeAttribute("hugeClient");
}
}
diff --git a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/handler/HubbleDisposableBean.java b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/handler/HubbleDisposableBean.java
index beaccc1c..bba10faa 100644
--- a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/handler/HubbleDisposableBean.java
+++ b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/handler/HubbleDisposableBean.java
@@ -27,7 +27,7 @@ import org.springframework.stereotype.Component;
@Log4j2
@Component
-@DependsOn("getAsyncExecutor")
+@DependsOn({"getAsyncExecutor", "maintenanceScheduler"})
public class HubbleDisposableBean implements DisposableBean {
@Autowired
diff --git a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/license/ExtraParam.java b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/license/ExtraParam.java
deleted file mode 100644
index 4095f651..00000000
--- a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/license/ExtraParam.java
+++ /dev/null
@@ -1,132 +0,0 @@
-/*
- *
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with this
- * work for additional information regarding copyright ownership. The ASF
- * licenses this file to You under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
- * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
- * License for the specific language governing permissions and limitations
- * under the License.
- */
-
-package org.apache.hugegraph.license;
-
-import com.fasterxml.jackson.annotation.JsonProperty;
-
-public class ExtraParam {
-
- @JsonProperty("username")
- private String username;
-
- @JsonProperty("license_type")
- private String licenseType;
-
- @JsonProperty("id")
- private String id;
-
- @JsonProperty("version")
- private String version;
-
- @JsonProperty("graphs")
- private int graphs;
-
- @JsonProperty("ip")
- private String ip;
-
- @JsonProperty("mac")
- private String mac;
-
- @JsonProperty("cpus")
- private int cpus;
-
- // The unit is MB
- @JsonProperty("ram")
- private int ram;
-
- @JsonProperty("threads")
- private int threads;
-
- // The unit is MB
- @JsonProperty("memory")
- private int memory;
-
- @JsonProperty("nodes")
- private int nodes;
-
- // The unit is MB
- @JsonProperty("data_size")
- private long dataSize;
-
- @JsonProperty("vertices")
- private long vertices;
-
- @JsonProperty("edges")
- private long edges;
-
- public String username() {
- return this.username;
- }
-
- public String licenseType() {
- return this.licenseType;
- }
-
- public String id() {
- return this.id;
- }
-
- public String version() {
- return this.version;
- }
-
- public int graphs() {
- return this.graphs;
- }
-
- public String ip() {
- return this.ip;
- }
-
- public String mac() {
- return this.mac;
- }
-
- public int cpus() {
- return this.cpus;
- }
-
- public int ram() {
- return this.ram;
- }
-
- public int threads() {
- return this.threads;
- }
-
- public int memory() {
- return this.memory;
- }
-
- public int nodes() {
- return this.nodes;
- }
-
- public long dataSize() {
- return this.dataSize;
- }
-
- public long vertices() {
- return this.vertices;
- }
-
- public long edges() {
- return this.edges;
- }
-}
diff --git a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/license/LicenseVerifyParam.java b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/license/LicenseVerifyParam.java
deleted file mode 100644
index dd109e45..00000000
--- a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/license/LicenseVerifyParam.java
+++ /dev/null
@@ -1,61 +0,0 @@
-/*
- *
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with this
- * work for additional information regarding copyright ownership. The ASF
- * licenses this file to You under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
- * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
- * License for the specific language governing permissions and limitations
- * under the License.
- */
-
-package org.apache.hugegraph.license;
-
-import com.fasterxml.jackson.annotation.JsonAlias;
-import com.fasterxml.jackson.annotation.JsonProperty;
-
-public class LicenseVerifyParam {
-
- @JsonProperty("subject")
- private String subject;
-
- @JsonProperty("public_alias")
- private String publicAlias;
-
- @JsonAlias("store_ticket")
- @JsonProperty("store_password")
- private String storePassword;
-
- @JsonProperty("publickey_path")
- private String publicKeyPath;
-
- @JsonProperty("license_path")
- private String licensePath;
-
- public String subject() {
- return this.subject;
- }
-
- public String publicAlias() {
- return this.publicAlias;
- }
-
- public String storePassword() {
- return this.storePassword;
- }
-
- public String licensePath() {
- return this.licensePath;
- }
-
- public String publicKeyPath() {
- return this.publicKeyPath;
- }
-}
diff --git a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/license/MachineInfo.java b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/license/MachineInfo.java
deleted file mode 100644
index a0fd29e5..00000000
--- a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/license/MachineInfo.java
+++ /dev/null
@@ -1,127 +0,0 @@
-/*
- *
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with this
- * work for additional information regarding copyright ownership. The ASF
- * licenses this file to You under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
- * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
- * License for the specific language governing permissions and limitations
- * under the License.
- */
-
-package org.apache.hugegraph.license;
-
-import java.net.InetAddress;
-import java.net.NetworkInterface;
-import java.net.SocketException;
-import java.util.ArrayList;
-import java.util.Enumeration;
-import java.util.HashSet;
-import java.util.List;
-import java.util.Set;
-import java.util.stream.Collectors;
-
-public class MachineInfo {
-
- private List<String> ipAddressList;
- private List<String> macAddressList;
-
- public MachineInfo() {
- this.ipAddressList = null;
- this.macAddressList = null;
- }
-
- public List<String> getIpAddress() {
- if (this.ipAddressList != null) {
- return this.ipAddressList;
- }
- this.ipAddressList = new ArrayList<>();
- List<InetAddress> inetAddresses = this.getLocalAllInetAddress();
- if (inetAddresses != null && !inetAddresses.isEmpty()) {
- this.ipAddressList = inetAddresses.stream()
- .map(InetAddress::getHostAddress)
- .distinct()
- .map(String::toLowerCase)
- .collect(Collectors.toList());
- }
- return this.ipAddressList;
- }
-
- public List<String> getMacAddress() {
- if (this.macAddressList != null) {
- return this.macAddressList;
- }
- this.macAddressList = new ArrayList<>();
- List<InetAddress> inetAddresses = this.getLocalAllInetAddress();
- if (inetAddresses != null && !inetAddresses.isEmpty()) {
- // Get the Mac address of all network interfaces
- List<String> list = new ArrayList<>();
- Set<String> uniqueValues = new HashSet<>();
- for (InetAddress inetAddress : inetAddresses) {
- String macByInetAddress = this.getMacByInetAddress(inetAddress);
- if (uniqueValues.add(macByInetAddress)) {
- list.add(macByInetAddress);
- }
- }
- this.macAddressList = list;
- }
- return this.macAddressList;
- }
-
- public List<InetAddress> getLocalAllInetAddress() {
- Enumeration<NetworkInterface> interfaces;
- try {
- interfaces = NetworkInterface.getNetworkInterfaces();
- } catch (SocketException e) {
- throw new RuntimeException("Failed to get network interfaces");
- }
-
- List<InetAddress> result = new ArrayList<>();
- while (interfaces.hasMoreElements()) {
- NetworkInterface nw = interfaces.nextElement();
- for (Enumeration<InetAddress> inetAddresses = nw.getInetAddresses();
- inetAddresses.hasMoreElements(); ) {
- InetAddress inetAddr = inetAddresses.nextElement();
- if (!inetAddr.isLoopbackAddress() &&
- !inetAddr.isLinkLocalAddress() &&
- !inetAddr.isMulticastAddress()) {
- result.add(inetAddr);
- }
- }
- }
- return result;
- }
-
- public String getMacByInetAddress(InetAddress inetAddr) {
- byte[] mac;
- try {
- mac = NetworkInterface.getByInetAddress(inetAddr)
- .getHardwareAddress();
- } catch (Exception e) {
- throw new RuntimeException(String.format(
- "Failed to get mac address for inet address '%s'",
- inetAddr));
- }
-
- StringBuilder sb = new StringBuilder();
- for (int i = 0; i < mac.length; i++) {
- if (i != 0) {
- sb.append("-");
- }
- String temp = Integer.toHexString(mac[i] & 0xff);
- if (temp.length() == 1) {
- sb.append("0").append(temp);
- } else {
- sb.append(temp);
- }
- }
- return sb.toString().toUpperCase();
- }
-}
diff --git a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/license/ServerInfo.java b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/license/ServerInfo.java
deleted file mode 100644
index f5779094..00000000
--- a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/license/ServerInfo.java
+++ /dev/null
@@ -1,38 +0,0 @@
-/*
- *
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with this
- * work for additional information regarding copyright ownership. The ASF
- * licenses this file to You under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
- * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
- * License for the specific language governing permissions and limitations
- * under the License.
- */
-
-package org.apache.hugegraph.license;
-
-public final class ServerInfo {
-
- private final String serverId;
- private final MachineInfo machineInfo;
-
- public ServerInfo(String serverId) {
- this.serverId = serverId;
- this.machineInfo = new MachineInfo();
- }
-
- public String serverId() {
- return this.serverId;
- }
-
- public MachineInfo machineInfo() {
- return this.machineInfo;
- }
-}
diff --git a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/options/HubbleOptions.java b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/options/HubbleOptions.java
index 40aa9408..db1a049a 100644
--- a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/options/HubbleOptions.java
+++ b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/options/HubbleOptions.java
@@ -77,14 +77,6 @@ public class HubbleOptions extends OptionHolder {
"data/text2gql-master.key"
);
- public static final ConfigOption<String> SERVER_ID =
- new ConfigOption<>(
- "server.id",
- "The id of hugegraph-hubble server.",
- disallowEmpty(),
- "hubble-1"
- );
-
public static final ConfigOption<String> SERVER_HOST =
new ConfigOption<>(
"server.host",
diff --git a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/service/auth/UserService.java b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/service/auth/UserService.java
index fde0b882..a6872e3d 100644
--- a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/service/auth/UserService.java
+++ b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/service/auth/UserService.java
@@ -18,13 +18,17 @@
package org.apache.hugegraph.service.auth;
+import java.io.BufferedReader;
import java.io.File;
-import java.nio.charset.Charset;
+import java.io.IOException;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
import java.util.ArrayList;
import java.util.Comparator;
import java.util.Collection;
import java.util.HashMap;
import java.util.HashSet;
+import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
@@ -53,7 +57,6 @@ import org.apache.hugegraph.util.HubbleUtil;
import org.apache.hugegraph.util.PageUtil;
import com.csvreader.CsvReader;
-
import org.springframework.web.multipart.MultipartFile;
@Log4j2
@@ -313,9 +316,7 @@ public class UserService extends AuthService {
public String addbatch(HugeClient client, MultipartFile csvFile) {
File file = multipartFileToFile(csvFile);
try {
- Map<String, Object> csv = readCsvByCsvReader(file);
- List<Map<String, String>> createBatchBody =
- HubbleUtil.uncheckedCast(csv.get("data"));
+ List<Map<String, String>> createBatchBody = this.readUserCsv(file);
Map<String, List<Map<String, String>>> result =
client.auth().createBatch(createBatchBody);
List<Map<String, String>> resultList = result.get("result");
@@ -366,44 +367,47 @@ public class UserService extends AuthService {
}
}
- public Map<String, Object> readCsvByCsvReader(File file) {
+ public List<Map<String, String>> readUserCsv(File file) {
if (file == null) {
throw new InternalException("auth.user.import-file.failed");
}
- Map<String, Object> mapData = new HashMap<>();
- String fileName = file.getName();
- mapData.put("sheetName", fileName);
-
- List<String> strList = new ArrayList<>();
- List<Map<String, Object>> list = new ArrayList<>();
- try {
- List<String[]> arrList = new ArrayList<String[]>();
- CsvReader reader = new CsvReader(file.getPath(), ',', Charset.forName("UTF-8"));
- // 读取表头
- reader.readHeaders();
- String[] headArray = reader.getHeaders();
- while (reader.readRecord()) {
- // 按行读取，并把每一行的数据添加到list集合
- arrList.add(reader.getValues());
- }
- reader.close();
- // 如果要返回 String[] 类型的 list 集合，则直接返回 arrList
- // 以下步骤是把 String[] 类型的 list 集合转化为 String 类型的 list 集合
- for (int i = 0; i < arrList.size(); i++) {
- // 组装String字符串
- // 如果不知道有多少列，则可再加一个循环
- Map<String, Object> map = new HashMap<>();
- for (int j = 0; j < arrList.get(0).length; j++) {
- map.put("" + headArray[j] + "", arrList.get(i)[j]);
+ try (BufferedReader reader = Files.newBufferedReader(file.toPath(),
+ StandardCharsets.UTF_8)) {
+ reader.mark(1);
+ if (reader.read() != '\uFEFF') {
+ reader.reset();
+ }
+ // Preserve the distinction between blank lines and quoted empty records.
+ CsvReader csv = new CsvReader(reader, ',');
+ if (!csv.readHeaders()) {
+ throw new InternalException("auth.user.import-csv.failed");
+ }
+ String[] headers = csv.getHeaders();
+ Set<String> names = new HashSet<>();
+ for (String header : headers) {
+ if (header.isBlank() || !names.add(header)) {
+ throw new InternalException("auth.user.import-csv.failed");
}
- list.add(map);
}
- } catch (Exception e) {
- log.error("Failed to parse the user import CSV file", e);
- throw new InternalException("auth.user.import-csv.failed", e);
+ List<Map<String, String>> users = new ArrayList<>();
+ while (csv.readRecord()) {
+ String[] values = csv.getValues();
+ if (values.length != headers.length) {
+ throw new InternalException("auth.user.import-csv.failed");
+ }
+ Map<String, String> user = new LinkedHashMap<>();
+ for (int column = 0; column < headers.length; column++) {
+ user.put(headers[column], values[column]);
+ }
+ users.add(user);
+ }
+ return users;
+ } catch (IOException | RuntimeException e) {
+ // Parser exceptions can contain CSV cells, including passwords.
+ log.warn("Failed to parse the user import CSV file ({})",
+ e.getClass().getSimpleName());
+ throw new InternalException("auth.user.import-csv.failed");
}
- mapData.put("data", list);
- return mapData;
}
public void delete(HugeClient hugeClient, String userId) {
diff --git a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/service/load/FileMappingService.java b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/service/load/FileMappingService.java
index 5ae13458..fdf935ee 100644
--- a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/service/load/FileMappingService.java
+++ b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/service/load/FileMappingService.java
@@ -18,12 +18,9 @@
package org.apache.hugegraph.service.load;
-import java.io.BufferedReader;
import java.io.File;
import java.io.FileInputStream;
-import java.io.FileNotFoundException;
import java.io.FileOutputStream;
-import java.io.FileReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
@@ -41,14 +38,12 @@ import java.util.Map;
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.locks.ReadWriteLock;
-import java.util.regex.Pattern;
import java.util.stream.Collectors;
import org.apache.commons.io.FileUtils;
import org.apache.commons.io.IOUtils;
import org.apache.commons.lang3.StringUtils;
import org.springframework.beans.factory.annotation.Autowired;
-import org.springframework.scheduling.annotation.Async;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Isolation;
@@ -65,10 +60,8 @@ import org.apache.hugegraph.exception.InternalException;
import org.apache.hugegraph.mapper.load.FileMappingMapper;
import org.apache.hugegraph.mapper.load.JobManagerMapper;
import org.apache.hugegraph.options.HubbleOptions;
-import org.apache.hugegraph.util.Ex;
import org.apache.hugegraph.util.HubbleUtil;
import org.apache.hugegraph.util.PageUtil;
-import org.apache.hugegraph.util.StringUtil;
import com.baomidou.mybatisplus.core.conditions.query.QueryWrapper;
import com.baomidou.mybatisplus.core.metadata.IPage;
import com.baomidou.mybatisplus.core.toolkit.Wrappers;
@@ -335,57 +328,10 @@ public class FileMappingService {
public void extractColumns(FileMapping mapping) {
File file = this.requirePathUnderUploadRoot(mapping.getPath());
- BufferedReader reader;
- try {
- reader = new BufferedReader(new FileReader(file));
- } catch (FileNotFoundException e) {
- throw new InternalException("The file '%s' is not found", file);
- }
FileSetting setting = mapping.getFileSetting();
- String delimiter = setting.getDelimiter();
- Pattern pattern = Pattern.compile(setting.getSkippedLine());
-
- String[] columnNames;
- String[] columnValues;
- try {
- String line;
- while ((line = reader.readLine()) != null) {
- if (!pattern.matcher(line).matches()) {
- break;
- }
- }
- Ex.check(line != null,
- "The file has no data line can treat as header");
-
- String[] firstLine = StringUtils.split(line, delimiter);
- if (setting.isHasHeader()) {
- // The first line as column names
- columnNames = firstLine;
- // The second line as column values
- line = reader.readLine();
- columnValues = StringUtil.split(line, delimiter);
- } else {
- // Let columns names as: column-1, column-2 ...
- columnNames = new String[firstLine.length];
- for (int i = 1; i <= firstLine.length; i++) {
- columnNames[i - 1] = "col-" + i;
- }
- // The first line as column values
- columnValues = firstLine;
- }
- } catch (IOException e) {
- throw new InternalException("Failed to read header and sample " +
- "data from file '%s'", file);
- } finally {
- try {
- reader.close();
- } catch (IOException ignored) {
- log.debug("Failed to close file mapping reader", ignored);
- }
- }
-
- setting.setColumnNames(Arrays.asList(columnNames));
- setting.setColumnValues(Arrays.asList(columnValues));
+ FilePreviewReader.Columns columns = FilePreviewReader.read(file.toPath(), setting);
+ setting.setColumnNames(columns.names());
+ setting.setColumnValues(columns.values());
}
public String moveToNextLevelDir(FileMapping mapping) {
@@ -454,14 +400,12 @@ public class FileMappingService {
}
}
- @Async
- @Scheduled(fixedRate = 10 * 60 * 1000)
+ @Scheduled(fixedRate = 10 * 60 * 1000, scheduler = "maintenanceScheduler")
public void deleteOrphanedJobFiles() {
this.cleanupMappings(this.listOrphanedJobFiles());
}
- @Async
- @Scheduled(fixedRate = 10 * 60 * 1000)
+ @Scheduled(fixedRate = 10 * 60 * 1000, scheduler = "maintenanceScheduler")
public void deleteUnfinishedFile() {
QueryWrapper<FileMapping> query = Wrappers.query();
query.in("file_status", FileMappingStatus.UPLOADING.getValue(),
diff --git a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/service/load/FilePreviewReader.java b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/service/load/FilePreviewReader.java
new file mode 100644
index 00000000..b9cd391d
--- /dev/null
+++ b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/service/load/FilePreviewReader.java
@@ -0,0 +1,125 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hugegraph.service.load;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.InputStreamReader;
+import java.nio.charset.Charset;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.regex.Pattern;
+
+import org.apache.hugegraph.entity.load.FileSetting;
+import org.apache.hugegraph.exception.InternalException;
+import org.apache.hugegraph.loader.parser.CsvLineParser;
+import org.apache.hugegraph.loader.parser.LineParser;
+import org.apache.hugegraph.loader.parser.TextLineParser;
+import org.apache.hugegraph.loader.source.file.FileFormat;
+import org.apache.hugegraph.util.Ex;
+
+import com.fasterxml.jackson.core.JsonParser;
+import com.fasterxml.jackson.core.type.TypeReference;
+import com.fasterxml.jackson.databind.ObjectMapper;
+
+/**
+ * Reads a header and one physical data line using the loader's delimiter semantics.
+ */
+public final class FilePreviewReader {
+
+ private static final ObjectMapper JSON_MAPPER =
+ new ObjectMapper().disable(JsonParser.Feature.INCLUDE_SOURCE_IN_LOCATION);
+
+ private FilePreviewReader() {
+ }
+
+ public static Columns read(Path file, FileSetting setting) {
+ Charset charset = Charset.forName(setting.getCharset());
+ Pattern skippedLine = Pattern.compile(setting.getSkippedLine());
+ FileFormat format = FileFormat.valueOf(setting.getFormat());
+ // Upload previews run before users can select the encoding. Preserve undecodable
+ // bytes as replacement characters so the file settings can still be corrected.
+ try (BufferedReader reader = new BufferedReader(
+ new InputStreamReader(Files.newInputStream(file), charset))) {
+ String firstLine = nextDataLine(reader, skippedLine);
+ Ex.check(firstLine != null, "The file has no data line can treat as header");
+ return switch (format) {
+ case CSV -> readDelimited(reader, firstLine, skippedLine,
+ new CsvLineParser(), setting.isHasHeader());
+ case TEXT -> readDelimited(reader, firstLine, skippedLine,
+ new TextLineParser(setting.getDelimiter()), setting.isHasHeader());
+ case JSON -> readJson(firstLine, file);
+ };
+ } catch (IOException e) {
+ throw new InternalException("Failed to read columns from file %s", e, file);
+ }
+ }
+
+ private static Columns readDelimited(BufferedReader reader, String firstLine,
+ Pattern skippedLine, LineParser parser,
+ boolean hasHeader) throws IOException {
+ List<String> firstValues = Arrays.asList(parser.split(firstLine));
+ if (hasHeader) {
+ String sample = nextDataLine(reader, skippedLine);
+ List<String> values = sample == null ? List.of() : Arrays.asList(parser.split(sample));
+ return new Columns(firstValues, values);
+ }
+ List<String> names = new ArrayList<>(firstValues.size());
+ for (int i = 1; i <= firstValues.size(); i++) {
+ names.add("col-" + i);
+ }
+ return new Columns(names, firstValues);
+ }
+
+ private static String nextDataLine(BufferedReader reader, Pattern skippedLine) throws IOException {
+ String line;
+ while ((line = reader.readLine()) != null) {
+ if (!line.isEmpty() && !skippedLine.matcher(line).matches()) {
+ return line;
+ }
+ }
+ return null;
+ }
+
+ private static Columns readJson(String line, Path file) {
+ try {
+ Map<String, Object> fields = JSON_MAPPER.readValue(
+ line, new TypeReference<LinkedHashMap<String, Object>>() {
+ });
+ List<String> names = new ArrayList<>(fields.keySet());
+ List<String> values = names.stream().map(fields::get).map(String::valueOf).toList();
+ return new Columns(names, values);
+ } catch (IOException ignored) {
+ // Parser errors can contain imported values; keep them out of responses and logs.
+ throw new InternalException("Failed to read JSON fields from file %s", file);
+ }
+ }
+
+ public record Columns(List<String> names, List<String> values) {
+
+ public Columns {
+ names = List.copyOf(names);
+ values = List.copyOf(values);
+ }
+ }
+}
diff --git a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/service/load/LoadTaskService.java b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/service/load/LoadTaskService.java
index bc2a1796..9c335d84 100644
--- a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/service/load/LoadTaskService.java
+++ b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/service/load/LoadTaskService.java
@@ -62,7 +62,6 @@ import org.apache.hugegraph.util.Ex;
import org.apache.hugegraph.util.HubbleUtil;
import org.apache.hugegraph.util.PageUtil;
import org.springframework.beans.factory.annotation.Autowired;
-import org.springframework.scheduling.annotation.Async;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Service;
import org.springframework.transaction.PlatformTransactionManager;
@@ -1174,8 +1173,7 @@ public class LoadTaskService {
/**
* Update progress periodically
*/
- @Async
- @Scheduled(fixedDelay = 1 * 1000)
+ @Scheduled(fixedDelay = 1 * 1000, scheduler = "maintenanceScheduler")
@Transactional(isolation = Isolation.READ_COMMITTED)
public void updateLoadTaskProgress() {
for (LoadTask task : this.runningTaskContainer.values()) {
diff --git a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/service/query/ExecuteHistoryService.java b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/service/query/ExecuteHistoryService.java
index 1af1a9bb..45ae7322 100644
--- a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/service/query/ExecuteHistoryService.java
+++ b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/service/query/ExecuteHistoryService.java
@@ -24,7 +24,6 @@ import java.time.temporal.ChronoField;
import org.apache.hugegraph.util.Ex;
import com.google.common.collect.ImmutableMap;
import org.springframework.beans.factory.annotation.Autowired;
-import org.springframework.scheduling.annotation.Async;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Isolation;
@@ -183,8 +182,7 @@ public class ExecuteHistoryService {
}
}
- @Async
- @Scheduled(fixedDelay = 24 * 60 * 60 * 1000)
+ @Scheduled(fixedDelay = 24 * 60 * 60 * 1000, scheduler = "maintenanceScheduler")
public void removeExceedLimit() {
// Do not wrap the complete cleanup in one transaction. Without an
// outer transaction, each bounded mapper DELETE commits independently.
diff --git a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/service/query/QueryService.java b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/service/query/QueryService.java
index 0def0ab4..3fa678fe 100644
--- a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/service/query/QueryService.java
+++ b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/service/query/QueryService.java
@@ -24,8 +24,6 @@ import com.google.common.collect.Iterables;
import lombok.extern.log4j.Log4j2;
import org.apache.commons.lang3.StringUtils;
import org.apache.hugegraph.api.gremlin.GremlinRequest;
-// TODO fix import
-//import org.apache.hugegraph.client.api.gremlin.GremlinRequest;
import org.apache.hugegraph.config.HugeConfig;
import org.apache.hugegraph.driver.HugeClient;
import org.apache.hugegraph.entity.query.AdjacentQuery;
@@ -115,20 +113,7 @@ public class QueryService {
log.debug("Gremlin query optimized");
// Execute gremlin query
ResultSet resultSet = this.executeGremlin(gremlin, client);
- // Scan data, vote the result type
- TypedResult typedResult = this.parseResults(resultSet);
- // Build json view
- JsonView jsonView = new JsonView(typedResult.getData());
- // Build table view
- TableView tableView = this.buildTableView(typedResult);
- // Build graph view
- GraphView graphView = this.buildGraphView(typedResult, client);
- return GremlinResult.builder()
- .type(typedResult.getType())
- .jsonView(jsonView)
- .tableView(tableView)
- .graphView(graphView)
- .build();
+ return this.buildResult(resultSet, client);
}
public JsonView executeSingleGremlinQuery(HugeClient client, GremlinQuery query) {
@@ -148,20 +133,17 @@ public class QueryService {
public GremlinResult executeCypherQuery(HugeClient client, String query) {
// Execute cypher query
ResultSet resultSet = this.executeCypher(query, client);
- // Scan data, vote the result type
+ return this.buildResult(resultSet, client);
+ }
+
+ private GremlinResult buildResult(ResultSet resultSet, HugeClient client) {
TypedResult typedResult = this.parseResults(resultSet);
- // Build json view
- JsonView jsonView = new JsonView(typedResult.getData());
- // Build table view
- TableView tableView = this.buildTableView(typedResult);
- // Build graph view
- GraphView graphView = this.buildGraphView(typedResult, client);
return GremlinResult.builder()
- .type(typedResult.getType())
- .jsonView(jsonView)
- .tableView(tableView)
- .graphView(graphView)
- .build();
+ .type(typedResult.getType())
+ .jsonView(new JsonView(typedResult.getData()))
+ .tableView(this.buildTableView(typedResult))
+ .graphView(this.buildGraphView(typedResult, client))
+ .build();
}
private ResultSet executeCypher(String cypher, HugeClient client) {
@@ -337,9 +319,8 @@ public class QueryService {
continue;
}
Object object = result.getObject();
- if (object instanceof Map) {
- List<Object> elements =
- unwrapCypherElements((Map<?, ?>) object);
+ if (object instanceof Map<?, ?> row) {
+ List<Object> elements = unwrapCypherElements(row);
if (elements != null) {
for (Object element : elements) {
Type elementType = element instanceof Vertex ?
@@ -380,61 +361,42 @@ public class QueryService {
private TableView buildTableView(TypedResult typedResult) {
List<Object> data = typedResult.getData();
if (CollectionUtils.isEmpty(data)) {
- return TableView.EMPTY;
+ return new TableView();
}
- switch (typedResult.getType()) {
- case EMPTY:
- return TableView.EMPTY;
- case GENERAL:
- // result
- List<Object> results = new ArrayList<>(data.size());
- data.forEach(object -> {
- results.add(ImmutableMap.of("result", object));
- });
- return new TableView(TableView.GENERAL_HEADER, results);
- case VERTEX:
- // id, label, properties
- List<Object> vertices = new ArrayList<>(data.size());
- data.forEach(object -> {
- if (object instanceof Vertex) {
- vertices.add(object);
- }
- });
- return new TableView(TableView.VERTEX_HEADER, vertices);
- case EDGE:
- // id, label, source, target, properties
- List<Object> edges = new ArrayList<>(data.size());
- data.forEach(object -> {
- if (object instanceof Edge) {
- edges.add(object);
- }
- });
- return new TableView(TableView.EDGE_HEADER, edges);
- case PATH:
- // path, only fill vertex/edge id
- List<Object> paths = new ArrayList<>(data.size());
- data.forEach(object -> {
- if (object instanceof Path) {
- Path path = (Path) object;
- List<Object> ids = new ArrayList<>();
- path.objects().forEach(element -> {
- if (element instanceof Vertex) {
- ids.add(((Vertex) element).id());
- } else if (element instanceof Edge) {
- ids.add(((Edge) element).id());
- } else {
- ids.add(element);
- }
- });
- paths.add(ImmutableMap.of("path", ids));
+ return switch (typedResult.getType()) {
+ case EMPTY -> new TableView();
+ case GENERAL -> new TableView(TableView.GENERAL_HEADER,
+ data.stream().map(object -> ImmutableMap.of("result", object))
+ .collect(Collectors.toCollection(ArrayList::new)));
+ case VERTEX -> new TableView(TableView.VERTEX_HEADER,
+ data.stream().filter(Vertex.class::isInstance)
+ .collect(Collectors.toCollection(ArrayList::new)));
+ case EDGE -> new TableView(TableView.EDGE_HEADER,
+ data.stream().filter(Edge.class::isInstance)
+ .collect(Collectors.toCollection(ArrayList::new)));
+ case PATH -> this.buildPathTableView(data);
+ };
+ }
+
+ private TableView buildPathTableView(List<Object> data) {
+ List<Object> paths = new ArrayList<>(data.size());
+ for (Object object : data) {
+ if (object instanceof Path path) {
+ List<Object> ids = new ArrayList<>(path.size());
+ for (Object element : path.objects()) {
+ if (element instanceof Vertex vertex) {
+ ids.add(vertex.id());
+ } else if (element instanceof Edge edge) {
+ ids.add(edge.id());
+ } else {
+ ids.add(element);
}
- });
- return new TableView(TableView.PATH_HEADER, paths);
- default:
- throw new AssertionError(String.format(
- "Unknown result type '%s'", typedResult.getType()));
+ }
+ paths.add(ImmutableMap.of("path", ids));
+ }
}
+ return new TableView(TableView.PATH_HEADER, paths);
}
/**
@@ -459,10 +421,9 @@ public class QueryService {
}
private static Object toGraphElement(Object value) {
- if (!(value instanceof Map)) {
+ if (!(value instanceof Map<?, ?> map)) {
return null;
}
- Map<?, ?> map = (Map<?, ?>) value;
Object type = map.get("_type");
if ("node".equals(type)) {
Vertex vertex = new Vertex(String.valueOf(map.get("_label")));
@@ -495,29 +456,24 @@ public class QueryService {
private GraphView buildGraphView(TypedResult result, HugeClient client) {
List<Object> data = result.getData();
if (!result.getType().isGraph() || CollectionUtils.isEmpty(data)) {
- return GraphView.EMPTY;
+ return new GraphView();
}
Map<Object, Vertex> vertices = new HashMap<>();
Map<String, Edge> edges = new HashMap<>();
for (Object object : data) {
- if (object instanceof Vertex) {
- Vertex vertex = (Vertex) object;
+ if (object instanceof Vertex vertex) {
vertices.put(vertex.id(), vertex);
- } else if (object instanceof Edge) {
- Edge edge = (Edge) object;
+ } else if (object instanceof Edge edge) {
edges.put(edge.id(), edge);
- } else if (object instanceof Path) {
- List<Object> elements = ((Path) object).objects();
- for (Object element : elements) {
- if (element instanceof Vertex) {
- Vertex vertex = (Vertex) element;
+ } else if (object instanceof Path path) {
+ for (Object element : path.objects()) {
+ if (element instanceof Vertex vertex) {
vertices.put(vertex.id(), vertex);
- } else if (element instanceof Edge) {
- Edge edge = (Edge) element;
+ } else if (element instanceof Edge edge) {
edges.put(edge.id(), edge);
} else {
- return GraphView.EMPTY;
+ return new GraphView();
}
}
}
diff --git a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/service/space/GraphSpaceService.java b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/service/space/GraphSpaceService.java
index 61ab4429..5ae9660c 100644
--- a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/service/space/GraphSpaceService.java
+++ b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/service/space/GraphSpaceService.java
@@ -44,9 +44,7 @@ import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpStatus;
import org.springframework.stereotype.Service;
-import java.text.ParseException;
-import java.text.SimpleDateFormat;
-import java.util.Date;
+import java.time.LocalDate;
import java.util.Comparator;
import java.util.Collections;
import java.util.HashMap;
@@ -497,17 +495,9 @@ public class GraphSpaceService {
private Map<String, Object> preDayTaskCount(HugeClient client,
String graphSpace) {
LOG.info("Task count in [{}]", graphSpace);
- String lastDay = HubbleUtil.dateFormatLastDay();
- Date start = null;
- Date end = null;
- try {
- SimpleDateFormat simpleDateFormat =
- new SimpleDateFormat("yyyyMMdd HH:mm:ss");
- start = simpleDateFormat.parse(lastDay + " 00:00:00");
- end = simpleDateFormat.parse(lastDay + " 23:59:59");
- } catch (ParseException e) {
- throw new InternalException("parse date error" + e.getMessage());
- }
+ LocalDate lastDay = LocalDate.now(HubbleUtil.TIME_ZONE).minusDays(1);
+ long start = lastDay.atStartOfDay().toInstant(HubbleUtil.TIME_ZONE).toEpochMilli();
+ long end = lastDay.plusDays(1).atStartOfDay().toInstant(HubbleUtil.TIME_ZONE).toEpochMilli();
Map<String, Object> result = new HashMap<>();
Set<String> graphs =
@@ -520,8 +510,7 @@ public class GraphSpaceService {
TasksWithPage pageTask =
client.task().list(null, page, 1000);
for (Task task : pageTask.tasks()) {
- if (task.createTime() >= start.getTime() &&
- task.createTime() <= end.getTime()) {
+ if (task.createTime() >= start && task.createTime() < end) {
preDayTaskCount += 1;
}
}
diff --git a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/util/ESUtil.java b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/util/ESUtil.java
index 0cdb5527..d59326ed 100644
--- a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/util/ESUtil.java
+++ b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/util/ESUtil.java
@@ -20,13 +20,18 @@ package org.apache.hugegraph.util;
import org.apache.commons.lang3.StringUtils;
-import java.time.ZoneId;
import java.time.ZonedDateTime;
import java.time.format.DateTimeFormatter;
import java.util.Arrays;
+import java.util.Locale;
import java.util.Map;
public class ESUtil {
+
+ private static final DateTimeFormatter TIMESTAMP_FORMAT =
+ DateTimeFormatter.ofPattern("uuuu-MM-dd HH:mm:ss", Locale.ROOT)
+ .withZone(HubbleUtil.TIME_ZONE);
+
public static Object getValueByPath(Map<String, Object> map,
String[] keys) {
if (keys.length == 1) {
@@ -43,18 +48,13 @@ public class ESUtil {
}
}
- public static String parseTimestamp(String esTimestatmp) {
- // 转换ES中的@timestamp为当前时区时间戳
- if (StringUtils.isEmpty(esTimestatmp)) {
+ public static String parseTimestamp(String esTimestamp) {
+ if (StringUtils.isEmpty(esTimestamp)) {
return null;
}
ZonedDateTime dateTime =
- ZonedDateTime.parse(esTimestatmp,
+ ZonedDateTime.parse(esTimestamp,
DateTimeFormatter.ISO_DATE_TIME);
- dateTime = dateTime.withZoneSameInstant(ZoneId.systemDefault());
-
- DateTimeFormatter dtf = DateTimeFormatter.ofPattern("yyyy-MM-dd " +
- "HH:mm:ss");
- return dateTime.format(dtf);
+ return TIMESTAMP_FORMAT.format(dateTime);
}
}
diff --git a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/util/HubbleUtil.java b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/util/HubbleUtil.java
index 374f3a5a..29b56029 100644
--- a/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/util/HubbleUtil.java
+++ b/hugegraph-hubble/hubble-be/src/main/java/org/apache/hugegraph/util/HubbleUtil.java
@@ -18,33 +18,40 @@
package org.apache.hugegraph.util;
-import java.text.DateFormat;
-import java.text.ParseException;
-import java.text.SimpleDateFormat;
+import java.time.Clock;
+import java.time.Duration;
import java.time.Instant;
+import java.time.LocalDate;
+import java.time.LocalDateTime;
import java.time.ZoneOffset;
-import java.util.TimeZone;
+import java.time.format.DateTimeFormatter;
+import java.time.format.ResolverStyle;
+import java.util.Collection;
import java.util.Date;
+import java.util.Locale;
import java.util.UUID;
-import java.util.Calendar;
-import java.util.Collection;
import java.util.regex.Pattern;
import org.apache.commons.codec.digest.DigestUtils;
import org.apache.commons.collections.CollectionUtils;
public final class HubbleUtil {
- static {
- TimeZone.setDefault(TimeZone.getTimeZone(ZoneOffset.of("+8")));
- }
+ // Preserve Hubble's business dates without changing the JVM's time zone.
+ public static final ZoneOffset TIME_ZONE = ZoneOffset.ofHours(8);
+ private static final Clock CLOCK = Clock.system(TIME_ZONE);
+ private static final DateTimeFormatter DATE_TIME_FORMAT = formatter("uuuu-MM-dd HH:mm:ss");
+ private static final DateTimeFormatter DAY_FORMAT = formatter("uuuuMMdd");
+ private static final DateTimeFormatter COMPACT_DATE_TIME_FORMAT = formatter("uuuuMMdd HH:mm:ss");
public static final Pattern HOST_PATTERN =
Pattern.compile("(([0-9]{1,3}\\.){3}[0-9]{1,3}|" +
"([0-9A-Za-z_!~*'()-]+\\.)*[0-9A-Za-z_!~*'()-]+)$");
- private static final String DF = "yyyy-MM-dd HH:mm:ss";
- private static final String M_FORMAT = "yyyyMMdd'T'HHmmssSSS";
- public static final DateFormat DATE_FORMAT = new SimpleDateFormat(DF);
+ private static DateTimeFormatter formatter(String pattern) {
+ return DateTimeFormatter.ofPattern(pattern, Locale.ROOT)
+ .withResolverStyle(ResolverStyle.STRICT)
+ .withZone(TIME_ZONE);
+ }
/**
* Isolates casts required by legacy client APIs that return raw JSON
@@ -56,149 +63,39 @@ public final class HubbleUtil {
}
public static String dateFormat() {
- return DATE_FORMAT.format(new Date());
- }
-
- public static String dateFormatMillis() {
- return (new SimpleDateFormat(M_FORMAT)).format(new Date());
- }
-
- public static String dateFormatMonth(Date date) {
- return (new SimpleDateFormat("yyyyMM").format(date));
+ return DATE_TIME_FORMAT.format(CLOCK.instant());
}
public static String dateFormatDay(Date date) {
- return (new SimpleDateFormat("yyyyMMdd").format(date));
+ return DAY_FORMAT.format(date.toInstant());
}
public static String dateFormatDay(String date) {
- SimpleDateFormat sdf = new SimpleDateFormat("yyyyMMdd");
- Date da = null;
- try {
- da = sdf.parse(date);
- } catch (ParseException e) {
- throw new RuntimeException(e);
- }
- return (new SimpleDateFormat("yyyy-MM-dd").format(da));
- }
-
- public static String dateFormatLastMonth() {
- Calendar cal = Calendar.getInstance();
- int month = cal.get(Calendar.MONTH);
-
- cal.set(Calendar.MONTH, month - 1);
- cal.set(Calendar.DAY_OF_MONTH, cal.getActualMaximum(Calendar.DAY_OF_MONTH));
- return (new SimpleDateFormat("yyyyMM").format(cal.getTime()));
+ return LocalDate.parse(date, DAY_FORMAT).format(DateTimeFormatter.ISO_LOCAL_DATE);
}
public static String dateFormatLastDay() {
- Calendar cal = Calendar.getInstance();
- int da = cal.get(Calendar.DATE);
- cal.set(Calendar.DATE, da - 1);
- return (new SimpleDateFormat("yyyyMMdd").format(cal.getTime()));
+ return dateFormatLastDay(CLOCK);
}
- /**
- * 获取当前时间往前推7天的时间戳数组(单位/秒)
- * @return 包含前7天时间戳和当前时间戳的数组
- */
- public static long[] getTimestampsBefore7Days() {
- Calendar cal = Calendar.getInstance();
- long currentTimestamp = cal.getTimeInMillis();
- // 将时间设置为当前时间往前推24小时
- cal.add(Calendar.HOUR_OF_DAY, -24 * 7);
- long timestampBefore24Hours = cal.getTimeInMillis();
- return new long[]{timestampBefore24Hours / 1000,
- currentTimestamp / 1000};
+ static String dateFormatLastDay(Clock clock) {
+ return LocalDate.now(clock.withZone(TIME_ZONE)).minusDays(1).format(DAY_FORMAT);
}
/**
- * 获取当前时间往前推24小时的时间戳数组(单位/秒)
- * @return 包含前24小时时间戳和当前时间戳的数组
+ * Return the previous 24 hours and the current time in epoch seconds.
*/
public static long[] getTimestampsBefore24Hours() {
- Calendar cal = Calendar.getInstance();
- long currentTimestamp = cal.getTimeInMillis();
- // 将时间设置为当前时间往前推24小时
- cal.add(Calendar.HOUR_OF_DAY, -24);
- long timestampBefore24Hours = cal.getTimeInMillis();
- return new long[]{timestampBefore24Hours / 1000,
- currentTimestamp / 1000};
+ return timestampsBefore(CLOCK.instant(), Duration.ofHours(24));
}
public static long[] getTimestampsBefore24Hours(String time) {
- Date date = null;
- try {
- SimpleDateFormat dateFormat = new SimpleDateFormat("yyyyMMdd HH:mm:ss");
- date = dateFormat.parse(time);
- } catch (ParseException e) {
- throw new RuntimeException(e);
- }
- return getTimestampsBefore24Hours(date.getTime());
+ Instant end = LocalDateTime.parse(time, COMPACT_DATE_TIME_FORMAT).toInstant(TIME_ZONE);
+ return timestampsBefore(end, Duration.ofHours(24));
}
- /**
- * 获取timestamp时间往前推24小时的时间戳数组(单位/秒)
- * @param timestamp 给定时间戳(单位/ms)
- * @return 包含前24小时时间戳和当前时间戳的数组
- */
- private static long[] getTimestampsBefore24Hours(long timestamp) {
- Calendar cal = Calendar.getInstance();
- cal.setTimeInMillis(timestamp);
- // 将时间设置为当前时间往前推24小时
- cal.add(Calendar.HOUR_OF_DAY, -24);
- return new long[]{cal.getTimeInMillis() / 1000, timestamp / 1000};
- }
-
- /**
- * 获取给定日期所在周的周一和周日的时间戳数组(单位/秒)
- * @param date 给定日期
- * @return 包含周一和周日时间戳的数组
- */
- public static long[] getWeekTimestamps(Date date) {
- Calendar cal = Calendar.getInstance();
- cal.setTime(date);
- cal.set(Calendar.DAY_OF_WEEK, Calendar.MONDAY);
- resetTime(cal);
- long startOfWeek = cal.getTimeInMillis();
- // 将日期调整到下周一
- cal.add(Calendar.DAY_OF_WEEK, 7);
- resetTime(cal);
- cal.add(Calendar.SECOND, -1);
- long endOfWeek = cal.getTimeInMillis();
- return new long[]{startOfWeek / 1000, endOfWeek / 1000};
- }
-
- /**
- * 获取给定日期所在月份的第一天和最后一天的时间戳数组(单位/秒)
- * @param date 给定日期
- * @return 包含月份第一天和最后一天时间戳的数组
- */
- public static long[] getMonthTimestamps(Date date) {
- Calendar cal = Calendar.getInstance();
- cal.setTime(date);
- // 将日期调整到该月的第一天
- cal.set(Calendar.DAY_OF_MONTH, 1);
- resetTime(cal);
- long startOfMonth = cal.getTimeInMillis();
- // 将日期调整到下个月的第一天
- cal.add(Calendar.MONTH, 1);
- resetTime(cal);
- // 将日期调整到本月的最后一天
- cal.add(Calendar.SECOND, -1);
- // 获取月份的结束时间戳
- long endOfMonth = cal.getTimeInMillis();
- return new long[]{startOfMonth, endOfMonth};
- }
-
- /**
- * 重置时间为当天的开始时间（即00:00:00）
- */
- private static void resetTime(Calendar cal) {
- cal.set(Calendar.HOUR_OF_DAY, 0);
- cal.set(Calendar.MINUTE, 0);
- cal.set(Calendar.SECOND, 0);
- cal.set(Calendar.MILLISECOND, 0);
+ private static long[] timestampsBefore(Instant end, Duration duration) {
+ return new long[]{end.minus(duration).getEpochSecond(), end.getEpochSecond()};
}
public static Date nowDate() {
diff --git a/hugegraph-hubble/hubble-be/src/main/resources/i18n/messages.properties b/hugegraph-hubble/hubble-be/src/main/resources/i18n/messages.properties
index 45200994..915d6473 100644
--- a/hugegraph-hubble/hubble-be/src/main/resources/i18n/messages.properties
+++ b/hugegraph-hubble/hubble-be/src/main/resources/i18n/messages.properties
@@ -239,21 +239,6 @@ execute.predication.error=Execute predication error
reflect.new-instance.failed=Failed to new instance of class {0}
reflect.access-field.failed=Failed to access field {0} of class {1}
-license.verify.graph-connection.failed.preifx=Cannot continue to access
-license.verify.graph-connection.failed.suffix=please delete and try again
-license.install.failed=Failed to install license
-license.public-cert.invalid=Invalid public cert
-license.read.public-cert.failed=Failed to read public cert
-license.read.failed=Failed to read license content
-license.verify.failed=Failed to verify license
-license.verify.server-id.unmatch=The hugegraph-hubble's server id {0} doesn't match the authorized
-license.verify.version.unmatch=The hugegraph-hubble's version {0} exceeded the authorized {1}
-license.verify.graphs.exceed=The hugegraph-hubble's graphs {0} exceeded the authorized {1}
-license.verify.datasize.exceed=The hugegraph-hubble's graph {0} data size {1}MB exceeded the authorized {2}MB
-license.datasize.no-limit=no limit
-license.verify.ip.unauthorized=The hugegraph-hubble's IP {0} doesn't match the authorized {1}
-license.verfiy.mac-unmatch-ip=Failed to get mac address for IP {0}
-license.verify.mac.unauthorized=The hugegraph-hubble's mac {0} doesn't match the authorized {1}
https.load.truststore.error=Failed to load https trusted certificate
diff --git a/hugegraph-hubble/hubble-be/src/main/resources/i18n/messages_zh_CN.properties b/hugegraph-hubble/hubble-be/src/main/resources/i18n/messages_zh_CN.properties
index c4d97b05..b8af9bb1 100644
--- a/hugegraph-hubble/hubble-be/src/main/resources/i18n/messages_zh_CN.properties
+++ b/hugegraph-hubble/hubble-be/src/main/resources/i18n/messages_zh_CN.properties
@@ -239,21 +239,6 @@ execute.predication.error=执行判断语句出错
reflect.new-instance.failed=通过反射构造类 {0} 的实体失败
reflect.access-field.failed=通过反射访问类 {1} 的域 {0} 失败
-license.verify.graph-connection.failed.preifx=部分图无法继续访问
-license.verify.graph-connection.failed.suffix=请删除后重试
-license.install.failed=安装证书失败
-license.public-cert.invalid=非法的公钥文件
-license.read.public-cert.failed=读取公钥信息失败
-license.read.failed=读取证书信息失败
-license.verify.failed=验证证书失败
-license.verify.server-id.unmatch=hugegraph-hubble 服务的 id {0} 不在已授权范围内
-license.verify.version.unmatch=hugegraph-hubble 的当前版本 {0} 不在已授权范围内 <= {1}
-license.verify.graphs.exceed=图连接个数 {0} 超过上限 {1} 个
-license.verify.datasize.exceed=图 {0} 已占用磁盘 {1}MB 超过上限 {2}MB
-license.datasize.no-limit=无限制
-license.verify.ip.unauthorized=hugegraph-hubble 机器的 IP {0} 不在已授权范围内 {1}
-license.verfiy.mac-unmatch-ip=hugegraph-hubble 机器的 MAC 与 IP {0} 不匹配
-license.verify.mac.unauthorized=hugegraph-hubble 机器的 MAC {0} 不在已授权范围内 {1}
https.load.truststore.error=https 可信证书加载失败
diff --git a/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/controller/ingest/IngestControllerTest.java b/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/controller/ingest/IngestControllerTest.java
index d3f765bf..54172eea 100644
--- a/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/controller/ingest/IngestControllerTest.java
+++ b/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/controller/ingest/IngestControllerTest.java
@@ -20,6 +20,7 @@ package org.apache.hugegraph.controller.ingest;
import java.io.File;
import java.lang.reflect.Field;
+import java.nio.charset.Charset;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.Collections;
@@ -167,6 +168,32 @@ public class IngestControllerTest {
response.getData());
}
+ @Test
+ public void testDatasourceSchemaUsesConfiguredEncodingAndCsvQuoting() throws Exception {
+ Path dataFile = Files.createTempFile("hubble-ingest-schema", ".csv");
+ try {
+ Files.writeString(dataFile, "\"姓名,别名\",城市\n李明,北京\n", Charset.forName("GB18030"));
+ IngestController controller = new IngestController();
+ DatasourceService datasourceService = Mockito.mock(DatasourceService.class);
+ FileMappingService fileMappingService = Mockito.mock(FileMappingService.class);
+ this.setField(controller, "datasourceService", datasourceService);
+ this.setField(controller, "fileMappingService", fileMappingService);
+ Datasource datasource = new Datasource();
+ datasource.setDatasourceConfig(Map.of("type", "FILE", "path", dataFile.toString(),
+ "format", "CSV", "charset", "GB18030"));
+ Mockito.when(datasourceService.get(1)).thenReturn(datasource);
+ Mockito.when(fileMappingService.requirePathUnderUploadRoot(dataFile.toString()))
+ .thenReturn(dataFile.toFile());
+
+ Response response = controller.datasourceSchema(1);
+
+ Assert.assertEquals(Constant.STATUS_OK, response.getStatus());
+ Assert.assertEquals(List.of("姓名,别名", "城市"), response.getData());
+ } finally {
+ Files.deleteIfExists(dataFile);
+ }
+ }
+
@Test
public void testDatasourceSchemaUsesJsonObjectKeys()
throws Exception {
diff --git a/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/controller/op/LogControllerTest.java b/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/controller/op/LogControllerTest.java
new file mode 100644
index 00000000..cc3bedb4
--- /dev/null
+++ b/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/controller/op/LogControllerTest.java
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hugegraph.controller.op;
+
+import java.time.Instant;
+import java.util.Date;
+import java.util.List;
+import java.util.TimeZone;
+
+import org.apache.hugegraph.service.op.LogService;
+import org.junit.Assert;
+import org.junit.Test;
+import org.mockito.Mockito;
+import org.springframework.mock.web.MockHttpServletResponse;
+
+public class LogControllerTest {
+
+ @Test
+ public void testExportFilenameUsesBusinessTimezoneWhenJvmUsesUtc() throws Exception {
+ TimeZone original = TimeZone.getDefault();
+ try {
+ TimeZone.setDefault(TimeZone.getTimeZone("UTC"));
+ LogController controller = new LogController();
+ controller.logService = Mockito.mock(LogService.class);
+ LogService.LogReq request = new LogService.LogReq();
+ request.startDatetime = Date.from(Instant.parse("2026-07-12T16:00:00Z"));
+ request.endDatetime = Date.from(Instant.parse("2026-07-12T17:02:03Z"));
+ Mockito.when(controller.logService.export(request)).thenReturn(List.of());
+ MockHttpServletResponse response = new MockHttpServletResponse();
+
+ controller.export(response, request);
+
+ Assert.assertEquals("attachment;filename=log_20260713000000_20260713010203.txt",
+ response.getHeader("Content-Disposition"));
+ } finally {
+ TimeZone.setDefault(original);
+ }
+ }
+
+ @Test
+ public void testExportFilenameRetainsOpenEndedRange() throws Exception {
+ LogController controller = new LogController();
+ controller.logService = Mockito.mock(LogService.class);
+ LogService.LogReq request = new LogService.LogReq();
+ Mockito.when(controller.logService.export(request)).thenReturn(List.of());
+ MockHttpServletResponse response = new MockHttpServletResponse();
+
+ controller.export(response, request);
+
+ Assert.assertEquals("attachment;filename=log_begin_end.txt", response.getHeader("Content-Disposition"));
+ }
+}
diff --git a/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/service/load/FilePreviewReaderTest.java b/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/service/load/FilePreviewReaderTest.java
new file mode 100644
index 00000000..c9e8e332
--- /dev/null
+++ b/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/service/load/FilePreviewReaderTest.java
@@ -0,0 +1,218 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hugegraph.service.load;
+
+import java.nio.charset.Charset;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.List;
+import java.util.regex.PatternSyntaxException;
+
+import org.apache.hugegraph.entity.load.FileMapping;
+import org.apache.hugegraph.entity.load.FileSetting;
+import org.apache.hugegraph.exception.ExternalException;
+import org.apache.hugegraph.exception.InternalException;
+import org.apache.hugegraph.testutil.Assert;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.rules.TemporaryFolder;
+import org.mockito.Mockito;
+
+public class FilePreviewReaderTest {
+
+ @Rule
+ public TemporaryFolder folder = new TemporaryFolder();
+
+ @Test
+ public void testCsvPreservesQuotedAndEmptyCells() throws Exception {
+ FileSetting setting = new FileSetting();
+ setting.setHasHeader(true);
+ Path file = this.write("\"family,name\",nickname,age\n\"Lee, Mei\",,18\n", setting);
+
+ FilePreviewReader.Columns columns = FilePreviewReader.read(file, setting);
+
+ Assert.assertEquals(List.of("family,name", "nickname", "age"), columns.names());
+ Assert.assertEquals(List.of("Lee, Mei", "", "18"), columns.values());
+ }
+
+ @Test
+ public void testCsvWithoutHeaderKeepsLeadingAndTrailingEmptyCells() throws Exception {
+ FileSetting setting = new FileSetting();
+ Path file = this.write(",\"value,with comma\",\n", setting);
+
+ FilePreviewReader.Columns columns = FilePreviewReader.read(file, setting);
+
+ Assert.assertEquals(List.of("col-1", "col-2", "col-3"), columns.names());
+ Assert.assertEquals(List.of("", "value,with comma", ""), columns.values());
+ }
+
+ @Test
+ public void testCsvSkipsEmptyPhysicalLinesBeforeHeaderAndSample() throws Exception {
+ FileSetting setting = new FileSetting();
+ setting.setHasHeader(true);
+ for (String data : List.of("a,b\n\n1,2\n", "\n\na,b\n\n1,2\n")) {
+ FilePreviewReader.Columns columns = FilePreviewReader.read(this.write(data, setting), setting);
+
+ Assert.assertEquals(List.of("a", "b"), columns.names());
+ Assert.assertEquals(List.of("1", "2"), columns.values());
+ }
+ }
+
+ @Test
+ public void testCsvQuotedEmptyFieldRemainsADataLine() throws Exception {
+ FileSetting setting = new FileSetting();
+ Path file = this.write("\n\"\"\nvalue\n", setting);
+
+ FilePreviewReader.Columns columns = FilePreviewReader.read(file, setting);
+
+ Assert.assertEquals(List.of("col-1"), columns.names());
+ Assert.assertEquals(List.of(""), columns.values());
+ }
+
+ @Test
+ public void testTextTreatsMultiCharacterDelimiterAsOneSeparator() throws Exception {
+ FileSetting setting = new FileSetting();
+ setting.setFormat("TEXT");
+ setting.setDelimiter("::");
+ Path file = this.write("a:b::::c\n", setting);
+
+ FilePreviewReader.Columns columns = FilePreviewReader.read(file, setting);
+
+ Assert.assertEquals(List.of("col-1", "col-2", "col-3"), columns.names());
+ Assert.assertEquals(List.of("a:b", "", "c"), columns.values());
+ }
+
+ @Test
+ public void testConfiguredCharsetAppliesToHeaderAndSample() throws Exception {
+ FileSetting setting = new FileSetting();
+ setting.setHasHeader(true);
+ setting.setCharset("GB18030");
+ Path file = this.write("姓名,城市\n李明,北京\n", setting);
+
+ FilePreviewReader.Columns columns = FilePreviewReader.read(file, setting);
+
+ Assert.assertEquals(List.of("姓名", "城市"), columns.names());
+ Assert.assertEquals(List.of("李明", "北京"), columns.values());
+ }
+
+ @Test
+ public void testPreviewAllowsSelectingCharsetAfterInitialUpload() throws Exception {
+ FileSetting setting = new FileSetting();
+ setting.setHasHeader(true);
+ Path file = this.folder.newFile().toPath();
+ Files.writeString(file, "姓名,城市\n李明,北京\n", Charset.forName("GB18030"));
+
+ FilePreviewReader.Columns initial = FilePreviewReader.read(file, setting);
+ Assert.assertTrue(String.join(",", initial.names()).contains("\uFFFD"));
+
+ setting.setCharset("GB18030");
+ FilePreviewReader.Columns corrected = FilePreviewReader.read(file, setting);
+ Assert.assertEquals(List.of("姓名", "城市"), corrected.names());
+ Assert.assertEquals(List.of("李明", "北京"), corrected.values());
+ }
+
+ @Test
+ public void testSkippedLinesAreIgnoredBeforeHeaderAndSample() throws Exception {
+ FileSetting setting = new FileSetting();
+ setting.setHasHeader(true);
+ Path file = this.write("# file description\nname,age\n// sample comment\n# more\nMei,18\n", setting);
+
+ FilePreviewReader.Columns columns = FilePreviewReader.read(file, setting);
+
+ Assert.assertEquals(List.of("name", "age"), columns.names());
+ Assert.assertEquals(List.of("Mei", "18"), columns.values());
+ }
+
+ @Test
+ public void testHeaderOnlyHasEmptySample() throws Exception {
+ FileSetting setting = new FileSetting();
+ setting.setHasHeader(true);
+ Path file = this.write("name,age\n\n# no data yet\n", setting);
+
+ FilePreviewReader.Columns columns = FilePreviewReader.read(file, setting);
+
+ Assert.assertEquals(List.of("name", "age"), columns.names());
+ Assert.assertEquals(List.of(), columns.values());
+ }
+
+ @Test
+ public void testEmptyAndCommentOnlyFilesAreRejected() throws Exception {
+ FileSetting setting = new FileSetting();
+ for (String data : List.of("", "\n\n", "# comment\n// comment\n")) {
+ Path file = this.write(data, setting);
+ Assert.assertThrows(ExternalException.class, () -> FilePreviewReader.read(file, setting));
+ }
+ }
+
+ @Test
+ public void testInvalidSkipPatternIsRejectedBeforeOpeningFile() {
+ FileSetting setting = new FileSetting();
+ setting.setSkippedLine("[");
+ Path missingFile = this.folder.getRoot().toPath().resolve("missing.csv");
+
+ Assert.assertThrows(PatternSyntaxException.class, () -> FilePreviewReader.read(missingFile, setting));
+ }
+
+ @Test
+ public void testJsonRetainsFieldOrderAndValues() throws Exception {
+ FileSetting setting = new FileSetting();
+ setting.setFormat("JSON");
+ Path file = this.write("{\"name\":\"Mei\",\"age\":18,\"nickname\":null}\n", setting);
+
+ FilePreviewReader.Columns columns = FilePreviewReader.read(file, setting);
+
+ Assert.assertEquals(List.of("name", "age", "nickname"), columns.names());
+ Assert.assertEquals(List.of("Mei", "18", "null"), columns.values());
+ }
+
+ @Test
+ public void testMalformedJsonDoesNotExposeImportedValues() throws Exception {
+ FileSetting setting = new FileSetting();
+ setting.setFormat("JSON");
+ Path file = this.write("{\"password\":\"private-preview-canary\"", setting);
+
+ Throwable error = Assert.assertThrows(
+ InternalException.class, () -> FilePreviewReader.read(file, setting));
+
+ Assert.assertFalse(error.toString().contains("private-preview-canary"));
+ Assert.assertNull(error.getCause());
+ }
+
+ @Test
+ public void testFileMappingUsesSharedPreview() throws Exception {
+ FileSetting setting = new FileSetting();
+ setting.setHasHeader(true);
+ setting.setCharset("GB18030");
+ Path file = this.write("\"姓名,别名\",城市\n# comment\n\"李明,小李\",\n", setting);
+ FileMapping mapping = new FileMapping("DEFAULT", "hugegraph", "data.csv", file.toString());
+ mapping.setFileSetting(setting);
+ FileMappingService service = Mockito.spy(new FileMappingService());
+ Mockito.doReturn(file.toFile()).when(service).requirePathUnderUploadRoot(file.toString());
+
+ service.extractColumns(mapping);
+
+ Assert.assertEquals(List.of("姓名,别名", "城市"), setting.getColumnNames());
+ Assert.assertEquals(List.of("李明,小李", ""), setting.getColumnValues());
+ Mockito.verify(service).requirePathUnderUploadRoot(file.toString());
+ }
+
+ private Path write(String data, FileSetting setting) throws Exception {
+ Path file = this.folder.newFile().toPath();
+ return Files.writeString(file, data, Charset.forName(setting.getCharset()));
+ }
+}
diff --git a/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/service/load/IngestTransactionIntegrationTest.java b/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/service/load/IngestTransactionIntegrationTest.java
index 16986aa3..0d656492 100644
--- a/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/service/load/IngestTransactionIntegrationTest.java
+++ b/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/service/load/IngestTransactionIntegrationTest.java
@@ -46,7 +46,7 @@ import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.SpringBootConfiguration;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.test.context.SpringBootTest;
-import org.springframework.boot.test.mock.mockito.MockBean;
+import org.springframework.test.context.bean.override.mockito.MockitoBean;
import org.springframework.context.annotation.Bean;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.test.context.junit4.SpringRunner;
@@ -126,9 +126,9 @@ public class IngestTransactionIntegrationTest {
private HugeConfig config;
@Autowired
private PlatformTransactionManager transactionManager;
- @MockBean
+ @MockitoBean
private VertexLabelService vertexLabelService;
- @MockBean
+ @MockitoBean
private EdgeLabelService edgeLabelService;
private JdbcTemplate jdbc;
diff --git a/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/service/load/JobAdmissionIntegrationTest.java b/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/service/load/JobAdmissionIntegrationTest.java
index 5d9e1334..41b69942 100644
--- a/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/service/load/JobAdmissionIntegrationTest.java
+++ b/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/service/load/JobAdmissionIntegrationTest.java
@@ -52,7 +52,7 @@ import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.SpringBootConfiguration;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.test.context.SpringBootTest;
-import org.springframework.boot.test.mock.mockito.MockBean;
+import org.springframework.test.context.bean.override.mockito.MockitoBean;
import org.springframework.context.annotation.Bean;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.mock.web.MockMultipartFile;
@@ -127,9 +127,9 @@ public class JobAdmissionIntegrationTest {
private ThreadPoolTaskExecutor executor;
@Autowired
private GateLoadTaskExecutor taskExecutor;
- @MockBean
+ @MockitoBean
private VertexLabelService vertexLabels;
- @MockBean
+ @MockitoBean
private EdgeLabelService edgeLabels;
private JdbcTemplate jdbc;
diff --git a/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/service/op/OperationsHttpClientTest.java b/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/service/op/OperationsHttpClientTest.java
index 5b1fd58f..d037374e 100644
--- a/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/service/op/OperationsHttpClientTest.java
+++ b/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/service/op/OperationsHttpClientTest.java
@@ -30,6 +30,8 @@ import java.util.concurrent.atomic.AtomicReference;
import com.sun.net.httpserver.HttpServer;
import org.junit.Test;
+import org.mockito.MockedStatic;
+import org.mockito.Mockito;
import org.apache.hugegraph.testutil.Assert;
@@ -129,15 +131,20 @@ public class OperationsHttpClientTest {
@Test
public void testDnsFailuresKeepOnlySafeRequestPaths() {
OperationsHttpClient client = new OperationsHttpClient(1000, 1000, 64);
- for (String path : new String[]{"/v1/cluster", "/v1/stores", "/actuator/prometheus"}) {
- UpstreamRequestException error = (UpstreamRequestException) Assert.assertThrows(
- UpstreamRequestException.class, () -> client.get(URI.create(
- "http://private-pd-host.invalid:8620" + path + "?token=private-query"),
- "private-user", "private-password"));
- Assert.assertEquals("upstream_unavailable", error.getMessage());
- Assert.assertTrue(error.getCause() instanceof UnknownHostException);
- Assert.assertEquals(Map.of("request_path", path), error.getDiagnostics());
- Assert.assertFalse(error.getDiagnostics().toString().contains("private-"));
+ String host = "private-pd-host.invalid";
+ try (MockedStatic<InetAddress> dns = Mockito.mockStatic(InetAddress.class)) {
+ dns.when(() -> InetAddress.getAllByName(host))
+ .thenThrow(new UnknownHostException(host));
+ for (String path : new String[]{"/v1/cluster", "/v1/stores", "/actuator/prometheus"}) {
+ UpstreamRequestException error = (UpstreamRequestException) Assert.assertThrows(
+ UpstreamRequestException.class, () -> client.get(URI.create(
+ "http://" + host + ":8620" + path + "?token=private-query"),
+ "private-user", "private-password"));
+ Assert.assertEquals("upstream_unavailable", error.getMessage());
+ Assert.assertTrue(error.getCause() instanceof UnknownHostException);
+ Assert.assertEquals(Map.of("request_path", path), error.getDiagnostics());
+ Assert.assertFalse(error.getDiagnostics().toString().contains("private-"));
+ }
}
}
diff --git a/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/service/space/GraphSpaceServiceTest.java b/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/service/space/GraphSpaceServiceTest.java
index 146f18a9..dffa4352 100644
--- a/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/service/space/GraphSpaceServiceTest.java
+++ b/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/service/space/GraphSpaceServiceTest.java
@@ -17,23 +17,30 @@
package org.apache.hugegraph.service.space;
+import java.time.LocalDate;
+import java.time.ZoneOffset;
import java.util.HashMap;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Map;
+import java.util.Set;
+import java.util.TimeZone;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.stream.Collectors;
import java.util.stream.IntStream;
import com.baomidou.mybatisplus.core.metadata.IPage;
+import org.apache.hugegraph.api.task.TasksWithPage;
import org.apache.hugegraph.driver.AuthManager;
import org.apache.hugegraph.driver.GraphSpaceManager;
import org.apache.hugegraph.driver.HugeClient;
+import org.apache.hugegraph.driver.TaskManager;
import org.apache.hugegraph.entity.space.GraphSpaceEntity;
import org.apache.hugegraph.exception.ExternalException;
import org.apache.hugegraph.exception.ServerException;
import org.apache.hugegraph.service.auth.UserService;
import org.apache.hugegraph.service.graphs.GraphsService;
+import org.apache.hugegraph.structure.Task;
import org.apache.hugegraph.structure.space.GraphSpace;
import org.apache.hugegraph.util.PageUtil;
import org.junit.Assert;
@@ -587,6 +594,41 @@ public class GraphSpaceServiceTest {
Assert.assertEquals(Long.valueOf(3L), result.get("eCount"));
}
+ @Test
+ public void testPreviousDayTaskCountUsesBusinessTimezoneAndIncludesLastMillisecond() {
+ TimeZone original = TimeZone.getDefault();
+ try {
+ ZoneOffset businessZone = ZoneOffset.ofHours(8);
+ LocalDate yesterday = LocalDate.now(businessZone).minusDays(1);
+ long start = yesterday.atStartOfDay().toInstant(businessZone).toEpochMilli();
+ long end = yesterday.plusDays(1).atStartOfDay().toInstant(businessZone).toEpochMilli();
+ TaskManager tasks = Mockito.mock(TaskManager.class);
+ TasksWithPage page = Mockito.mock(TasksWithPage.class);
+ Task task = Mockito.mock(Task.class);
+ Mockito.when(this.graphsService.listGraphNames(this.client, "space", ""))
+ .thenReturn(Set.of("graph"));
+ Mockito.when(this.client.task()).thenReturn(tasks);
+ Mockito.when(tasks.list(null, "", 1000)).thenReturn(page);
+ Mockito.when(page.tasks()).thenReturn(List.of(task));
+ Mockito.when(page.page()).thenReturn(null);
+
+ for (String zone : List.of("UTC", "GMT+08:00")) {
+ TimeZone.setDefault(TimeZone.getTimeZone(zone));
+ for (long[] boundary : new long[][]{{start - 1, 0}, {start, 1}, {end - 1, 1}, {end, 0}}) {
+ Mockito.when(task.createTime()).thenReturn(boundary[0]);
+
+ Map<String, Object> result = ReflectionTestUtils.invokeMethod(
+ this.service, "preDayTaskCount", this.client, "space");
+
+ Assert.assertEquals("Task boundary in JVM timezone " + zone,
+ boundary[1], result.get("task"));
+ }
+ }
+ } finally {
+ TimeZone.setDefault(original);
+ }
+ }
+
@Test
public void testMetricsPreservesUnavailableElementCounts() {
GraphSpaceService spy = Mockito.spy(this.service);
diff --git a/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/unit/AsyncConfigLifecycleTest.java b/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/unit/AsyncConfigLifecycleTest.java
index a1b75006..ae464958 100644
--- a/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/unit/AsyncConfigLifecycleTest.java
+++ b/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/unit/AsyncConfigLifecycleTest.java
@@ -35,7 +35,7 @@ import java.util.concurrent.CountDownLatch;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
-import java.util.concurrent.ThreadPoolExecutor;
+import java.util.concurrent.ScheduledThreadPoolExecutor;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicReference;
import java.util.concurrent.TimeUnit;
@@ -66,8 +66,11 @@ import org.springframework.aop.Advisor;
import org.springframework.aop.framework.Advised;
import org.springframework.aop.interceptor.AsyncExecutionAspectSupport;
import org.springframework.context.annotation.AnnotationConfigApplicationContext;
+import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.annotation.Async;
+import org.springframework.scheduling.annotation.EnableScheduling;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;
+import org.springframework.scheduling.concurrent.ThreadPoolTaskScheduler;
import org.springframework.test.util.ReflectionTestUtils;
import org.springframework.util.function.SingletonSupplier;
@@ -171,7 +174,7 @@ public class AsyncConfigLifecycleTest {
private void closeScheduledMetadata(boolean interruptClose) throws Throwable {
AnnotationConfigApplicationContext context = new AnnotationConfigApplicationContext();
DataSourceProperties properties = new DataSourceProperties();
- properties.setUrl("jdbc:h2:mem:shared-async-drain-" + interruptClose);
+ properties.setUrl("jdbc:h2:mem:maintenance-drain-" + interruptClose);
properties.setDriverClassName("org.h2.Driver");
properties.setUsername("sa");
properties.setPassword("");
@@ -206,7 +209,8 @@ public class AsyncConfigLifecycleTest {
writesFinished.countDown();
}
};
- Mockito.when(files.selectList(Mockito.any())).thenAnswer(invocation -> {
+ Mockito.when(files.selectList(null)).thenReturn(Collections.emptyList());
+ Mockito.when(files.selectList(Mockito.isNotNull())).thenAnswer(invocation -> {
metadataWrite.run();
return Collections.emptyList();
});
@@ -225,31 +229,31 @@ public class AsyncConfigLifecycleTest {
context.getBeanFactory().registerSingleton("vertexLabelService", Mockito.mock(VertexLabelService.class));
context.getBeanFactory().registerSingleton("edgeLabelService", Mockito.mock(EdgeLabelService.class));
context.register(HubbleDisposableBean.class, LoadTaskService.class, LoadTaskExecutor.class,
- FileMappingService.class, ExecuteHistoryService.class, AsyncConfig.class);
+ FileMappingService.class, ExecuteHistoryService.class, AsyncConfig.class,
+ SchedulingTestConfig.class);
context.refresh();
ThreadPoolTaskExecutor executor = context.getBean("getAsyncExecutor", ThreadPoolTaskExecutor.class);
+ ThreadPoolTaskScheduler scheduler = context.getBean("maintenanceScheduler", ThreadPoolTaskScheduler.class);
LoadTaskService loadService = context.getBean(LoadTaskService.class);
- ThreadPoolExecutor actualPool = executor.getThreadPoolExecutor();
+ ScheduledThreadPoolExecutor actualPool = scheduler.getScheduledThreadPoolExecutor();
// Delegate every operation to the actual native pool. Only observe entry
// to its blocking await API; a signal before shutdown is insufficient.
- ThreadPoolExecutor observedPool = Mockito.mock(ThreadPoolExecutor.class,
+ ScheduledThreadPoolExecutor observedPool = Mockito.mock(ScheduledThreadPoolExecutor.class,
AdditionalAnswers.delegatesTo(actualPool));
Mockito.doAnswer(invocation -> {
nativeAwaitEntered.countDown();
return actualPool.awaitTermination(invocation.getArgument(0), invocation.getArgument(1));
}).when(observedPool).awaitTermination(Mockito.anyLong(), Mockito.any(TimeUnit.class));
- ReflectionTestUtils.setField(executor, "threadPoolExecutor", observedPool);
- ReflectionTestUtils.setField(executor, "executor", observedPool);
+ ReflectionTestUtils.setField(scheduler, "scheduledExecutor", observedPool);
+ ReflectionTestUtils.setField(scheduler, "executor", observedPool);
ExecutorService closer = Executors.newSingleThreadExecutor();
AtomicReference<Thread> closeThread = new AtomicReference<>();
AtomicBoolean closeInterrupted = new AtomicBoolean();
Future<?> closing = null;
Throwable primary = null;
try {
- // Invoke the actual @Scheduled methods through their real @Async
- // proxies. No timer delays or filesystem deletions are involved.
- context.getBean(FileMappingService.class).deleteUnfinishedFile();
- context.getBean(ExecuteHistoryService.class).removeExceedLimit();
+ // The real @Scheduled registration starts both metadata methods
+ // immediately; the other scheduled methods have no records to process.
assertTrue(writesEntered.await(10, TimeUnit.SECONDS));
closing = closer.submit(() -> {
closeThread.set(Thread.currentThread());
@@ -270,6 +274,7 @@ public class AsyncConfigLifecycleTest {
assertEquals(interruptClose, closeInterrupted.get());
assertEquals(null, writeFailure.get());
assertTrue(actualPool.isTerminated());
+ assertTrue(executor.getThreadPoolExecutor().isTerminated());
assertTrue(datasource.isClosed());
// Check the two physical H2 connections actually used by this
// fixture; Hikari.isClosed alone does not prove physical closure.
@@ -289,8 +294,10 @@ public class AsyncConfigLifecycleTest {
} else {
closing.get(10, TimeUnit.SECONDS);
}
+ scheduler.shutdown();
executor.shutdown();
assertTrue(actualPool.awaitTermination(10, TimeUnit.SECONDS));
+ assertTrue(executor.getThreadPoolExecutor().awaitTermination(10, TimeUnit.SECONDS));
} catch (Throwable failure) {
cleanupFailure = failure;
} finally {
@@ -314,6 +321,11 @@ public class AsyncConfigLifecycleTest {
}
}
+ @Configuration(proxyBeanMethods = false)
+ @EnableScheduling
+ static class SchedulingTestConfig {
+ }
+
private static ThreadPoolTaskExecutor configuredExecutor(AsyncProbe probe) {
for (Advisor advisor : ((Advised) probe).getAdvisors()) {
if (advisor.getAdvice() instanceof AsyncExecutionAspectSupport) {
diff --git a/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/unit/GraphMetricsControllerTest.java b/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/unit/GraphMetricsControllerTest.java
index 34be2371..11aa3005 100644
--- a/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/unit/GraphMetricsControllerTest.java
+++ b/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/unit/GraphMetricsControllerTest.java
@@ -18,18 +18,61 @@
package org.apache.hugegraph.unit;
+import java.time.format.DateTimeParseException;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
+import java.util.HashMap;
+import java.util.Locale;
+import java.util.Map;
import org.junit.Test;
+import org.apache.hugegraph.api.graph.GraphMetricsAPI.TypeCount;
import org.apache.hugegraph.controller.saas.GraphMetricsController;
import org.apache.hugegraph.controller.saas.GraphMetricsController.SchemaCount;
import org.apache.hugegraph.testutil.Assert;
public class GraphMetricsControllerTest {
+ @Test
+ public void testGrowthRatesAreIndependentOfLocaleAndAcceptPartialSchemas() {
+ Locale previous = Locale.getDefault(Locale.Category.FORMAT);
+ try {
+ Locale.setDefault(Locale.Category.FORMAT, Locale.GERMANY);
+ List<String> dates = GraphMetricsController.getRangeDate("20260714", -13);
+ Map<String, TypeCount> counts = new HashMap<>();
+ counts.put(dates.get(0), TypeCount.builder().schemas(
+ Map.of("pk_count", 8L, "vl_count", 4L, "el_count", 3L)).build());
+ counts.put(dates.get(1), new TypeCount());
+ counts.put(dates.get(7), TypeCount.builder().schemas(Map.of("pk_count", 2L)).build());
+ SchemaCount<Double> result = GraphMetricsController.weeklyGrowthRate(
+ counts, dates, new SchemaCount<>(8, 3, 2, 1));
+
+ Assert.assertEquals(0.25, result.pk, 0.0);
+ Assert.assertEquals(-0.25, result.vl, 0.0);
+ Assert.assertEquals(-0.33, result.el, 0.0);
+ Assert.assertEquals(1.0, result.il, 0.0);
+ } finally {
+ Locale.setDefault(Locale.Category.FORMAT, previous);
+ }
+ }
+
+ @Test
+ public void testDateRangesCrossLeapDayAndYearBoundary() {
+ Assert.assertEquals(List.of("20240228", "20240229", "20240301"),
+ GraphMetricsController.getRangeDate("20240301", -2));
+ Assert.assertEquals(List.of("20251231", "20260101", "20260102"),
+ GraphMetricsController.getRangeDate("20251231", 2));
+ Assert.assertEquals(List.of("20260101"),
+ GraphMetricsController.getRangeDate("20260101", 0));
+ }
+
+ @Test(expected = DateTimeParseException.class)
+ public void testInvalidDateIsRejected() {
+ GraphMetricsController.getRangeDate("20260229", -1);
+ }
+
@Test
public void testWeeklyGrowthRateAcceptsSparseTypeCounts() {
List<String> dateRange = Arrays.asList(
diff --git a/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/unit/HubbleUtilTest.java b/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/unit/HubbleUtilTest.java
index 2653dcaf..a6fea640 100644
--- a/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/unit/HubbleUtilTest.java
+++ b/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/unit/HubbleUtilTest.java
@@ -17,12 +17,11 @@
*/
package org.apache.hugegraph.unit;
+import java.time.Instant;
+import java.time.format.DateTimeParseException;
import java.util.Arrays;
-import java.util.Calendar;
import java.util.Collections;
import java.util.Date;
-import java.util.GregorianCalendar;
-import java.util.TimeZone;
import org.apache.hugegraph.testutil.Assert;
import org.apache.hugegraph.util.HubbleUtil;
@@ -54,27 +53,22 @@ public class HubbleUtilTest {
public void testTimestampsBefore24Hours() {
long[] timestamps = HubbleUtil.getTimestampsBefore24Hours("20240115 12:00:00");
- // 2024-01-15 12:00:00 is parsed in the default time zone
- long offset = TimeZone.getDefault().getOffset(1705320000000L) / 1000L;
- long expected = 1705320000L - offset;
+ long expected = Instant.parse("2024-01-15T04:00:00Z").getEpochSecond();
Assert.assertArrayEquals(new long[]{expected - 86400L, expected}, timestamps);
long[] now = HubbleUtil.getTimestampsBefore24Hours();
Assert.assertEquals(86400L, now[1] - now[0]);
- Assert.assertThrows(RuntimeException.class, () -> {
+ Assert.assertThrows(DateTimeParseException.class, () -> {
HubbleUtil.getTimestampsBefore24Hours("2024-01-15 12:00:00");
- }, e -> {
- Assert.assertTrue(e.getCause() instanceof java.text.ParseException);
});
}
@Test
public void testDateFormatDay() {
Assert.assertEquals("2024-01-15", HubbleUtil.dateFormatDay("20240115"));
- Date day = new GregorianCalendar(2024, Calendar.JANUARY, 15, 12, 0).getTime();
+ Date day = Date.from(Instant.parse("2024-01-15T04:00:00Z"));
Assert.assertEquals("20240115", HubbleUtil.dateFormatDay(day));
- Assert.assertEquals("202401", HubbleUtil.dateFormatMonth(day));
Assert.assertThrows(RuntimeException.class, () -> {
HubbleUtil.dateFormatDay("2024/01/15");
});
diff --git a/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/unit/JacksonConfigTest.java b/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/unit/JacksonConfigTest.java
new file mode 100644
index 00000000..0529193b
--- /dev/null
+++ b/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/unit/JacksonConfigTest.java
@@ -0,0 +1,124 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hugegraph.unit;
+
+import java.time.Instant;
+import java.time.LocalDate;
+import java.util.Date;
+import java.util.LinkedHashMap;
+
+import com.fasterxml.jackson.databind.ObjectMapper;
+import org.apache.hugegraph.common.Response;
+import org.apache.hugegraph.config.JacksonConfig;
+import org.apache.hugegraph.entity.graph.VertexQueryEntity;
+import org.apache.hugegraph.structure.graph.Edge;
+import org.apache.hugegraph.structure.graph.Vertex;
+import org.junit.Assert;
+import org.junit.Test;
+import org.springframework.boot.autoconfigure.AutoConfigurations;
+import org.springframework.boot.autoconfigure.jackson.JacksonAutoConfiguration;
+import org.springframework.boot.test.context.runner.ApplicationContextRunner;
+
+public class JacksonConfigTest {
+
+ private static final long LARGE_LONG = 9007199254740993L;
+
+ private final ApplicationContextRunner runner = new ApplicationContextRunner()
+ .withConfiguration(AutoConfigurations.of(JacksonAutoConfiguration.class))
+ .withUserConfiguration(JacksonConfig.class);
+
+ @Test
+ public void testBootMapperPreservesResponseAndVertexContract() {
+ this.runner.run(context -> {
+ Assert.assertNull(context.getStartupFailure());
+ Assert.assertEquals(1, context.getBeansOfType(ObjectMapper.class).size());
+ ObjectMapper mapper = context.getBean(ObjectMapper.class);
+ Assert.assertSame(mapper, context.getBean("jacksonObjectMapper"));
+ Vertex vertex = new Vertex("person");
+ vertex.id(LARGE_LONG);
+ vertex.properties().put("balance", LARGE_LONG);
+ vertex.properties().put("age", 42);
+ Response response = Response.builder().status(200).data(vertex)
+ .message("Success").build();
+
+ Assert.assertEquals(mapper.readTree("""
+ {"status":200,"data":{"id":"9007199254740993","label":"person",
+ "properties":{"balance":"9007199254740993","age":42}},
+ "message":"Success","cause":null}
+ """), mapper.readTree(mapper.writeValueAsString(response)));
+ });
+ }
+
+ @Test
+ public void testQueryVertexPreservesStatisticsAndNullProperties() {
+ this.runner.run(context -> {
+ ObjectMapper mapper = context.getBean(ObjectMapper.class);
+ VertexQueryEntity vertex = new VertexQueryEntity("person");
+ vertex.id("person/1");
+ vertex.setProperties(new LinkedHashMap<>());
+ vertex.properties().put("nickname", null);
+ vertex.getStatistics().put("degree", LARGE_LONG);
+ vertex.getStatistics().put("optional", null);
+
+ Assert.assertEquals(mapper.readTree("""
+ {"id":"person/1","label":"person","properties":{"nickname":null},
+ "statistics":{"degree":"9007199254740993","optional":null}}
+ """), mapper.readTree(mapper.writeValueAsString(vertex)));
+ });
+ }
+
+ @Test
+ public void testEdgesPreserveStringIdsAndLongProperties() {
+ this.runner.run(context -> {
+ ObjectMapper mapper = context.getBean(ObjectMapper.class);
+ Edge edge = new Edge("knows");
+ edge.id("edge-1");
+ edge.sourceId(LARGE_LONG);
+ edge.targetId("person/2");
+ edge.properties().put("weight", LARGE_LONG);
+
+ Assert.assertEquals(mapper.readTree("""
+ {"id":"edge-1","label":"knows","source":"9007199254740993",
+ "target":"person/2","properties":{"weight":"9007199254740993"}}
+ """), mapper.readTree(mapper.writeValueAsString(edge)));
+ edge.id(null);
+ Assert.assertFalse(mapper.readTree(mapper.writeValueAsString(edge)).has("id"));
+ });
+ }
+
+ @Test
+ public void testBootPropertiesAndJavaTimeModuleRemainAvailable() {
+ this.runner.withPropertyValues(
+ "spring.jackson.property-naming-strategy=SNAKE_CASE",
+ "spring.jackson.default-property-inclusion=non_null",
+ "spring.jackson.date-format=yyyy-MM-dd HH:mm:ss",
+ "spring.jackson.time-zone=GMT+8")
+ .run(context -> {
+ ObjectMapper mapper = context.getBean(ObjectMapper.class);
+ TimeValue value = new TimeValue(LocalDate.of(2026, 10, 23),
+ Date.from(Instant.parse("2026-01-01T16:05:06Z")), null);
+
+ Assert.assertEquals("{\"creation_date\":\"2026-10-23\",\"update_time\":\"2026-01-02 00:05:06\"}",
+ mapper.writeValueAsString(value));
+ });
+ }
+
+ private record TimeValue(LocalDate creationDate, Date updateTime, String missingValue) {
+ }
+}
diff --git a/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/unit/MaintenanceSchedulingTest.java b/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/unit/MaintenanceSchedulingTest.java
new file mode 100644
index 00000000..2c2d9aa4
--- /dev/null
+++ b/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/unit/MaintenanceSchedulingTest.java
@@ -0,0 +1,288 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hugegraph.unit;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertTrue;
+
+import java.time.Duration;
+import java.util.Collections;
+import java.util.Map;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.ScheduledFuture;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import org.apache.hugegraph.config.AsyncConfig;
+import org.apache.hugegraph.config.HugeConfig;
+import org.apache.hugegraph.config.Text2GqlStreamConfig;
+import org.apache.hugegraph.entity.enums.LoadStatus;
+import org.apache.hugegraph.entity.load.LoadTask;
+import org.apache.hugegraph.handler.LoadTaskExecutor;
+import org.apache.hugegraph.loader.HugeGraphLoader;
+import org.apache.hugegraph.loader.builder.SchemaCache;
+import org.apache.hugegraph.loader.executor.LoadContext;
+import org.apache.hugegraph.loader.executor.LoadOptions;
+import org.apache.hugegraph.mapper.load.FileMappingMapper;
+import org.apache.hugegraph.mapper.load.JobManagerMapper;
+import org.apache.hugegraph.mapper.load.LoadTaskMapper;
+import org.apache.hugegraph.mapper.query.ExecuteHistoryMapper;
+import org.apache.hugegraph.options.HubbleOptions;
+import org.apache.hugegraph.service.load.FileMappingService;
+import org.apache.hugegraph.service.load.LoadTaskService;
+import org.apache.hugegraph.service.query.ExecuteHistoryService;
+import org.apache.hugegraph.service.schema.EdgeLabelService;
+import org.apache.hugegraph.service.schema.VertexLabelService;
+import org.junit.Test;
+import org.mockito.Mockito;
+import org.springframework.aop.support.AopUtils;
+import org.springframework.beans.factory.SmartInitializingSingleton;
+import org.springframework.context.annotation.AnnotationConfigApplicationContext;
+import org.springframework.context.annotation.Configuration;
+import org.springframework.jdbc.core.JdbcTemplate;
+import org.springframework.jdbc.datasource.DataSourceTransactionManager;
+import org.springframework.scheduling.annotation.EnableScheduling;
+import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;
+import org.springframework.scheduling.concurrent.ThreadPoolTaskScheduler;
+import org.springframework.test.util.ReflectionTestUtils;
+import org.springframework.transaction.annotation.EnableTransactionManagement;
+import org.springframework.transaction.support.TransactionSynchronization;
+import org.springframework.transaction.support.TransactionSynchronizationManager;
+
+import com.zaxxer.hikari.HikariDataSource;
+
+public class MaintenanceSchedulingTest {
+
+ @Test
+ public void testScheduledMaintenanceRunsWhileLoaderWorkersAreOccupied() throws Exception {
+ try (Fixture fixture = new Fixture(false)) {
+ fixture.start();
+
+ await(fixture.metadataWritten, "Progress and history cleanup must run before the loaders finish");
+ assertTrue(fixture.rows("progress") > 0);
+ assertTrue(fixture.rows("history") > 0);
+ assertEquals(8L, fixture.loadersFinished.getCount());
+ assertEquals(8, fixture.loaderExecutor().getActiveCount());
+ assertEquals("Maintenance must not enter the Loader queue", 0,
+ fixture.loaderExecutor().getThreadPoolExecutor().getQueue().size());
+ }
+ }
+
+ @Test
+ public void testSlowMaintenanceDoesNotOverlapOrDelayHeartbeats() throws Exception {
+ try (Fixture fixture = new Fixture(true)) {
+ fixture.start();
+ await(fixture.maintenanceEntered, "All four actual scheduled methods must start independently");
+ ThreadPoolTaskScheduler scheduler = fixture.context.getBean(
+ "maintenanceScheduler", ThreadPoolTaskScheduler.class);
+
+ assertEquals(4, scheduler.getActiveCount());
+ fixture.observeHeartbeats();
+ assertEquals("Blocked periodic tasks must not accumulate subsequent executions", 0,
+ scheduler.getScheduledThreadPoolExecutor().getQueue().size());
+
+ // Leave only progress blocked, with spare maintenance workers available.
+ fixture.releaseCleanup.countDown();
+ await(fixture.cleanupFinished, "Other maintenance methods must finish independently of progress");
+ fixture.observeHeartbeats();
+ assertEquals("A slow progress update must not overlap itself", 1,
+ fixture.calls.get("progress").get());
+ assertEquals("No duplicate progress invocation may wait for the task lock", 1,
+ scheduler.getActiveCount());
+ assertEquals("No maintenance work may accumulate behind the long-running loaders", 0,
+ fixture.loaderExecutor().getThreadPoolExecutor().getQueue().size());
+ }
+ }
+
+ private static void await(CountDownLatch latch, String message) {
+ await(latch, message, 10);
+ }
+
+ private static void await(CountDownLatch latch, String message, int seconds) {
+ try {
+ assertTrue(message, latch.await(seconds, TimeUnit.SECONDS));
+ } catch (InterruptedException e) {
+ Thread.currentThread().interrupt();
+ throw new AssertionError(message, e);
+ }
+ }
+
+ @Configuration
+ @EnableScheduling
+ @EnableTransactionManagement
+ static class Scheduling {
+ }
+
+ private static final class Fixture implements AutoCloseable {
+
+ private final AnnotationConfigApplicationContext context = new AnnotationConfigApplicationContext();
+ private final HikariDataSource datasource = new HikariDataSource();
+ private final JdbcTemplate jdbc = new JdbcTemplate(this.datasource);
+ private final CountDownLatch loadersStarted = new CountDownLatch(8);
+ private final CountDownLatch loadersFinished = new CountDownLatch(8);
+ private final CountDownLatch releaseLoaders = new CountDownLatch(1);
+ private final CountDownLatch maintenanceEntered = new CountDownLatch(4);
+ private final CountDownLatch metadataWritten = new CountDownLatch(2);
+ private final CountDownLatch cleanupFinished = new CountDownLatch(3);
+ private final CountDownLatch releaseProgress;
+ private final CountDownLatch releaseCleanup;
+ private final Map<String, AtomicInteger> calls = new ConcurrentHashMap<>();
+ private LoadContext nativeContext;
+
+ private Fixture(boolean blockMaintenance) {
+ this.releaseProgress = new CountDownLatch(blockMaintenance ? 1 : 0);
+ this.releaseCleanup = new CountDownLatch(blockMaintenance ? 1 : 0);
+ }
+
+ private void start() {
+ this.datasource.setJdbcUrl("jdbc:h2:mem:maintenance-" + UUID.randomUUID());
+ this.datasource.setMaximumPoolSize(4);
+ this.datasource.setMinimumIdle(0);
+ this.jdbc.execute("CREATE TABLE maintenance_probe(kind VARCHAR(24))");
+ this.context.registerBean("dataSource", HikariDataSource.class, () -> this.datasource);
+ this.context.registerBean("transactionManager", DataSourceTransactionManager.class,
+ () -> new DataSourceTransactionManager(this.datasource));
+
+ HugeConfig config = Mockito.mock(HugeConfig.class);
+ Mockito.when(config.get(HubbleOptions.EXECUTE_HISTORY_SHOW_LIMIT)).thenReturn(10);
+ Mockito.when(config.get(HubbleOptions.UPLOAD_FILE_MAX_TIME_CONSUMING)).thenReturn(3600L);
+ LoadTaskMapper loads = Mockito.mock(LoadTaskMapper.class);
+ Mockito.when(loads.updateById(Mockito.any(LoadTask.class))).thenAnswer(invocation -> {
+ this.maintenance("progress");
+ return 1;
+ });
+ ExecuteHistoryMapper histories = Mockito.mock(ExecuteHistoryMapper.class);
+ Mockito.doAnswer(invocation -> {
+ this.maintenance("history");
+ return null;
+ }).when(histories).deleteExceedLimit(Mockito.anyInt());
+ FileMappingMapper files = Mockito.mock(FileMappingMapper.class);
+ Mockito.when(files.selectList(Mockito.any())).thenAnswer(invocation -> {
+ this.maintenance(invocation.getArgument(0) == null ? "orphaned" : "unfinished");
+ return Collections.emptyList();
+ });
+ this.register("hugeConfig", config);
+ this.register("loadTaskMapper", loads);
+ this.register("executeHistoryMapper", histories);
+ this.register("fileMappingMapper", files);
+ this.register("jobManagerMapper", Mockito.mock(JobManagerMapper.class));
+ this.register("vertexLabelService", Mockito.mock(VertexLabelService.class));
+ this.register("edgeLabelService", Mockito.mock(EdgeLabelService.class));
+
+ this.nativeContext = LoadContext.forOffline(new LoadOptions(), new SchemaCache(null));
+ HugeGraphLoader loader = Mockito.mock(HugeGraphLoader.class);
+ Mockito.when(loader.context()).thenReturn(this.nativeContext);
+ LoadTask progress = new LoadTask();
+ progress.setId(991);
+ progress.setStatus(LoadStatus.RUNNING);
+ progress.setLoader(loader);
+ LoadTaskService service = new LoadTaskService();
+ ReflectionTestUtils.setField(service, "runningTaskContainer", Map.of(991, progress));
+ this.context.registerBean(LoadTaskService.class, () -> service);
+ this.context.register(AsyncConfig.class, Text2GqlStreamConfig.class, Scheduling.class,
+ LoadTaskExecutor.class, FileMappingService.class, ExecuteHistoryService.class);
+ // Occupy the real @Async executor before ContextRefreshedEvent starts @Scheduled methods.
+ this.context.registerBean("occupyLoader", SmartInitializingSingleton.class, () -> this::occupyLoader);
+ this.context.refresh();
+ }
+
+ private void register(String name, Object bean) {
+ this.context.getBeanFactory().registerSingleton(name, bean);
+ }
+
+ private void occupyLoader() {
+ LoadTaskExecutor executor = this.context.getBean(LoadTaskExecutor.class);
+ assertTrue("Use the production @Async dispatcher", AopUtils.isAopProxy(executor));
+ for (int i = 0; i < 8; i++) {
+ LoadTask task = new LoadTask() {
+ @Override
+ public void run() {
+ Fixture.this.loadersStarted.countDown();
+ await(Fixture.this.releaseLoaders, "Release the controlled long-running Loader task", 30);
+ }
+ };
+ task.setId(i);
+ executor.execute(task, this.loadersFinished::countDown);
+ }
+ await(this.loadersStarted, "All eight Loader core workers must be occupied");
+ }
+
+ private void maintenance(String kind) {
+ this.calls.computeIfAbsent(kind, ignored -> new AtomicInteger()).incrementAndGet();
+ this.maintenanceEntered.countDown();
+ await("progress".equals(kind) ? this.releaseProgress : this.releaseCleanup,
+ "Release the controlled metadata operation", 30);
+ // Only the Mapper boundary is controlled; the scheduled service body and H2 write are real.
+ this.jdbc.update("INSERT INTO maintenance_probe VALUES (?)", kind);
+ if ("progress".equals(kind)) {
+ assertTrue("The scheduled progress method must retain its transaction",
+ TransactionSynchronizationManager.isActualTransactionActive());
+ TransactionSynchronizationManager.registerSynchronization(new TransactionSynchronization() {
+ @Override
+ public void afterCommit() {
+ Fixture.this.metadataWritten.countDown();
+ }
+ });
+ } else if ("history".equals(kind)) {
+ this.metadataWritten.countDown();
+ }
+ if (!"progress".equals(kind)) {
+ this.cleanupFinished.countDown();
+ }
+ }
+
+ private int rows(String kind) {
+ return this.jdbc.queryForObject("SELECT COUNT(*) FROM maintenance_probe WHERE kind = ?",
+ Integer.class, kind);
+ }
+
+ private ThreadPoolTaskExecutor loaderExecutor() {
+ return this.context.getBean("getAsyncExecutor", ThreadPoolTaskExecutor.class);
+ }
+
+ private void observeHeartbeats() {
+ ThreadPoolTaskScheduler heartbeats = this.context.getBean(
+ "text2GqlHeartbeats", ThreadPoolTaskScheduler.class);
+ CountDownLatch ticks = new CountDownLatch(3);
+ ScheduledFuture<?> heartbeat = heartbeats.scheduleAtFixedRate(ticks::countDown, Duration.ofSeconds(1));
+ try {
+ await(ticks, "The SSE heartbeat scheduler must remain responsive across progress periods");
+ } finally {
+ heartbeat.cancel(false);
+ }
+ }
+
+ @Override
+ public void close() {
+ this.releaseLoaders.countDown();
+ this.releaseProgress.countDown();
+ this.releaseCleanup.countDown();
+ try {
+ this.context.close();
+ } finally {
+ this.datasource.close();
+ if (this.nativeContext != null) {
+ this.nativeContext.close(false);
+ }
+ }
+ }
+ }
+}
diff --git a/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/unit/QueryServiceTest.java b/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/unit/QueryServiceTest.java
index 66c765b6..18883e4c 100644
--- a/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/unit/QueryServiceTest.java
+++ b/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/unit/QueryServiceTest.java
@@ -21,6 +21,10 @@ package org.apache.hugegraph.unit;
import java.lang.reflect.Field;
import java.util.Arrays;
import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+
+import com.fasterxml.jackson.databind.ObjectMapper;
import org.junit.Test;
import org.mockito.Mockito;
@@ -42,6 +46,8 @@ import org.apache.hugegraph.service.query.QueryService;
import org.apache.hugegraph.service.schema.VertexLabelService;
import org.apache.hugegraph.structure.constant.Direction;
import org.apache.hugegraph.structure.constant.IdStrategy;
+import org.apache.hugegraph.structure.graph.Edge;
+import org.apache.hugegraph.structure.graph.Path;
import org.apache.hugegraph.structure.graph.Vertex;
import org.apache.hugegraph.structure.gremlin.ResultSet;
import org.apache.hugegraph.testutil.Assert;
@@ -259,6 +265,125 @@ public class QueryServiceTest {
cypherError.getMessage());
}
+ @Test
+ public void testEmptyResultsAreIsolatedAcrossQueryLanguages() throws Exception {
+ HugeClient client = this.mockClient(this.mockGremlin());
+ CypherManager cypher = Mockito.mock(CypherManager.class);
+ Mockito.when(cypher.cypher(Mockito.anyString())).thenReturn(this.resultSet());
+ Mockito.when(client.cypher()).thenReturn(cypher);
+ QueryService service = this.serviceWithConfig();
+
+ GremlinResult first = service.executeGremlinQuery(client, new GremlinQuery("g.V()"));
+ first.getGraphView().getStatistics().put("stale", true);
+ first.getTableView().setHeader(List.of("stale"));
+ first.getTableView().setRows(List.of("stale"));
+
+ GremlinResult second = service.executeCypherQuery(client, "MATCH (n) RETURN n");
+ GremlinResult third = service.executeGremlinQuery(client, new GremlinQuery("g.V()"));
+ ObjectMapper mapper = new ObjectMapper();
+ for (GremlinResult result : List.of(second, third)) {
+ Assert.assertEquals(GremlinResult.Type.EMPTY, result.getType());
+ Assert.assertNull(result.getJsonView().getData());
+ Assert.assertEquals(mapper.readTree("{\"header\":null,\"rows\":null}"),
+ mapper.valueToTree(result.getTableView()));
+ Assert.assertEquals(mapper.readTree("{\"vertices\":null,\"edges\":null,\"statistics\":{}}"),
+ mapper.valueToTree(result.getGraphView()));
+ Assert.assertNotSame(first.getTableView(), result.getTableView());
+ Assert.assertNotSame(first.getGraphView(), result.getGraphView());
+ }
+ }
+
+ @Test
+ public void testScalarQueriesHaveTheSameResultShape() throws Exception {
+ ResultSet data = this.resultSet(1, "name", Map.of("count", 2));
+ HugeClient client = this.mockClient(this.mockGremlin(data));
+ CypherManager cypher = Mockito.mock(CypherManager.class);
+ Mockito.when(cypher.cypher(Mockito.anyString())).thenReturn(data);
+ Mockito.when(client.cypher()).thenReturn(cypher);
+ QueryService service = this.serviceWithConfig();
+
+ GremlinResult gremlin = service.executeGremlinQuery(client, new GremlinQuery("g.V().count()"));
+ GremlinResult cypherResult = service.executeCypherQuery(client, "RETURN 1");
+
+ Assert.assertEquals(GremlinResult.Type.GENERAL, gremlin.getType());
+ Assert.assertEquals(List.of("result"), gremlin.getTableView().getHeader());
+ Assert.assertEquals(List.of(Map.of("result", 1), Map.of("result", "name"),
+ Map.of("result", Map.of("count", 2))), gremlin.getTableView().getRows());
+ ObjectMapper mapper = new ObjectMapper();
+ Assert.assertEquals(mapper.valueToTree(gremlin), mapper.valueToTree(cypherResult));
+ }
+
+ @Test
+ public void testPathTablePreservesScalarAndNullValues() throws Exception {
+ Vertex vertex = new Vertex("person");
+ vertex.id("marko");
+ Edge edge = new Edge("knows");
+ edge.id("edge1");
+ Path path = new Path(Arrays.asList(vertex, edge, "step", null));
+ HugeClient client = this.mockClient(this.mockGremlin(this.resultSet(path)));
+ QueryService service = this.serviceWithConfig();
+
+ GremlinResult result = service.executeGremlinQuery(client, new GremlinQuery("g.V().path()"));
+
+ Assert.assertEquals(GremlinResult.Type.PATH, result.getType());
+ Assert.assertEquals(List.of("path"), result.getTableView().getHeader());
+ Assert.assertEquals(List.of(Map.of("path", Arrays.asList("marko", "edge1", "step", null))),
+ result.getTableView().getRows());
+ Assert.assertEquals(List.of(path), result.getJsonView().getData());
+ Assert.assertNull(result.getGraphView().getVertices());
+ Assert.assertTrue(result.getGraphView().getStatistics().isEmpty());
+ }
+
+ @Test
+ public void testEdgeTableFiltersOtherResultTypes() throws Exception {
+ Vertex source = new Vertex("person");
+ source.id("marko");
+ Vertex target = new Vertex("person");
+ target.id("vadas");
+ Edge first = new Edge("knows");
+ first.id("edge1");
+ first.sourceId(source.id());
+ first.targetId(target.id());
+ Edge second = new Edge("knows");
+ second.id("edge2");
+ second.sourceId(source.id());
+ second.targetId(target.id());
+ HugeClient client = this.mockClient(this.mockGremlin(this.resultSet(first, second, "scalar"),
+ this.resultSet(source, target)));
+ QueryService service = this.serviceWithConfig();
+
+ GremlinResult result = service.executeGremlinQuery(client, new GremlinQuery("g.E()"));
+
+ Assert.assertEquals(GremlinResult.Type.EDGE, result.getType());
+ Assert.assertEquals(List.of("id", "label", "source", "target", "properties"),
+ result.getTableView().getHeader());
+ Assert.assertEquals(List.of(first, second), result.getTableView().getRows());
+ Assert.assertEquals(List.of(first, second, "scalar"), result.getJsonView().getData());
+ Assert.assertEquals(2, result.getGraphView().getVertices().size());
+ Assert.assertEquals(2, result.getGraphView().getEdges().size());
+ }
+
+ @Test
+ public void testCypherNodeRowsBuildVertexViews() throws Exception {
+ Map<String, Object> node = Map.of("_type", "node", "_id", "marko", "_label", "person", "name", "Marko");
+ HugeClient client = this.mockClient(this.mockGremlin());
+ CypherManager cypher = Mockito.mock(CypherManager.class);
+ Mockito.when(cypher.cypher(Mockito.anyString())).thenReturn(this.resultSet(Map.of("n", node)));
+ Mockito.when(client.cypher()).thenReturn(cypher);
+ QueryService service = this.serviceWithConfig();
+
+ GremlinResult result = service.executeCypherQuery(client, "MATCH (n) RETURN n");
+
+ Assert.assertEquals(GremlinResult.Type.VERTEX, result.getType());
+ Assert.assertEquals(List.of("id", "label", "properties"), result.getTableView().getHeader());
+ Vertex vertex = (Vertex) result.getTableView().getRows().get(0);
+ Assert.assertEquals("marko", vertex.id());
+ Assert.assertEquals("person", vertex.label());
+ Assert.assertEquals("Marko", vertex.property("name"));
+ Assert.assertEquals(1, result.getGraphView().getVertices().size());
+ Assert.assertTrue(result.getGraphView().getEdges().isEmpty());
+ }
+
private QueryService serviceWithConfig() throws Exception {
QueryService service = new QueryService();
HugeConfig config = Mockito.mock(HugeConfig.class);
diff --git a/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/unit/UnitTestSuite.java b/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/unit/UnitTestSuite.java
index cb6b3c98..bc95958a 100644
--- a/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/unit/UnitTestSuite.java
+++ b/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/unit/UnitTestSuite.java
@@ -56,6 +56,7 @@ import org.junit.runners.Suite;
@Suite.SuiteClasses({
AccountMutationAuthorizationTest.class,
AsyncConfigLifecycleTest.class,
+ MaintenanceSchedulingTest.class,
AsyncLoadShutdownTest.class,
AuthContextServiceTest.class,
AuthModeServiceTest.class,
@@ -68,6 +69,7 @@ import org.junit.runners.Suite;
ConfigControllerTest.class,
EmptyCatchTest.class,
FileMappingSchemaTest.class,
+ org.apache.hugegraph.service.load.FilePreviewReaderTest.class,
FileUploadControllerTest.class,
FileUtilTest.class,
GraphServiceImportTest.class,
@@ -77,6 +79,7 @@ import org.junit.runners.Suite;
HubbleDataSourceTest.class,
AvaticaRuntimeTest.class,
JaxbRuntimeTest.class,
+ JacksonConfigTest.class,
GraphSpaceControllerTest.class,
GraphSpaceAuthMutationAuthorizationTest.class,
GraphSpaceAuthOwnershipTest.class,
@@ -115,6 +118,7 @@ import org.junit.runners.Suite;
LoadTaskCheckpointTest.class,
LoaderScopeControllerTest.class,
LoginAttemptGuardTest.class,
+ org.apache.hugegraph.controller.op.LogControllerTest.class,
OltpAlgoControllerTest.class,
OltpAlgoServiceTest.class,
OperationsCapabilityServiceTest.class,
@@ -129,7 +133,9 @@ import org.junit.runners.Suite;
QueryServiceTest.class,
ResponseAdvisorStatusTest.class,
UrlUtilTest.class,
+ UserCsvImportTest.class,
EntityUtilTest.class,
+ org.apache.hugegraph.util.HubbleDateTimeTest.class,
HubbleUtilTest.class
})
public class UnitTestSuite {
diff --git a/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/unit/UserCsvImportTest.java b/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/unit/UserCsvImportTest.java
new file mode 100644
index 00000000..47df4e12
--- /dev/null
+++ b/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/unit/UserCsvImportTest.java
@@ -0,0 +1,128 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hugegraph.unit;
+
+import java.io.File;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.hugegraph.driver.HugeClient;
+import org.apache.hugegraph.exception.InternalException;
+import org.apache.hugegraph.service.auth.UserService;
+import org.junit.Assert;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.rules.TemporaryFolder;
+import org.mockito.Mockito;
+import org.springframework.mock.web.MockMultipartFile;
+
+public class UserCsvImportTest {
+
+ @Rule
+ public TemporaryFolder temporary = new TemporaryFolder();
+
+ @Test
+ public void testQuotedUnicodeMultilineAndEmptyValues() throws Exception {
+ File file = this.csv("user_name,user_password,user_email,user_description\r\n" +
+ "张三,\" p,\"\"secret\"\"\n终 \",,\"first\nsecond\"\r\n");
+
+ List<Map<String, String>> users = new UserService().readUserCsv(file);
+
+ Assert.assertEquals(1, users.size());
+ Assert.assertEquals(Map.of("user_name", "张三",
+ "user_password", " p,\"secret\"\n终 ",
+ "user_email", "",
+ "user_description", "first\nsecond"), users.get(0));
+ }
+
+ @Test
+ public void testBomWhitespaceAndBlankLines() throws Exception {
+ File file = this.csv("\uFEFF\"user_name\", user_password \n\n alice , secret \n\n");
+
+ Assert.assertEquals(List.of(Map.of("user_name", "alice", "user_password", "secret")),
+ new UserService().readUserCsv(file));
+ }
+
+ @Test
+ public void testRejectsMissingDuplicateAndBlankHeaders() throws Exception {
+ for (String content : List.of("", "\n\n", "user_name,user_name\nalice,bob\n",
+ "user_name, \nalice,secret\n", ",user_password\nalice,secret\n")) {
+ this.assertInvalid(this.csv(content));
+ }
+ }
+
+ @Test
+ public void testRejectsEveryRecordWithMismatchedWidth() throws Exception {
+ for (String records : List.of("alice\n", "alice,secret,extra\n",
+ "alice,secret\nbob\n", "alice,secret\nbob,secret,extra\n",
+ "\"\"\nalice,secret\n", "alice,secret\n\"\"\n")) {
+ this.assertInvalid(this.csv("user_name,user_password\n" + records));
+ }
+ }
+
+ @Test
+ public void testValidatesWholeFileBeforeCreatingAnyAccounts() {
+ HugeClient client = Mockito.mock(HugeClient.class);
+ MockMultipartFile upload = new MockMultipartFile(
+ "file", "users.csv", "text/csv",
+ "user_name,user_password\nalice,secret\nbob,secret,unexpected\n"
+ .getBytes(StandardCharsets.UTF_8));
+
+ try {
+ new UserService().addbatch(client, upload);
+ Assert.fail("Expected invalid CSV to be rejected");
+ } catch (InternalException error) {
+ Assert.assertEquals("auth.user.import-csv.failed", error.getMessage());
+ Assert.assertNull(error.getCause());
+ }
+ Mockito.verifyNoInteractions(client);
+ }
+
+ @Test
+ public void testInvalidEncodingDoesNotExposeCsvContents() throws Exception {
+ File file = this.temporary.newFile();
+ Files.write(file.toPath(), new byte[]{(byte) 0xC3, (byte) 0x28});
+
+ this.assertInvalid(file);
+ }
+
+ @Test
+ public void testHeaderOnlyFileHasNoAccounts() throws Exception {
+ Assert.assertTrue(new UserService().readUserCsv(
+ this.csv("user_name,user_password\n")).isEmpty());
+ }
+
+ private File csv(String content) throws Exception {
+ File file = this.temporary.newFile();
+ Files.writeString(file.toPath(), content, StandardCharsets.UTF_8);
+ return file;
+ }
+
+ private void assertInvalid(File file) {
+ try {
+ new UserService().readUserCsv(file);
+ Assert.fail("Expected invalid CSV to be rejected");
+ } catch (InternalException error) {
+ Assert.assertEquals("auth.user.import-csv.failed", error.getMessage());
+ Assert.assertNull(error.getCause());
+ }
+ }
+}
diff --git a/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/unit/UserImportSafetyTest.java b/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/unit/UserImportSafetyTest.java
index 4bac0b1b..84992d13 100644
--- a/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/unit/UserImportSafetyTest.java
+++ b/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/unit/UserImportSafetyTest.java
@@ -76,7 +76,7 @@ public class UserImportSafetyTest {
@Test(expected = InternalException.class)
public void testNullCsvFileHasActionableFailure() {
- new UserService().readCsvByCsvReader(null);
+ new UserService().readUserCsv(null);
}
private static void assertInternalException(ThrowingRunnable runnable)
diff --git a/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/util/HubbleDateTimeTest.java b/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/util/HubbleDateTimeTest.java
new file mode 100644
index 00000000..e42d0fe9
--- /dev/null
+++ b/hugegraph-hubble/hubble-be/src/test/java/org/apache/hugegraph/util/HubbleDateTimeTest.java
@@ -0,0 +1,110 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.hugegraph.util;
+
+import java.net.URL;
+import java.net.URLClassLoader;
+import java.time.Clock;
+import java.time.Instant;
+import java.time.ZoneOffset;
+import java.time.format.DateTimeParseException;
+import java.util.Date;
+import java.util.TimeZone;
+
+import org.apache.hugegraph.testutil.Assert;
+import org.junit.Test;
+
+public class HubbleDateTimeTest {
+
+ @Test
+ public void testInitializationDoesNotChangeJvmTimeZone() throws Exception {
+ TimeZone original = TimeZone.getDefault();
+ URL classes = HubbleUtil.class.getProtectionDomain().getCodeSource().getLocation();
+ try (URLClassLoader loader = new URLClassLoader(new URL[]{classes},
+ ClassLoader.getPlatformClassLoader())) {
+ TimeZone.setDefault(TimeZone.getTimeZone("UTC"));
+ // Initialize a fresh copy even when another test already used HubbleUtil.
+ Class.forName(HubbleUtil.class.getName(), true, loader);
+ Assert.assertEquals("UTC", TimeZone.getDefault().getID());
+ } finally {
+ TimeZone.setDefault(original);
+ }
+ }
+
+ @Test
+ public void testBusinessDatesDoNotDependOnJvmTimeZone() {
+ TimeZone original = TimeZone.getDefault();
+ try {
+ TimeZone.setDefault(TimeZone.getTimeZone("America/Los_Angeles"));
+ Date date = date("2024-01-31T20:00:00Z");
+ Assert.assertEquals("20240201", HubbleUtil.dateFormatDay(date));
+ Assert.assertEquals("2024-02-01", HubbleUtil.dateFormatDay("20240201"));
+ Assert.assertEquals("America/Los_Angeles", TimeZone.getDefault().getID());
+ } finally {
+ TimeZone.setDefault(original);
+ }
+ }
+
+ @Test
+ public void testElasticsearchTimestampUsesBusinessTimeZone() {
+ TimeZone original = TimeZone.getDefault();
+ try {
+ TimeZone.setDefault(TimeZone.getTimeZone("UTC"));
+ Assert.assertEquals("2024-02-01 04:00:00", ESUtil.parseTimestamp("2024-01-31T20:00:00Z"));
+ Assert.assertEquals("2024-02-01 04:00:00", ESUtil.parseTimestamp("2024-02-01T05:00:00+09:00"));
+ Assert.assertNull(ESUtil.parseTimestamp(null));
+ Assert.assertNull(ESUtil.parseTimestamp(""));
+ Assert.assertEquals("UTC", TimeZone.getDefault().getID());
+ } finally {
+ TimeZone.setDefault(original);
+ }
+ }
+
+ @Test
+ public void testPreviousDayUsesBusinessMidnight() {
+ Assert.assertEquals("20231231", HubbleUtil.dateFormatLastDay(clock("2023-12-31T16:00:00Z")));
+ Assert.assertEquals("20240229", HubbleUtil.dateFormatLastDay(clock("2024-02-29T16:00:00Z")));
+ }
+
+ @Test
+ public void testInvalidDatesAreNotSilentlyNormalized() {
+ for (String invalid : new String[]{"20230229", "20240230", "20240101extra", "20241301"}) {
+ Assert.assertThrows(DateTimeParseException.class, () -> HubbleUtil.dateFormatDay(invalid));
+ }
+ Assert.assertThrows(DateTimeParseException.class, () -> {
+ HubbleUtil.getTimestampsBefore24Hours("20240230 12:00:00");
+ });
+ Assert.assertThrows(DateTimeParseException.class, () -> {
+ HubbleUtil.getTimestampsBefore24Hours("20240229 24:00:00");
+ });
+ Assert.assertArrayEquals(seconds("2024-02-28T04:00:00Z", "2024-02-29T04:00:00Z"),
+ HubbleUtil.getTimestampsBefore24Hours("20240229 12:00:00"));
+ }
+
+ private static Date date(String value) {
+ return Date.from(Instant.parse(value));
+ }
+
+ private static Clock clock(String value) {
+ return Clock.fixed(Instant.parse(value), ZoneOffset.UTC);
+ }
+
+ private static long[] seconds(String start, String end) {
+ return new long[]{Instant.parse(start).getEpochSecond(), Instant.parse(end).getEpochSecond()};
+ }
+}
diff --git a/hugegraph-hubble/pom.xml b/hugegraph-hubble/pom.xml
index 2615919b..13d6f5df 100644
--- a/hugegraph-hubble/pom.xml
+++ b/hugegraph-hubble/pom.xml
@@ -46,9 +46,7 @@
<springboot.version>3.5.16</springboot.version>
<exec-maven-plugin.version>3.6.4</exec-maven-plugin.version>
<okhttp.version>4.12.0</okhttp.version>
- <kotlin.version>1.9.25</kotlin.version>
<es.version>7.16.1</es.version>
- <caffeine.version>2.8.0</caffeine.version>
<avatica.version>1.22.0</avatica.version>
<!-- Align inherited dependency management with the Boot 3 BOM. -->
<jackson.version>2.21.4</jackson.version>
@@ -80,11 +78,6 @@
<artifactId>elasticsearch-java</artifactId>
<version>${es.version}</version>
</dependency>
- <dependency>
- <groupId>com.github.ben-manes.caffeine</groupId>
- <artifactId>caffeine</artifactId>
- <version>${caffeine.version}</version>
- </dependency>
<!-- Keep all logging bridges aligned with the shared Log4j version. -->
<dependency>
<groupId>org.apache.logging.log4j</groupId>
@@ -113,17 +106,6 @@
<version>${springboot.version}</version>
<type>pom</type>
<scope>import</scope>
- <exclusions>
- <exclusion>
- <groupId>org.jetbrains.kotlin</groupId>
- <artifactId>kotlin-stdlib</artifactId>
- </exclusion>
- </exclusions>
- </dependency>
- <dependency>
- <groupId>org.jetbrains.kotlin</groupId>
- <artifactId>kotlin-stdlib</artifactId>
- <version>${kotlin.version}</version>
</dependency>
<dependency>
<groupId>com.squareup.okhttp3</groupId>
@@ -132,11 +114,6 @@
<type>pom</type>
<scope>import</scope>
</dependency>
- <dependency>
- <groupId>org.jetbrains.kotlin</groupId>
- <artifactId>kotlin-stdlib-common</artifactId>
- <version>${kotlin.version}</version>
- </dependency>
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>