You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@inlong.apache.org by "ZhenyuePan (via GitHub)" <gi...@apache.org> on 2025/09/13 16:17:33 UTC
[PR] [INLONG-11999][Audit] Add alert evaluation and periodic audit check task [inlong]
ZhenyuePan opened a new pull request, #12002:
URL: https://github.com/apache/inlong/pull/12002
[INLONG-12000][Audit]Implement API for obtaining audit data in Audit Tool
Fixes #11999
**Motivation**
This component provides the core functionality for periodic audit data inspection and alert evaluation. It enables the system to automatically check data consistency between source and sink components at configurable intervals, detect anomalies based on predefined threshold rules, and trigger alerts when discrepancies exceed acceptable limits.
**Modifications**
1. **Periodic Check Task**
- Added `AuditCheckTask` class implementing scheduled background job
- Configurable execution interval via `executionIntervalTime` parameter
- Supports time-window-based audit metric querying with `startLogTs` and `endLogTs`
2. **Data Query Service Integration**
- Integrated with `AuditMetricService` for fetching audit metrics from storage
- Handles source and sink metric retrieval using `getStorageAuditMetrics()` method
- Supports multiple audit IDs through `auditAlertRuleManager.getAuditIds()`
3. **Alert Evaluation Integration**
- Incorporated `AlertEvaluator` for rule-based anomaly detection
- Evaluates all configured alert rules from `auditAlertRuleManager.getAuditAlertRuleList()`
- Processes comparisons between source and sink metrics for each audit ID
4. **Configuration Management**
- Added configuration parameters through `AppConfig`:
- `sourceAuditId`: Source audit identifier (default: "5")
- `executionIntervalTime`: Task execution interval in minutes (default: 1)
- `intervalTimeMinute`: Time window for metric querying in minutes (default: 1)
- Implemented fallback to default values on configuration read failures
5. **Task Lifecycle Management**
- Added `start()` method for initiating periodic checks
- Implemented `stop()` method for graceful shutdown with termination timeout
- Used `ScheduledExecutorService` for robust task scheduling
**Verification**
- [ ] Requires unit tests for periodic task scheduling and interval configuration
- [ ] Needs integration tests for metric querying and alert evaluation flow
- [ ] Should validate configuration fallback behavior and error handling
- [ ] Must verify time window calculation correctness for different interval settings
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-11999][Audit] Add alert evaluation and periodic audit check task [inlong]
Posted by "fuweng11 (via GitHub)" <gi...@apache.org>.
fuweng11 commented on code in PR #12002:
URL: https://github.com/apache/inlong/pull/12002#discussion_r2347101054
##########
inlong-audit/audit-tool/src/main/resources/application.properties:
##########
@@ -0,0 +1,53 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Application properties for the audit tool
+# Configuration for connecting to the manager service
+manager.url=http://localhost:8083
+manager.api.key=your_api_key
+
+# Alert policy configuration
+alert.policy.threshold=5
+alert.policy.check.interval=60
+
+# Prometheus reporting configuration
+prometheus.enabled=true
+prometheus.endpoint=http://localhost:9090/api/v1/write
+prometheus.port=9091
+
+# OpenTelemetry reporting configuration
+opentelemetry.enabled=true
+opentelemetry.endpoint=http://localhost:4317
+opentelemetry.service.name=audit-tool
+
+# Logging configuration
+logging.level.root=INFO
+logging.file.path=/var/log/inlong/audit-tool.log
+
+# Audit Database settings
+audit.data.source=apache_inlong_audit
+audit.data.source.url=jdbc:mysql://192.168.10.3:3306/apache_inlong_audit
+audit.data.source.username=root
+audit.data.source.password=inlong
+audit.data.time.interval.minute=1
+audit.data.time.delay.minute=1
+audit.id.source=5
+
+# Audit SecretId SecretKey
+audit.secretId = admin
+audit.secretKey = inlong
Review Comment:
```suggestion
audit.secretId=admin
audit.secretKey=inlong
```
##########
inlong-audit/audit-tool/src/main/resources/application.properties:
##########
@@ -0,0 +1,53 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Application properties for the audit tool
+# Configuration for connecting to the manager service
+manager.url=http://localhost:8083
+manager.api.key=your_api_key
+
+# Alert policy configuration
+alert.policy.threshold=5
+alert.policy.check.interval=60
+
+# Prometheus reporting configuration
+prometheus.enabled=true
+prometheus.endpoint=http://localhost:9090/api/v1/write
+prometheus.port=9091
+
+# OpenTelemetry reporting configuration
+opentelemetry.enabled=true
+opentelemetry.endpoint=http://localhost:4317
+opentelemetry.service.name=audit-tool
+
+# Logging configuration
+logging.level.root=INFO
+logging.file.path=/var/log/inlong/audit-tool.log
+
+# Audit Database settings
+audit.data.source=apache_inlong_audit
+audit.data.source.url=jdbc:mysql://192.168.10.3:3306/apache_inlong_audit
Review Comment:
Please do not include real IP addresses in the code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-11999][Audit] Add alert evaluation and periodic audit check task [inlong]
Posted by "ZhenyuePan (via GitHub)" <gi...@apache.org>.
ZhenyuePan commented on code in PR #12002:
URL: https://github.com/apache/inlong/pull/12002#discussion_r2347383768
##########
inlong-audit/audit-tool/src/main/resources/application.properties:
##########
@@ -0,0 +1,53 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Application properties for the audit tool
+# Configuration for connecting to the manager service
+manager.url=http://localhost:8083
+manager.api.key=your_api_key
+
+# Alert policy configuration
+alert.policy.threshold=5
+alert.policy.check.interval=60
+
+# Prometheus reporting configuration
+prometheus.enabled=true
+prometheus.endpoint=http://localhost:9090/api/v1/write
+prometheus.port=9091
+
+# OpenTelemetry reporting configuration
+opentelemetry.enabled=true
+opentelemetry.endpoint=http://localhost:4317
+opentelemetry.service.name=audit-tool
+
+# Logging configuration
+logging.level.root=INFO
+logging.file.path=/var/log/inlong/audit-tool.log
+
+# Audit Database settings
+audit.data.source=apache_inlong_audit
+audit.data.source.url=jdbc:mysql://192.168.10.3:3306/apache_inlong_audit
+audit.data.source.username=root
+audit.data.source.password=inlong
+audit.data.time.interval.minute=1
+audit.data.time.delay.minute=1
+audit.id.source=5
+
+# Audit SecretId SecretKey
+audit.secretId = admin
+audit.secretKey = inlong
Review Comment:
already resolved
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-11999][Audit] Add alert evaluation and periodic audit check task [inlong]
Posted by "ZhenyuePan (via GitHub)" <gi...@apache.org>.
ZhenyuePan commented on code in PR #12002:
URL: https://github.com/apache/inlong/pull/12002#discussion_r2347409156
##########
inlong-audit/audit-tool/src/main/java/org/apache/inlong/audit/tool/AuditToolMain.java:
##########
@@ -17,9 +17,48 @@
package org.apache.inlong.audit.tool;
+import org.apache.inlong.audit.tool.config.AppConfig;
+import org.apache.inlong.audit.tool.evaluator.AlertEvaluator;
+import org.apache.inlong.audit.tool.manager.AuditAlertRuleManager;
+import org.apache.inlong.audit.tool.reporter.PrometheusReporter;
+import org.apache.inlong.audit.tool.task.AuditCheckTask;
+import org.apache.inlong.audit.tool.util.AuditSQLUtil;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
public class AuditToolMain {
+ private static final Logger LOGGER = LoggerFactory.getLogger(AuditAlertRuleManager.class);
+
public static void main(String[] args) {
+ // Load application configuration
+ AppConfig appConfig = new AppConfig();
+
+ // Initialize auditAlertRule Manager
+ AuditAlertRuleManager auditAlertRuleManager = AuditAlertRuleManager.getInstance();
+ auditAlertRuleManager.init(appConfig);
+ auditAlertRuleManager.schedule();
+
+ // Initialize reporters
+ PrometheusReporter prometheusReporter = new PrometheusReporter();
+ prometheusReporter.init(appConfig.getPrometheusConfig());
+
+ // Database query initialization
+ AuditSQLUtil.initialize(appConfig.getProperties());
+
+ // Initialize alert evaluator
+ AlertEvaluator alertEvaluator = new AlertEvaluator(prometheusReporter, auditAlertRuleManager);
+ AuditCheckTask auditCheckTask =
+ new AuditCheckTask(auditAlertRuleManager, alertEvaluator, appConfig);
+ auditCheckTask.start();
+
+ // Keep the application running
+ Runtime.getRuntime().addShutdownHook(new Thread(() -> {
+ auditCheckTask.stop();
+ LOGGER.error("Audit Tool stopped.");
Review Comment:
already resolved
##########
inlong-audit/audit-tool/src/main/java/org/apache/inlong/audit/tool/AuditToolMain.java:
##########
@@ -17,9 +17,48 @@
package org.apache.inlong.audit.tool;
+import org.apache.inlong.audit.tool.config.AppConfig;
+import org.apache.inlong.audit.tool.evaluator.AlertEvaluator;
+import org.apache.inlong.audit.tool.manager.AuditAlertRuleManager;
+import org.apache.inlong.audit.tool.reporter.PrometheusReporter;
+import org.apache.inlong.audit.tool.task.AuditCheckTask;
+import org.apache.inlong.audit.tool.util.AuditSQLUtil;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
public class AuditToolMain {
+ private static final Logger LOGGER = LoggerFactory.getLogger(AuditAlertRuleManager.class);
+
public static void main(String[] args) {
+ // Load application configuration
+ AppConfig appConfig = new AppConfig();
+
+ // Initialize auditAlertRule Manager
+ AuditAlertRuleManager auditAlertRuleManager = AuditAlertRuleManager.getInstance();
+ auditAlertRuleManager.init(appConfig);
+ auditAlertRuleManager.schedule();
+
+ // Initialize reporters
+ PrometheusReporter prometheusReporter = new PrometheusReporter();
+ prometheusReporter.init(appConfig.getPrometheusConfig());
+
+ // Database query initialization
+ AuditSQLUtil.initialize(appConfig.getProperties());
+
+ // Initialize alert evaluator
+ AlertEvaluator alertEvaluator = new AlertEvaluator(prometheusReporter, auditAlertRuleManager);
+ AuditCheckTask auditCheckTask =
+ new AuditCheckTask(auditAlertRuleManager, alertEvaluator, appConfig);
+ auditCheckTask.start();
+
+ // Keep the application running
+ Runtime.getRuntime().addShutdownHook(new Thread(() -> {
+ auditCheckTask.stop();
+ LOGGER.error("Audit Tool stopped.");
Review Comment:
already resolved
##########
inlong-audit/audit-tool/src/main/java/org/apache/inlong/audit/tool/AuditToolMain.java:
##########
@@ -17,9 +17,48 @@
package org.apache.inlong.audit.tool;
+import org.apache.inlong.audit.tool.config.AppConfig;
+import org.apache.inlong.audit.tool.evaluator.AlertEvaluator;
+import org.apache.inlong.audit.tool.manager.AuditAlertRuleManager;
+import org.apache.inlong.audit.tool.reporter.PrometheusReporter;
+import org.apache.inlong.audit.tool.task.AuditCheckTask;
+import org.apache.inlong.audit.tool.util.AuditSQLUtil;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
public class AuditToolMain {
+ private static final Logger LOGGER = LoggerFactory.getLogger(AuditAlertRuleManager.class);
+
public static void main(String[] args) {
+ // Load application configuration
+ AppConfig appConfig = new AppConfig();
+
+ // Initialize auditAlertRule Manager
+ AuditAlertRuleManager auditAlertRuleManager = AuditAlertRuleManager.getInstance();
+ auditAlertRuleManager.init(appConfig);
+ auditAlertRuleManager.schedule();
+
+ // Initialize reporters
+ PrometheusReporter prometheusReporter = new PrometheusReporter();
+ prometheusReporter.init(appConfig.getPrometheusConfig());
+
+ // Database query initialization
+ AuditSQLUtil.initialize(appConfig.getProperties());
+
+ // Initialize alert evaluator
+ AlertEvaluator alertEvaluator = new AlertEvaluator(prometheusReporter, auditAlertRuleManager);
+ AuditCheckTask auditCheckTask =
+ new AuditCheckTask(auditAlertRuleManager, alertEvaluator, appConfig);
+ auditCheckTask.start();
+
+ // Keep the application running
+ Runtime.getRuntime().addShutdownHook(new Thread(() -> {
+ auditCheckTask.stop();
+ LOGGER.error("Audit Tool stopped.");
+ }));
+ LOGGER.error("Audit Tool started.");
Review Comment:
already resolved
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-11999][Audit] Add alert evaluation and periodic audit check task [inlong]
Posted by "fuweng11 (via GitHub)" <gi...@apache.org>.
fuweng11 commented on code in PR #12002:
URL: https://github.com/apache/inlong/pull/12002#discussion_r2347401295
##########
inlong-audit/audit-tool/src/main/java/org/apache/inlong/audit/tool/task/AuditCheckTask.java:
##########
@@ -0,0 +1,149 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.inlong.audit.tool.task;
+
+import org.apache.inlong.audit.tool.config.AppConfig;
+import org.apache.inlong.audit.tool.config.ConfigConstants;
+import org.apache.inlong.audit.tool.dto.AuditAlertRule;
+import org.apache.inlong.audit.tool.entity.AuditMetric;
+import org.apache.inlong.audit.tool.evaluator.AlertEvaluator;
+import org.apache.inlong.audit.tool.manager.AuditAlertRuleManager;
+import org.apache.inlong.audit.tool.service.AuditMetricService;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.time.LocalDateTime;
+import java.time.format.DateTimeFormatter;
+import java.util.List;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * AuditCheckTask class: Periodically fetches audit data and evaluates alert policies.
+ */
+public class AuditCheckTask {
+
+ private final ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);
+ private final AlertEvaluator alertEvaluator;
+ private final AuditAlertRuleManager auditAlertRuleManager;
+ private static final Logger LOGGER = LoggerFactory.getLogger(AuditAlertRuleManager.class);
+ private final AuditMetricService auditMetricService;
+ private Integer executionIntervalTime;
+ private Integer intervalTimeMinute;
+ private final Integer delayTimeMinute;
+ private static final DateTimeFormatter LOGS_FMT = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss");
+ private String sourceAuditId;
+
+ public AuditCheckTask(
+ AuditAlertRuleManager auditAlertRuleManager, AlertEvaluator alertEvaluator, AppConfig appConfig) {
+ this.auditAlertRuleManager = auditAlertRuleManager;
+ this.alertEvaluator = alertEvaluator;
+ this.auditMetricService = new AuditMetricService();
+ try {
+ this.executionIntervalTime =
+ Integer.valueOf(appConfig.getProperties().getProperty(ConfigConstants.KEY_DELAY_TIME, "1"));
+ this.intervalTimeMinute =
+ Integer.parseInt(appConfig.getProperties().getProperty(ConfigConstants.KEY_INTERVAL_TIME, "1"));
+ this.sourceAuditId = appConfig.getProperties().getProperty(ConfigConstants.KEY_SOURCE_AUDIT_ID, "5");
+ } catch (Exception e) {
+ LOGGER.info(
Review Comment:
```suggestion
LOGGER.error(
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-11999][Audit] Add alert evaluation and periodic audit check task [inlong]
Posted by "fuweng11 (via GitHub)" <gi...@apache.org>.
fuweng11 commented on code in PR #12002:
URL: https://github.com/apache/inlong/pull/12002#discussion_r2347402147
##########
inlong-audit/audit-tool/src/test/java/org/apache/inlong/tool/service/AuditMetricServiceTest.java:
##########
@@ -19,54 +19,35 @@
import org.apache.inlong.audit.tool.config.AppConfig;
import org.apache.inlong.audit.tool.entity.AuditMetric;
-import org.apache.inlong.audit.tool.mapper.AuditMapper;
import org.apache.inlong.audit.tool.service.AuditMetricService;
import org.apache.inlong.audit.tool.util.AuditSQLUtil;
-import org.apache.ibatis.session.SqlSession;
import org.junit.jupiter.api.Test;
-import org.mockito.MockedStatic;
-import org.mockito.Mockito;
-import java.util.Collections;
+import java.time.LocalDateTime;
+import java.time.format.DateTimeFormatter;
import java.util.List;
-import static org.junit.jupiter.api.Assertions.*;
-
public class AuditMetricServiceTest {
@Test
- public void getStorageAuditMetricsReturnsEmptyListWhenNoDataFound() {
- AuditMetricService auditMetricService = new AuditMetricService();
-
- try (MockedStatic<AuditSQLUtil> sqlUtilMockedStatic = Mockito.mockStatic(AuditSQLUtil.class)) {
- SqlSession sqlSessionMock = Mockito.mock(SqlSession.class);
- AuditMapper auditMapperMock = Mockito.mock(AuditMapper.class);
-
- sqlUtilMockedStatic.when(AuditSQLUtil::getSqlSession).thenReturn(sqlSessionMock);
- Mockito.when(sqlSessionMock.getMapper(AuditMapper.class)).thenReturn(auditMapperMock);
- Mockito.when(auditMapperMock.getAuditMetrics(Mockito.anyString(), Mockito.anyString(), Mockito.anyString()))
- .thenReturn(Collections.emptyList());
-
- List<AuditMetric> result = auditMetricService.getStorageAuditMetrics("nonexistentId", "2023-01-01 00:00:00",
- "2023-01-01 01:00:00");
- assertTrue(result.isEmpty());
- }
- }
-
- @Test
- public void getStorageAuditMetricsHandlesInvalidTimestampsGracefully() {
+ public void testGetDataProxyAuditMetrics() {
// Query service initialization
AppConfig appConfig = new AppConfig();
AuditSQLUtil.initialize(appConfig.getProperties());
AuditMetricService auditMetricService = new AuditMetricService();
- String invalidStartLogTs = "invalid-timestamp";
- String invalidEndLogTs = "invalid-timestamp";
+ DateTimeFormatter formatter = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss");
+ String endLogTs = LocalDateTime.now().minusMinutes(1).format(formatter);
+ String startLogTs = LocalDateTime.now().minusMinutes(5).format(formatter);
- List<AuditMetric> result = auditMetricService.getStorageAuditMetrics("5", invalidStartLogTs, invalidEndLogTs);
+ // Search for relevant data
+ List<AuditMetric> dataproxyAuditMetrics =
+ auditMetricService.getStorageAuditMetrics("5", startLogTs, endLogTs);
- assertTrue(result.isEmpty());
+ for (AuditMetric auditMetric : dataproxyAuditMetrics) {
+ System.out.println(auditMetric.getInlongGroupId() + " " + auditMetric.getInlongStreamId() + " "
+ + auditMetric.getCount());
+ }
Review Comment:
Please refrain from using System.out.println and print valid logs instead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-11999][Audit] Add alert evaluation and periodic audit check task [inlong]
Posted by "doleyzi (via GitHub)" <gi...@apache.org>.
doleyzi merged PR #12002:
URL: https://github.com/apache/inlong/pull/12002
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-11999][Audit] Add alert evaluation and periodic audit check task [inlong]
Posted by "ZhenyuePan (via GitHub)" <gi...@apache.org>.
ZhenyuePan commented on code in PR #12002:
URL: https://github.com/apache/inlong/pull/12002#discussion_r2347409329
##########
inlong-audit/audit-tool/src/main/java/org/apache/inlong/audit/tool/task/AuditCheckTask.java:
##########
@@ -0,0 +1,149 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.inlong.audit.tool.task;
+
+import org.apache.inlong.audit.tool.config.AppConfig;
+import org.apache.inlong.audit.tool.config.ConfigConstants;
+import org.apache.inlong.audit.tool.dto.AuditAlertRule;
+import org.apache.inlong.audit.tool.entity.AuditMetric;
+import org.apache.inlong.audit.tool.evaluator.AlertEvaluator;
+import org.apache.inlong.audit.tool.manager.AuditAlertRuleManager;
+import org.apache.inlong.audit.tool.service.AuditMetricService;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.time.LocalDateTime;
+import java.time.format.DateTimeFormatter;
+import java.util.List;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * AuditCheckTask class: Periodically fetches audit data and evaluates alert policies.
+ */
+public class AuditCheckTask {
+
+ private final ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);
+ private final AlertEvaluator alertEvaluator;
+ private final AuditAlertRuleManager auditAlertRuleManager;
+ private static final Logger LOGGER = LoggerFactory.getLogger(AuditAlertRuleManager.class);
+ private final AuditMetricService auditMetricService;
+ private Integer executionIntervalTime;
+ private Integer intervalTimeMinute;
+ private final Integer delayTimeMinute;
+ private static final DateTimeFormatter LOGS_FMT = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss");
+ private String sourceAuditId;
+
+ public AuditCheckTask(
+ AuditAlertRuleManager auditAlertRuleManager, AlertEvaluator alertEvaluator, AppConfig appConfig) {
+ this.auditAlertRuleManager = auditAlertRuleManager;
+ this.alertEvaluator = alertEvaluator;
+ this.auditMetricService = new AuditMetricService();
+ try {
+ this.executionIntervalTime =
+ Integer.valueOf(appConfig.getProperties().getProperty(ConfigConstants.KEY_DELAY_TIME, "1"));
+ this.intervalTimeMinute =
+ Integer.parseInt(appConfig.getProperties().getProperty(ConfigConstants.KEY_INTERVAL_TIME, "1"));
+ this.sourceAuditId = appConfig.getProperties().getProperty(ConfigConstants.KEY_SOURCE_AUDIT_ID, "5");
+ } catch (Exception e) {
+ LOGGER.info(
Review Comment:
already resolved
##########
inlong-audit/audit-tool/src/main/java/org/apache/inlong/audit/tool/task/AuditCheckTask.java:
##########
@@ -0,0 +1,149 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.inlong.audit.tool.task;
+
+import org.apache.inlong.audit.tool.config.AppConfig;
+import org.apache.inlong.audit.tool.config.ConfigConstants;
+import org.apache.inlong.audit.tool.dto.AuditAlertRule;
+import org.apache.inlong.audit.tool.entity.AuditMetric;
+import org.apache.inlong.audit.tool.evaluator.AlertEvaluator;
+import org.apache.inlong.audit.tool.manager.AuditAlertRuleManager;
+import org.apache.inlong.audit.tool.service.AuditMetricService;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.time.LocalDateTime;
+import java.time.format.DateTimeFormatter;
+import java.util.List;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * AuditCheckTask class: Periodically fetches audit data and evaluates alert policies.
+ */
+public class AuditCheckTask {
+
+ private final ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);
+ private final AlertEvaluator alertEvaluator;
+ private final AuditAlertRuleManager auditAlertRuleManager;
+ private static final Logger LOGGER = LoggerFactory.getLogger(AuditAlertRuleManager.class);
Review Comment:
already resolved
##########
inlong-audit/audit-tool/src/main/java/org/apache/inlong/audit/tool/evaluator/AlertEvaluator.java:
##########
@@ -0,0 +1,112 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.inlong.audit.tool.evaluator;
+
+import org.apache.inlong.audit.tool.dto.AuditAlertCondition;
+import org.apache.inlong.audit.tool.dto.AuditAlertRule;
+import org.apache.inlong.audit.tool.entity.AuditMetric;
+import org.apache.inlong.audit.tool.manager.AuditAlertRuleManager;
+import org.apache.inlong.audit.tool.reporter.PrometheusReporter;
+
+import lombok.Getter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.List;
+import java.util.Objects;
+
+public class AlertEvaluator {
+
+ private final PrometheusReporter prometheusReporter;
+ private static final Logger LOGGER = LoggerFactory.getLogger(AuditAlertRuleManager.class);
Review Comment:
already resolved
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-11999][Audit] Add alert evaluation and periodic audit check task [inlong]
Posted by "ZhenyuePan (via GitHub)" <gi...@apache.org>.
ZhenyuePan commented on code in PR #12002:
URL: https://github.com/apache/inlong/pull/12002#discussion_r2347409434
##########
inlong-audit/audit-tool/src/main/java/org/apache/inlong/audit/tool/AuditToolMain.java:
##########
@@ -17,9 +17,48 @@
package org.apache.inlong.audit.tool;
+import org.apache.inlong.audit.tool.config.AppConfig;
+import org.apache.inlong.audit.tool.evaluator.AlertEvaluator;
+import org.apache.inlong.audit.tool.manager.AuditAlertRuleManager;
+import org.apache.inlong.audit.tool.reporter.PrometheusReporter;
+import org.apache.inlong.audit.tool.task.AuditCheckTask;
+import org.apache.inlong.audit.tool.util.AuditSQLUtil;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
public class AuditToolMain {
+ private static final Logger LOGGER = LoggerFactory.getLogger(AuditAlertRuleManager.class);
Review Comment:
already resolved
##########
inlong-audit/audit-tool/src/main/java/org/apache/inlong/audit/tool/task/AuditCheckTask.java:
##########
@@ -0,0 +1,149 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.inlong.audit.tool.task;
+
+import org.apache.inlong.audit.tool.config.AppConfig;
+import org.apache.inlong.audit.tool.config.ConfigConstants;
+import org.apache.inlong.audit.tool.dto.AuditAlertRule;
+import org.apache.inlong.audit.tool.entity.AuditMetric;
+import org.apache.inlong.audit.tool.evaluator.AlertEvaluator;
+import org.apache.inlong.audit.tool.manager.AuditAlertRuleManager;
+import org.apache.inlong.audit.tool.service.AuditMetricService;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.time.LocalDateTime;
+import java.time.format.DateTimeFormatter;
+import java.util.List;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * AuditCheckTask class: Periodically fetches audit data and evaluates alert policies.
+ */
+public class AuditCheckTask {
+
+ private final ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);
+ private final AlertEvaluator alertEvaluator;
+ private final AuditAlertRuleManager auditAlertRuleManager;
+ private static final Logger LOGGER = LoggerFactory.getLogger(AuditAlertRuleManager.class);
+ private final AuditMetricService auditMetricService;
+ private Integer executionIntervalTime;
+ private Integer intervalTimeMinute;
+ private final Integer delayTimeMinute;
+ private static final DateTimeFormatter LOGS_FMT = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss");
+ private String sourceAuditId;
+
+ public AuditCheckTask(
+ AuditAlertRuleManager auditAlertRuleManager, AlertEvaluator alertEvaluator, AppConfig appConfig) {
+ this.auditAlertRuleManager = auditAlertRuleManager;
+ this.alertEvaluator = alertEvaluator;
+ this.auditMetricService = new AuditMetricService();
+ try {
+ this.executionIntervalTime =
+ Integer.valueOf(appConfig.getProperties().getProperty(ConfigConstants.KEY_DELAY_TIME, "1"));
+ this.intervalTimeMinute =
+ Integer.parseInt(appConfig.getProperties().getProperty(ConfigConstants.KEY_INTERVAL_TIME, "1"));
+ this.sourceAuditId = appConfig.getProperties().getProperty(ConfigConstants.KEY_SOURCE_AUDIT_ID, "5");
+ } catch (Exception e) {
+ LOGGER.info(
+ "Failed to read configuration information, default source AuditId is 5, delay execution time is 1, time interval is 1");
+ this.executionIntervalTime = 1;
+ this.intervalTimeMinute = 1;
+ this.sourceAuditId = "5";
Review Comment:
already resolved
##########
inlong-audit/audit-tool/src/main/java/org/apache/inlong/audit/tool/task/AuditCheckTask.java:
##########
@@ -0,0 +1,149 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.inlong.audit.tool.task;
+
+import org.apache.inlong.audit.tool.config.AppConfig;
+import org.apache.inlong.audit.tool.config.ConfigConstants;
+import org.apache.inlong.audit.tool.dto.AuditAlertRule;
+import org.apache.inlong.audit.tool.entity.AuditMetric;
+import org.apache.inlong.audit.tool.evaluator.AlertEvaluator;
+import org.apache.inlong.audit.tool.manager.AuditAlertRuleManager;
+import org.apache.inlong.audit.tool.service.AuditMetricService;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.time.LocalDateTime;
+import java.time.format.DateTimeFormatter;
+import java.util.List;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * AuditCheckTask class: Periodically fetches audit data and evaluates alert policies.
+ */
+public class AuditCheckTask {
+
+ private final ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);
+ private final AlertEvaluator alertEvaluator;
+ private final AuditAlertRuleManager auditAlertRuleManager;
+ private static final Logger LOGGER = LoggerFactory.getLogger(AuditAlertRuleManager.class);
+ private final AuditMetricService auditMetricService;
+ private Integer executionIntervalTime;
Review Comment:
already resolved
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-11999][Audit] Add alert evaluation and periodic audit check task [inlong]
Posted by "fuweng11 (via GitHub)" <gi...@apache.org>.
fuweng11 commented on code in PR #12002:
URL: https://github.com/apache/inlong/pull/12002#discussion_r2347397871
##########
inlong-audit/audit-tool/src/main/java/org/apache/inlong/audit/tool/task/AuditCheckTask.java:
##########
@@ -0,0 +1,149 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.inlong.audit.tool.task;
+
+import org.apache.inlong.audit.tool.config.AppConfig;
+import org.apache.inlong.audit.tool.config.ConfigConstants;
+import org.apache.inlong.audit.tool.dto.AuditAlertRule;
+import org.apache.inlong.audit.tool.entity.AuditMetric;
+import org.apache.inlong.audit.tool.evaluator.AlertEvaluator;
+import org.apache.inlong.audit.tool.manager.AuditAlertRuleManager;
+import org.apache.inlong.audit.tool.service.AuditMetricService;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.time.LocalDateTime;
+import java.time.format.DateTimeFormatter;
+import java.util.List;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * AuditCheckTask class: Periodically fetches audit data and evaluates alert policies.
+ */
+public class AuditCheckTask {
+
+ private final ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);
+ private final AlertEvaluator alertEvaluator;
+ private final AuditAlertRuleManager auditAlertRuleManager;
+ private static final Logger LOGGER = LoggerFactory.getLogger(AuditAlertRuleManager.class);
+ private final AuditMetricService auditMetricService;
+ private Integer executionIntervalTime;
+ private Integer intervalTimeMinute;
+ private final Integer delayTimeMinute;
+ private static final DateTimeFormatter LOGS_FMT = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss");
+ private String sourceAuditId;
+
+ public AuditCheckTask(
+ AuditAlertRuleManager auditAlertRuleManager, AlertEvaluator alertEvaluator, AppConfig appConfig) {
+ this.auditAlertRuleManager = auditAlertRuleManager;
+ this.alertEvaluator = alertEvaluator;
+ this.auditMetricService = new AuditMetricService();
+ try {
+ this.executionIntervalTime =
+ Integer.valueOf(appConfig.getProperties().getProperty(ConfigConstants.KEY_DELAY_TIME, "1"));
+ this.intervalTimeMinute =
+ Integer.parseInt(appConfig.getProperties().getProperty(ConfigConstants.KEY_INTERVAL_TIME, "1"));
+ this.sourceAuditId = appConfig.getProperties().getProperty(ConfigConstants.KEY_SOURCE_AUDIT_ID, "5");
+ } catch (Exception e) {
+ LOGGER.info(
+ "Failed to read configuration information, default source AuditId is 5, delay execution time is 1, time interval is 1");
+ this.executionIntervalTime = 1;
+ this.intervalTimeMinute = 1;
+ this.sourceAuditId = "5";
Review Comment:
Remove it.
##########
inlong-audit/audit-tool/src/main/java/org/apache/inlong/audit/tool/task/AuditCheckTask.java:
##########
@@ -0,0 +1,149 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.inlong.audit.tool.task;
+
+import org.apache.inlong.audit.tool.config.AppConfig;
+import org.apache.inlong.audit.tool.config.ConfigConstants;
+import org.apache.inlong.audit.tool.dto.AuditAlertRule;
+import org.apache.inlong.audit.tool.entity.AuditMetric;
+import org.apache.inlong.audit.tool.evaluator.AlertEvaluator;
+import org.apache.inlong.audit.tool.manager.AuditAlertRuleManager;
+import org.apache.inlong.audit.tool.service.AuditMetricService;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.time.LocalDateTime;
+import java.time.format.DateTimeFormatter;
+import java.util.List;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * AuditCheckTask class: Periodically fetches audit data and evaluates alert policies.
+ */
+public class AuditCheckTask {
+
+ private final ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);
+ private final AlertEvaluator alertEvaluator;
+ private final AuditAlertRuleManager auditAlertRuleManager;
+ private static final Logger LOGGER = LoggerFactory.getLogger(AuditAlertRuleManager.class);
+ private final AuditMetricService auditMetricService;
+ private Integer executionIntervalTime;
+ private Integer intervalTimeMinute;
Review Comment:
```suggestion
private Integer intervalTimeMinute = 1;
```
##########
inlong-audit/audit-tool/src/main/java/org/apache/inlong/audit/tool/task/AuditCheckTask.java:
##########
@@ -0,0 +1,149 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.inlong.audit.tool.task;
+
+import org.apache.inlong.audit.tool.config.AppConfig;
+import org.apache.inlong.audit.tool.config.ConfigConstants;
+import org.apache.inlong.audit.tool.dto.AuditAlertRule;
+import org.apache.inlong.audit.tool.entity.AuditMetric;
+import org.apache.inlong.audit.tool.evaluator.AlertEvaluator;
+import org.apache.inlong.audit.tool.manager.AuditAlertRuleManager;
+import org.apache.inlong.audit.tool.service.AuditMetricService;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.time.LocalDateTime;
+import java.time.format.DateTimeFormatter;
+import java.util.List;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * AuditCheckTask class: Periodically fetches audit data and evaluates alert policies.
+ */
+public class AuditCheckTask {
+
+ private final ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);
+ private final AlertEvaluator alertEvaluator;
+ private final AuditAlertRuleManager auditAlertRuleManager;
+ private static final Logger LOGGER = LoggerFactory.getLogger(AuditAlertRuleManager.class);
+ private final AuditMetricService auditMetricService;
+ private Integer executionIntervalTime;
Review Comment:
```suggestion
private Integer executionIntervalTime = 1;
```
##########
inlong-audit/audit-tool/src/main/java/org/apache/inlong/audit/tool/task/AuditCheckTask.java:
##########
@@ -0,0 +1,149 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.inlong.audit.tool.task;
+
+import org.apache.inlong.audit.tool.config.AppConfig;
+import org.apache.inlong.audit.tool.config.ConfigConstants;
+import org.apache.inlong.audit.tool.dto.AuditAlertRule;
+import org.apache.inlong.audit.tool.entity.AuditMetric;
+import org.apache.inlong.audit.tool.evaluator.AlertEvaluator;
+import org.apache.inlong.audit.tool.manager.AuditAlertRuleManager;
+import org.apache.inlong.audit.tool.service.AuditMetricService;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.time.LocalDateTime;
+import java.time.format.DateTimeFormatter;
+import java.util.List;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * AuditCheckTask class: Periodically fetches audit data and evaluates alert policies.
+ */
+public class AuditCheckTask {
+
+ private final ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);
+ private final AlertEvaluator alertEvaluator;
+ private final AuditAlertRuleManager auditAlertRuleManager;
+ private static final Logger LOGGER = LoggerFactory.getLogger(AuditAlertRuleManager.class);
+ private final AuditMetricService auditMetricService;
+ private Integer executionIntervalTime;
+ private Integer intervalTimeMinute;
+ private final Integer delayTimeMinute;
+ private static final DateTimeFormatter LOGS_FMT = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss");
+ private String sourceAuditId;
Review Comment:
```suggestion
private String sourceAuditId = 5;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-11999][Audit] Add alert evaluation and periodic audit check task [inlong]
Posted by "fuweng11 (via GitHub)" <gi...@apache.org>.
fuweng11 commented on code in PR #12002:
URL: https://github.com/apache/inlong/pull/12002#discussion_r2347399871
##########
inlong-audit/audit-tool/src/main/java/org/apache/inlong/audit/tool/AuditToolMain.java:
##########
@@ -17,9 +17,48 @@
package org.apache.inlong.audit.tool;
+import org.apache.inlong.audit.tool.config.AppConfig;
+import org.apache.inlong.audit.tool.evaluator.AlertEvaluator;
+import org.apache.inlong.audit.tool.manager.AuditAlertRuleManager;
+import org.apache.inlong.audit.tool.reporter.PrometheusReporter;
+import org.apache.inlong.audit.tool.task.AuditCheckTask;
+import org.apache.inlong.audit.tool.util.AuditSQLUtil;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
public class AuditToolMain {
+ private static final Logger LOGGER = LoggerFactory.getLogger(AuditAlertRuleManager.class);
Review Comment:
```suggestion
private static final Logger LOGGER = LoggerFactory.getLogger(AuditToolMain.class);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-11999][Audit] Add alert evaluation and periodic audit check task [inlong]
Posted by "fuweng11 (via GitHub)" <gi...@apache.org>.
fuweng11 commented on code in PR #12002:
URL: https://github.com/apache/inlong/pull/12002#discussion_r2347400306
##########
inlong-audit/audit-tool/src/main/java/org/apache/inlong/audit/tool/task/AuditCheckTask.java:
##########
@@ -0,0 +1,149 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.inlong.audit.tool.task;
+
+import org.apache.inlong.audit.tool.config.AppConfig;
+import org.apache.inlong.audit.tool.config.ConfigConstants;
+import org.apache.inlong.audit.tool.dto.AuditAlertRule;
+import org.apache.inlong.audit.tool.entity.AuditMetric;
+import org.apache.inlong.audit.tool.evaluator.AlertEvaluator;
+import org.apache.inlong.audit.tool.manager.AuditAlertRuleManager;
+import org.apache.inlong.audit.tool.service.AuditMetricService;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.time.LocalDateTime;
+import java.time.format.DateTimeFormatter;
+import java.util.List;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * AuditCheckTask class: Periodically fetches audit data and evaluates alert policies.
+ */
+public class AuditCheckTask {
+
+ private final ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);
+ private final AlertEvaluator alertEvaluator;
+ private final AuditAlertRuleManager auditAlertRuleManager;
+ private static final Logger LOGGER = LoggerFactory.getLogger(AuditAlertRuleManager.class);
Review Comment:
```suggestion
private static final Logger LOGGER = LoggerFactory.getLogger(AuditCheckTask.class);
```
##########
inlong-audit/audit-tool/src/main/java/org/apache/inlong/audit/tool/evaluator/AlertEvaluator.java:
##########
@@ -0,0 +1,112 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.inlong.audit.tool.evaluator;
+
+import org.apache.inlong.audit.tool.dto.AuditAlertCondition;
+import org.apache.inlong.audit.tool.dto.AuditAlertRule;
+import org.apache.inlong.audit.tool.entity.AuditMetric;
+import org.apache.inlong.audit.tool.manager.AuditAlertRuleManager;
+import org.apache.inlong.audit.tool.reporter.PrometheusReporter;
+
+import lombok.Getter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.List;
+import java.util.Objects;
+
+public class AlertEvaluator {
+
+ private final PrometheusReporter prometheusReporter;
+ private static final Logger LOGGER = LoggerFactory.getLogger(AuditAlertRuleManager.class);
Review Comment:
```suggestion
private static final Logger LOGGER = LoggerFactory.getLogger(AlertEvaluator.class);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-11999][Audit] Add alert evaluation and periodic audit check task [inlong]
Posted by "ZhenyuePan (via GitHub)" <gi...@apache.org>.
ZhenyuePan commented on code in PR #12002:
URL: https://github.com/apache/inlong/pull/12002#discussion_r2347383610
##########
inlong-audit/audit-tool/src/main/resources/application.properties:
##########
@@ -0,0 +1,53 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Application properties for the audit tool
+# Configuration for connecting to the manager service
+manager.url=http://localhost:8083
+manager.api.key=your_api_key
+
+# Alert policy configuration
+alert.policy.threshold=5
+alert.policy.check.interval=60
+
+# Prometheus reporting configuration
+prometheus.enabled=true
+prometheus.endpoint=http://localhost:9090/api/v1/write
+prometheus.port=9091
+
+# OpenTelemetry reporting configuration
+opentelemetry.enabled=true
+opentelemetry.endpoint=http://localhost:4317
+opentelemetry.service.name=audit-tool
+
+# Logging configuration
+logging.level.root=INFO
+logging.file.path=/var/log/inlong/audit-tool.log
+
+# Audit Database settings
+audit.data.source=apache_inlong_audit
+audit.data.source.url=jdbc:mysql://192.168.10.3:3306/apache_inlong_audit
Review Comment:
already resolved
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-11999][Audit] Add alert evaluation and periodic audit check task [inlong]
Posted by "ZhenyuePan (via GitHub)" <gi...@apache.org>.
ZhenyuePan commented on code in PR #12002:
URL: https://github.com/apache/inlong/pull/12002#discussion_r2347401700
##########
inlong-audit/audit-tool/src/main/java/org/apache/inlong/audit/tool/task/AuditCheckTask.java:
##########
@@ -0,0 +1,149 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.inlong.audit.tool.task;
+
+import org.apache.inlong.audit.tool.config.AppConfig;
+import org.apache.inlong.audit.tool.config.ConfigConstants;
+import org.apache.inlong.audit.tool.dto.AuditAlertRule;
+import org.apache.inlong.audit.tool.entity.AuditMetric;
+import org.apache.inlong.audit.tool.evaluator.AlertEvaluator;
+import org.apache.inlong.audit.tool.manager.AuditAlertRuleManager;
+import org.apache.inlong.audit.tool.service.AuditMetricService;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.time.LocalDateTime;
+import java.time.format.DateTimeFormatter;
+import java.util.List;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * AuditCheckTask class: Periodically fetches audit data and evaluates alert policies.
+ */
+public class AuditCheckTask {
+
+ private final ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);
+ private final AlertEvaluator alertEvaluator;
+ private final AuditAlertRuleManager auditAlertRuleManager;
+ private static final Logger LOGGER = LoggerFactory.getLogger(AuditAlertRuleManager.class);
+ private final AuditMetricService auditMetricService;
+ private Integer executionIntervalTime;
+ private Integer intervalTimeMinute;
+ private final Integer delayTimeMinute;
+ private static final DateTimeFormatter LOGS_FMT = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss");
+ private String sourceAuditId;
Review Comment:
already resolved
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-11999][Audit] Add alert evaluation and periodic audit check task [inlong]
Posted by "ZhenyuePan (via GitHub)" <gi...@apache.org>.
ZhenyuePan commented on code in PR #12002:
URL: https://github.com/apache/inlong/pull/12002#discussion_r2347409515
##########
inlong-audit/audit-tool/src/main/java/org/apache/inlong/audit/tool/task/AuditCheckTask.java:
##########
@@ -0,0 +1,149 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.inlong.audit.tool.task;
+
+import org.apache.inlong.audit.tool.config.AppConfig;
+import org.apache.inlong.audit.tool.config.ConfigConstants;
+import org.apache.inlong.audit.tool.dto.AuditAlertRule;
+import org.apache.inlong.audit.tool.entity.AuditMetric;
+import org.apache.inlong.audit.tool.evaluator.AlertEvaluator;
+import org.apache.inlong.audit.tool.manager.AuditAlertRuleManager;
+import org.apache.inlong.audit.tool.service.AuditMetricService;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.time.LocalDateTime;
+import java.time.format.DateTimeFormatter;
+import java.util.List;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * AuditCheckTask class: Periodically fetches audit data and evaluates alert policies.
+ */
+public class AuditCheckTask {
+
+ private final ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);
+ private final AlertEvaluator alertEvaluator;
+ private final AuditAlertRuleManager auditAlertRuleManager;
+ private static final Logger LOGGER = LoggerFactory.getLogger(AuditAlertRuleManager.class);
+ private final AuditMetricService auditMetricService;
+ private Integer executionIntervalTime;
+ private Integer intervalTimeMinute;
Review Comment:
already resolved
##########
inlong-audit/audit-tool/src/main/java/org/apache/inlong/audit/tool/task/AuditCheckTask.java:
##########
@@ -0,0 +1,149 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.inlong.audit.tool.task;
+
+import org.apache.inlong.audit.tool.config.AppConfig;
+import org.apache.inlong.audit.tool.config.ConfigConstants;
+import org.apache.inlong.audit.tool.dto.AuditAlertRule;
+import org.apache.inlong.audit.tool.entity.AuditMetric;
+import org.apache.inlong.audit.tool.evaluator.AlertEvaluator;
+import org.apache.inlong.audit.tool.manager.AuditAlertRuleManager;
+import org.apache.inlong.audit.tool.service.AuditMetricService;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.time.LocalDateTime;
+import java.time.format.DateTimeFormatter;
+import java.util.List;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * AuditCheckTask class: Periodically fetches audit data and evaluates alert policies.
+ */
+public class AuditCheckTask {
+
+ private final ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);
+ private final AlertEvaluator alertEvaluator;
+ private final AuditAlertRuleManager auditAlertRuleManager;
+ private static final Logger LOGGER = LoggerFactory.getLogger(AuditAlertRuleManager.class);
+ private final AuditMetricService auditMetricService;
+ private Integer executionIntervalTime;
+ private Integer intervalTimeMinute;
+ private final Integer delayTimeMinute;
+ private static final DateTimeFormatter LOGS_FMT = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss");
+ private String sourceAuditId;
Review Comment:
already resolved
##########
inlong-audit/audit-tool/src/main/java/org/apache/inlong/audit/tool/task/AuditCheckTask.java:
##########
@@ -0,0 +1,149 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.inlong.audit.tool.task;
+
+import org.apache.inlong.audit.tool.config.AppConfig;
+import org.apache.inlong.audit.tool.config.ConfigConstants;
+import org.apache.inlong.audit.tool.dto.AuditAlertRule;
+import org.apache.inlong.audit.tool.entity.AuditMetric;
+import org.apache.inlong.audit.tool.evaluator.AlertEvaluator;
+import org.apache.inlong.audit.tool.manager.AuditAlertRuleManager;
+import org.apache.inlong.audit.tool.service.AuditMetricService;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.time.LocalDateTime;
+import java.time.format.DateTimeFormatter;
+import java.util.List;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * AuditCheckTask class: Periodically fetches audit data and evaluates alert policies.
+ */
+public class AuditCheckTask {
+
+ private final ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);
+ private final AlertEvaluator alertEvaluator;
+ private final AuditAlertRuleManager auditAlertRuleManager;
+ private static final Logger LOGGER = LoggerFactory.getLogger(AuditAlertRuleManager.class);
+ private final AuditMetricService auditMetricService;
+ private Integer executionIntervalTime;
+ private Integer intervalTimeMinute;
+ private final Integer delayTimeMinute;
+ private static final DateTimeFormatter LOGS_FMT = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss");
+ private String sourceAuditId;
Review Comment:
already resolved
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-11999][Audit] Add alert evaluation and periodic audit check task [inlong]
Posted by "ZhenyuePan (via GitHub)" <gi...@apache.org>.
ZhenyuePan commented on code in PR #12002:
URL: https://github.com/apache/inlong/pull/12002#discussion_r2347408974
##########
inlong-audit/audit-tool/src/main/java/org/apache/inlong/audit/tool/AuditToolMain.java:
##########
@@ -17,9 +17,48 @@
package org.apache.inlong.audit.tool;
+import org.apache.inlong.audit.tool.config.AppConfig;
+import org.apache.inlong.audit.tool.evaluator.AlertEvaluator;
+import org.apache.inlong.audit.tool.manager.AuditAlertRuleManager;
+import org.apache.inlong.audit.tool.reporter.PrometheusReporter;
+import org.apache.inlong.audit.tool.task.AuditCheckTask;
+import org.apache.inlong.audit.tool.util.AuditSQLUtil;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
public class AuditToolMain {
+ private static final Logger LOGGER = LoggerFactory.getLogger(AuditAlertRuleManager.class);
+
public static void main(String[] args) {
+ // Load application configuration
+ AppConfig appConfig = new AppConfig();
+
+ // Initialize auditAlertRule Manager
+ AuditAlertRuleManager auditAlertRuleManager = AuditAlertRuleManager.getInstance();
+ auditAlertRuleManager.init(appConfig);
+ auditAlertRuleManager.schedule();
+
+ // Initialize reporters
+ PrometheusReporter prometheusReporter = new PrometheusReporter();
+ prometheusReporter.init(appConfig.getPrometheusConfig());
+
+ // Database query initialization
+ AuditSQLUtil.initialize(appConfig.getProperties());
+
+ // Initialize alert evaluator
+ AlertEvaluator alertEvaluator = new AlertEvaluator(prometheusReporter, auditAlertRuleManager);
+ AuditCheckTask auditCheckTask =
+ new AuditCheckTask(auditAlertRuleManager, alertEvaluator, appConfig);
+ auditCheckTask.start();
+
+ // Keep the application running
+ Runtime.getRuntime().addShutdownHook(new Thread(() -> {
+ auditCheckTask.stop();
+ LOGGER.error("Audit Tool stopped.");
Review Comment:
already resolved
##########
inlong-audit/audit-tool/src/test/java/org/apache/inlong/tool/service/AuditMetricServiceTest.java:
##########
@@ -19,54 +19,35 @@
import org.apache.inlong.audit.tool.config.AppConfig;
import org.apache.inlong.audit.tool.entity.AuditMetric;
-import org.apache.inlong.audit.tool.mapper.AuditMapper;
import org.apache.inlong.audit.tool.service.AuditMetricService;
import org.apache.inlong.audit.tool.util.AuditSQLUtil;
-import org.apache.ibatis.session.SqlSession;
import org.junit.jupiter.api.Test;
-import org.mockito.MockedStatic;
-import org.mockito.Mockito;
-import java.util.Collections;
+import java.time.LocalDateTime;
+import java.time.format.DateTimeFormatter;
import java.util.List;
-import static org.junit.jupiter.api.Assertions.*;
-
public class AuditMetricServiceTest {
@Test
- public void getStorageAuditMetricsReturnsEmptyListWhenNoDataFound() {
- AuditMetricService auditMetricService = new AuditMetricService();
-
- try (MockedStatic<AuditSQLUtil> sqlUtilMockedStatic = Mockito.mockStatic(AuditSQLUtil.class)) {
- SqlSession sqlSessionMock = Mockito.mock(SqlSession.class);
- AuditMapper auditMapperMock = Mockito.mock(AuditMapper.class);
-
- sqlUtilMockedStatic.when(AuditSQLUtil::getSqlSession).thenReturn(sqlSessionMock);
- Mockito.when(sqlSessionMock.getMapper(AuditMapper.class)).thenReturn(auditMapperMock);
- Mockito.when(auditMapperMock.getAuditMetrics(Mockito.anyString(), Mockito.anyString(), Mockito.anyString()))
- .thenReturn(Collections.emptyList());
-
- List<AuditMetric> result = auditMetricService.getStorageAuditMetrics("nonexistentId", "2023-01-01 00:00:00",
- "2023-01-01 01:00:00");
- assertTrue(result.isEmpty());
- }
- }
-
- @Test
- public void getStorageAuditMetricsHandlesInvalidTimestampsGracefully() {
+ public void testGetDataProxyAuditMetrics() {
// Query service initialization
AppConfig appConfig = new AppConfig();
AuditSQLUtil.initialize(appConfig.getProperties());
AuditMetricService auditMetricService = new AuditMetricService();
- String invalidStartLogTs = "invalid-timestamp";
- String invalidEndLogTs = "invalid-timestamp";
+ DateTimeFormatter formatter = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss");
+ String endLogTs = LocalDateTime.now().minusMinutes(1).format(formatter);
+ String startLogTs = LocalDateTime.now().minusMinutes(5).format(formatter);
- List<AuditMetric> result = auditMetricService.getStorageAuditMetrics("5", invalidStartLogTs, invalidEndLogTs);
+ // Search for relevant data
+ List<AuditMetric> dataproxyAuditMetrics =
+ auditMetricService.getStorageAuditMetrics("5", startLogTs, endLogTs);
- assertTrue(result.isEmpty());
+ for (AuditMetric auditMetric : dataproxyAuditMetrics) {
+ System.out.println(auditMetric.getInlongGroupId() + " " + auditMetric.getInlongStreamId() + " "
+ + auditMetric.getCount());
+ }
Review Comment:
already resolved
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-11999][Audit] Add alert evaluation and periodic audit check task [inlong]
Posted by "fuweng11 (via GitHub)" <gi...@apache.org>.
fuweng11 commented on code in PR #12002:
URL: https://github.com/apache/inlong/pull/12002#discussion_r2347401441
##########
inlong-audit/audit-tool/src/main/java/org/apache/inlong/audit/tool/AuditToolMain.java:
##########
@@ -17,9 +17,48 @@
package org.apache.inlong.audit.tool;
+import org.apache.inlong.audit.tool.config.AppConfig;
+import org.apache.inlong.audit.tool.evaluator.AlertEvaluator;
+import org.apache.inlong.audit.tool.manager.AuditAlertRuleManager;
+import org.apache.inlong.audit.tool.reporter.PrometheusReporter;
+import org.apache.inlong.audit.tool.task.AuditCheckTask;
+import org.apache.inlong.audit.tool.util.AuditSQLUtil;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
public class AuditToolMain {
+ private static final Logger LOGGER = LoggerFactory.getLogger(AuditAlertRuleManager.class);
+
public static void main(String[] args) {
+ // Load application configuration
+ AppConfig appConfig = new AppConfig();
+
+ // Initialize auditAlertRule Manager
+ AuditAlertRuleManager auditAlertRuleManager = AuditAlertRuleManager.getInstance();
+ auditAlertRuleManager.init(appConfig);
+ auditAlertRuleManager.schedule();
+
+ // Initialize reporters
+ PrometheusReporter prometheusReporter = new PrometheusReporter();
+ prometheusReporter.init(appConfig.getPrometheusConfig());
+
+ // Database query initialization
+ AuditSQLUtil.initialize(appConfig.getProperties());
+
+ // Initialize alert evaluator
+ AlertEvaluator alertEvaluator = new AlertEvaluator(prometheusReporter, auditAlertRuleManager);
+ AuditCheckTask auditCheckTask =
+ new AuditCheckTask(auditAlertRuleManager, alertEvaluator, appConfig);
+ auditCheckTask.start();
+
+ // Keep the application running
+ Runtime.getRuntime().addShutdownHook(new Thread(() -> {
+ auditCheckTask.stop();
+ LOGGER.error("Audit Tool stopped.");
Review Comment:
```suggestion
LOGGER.info("Audit Tool stopped.");
```
##########
inlong-audit/audit-tool/src/main/java/org/apache/inlong/audit/tool/AuditToolMain.java:
##########
@@ -17,9 +17,48 @@
package org.apache.inlong.audit.tool;
+import org.apache.inlong.audit.tool.config.AppConfig;
+import org.apache.inlong.audit.tool.evaluator.AlertEvaluator;
+import org.apache.inlong.audit.tool.manager.AuditAlertRuleManager;
+import org.apache.inlong.audit.tool.reporter.PrometheusReporter;
+import org.apache.inlong.audit.tool.task.AuditCheckTask;
+import org.apache.inlong.audit.tool.util.AuditSQLUtil;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
public class AuditToolMain {
+ private static final Logger LOGGER = LoggerFactory.getLogger(AuditAlertRuleManager.class);
+
public static void main(String[] args) {
+ // Load application configuration
+ AppConfig appConfig = new AppConfig();
+
+ // Initialize auditAlertRule Manager
+ AuditAlertRuleManager auditAlertRuleManager = AuditAlertRuleManager.getInstance();
+ auditAlertRuleManager.init(appConfig);
+ auditAlertRuleManager.schedule();
+
+ // Initialize reporters
+ PrometheusReporter prometheusReporter = new PrometheusReporter();
+ prometheusReporter.init(appConfig.getPrometheusConfig());
+
+ // Database query initialization
+ AuditSQLUtil.initialize(appConfig.getProperties());
+
+ // Initialize alert evaluator
+ AlertEvaluator alertEvaluator = new AlertEvaluator(prometheusReporter, auditAlertRuleManager);
+ AuditCheckTask auditCheckTask =
+ new AuditCheckTask(auditAlertRuleManager, alertEvaluator, appConfig);
+ auditCheckTask.start();
+
+ // Keep the application running
+ Runtime.getRuntime().addShutdownHook(new Thread(() -> {
+ auditCheckTask.stop();
+ LOGGER.error("Audit Tool stopped.");
+ }));
+ LOGGER.error("Audit Tool started.");
Review Comment:
```suggestion
LOGGER.info("Audit Tool started.");
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org