You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@inlong.apache.org by "flowers-59f (via GitHub)" <gi...@apache.org> on 2026/03/20 03:55:57 UTC
[PR] [INLONG-12104][Manager] Fix failure to logically delete associated data for data flow groups [inlong]
flowers-59f opened a new pull request, #12105:
URL: https://github.com/apache/inlong/pull/12105
Fixes https://github.com/apache/inlong/issues/12104
### Motivation
This PR ensures that when a data flow group is removed, all its associated records are deleted simultaneously.
### Modifications
1. Add "GroupStatus.CONFIG_DELETING" to the "allowedDeleteSubInfos" allowed state
```java
public static boolean allowedDeleteSubInfos(GroupStatus status) {
return status == GroupStatus.TO_BE_SUBMIT
|| status == GroupStatus.APPROVE_REJECTED
|| status == GroupStatus.CONFIG_DELETED
|| status == GroupStatus.CONFIG_DELETING;
```
In this way, the following code can be executed correctly.
```java
if (GroupStatus.allowedDeleteSubInfos(GroupStatus.forCode(entity.getStatus()))) {
streamService.logicDeleteAll(groupId, operator);
}
```
2. Cancel the judgment of the group's state when deleting the stream
In the original code you had written (StreamSouceServiceimpl.delete(...)) ), it seems that's exactly what was done.
```java
// Check if it can be delete
InlongGroupEntity groupEntity = groupMapper.selectByGroupId(entity.getInlongGroupId());
```
The logic for judging the status has been removed here.
Then here are my thoughts.
In many deletion operations of streams, the status check will be conducted, the status of the group is not allowed to be "GroupStatus.CONFIG_DELETING". I think this status check is unnecessary because at this moment, the status of the group is always "GroupStatus.CONFIG_DELETING". This judgment is meaningless and will cause the deletion process to be unable to proceed. And executing the deletion operation repeatedly does not seem to have any consequences, except for additional resource consumption. However, I think it should be restricted so that users are not allowed to click the delete button again while in deleting (currently, users can still click the delete button even when a group is deleting).However, will there be any other places where this deletion function is needed? Besides when deleting the data stream group, will these deletion functions of the "stream" be called when the "group" is in other status? If such a situation occurs, then modifying the method for judging th
ese status and allowing the "GroupStatus.CONFIG_DELETING" status would be a better approach.
3. Move the update operation of the data stream group after the last two deletion operations
```java
// logically delete the associated extension info
groupExtMapper.logicDeleteAllByGroupId(groupId);
// remove schedule
if (DATASYNC_OFFLINE_MODE.equals(entity.getInlongGroupMode())) {
try {
scheduleOperator.deleteByGroupIdOpt(entity.getInlongGroupId(), operator);
} catch (Exception e) {
LOGGER.warn("failed to delete schedule info for groupId={}, error msg: {}", groupId, e.getMessage());
}
}
entity.setIsDeleted(entity.getId());
entity.setStatus(GroupStatus.CONFIG_DELETED.getCode());
entity.setModifier(operator);
int rowCount = groupMapper.updateByIdentifierSelective(entity);
if (rowCount != InlongConstants.AFFECTED_ONE_ROW) {
LOGGER.error("inlong group has already updated for groupId={} curVersion={}", groupId, entity.getVersion());
throw new BusinessException(ErrorCodeEnum.CONFIG_EXPIRED);
}
```
In this way, the two deletion operations can be carried out correctly.
### Verifying this change
- [x] This change added tests and can be verified as follows:
I didn't write unit tests, but conducted a front-end and back-end integration test.
The results are as follows:
**Create a data flow group**
<img width="1556" height="204" alt="image" src="https://github.com/user-attachments/assets/ecf7ac58-ef51-4ba8-a53b-71be09c99b05" />
**Relevant data**
<img width="649" height="221" alt="image" src="https://github.com/user-attachments/assets/47ecf8c7-fa02-4338-a87e-7853241a2979" />
<img width="507" height="220" alt="image" src="https://github.com/user-attachments/assets/c7e6c987-7139-4a8a-939d-50103a76964c" />
<img width="596" height="222" alt="image" src="https://github.com/user-attachments/assets/08c232aa-2a90-455a-a092-1acfa789b0e1" />
**After deletion**
<img width="1557" height="302" alt="image" src="https://github.com/user-attachments/assets/e231d4b1-9de4-4ab8-be68-dcaa160e29e9" />
<img width="662" height="227" alt="image" src="https://github.com/user-attachments/assets/9795ba70-f314-4628-8d6e-f83b9b91955e" />
<img width="553" height="252" alt="image" src="https://github.com/user-attachments/assets/cf165a52-bb14-4619-a237-58c5ffe58516" />
<img width="500" height="222" alt="image" src="https://github.com/user-attachments/assets/c10b29f9-42b3-4967-8b6a-820773edbc22" />
The "is_deleted" field of the relevant data can be correctly modified.
### Documentation
- Does this pull request introduce a new feature? (no)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-12104][Manager] Fix failure to logically delete associated data for data flow groups [inlong]
Posted by "fuweng11 (via GitHub)" <gi...@apache.org>.
fuweng11 commented on PR #12105:
URL: https://github.com/apache/inlong/pull/12105#issuecomment-4095477789
The status check is essential and cannot be removed, as the deletion process involves asynchronous operations that interact with the Agent and sort. If the deletion command is executed repeatedly during this process, due to asynchronous operations, the Agent or sort task may make incorrect judgments. For instance, if a deletion is performed immediately after the Agent has just been issued a collection task, the requests received by the Agent may originate from different managers, temporarily causing data inconsistency. At this time, the order in which the Agent receives task deletion or addition commands may be inconsistent, leading to task anomalies.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-12104][Manager] Fix failure to logically delete associated data for data flow groups [inlong]
Posted by "flowers-59f (via GitHub)" <gi...@apache.org>.
flowers-59f commented on PR #12105:
URL: https://github.com/apache/inlong/pull/12105#issuecomment-4096790740
> Suggested modifications are as follows:
> `case DELETE: // delete process completed, then delete the group info groupService.updateStatus(groupId, GroupStatus.CONFIG_DELETED.getCode(), operator); groupService.delete(groupId, operator); break;`
OK, Thank you very much, I will modify my code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-12104][Manager] Fix failure to logically delete associated data for data flow groups [inlong]
Posted by "vernedeng (via GitHub)" <gi...@apache.org>.
vernedeng merged PR #12105:
URL: https://github.com/apache/inlong/pull/12105
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-12104][Manager] Fix failure to logically delete associated data for data flow groups [inlong]
Posted by "flowers-59f (via GitHub)" <gi...@apache.org>.
flowers-59f commented on PR #12105:
URL: https://github.com/apache/inlong/pull/12105#issuecomment-4096235068
> > > The status check is essential and cannot be removed, as the deletion process involves asynchronous operations that interact with the Agent and sort. If the deletion command is executed repeatedly during this process, due to asynchronous operations, the Agent or sort task may make incorrect judgments. For instance, if a deletion is performed immediately after the Agent has just been issued a collection task, the requests received by the Agent may originate from different managers, temporarily causing data inconsistency. At this time, the order in which the Agent receives task deletion or addition commands may be inconsistent, leading to task anomalies.
> >
> >
> > Then, when executing these deletion functions, can we check the current status and skip the status validation if it's already "GroupStatus.CONFIG_DELETING"? Or is there a better solution?
>
> Given that the following code is only referenced by group.delete, you can add "GroupStatus.CONFIG_DELETING" to this status check, or it is better to modify the group status to "GroupStatus.CONFIG_DELETED" before performing this status check `if (GroupStatus.allowedDeleteSubInfos(GroupStatus.forCode(entity.getStatus()))) { streamService.logicDeleteAll(groupId, operator); }`
I think it would be a good approach to change the status of the group to "GroupStatus.CONFIG_DELETED" in advance. Issues caused by the status check during streamService.logicDeleteAll(...) will also be resolved. But which of the following two positions is better for updating this status?
UpdateGroupCompleteListener.listen(...)
```java
case DELETE:
// delete process completed, then delete the group info
// first position
groupService.delete(groupId, operator);
break;
```
Because there are no transactions, if the status is changed in the first position, problems may arise if an exception occurs during the subsequent execution process.
InlongGroupServiceImpl.delete(...)
```java
// second position
// before deleting an inlong group, delete all inlong streams, sources, sinks, and other info under it
if (GroupStatus.allowedDeleteSubInfos(GroupStatus.forCode(entity.getStatus()))) {
streamService.logicDeleteAll(groupId, operator);
}
```
I think it would be better to make the group status change at the second position. However, in this case, the subsequent judgment would lose its significance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-12104][Manager] Fix failure to logically delete associated data for data flow groups [inlong]
Posted by "fuweng11 (via GitHub)" <gi...@apache.org>.
fuweng11 commented on PR #12105:
URL: https://github.com/apache/inlong/pull/12105#issuecomment-4095959430
> > The status check is essential and cannot be removed, as the deletion process involves asynchronous operations that interact with the Agent and sort. If the deletion command is executed repeatedly during this process, due to asynchronous operations, the Agent or sort task may make incorrect judgments. For instance, if a deletion is performed immediately after the Agent has just been issued a collection task, the requests received by the Agent may originate from different managers, temporarily causing data inconsistency. At this time, the order in which the Agent receives task deletion or addition commands may be inconsistent, leading to task anomalies.
>
> Then, when executing these deletion functions, can we check the current status and skip the status validation if it's already "GroupStatus.CONFIG_DELETING"? Or is there a better solution?
Given that the following code is only referenced by group.delete, you can add "GroupStatus.CONFIG_DELETING" to this status check, or it is better to modify the group status to "GroupStatus.CONFIG_DELETED" before performing this status check
`if (GroupStatus.allowedDeleteSubInfos(GroupStatus.forCode(entity.getStatus()))) {
streamService.logicDeleteAll(groupId, operator);
}`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-12104][Manager] Fix failure to logically delete associated data for data flow groups [inlong]
Posted by "fuweng11 (via GitHub)" <gi...@apache.org>.
fuweng11 commented on PR #12105:
URL: https://github.com/apache/inlong/pull/12105#issuecomment-4096758705
> ```java
> lowedDeleteSubInf
> ```
Suggested modifications are as follows:
`case DELETE:
// delete process completed, then delete the group info
groupService.updateStatus(groupId, GroupStatus.CONFIG_DELETED.getCode(), operator);
groupService.delete(groupId, operator);
break;`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-12104][Manager] Fix failure to logically delete associated data for data flow groups [inlong]
Posted by "fuweng11 (via GitHub)" <gi...@apache.org>.
fuweng11 commented on PR #12105:
URL: https://github.com/apache/inlong/pull/12105#issuecomment-4096760441
> > ```java
> > lowedDeleteSubInf
> > ```
>
> Suggested modifications are as follows:
`case DELETE:
// delete process completed, then delete the group info
groupService.updateStatus(groupId, GroupStatus.CONFIG_DELETED.getCode(), operator); groupService.delete(groupId, operator); break;`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [INLONG-12104][Manager] Fix failure to logically delete associated data for data flow groups [inlong]
Posted by "flowers-59f (via GitHub)" <gi...@apache.org>.
flowers-59f commented on PR #12105:
URL: https://github.com/apache/inlong/pull/12105#issuecomment-4095559971
> The status check is essential and cannot be removed, as the deletion process involves asynchronous operations that interact with the Agent and sort. If the deletion command is executed repeatedly during this process, due to asynchronous operations, the Agent or sort task may make incorrect judgments. For instance, if a deletion is performed immediately after the Agent has just been issued a collection task, the requests received by the Agent may originate from different managers, temporarily causing data inconsistency. At this time, the order in which the Agent receives task deletion or addition commands may be inconsistent, leading to task anomalies.
Then, when executing these deletion functions, can we check the current status and skip the status validation if it's already "GroupStatus.CONFIG_DELETING"? Or is there a better solution?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org