You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/12/30 03:59:09 UTC
[GitHub] [hudi] boneanxs opened a new pull request, #7582: [HUDI-5488]Make sure Discrupt queue start first, then insert records
boneanxs opened a new pull request, #7582:
URL: https://github.com/apache/hudi/pull/7582
### Change Logs
_Describe context and summary for this change. Highlight if any code was copied._
We must to make sure to set up Disruptor's queue first, then producer can insert records to the queue. But currently we have no idea which thread start first, so this pr tries to fix it.
```java
CompletableFuture<Void> consuming = startConsumingAsync();
CompletableFuture<Void> producing = startProducingAsync();
```
Also, I think the test `TestDisruptorExecutionInSpark#testExecutor` and `TestDisruptorMessageQueue#testRecordReading` failures relate to this bug.
<img width="1585" alt="Screenshot 2022-12-29 at 10 07 21" src="https://user-images.githubusercontent.com/10115332/210033047-7b3573ec-c43b-44b3-a898-c4269b6bfd14.png">
### Impact
_Describe any public API or user-facing feature change or any performance impact._
none
### Risk level (write none, low medium or high below)
_If medium or high, explain what verification was done to mitigate the risks._
none
### Documentation Update
_Describe any necessary documentation update if there is any new feature, config, or user-facing change_
- _The config description must be updated if new configs are added or the default value of the configs are changed_
- _Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the
ticket number here and follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make
changes to the website._
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] boneanxs commented on a diff in pull request #7582: [HUDI-5488] Make sure Disrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
boneanxs commented on code in PR #7582:
URL: https://github.com/apache/hudi/pull/7582#discussion_r1070202016
##########
hudi-common/src/main/java/org/apache/hudi/common/util/queue/DisruptorMessageQueue.java:
##########
@@ -60,6 +61,10 @@ public long size() {
@Override
public void insertRecord(I value) throws Exception {
+ if (!isStart) {
Review Comment:
> We assert in insertRecord that the queue is started
Here just assert the queue is started, in case others use this queue without calling the `start`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7582: [HUDI-5488] Make sure Disrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7582:
URL: https://github.com/apache/hudi/pull/7582#issuecomment-1380070985
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14049",
"triggerID" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"triggerType" : "PUSH"
}, {
"hash" : "bb11dd9841a68f01289f371e3e6fc72ca84cbf25",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14269",
"triggerID" : "bb11dd9841a68f01289f371e3e6fc72ca84cbf25",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* bb11dd9841a68f01289f371e3e6fc72ca84cbf25 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14269)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7582: [HUDI-5488] Make sure Disrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7582:
URL: https://github.com/apache/hudi/pull/7582#issuecomment-1383232745
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14049",
"triggerID" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"triggerType" : "PUSH"
}, {
"hash" : "bb11dd9841a68f01289f371e3e6fc72ca84cbf25",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14269",
"triggerID" : "bb11dd9841a68f01289f371e3e6fc72ca84cbf25",
"triggerType" : "PUSH"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14316",
"triggerID" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"triggerType" : "PUSH"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14329",
"triggerID" : "1383184205",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 0f230ae48d0179365c2658c5db3fe7819eaca252 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14316) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14329)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7582: [HUDI-5488]Make sure Discrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7582:
URL: https://github.com/apache/hudi/pull/7582#issuecomment-1367712487
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 424e4a25b477f8aab3f8b4e5590023d20cca98f3 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7582: [HUDI-5488]Make sure Discrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7582:
URL: https://github.com/apache/hudi/pull/7582#issuecomment-1367714336
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14049",
"triggerID" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 424e4a25b477f8aab3f8b4e5590023d20cca98f3 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14049)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7582: [HUDI-5488] Make sure Disrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7582:
URL: https://github.com/apache/hudi/pull/7582#issuecomment-1384453173
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14049",
"triggerID" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"triggerType" : "PUSH"
}, {
"hash" : "bb11dd9841a68f01289f371e3e6fc72ca84cbf25",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14269",
"triggerID" : "bb11dd9841a68f01289f371e3e6fc72ca84cbf25",
"triggerType" : "PUSH"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14316",
"triggerID" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"triggerType" : "PUSH"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14329",
"triggerID" : "1383184205",
"triggerType" : "MANUAL"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14330",
"triggerID" : "1383334669",
"triggerType" : "MANUAL"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14343",
"triggerID" : "1383864028",
"triggerType" : "MANUAL"
}, {
"hash" : "a94ec9cf09ce55b684fa059ce1ede73bead0e991",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14344",
"triggerID" : "a94ec9cf09ce55b684fa059ce1ede73bead0e991",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a94ec9cf09ce55b684fa059ce1ede73bead0e991 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14344)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] boneanxs commented on pull request #7582: [HUDI-5488] Make sure Disrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
boneanxs commented on PR #7582:
URL: https://github.com/apache/hudi/pull/7582#issuecomment-1383184205
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] f7753 commented on pull request #7582: [HUDI-5488]Make sure Discrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
f7753 commented on PR #7582:
URL: https://github.com/apache/hudi/pull/7582#issuecomment-1368316681
typo in title
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7582: [HUDI-5488] Make sure Disrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7582:
URL: https://github.com/apache/hudi/pull/7582#issuecomment-1385273943
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14049",
"triggerID" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"triggerType" : "PUSH"
}, {
"hash" : "bb11dd9841a68f01289f371e3e6fc72ca84cbf25",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14269",
"triggerID" : "bb11dd9841a68f01289f371e3e6fc72ca84cbf25",
"triggerType" : "PUSH"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14316",
"triggerID" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"triggerType" : "PUSH"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14329",
"triggerID" : "1383184205",
"triggerType" : "MANUAL"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14330",
"triggerID" : "1383334669",
"triggerType" : "MANUAL"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14343",
"triggerID" : "1383864028",
"triggerType" : "MANUAL"
}, {
"hash" : "a94ec9cf09ce55b684fa059ce1ede73bead0e991",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14344",
"triggerID" : "a94ec9cf09ce55b684fa059ce1ede73bead0e991",
"triggerType" : "PUSH"
}, {
"hash" : "a94ec9cf09ce55b684fa059ce1ede73bead0e991",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14365",
"triggerID" : "1385170676",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* a94ec9cf09ce55b684fa059ce1ede73bead0e991 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14344) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14365)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] boneanxs commented on a diff in pull request #7582: [HUDI-5488] Make sure Disrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
boneanxs commented on code in PR #7582:
URL: https://github.com/apache/hudi/pull/7582#discussion_r1066505409
##########
hudi-common/src/main/java/org/apache/hudi/common/util/queue/DisruptorMessageQueue.java:
##########
@@ -60,6 +61,12 @@ public long size() {
@Override
public void insertRecord(I value) throws Exception {
+ if (!isStart) {
+ synchronized (this) {
+ wait();
+ }
+ }
Review Comment:
Sure, will change this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on pull request #7582: [HUDI-5488]Make sure Disrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on PR #7582:
URL: https://github.com/apache/hudi/pull/7582#issuecomment-1377612744
@boneanxs @zhangyue19921010
We should do what @zhangyue19921010 suggested in his review
1. Introduce empty `setup` method in the base class
2. Move current contents of the `doConsume` for Disruptor into `setup`
3. Keep `doConsume` for Disruptor empty
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7582: [HUDI-5488] Make sure Disrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7582:
URL: https://github.com/apache/hudi/pull/7582#issuecomment-1379883272
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14049",
"triggerID" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"triggerType" : "PUSH"
}, {
"hash" : "bb11dd9841a68f01289f371e3e6fc72ca84cbf25",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "bb11dd9841a68f01289f371e3e6fc72ca84cbf25",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 424e4a25b477f8aab3f8b4e5590023d20cca98f3 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14049)
* bb11dd9841a68f01289f371e3e6fc72ca84cbf25 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7582: [HUDI-5488] Make sure Disrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7582:
URL: https://github.com/apache/hudi/pull/7582#issuecomment-1382677671
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14049",
"triggerID" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"triggerType" : "PUSH"
}, {
"hash" : "bb11dd9841a68f01289f371e3e6fc72ca84cbf25",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14269",
"triggerID" : "bb11dd9841a68f01289f371e3e6fc72ca84cbf25",
"triggerType" : "PUSH"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14316",
"triggerID" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0f230ae48d0179365c2658c5db3fe7819eaca252 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14316)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7582: [HUDI-5488] Make sure Disrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7582:
URL: https://github.com/apache/hudi/pull/7582#issuecomment-1383472837
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14049",
"triggerID" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"triggerType" : "PUSH"
}, {
"hash" : "bb11dd9841a68f01289f371e3e6fc72ca84cbf25",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14269",
"triggerID" : "bb11dd9841a68f01289f371e3e6fc72ca84cbf25",
"triggerType" : "PUSH"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14316",
"triggerID" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"triggerType" : "PUSH"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14329",
"triggerID" : "1383184205",
"triggerType" : "MANUAL"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14330",
"triggerID" : "1383334669",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 0f230ae48d0179365c2658c5db3fe7819eaca252 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14316) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14329) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14330)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #7582: [HUDI-5488] Make sure Disrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on code in PR #7582:
URL: https://github.com/apache/hudi/pull/7582#discussion_r1072998596
##########
hudi-common/src/main/java/org/apache/hudi/common/util/queue/DisruptorMessageQueue.java:
##########
@@ -60,6 +61,10 @@ public long size() {
@Override
public void insertRecord(I value) throws Exception {
+ if (!isStart) {
Review Comment:
@boneanxs we need to keep in mind that this lies on the hot-path of being executed for every record, so we should keep the bar really high in terms of what we allow to be executed for every record.
That being said, i also see your point of adding it to make sure it would fail definitively unless queue has been started
##########
hudi-common/src/main/java/org/apache/hudi/common/util/queue/DisruptorMessageQueue.java:
##########
@@ -60,6 +61,10 @@ public long size() {
@Override
public void insertRecord(I value) throws Exception {
+ if (!isStart) {
Review Comment:
Let's keep it, please rename it to `isStarted`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7582: [HUDI-5488] Make sure Disrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7582:
URL: https://github.com/apache/hudi/pull/7582#issuecomment-1396342702
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14049",
"triggerID" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"triggerType" : "PUSH"
}, {
"hash" : "bb11dd9841a68f01289f371e3e6fc72ca84cbf25",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14269",
"triggerID" : "bb11dd9841a68f01289f371e3e6fc72ca84cbf25",
"triggerType" : "PUSH"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14316",
"triggerID" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"triggerType" : "PUSH"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14329",
"triggerID" : "1383184205",
"triggerType" : "MANUAL"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14330",
"triggerID" : "1383334669",
"triggerType" : "MANUAL"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14343",
"triggerID" : "1383864028",
"triggerType" : "MANUAL"
}, {
"hash" : "a94ec9cf09ce55b684fa059ce1ede73bead0e991",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14344",
"triggerID" : "a94ec9cf09ce55b684fa059ce1ede73bead0e991",
"triggerType" : "PUSH"
}, {
"hash" : "a94ec9cf09ce55b684fa059ce1ede73bead0e991",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14365",
"triggerID" : "1385170676",
"triggerType" : "MANUAL"
}, {
"hash" : "80fedf433f5b2c83c28f2b6ec4d50a92b6911e06",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14426",
"triggerID" : "80fedf433f5b2c83c28f2b6ec4d50a92b6911e06",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a94ec9cf09ce55b684fa059ce1ede73bead0e991 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14344) Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14365)
* 80fedf433f5b2c83c28f2b6ec4d50a92b6911e06 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14426)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] boneanxs commented on a diff in pull request #7582: [HUDI-5488]Make sure Discrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
boneanxs commented on code in PR #7582:
URL: https://github.com/apache/hudi/pull/7582#discussion_r1059702584
##########
hudi-common/src/main/java/org/apache/hudi/common/util/queue/DisruptorMessageQueue.java:
##########
@@ -60,6 +61,12 @@ public long size() {
@Override
public void insertRecord(I value) throws Exception {
+ if (!isStart) {
+ synchronized (this) {
+ wait();
+ }
+ }
Review Comment:
Oops,forgot to add a double check here.
```java
if(!isStart) {
synchronized(this) {
if(!isStart) {
wait();
}
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on pull request #7582: [HUDI-5488]Make sure Discrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on PR #7582:
URL: https://github.com/apache/hudi/pull/7582#issuecomment-1367801102
For example something like this
```
public class DisruptorExecutor<I, O, E> extends BaseHoodieQueueBasedExecutor<I, O, E> {
public DisruptorExecutor(final Option<Integer> bufferSize, final Iterator<I> inputItr,
HoodieConsumer<O, E> consumer, Function<I, O> transformFunction, Option<String> waitStrategy, Runnable preExecuteRunnable) {
this(bufferSize, Collections.singletonList(new IteratorBasedQueueProducer<>(inputItr)), Option.of(consumer),
transformFunction, waitStrategy, preExecuteRunnable);
}
public DisruptorExecutor(final Option<Integer> bufferSize, List<HoodieProducer<I>> producers,
Option<HoodieConsumer<O, E>> consumer, final Function<I, O> transformFunction,
final Option<String> waitStrategy, Runnable preExecuteRunnable) {
super(producers, consumer, new DisruptorMessageQueue<>(bufferSize, transformFunction, waitStrategy, producers.size(), preExecuteRunnable), preExecuteRunnable);
}
@Override
protected void doConsume(HoodieMessageQueue<I, O> queue, HoodieConsumer<O, E> consumer) {
// no-op
// will do consuming actions in disruptor
}
@Override
public void setup() {
DisruptorMessageQueue<I, O> disruptorQueue = (DisruptorMessageQueue<I, O>) queue;
// Before we start producing, we need to set up Disruptor's queue
disruptorQueue.setHandlers(consumer.get());
disruptorQueue.start();
}
}
```
and
```
public void setup(){}
/**
* Main API to run both production and consumption.
*/
@Override
public E execute() {
try {
checkState(this.consumer.isPresent());
setup();
// Start consuming/producing asynchronously
CompletableFuture<Void> consuming = startConsumingAsync();
CompletableFuture<Void> producing = startProducingAsync();
// NOTE: To properly support mode when there's no consumer, we have to fall back
// to producing future as the trigger for us to shut down the queue
return producing.thenCombine(consuming, (aVoid, anotherVoid) -> null)
.whenComplete((ignored, throwable) -> {
// Close the queue to release the resources
queue.close();
})
.thenApply(ignored -> consumer.get().finish())
// Block until producing and consuming both finish
.get();
} catch (Exception e) {
if (e instanceof InterruptedException) {
// In case {@code InterruptedException} was thrown, resetting the interrupted flag
// of the thread, we reset it (to true) again to permit subsequent handlers
// to be interrupted as well
Thread.currentThread().interrupt();
}
throw new HoodieException(e);
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on pull request #7582: [HUDI-5488] Make sure Disrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on PR #7582:
URL: https://github.com/apache/hudi/pull/7582#issuecomment-1383334669
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7582: [HUDI-5488] Make sure Disrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7582:
URL: https://github.com/apache/hudi/pull/7582#issuecomment-1383949622
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14049",
"triggerID" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"triggerType" : "PUSH"
}, {
"hash" : "bb11dd9841a68f01289f371e3e6fc72ca84cbf25",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14269",
"triggerID" : "bb11dd9841a68f01289f371e3e6fc72ca84cbf25",
"triggerType" : "PUSH"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14316",
"triggerID" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"triggerType" : "PUSH"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14329",
"triggerID" : "1383184205",
"triggerType" : "MANUAL"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14330",
"triggerID" : "1383334669",
"triggerType" : "MANUAL"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14343",
"triggerID" : "1383864028",
"triggerType" : "MANUAL"
}, {
"hash" : "a94ec9cf09ce55b684fa059ce1ede73bead0e991",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14344",
"triggerID" : "a94ec9cf09ce55b684fa059ce1ede73bead0e991",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0f230ae48d0179365c2658c5db3fe7819eaca252 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14316) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14329) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14330) Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14343)
* a94ec9cf09ce55b684fa059ce1ede73bead0e991 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14344)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] boneanxs commented on pull request #7582: [HUDI-5488]Make sure Disrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
boneanxs commented on PR #7582:
URL: https://github.com/apache/hudi/pull/7582#issuecomment-1376703720
Gentle ping @alexeykudinkin
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #7582: [HUDI-5488]Make sure Disrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on code in PR #7582:
URL: https://github.com/apache/hudi/pull/7582#discussion_r1066070324
##########
hudi-common/src/main/java/org/apache/hudi/common/util/queue/DisruptorMessageQueue.java:
##########
@@ -60,6 +61,12 @@ public long size() {
@Override
public void insertRecord(I value) throws Exception {
+ if (!isStart) {
+ synchronized (this) {
+ wait();
+ }
+ }
Review Comment:
@boneanxs Disruptor is a *lock-free* message queue, let's avoid taking the locks in the hot-path to avoid defeating its purpose.
Instead we should do the following:
1. We set the flag `started` in the `start` method
2. We assert in `insertRecord` that the queue is started
3. We fix the start-up sequence for Disruptor to make sure we start consumer before producer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] boneanxs commented on pull request #7582: [HUDI-5488]Make sure Discrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
boneanxs commented on PR #7582:
URL: https://github.com/apache/hudi/pull/7582#issuecomment-1367711026
@alexeykudinkin @zhangyue19921010 Could you please help to take a look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7582: [HUDI-5488]Make sure Discrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7582:
URL: https://github.com/apache/hudi/pull/7582#issuecomment-1367754215
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14049",
"triggerID" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 424e4a25b477f8aab3f8b4e5590023d20cca98f3 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14049)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] boneanxs commented on pull request #7582: [HUDI-5488]Make sure Discrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
boneanxs commented on PR #7582:
URL: https://github.com/apache/hudi/pull/7582#issuecomment-1368159010
> But do we need to have a synchronized check for each record consuming(performance concern)?
This fix won't do the synchronize check for each record, it will only block the consuming when the queue is not start yet.
But I think adding a new `setup()` method is also an appropriate way. Pending this for a while if @alexeykudinkin has other suggestion. Or I'll add the `setup()` to fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7582: [HUDI-5488] Make sure Disrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7582:
URL: https://github.com/apache/hudi/pull/7582#issuecomment-1383930041
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14049",
"triggerID" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"triggerType" : "PUSH"
}, {
"hash" : "bb11dd9841a68f01289f371e3e6fc72ca84cbf25",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14269",
"triggerID" : "bb11dd9841a68f01289f371e3e6fc72ca84cbf25",
"triggerType" : "PUSH"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14316",
"triggerID" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"triggerType" : "PUSH"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14329",
"triggerID" : "1383184205",
"triggerType" : "MANUAL"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14330",
"triggerID" : "1383334669",
"triggerType" : "MANUAL"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14343",
"triggerID" : "1383864028",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 0f230ae48d0179365c2658c5db3fe7819eaca252 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14316) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14329) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14330) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14343)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7582: [HUDI-5488] Make sure Disrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7582:
URL: https://github.com/apache/hudi/pull/7582#issuecomment-1382654877
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14049",
"triggerID" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"triggerType" : "PUSH"
}, {
"hash" : "bb11dd9841a68f01289f371e3e6fc72ca84cbf25",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14269",
"triggerID" : "bb11dd9841a68f01289f371e3e6fc72ca84cbf25",
"triggerType" : "PUSH"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* bb11dd9841a68f01289f371e3e6fc72ca84cbf25 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14269)
* 0f230ae48d0179365c2658c5db3fe7819eaca252 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7582: [HUDI-5488] Make sure Disrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7582:
URL: https://github.com/apache/hudi/pull/7582#issuecomment-1383940824
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14049",
"triggerID" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"triggerType" : "PUSH"
}, {
"hash" : "bb11dd9841a68f01289f371e3e6fc72ca84cbf25",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14269",
"triggerID" : "bb11dd9841a68f01289f371e3e6fc72ca84cbf25",
"triggerType" : "PUSH"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14316",
"triggerID" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"triggerType" : "PUSH"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14329",
"triggerID" : "1383184205",
"triggerType" : "MANUAL"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14330",
"triggerID" : "1383334669",
"triggerType" : "MANUAL"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14343",
"triggerID" : "1383864028",
"triggerType" : "MANUAL"
}, {
"hash" : "a94ec9cf09ce55b684fa059ce1ede73bead0e991",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a94ec9cf09ce55b684fa059ce1ede73bead0e991",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0f230ae48d0179365c2658c5db3fe7819eaca252 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14316) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14329) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14330) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14343)
* a94ec9cf09ce55b684fa059ce1ede73bead0e991 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7582: [HUDI-5488] Make sure Disrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7582:
URL: https://github.com/apache/hudi/pull/7582#issuecomment-1379888331
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14049",
"triggerID" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"triggerType" : "PUSH"
}, {
"hash" : "bb11dd9841a68f01289f371e3e6fc72ca84cbf25",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14269",
"triggerID" : "bb11dd9841a68f01289f371e3e6fc72ca84cbf25",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 424e4a25b477f8aab3f8b4e5590023d20cca98f3 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14049)
* bb11dd9841a68f01289f371e3e6fc72ca84cbf25 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14269)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #7582: [HUDI-5488] Make sure Disrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on code in PR #7582:
URL: https://github.com/apache/hudi/pull/7582#discussion_r1069766400
##########
hudi-common/src/main/java/org/apache/hudi/common/util/queue/DisruptorMessageQueue.java:
##########
@@ -60,6 +61,10 @@ public long size() {
@Override
public void insertRecord(I value) throws Exception {
+ if (!isStart) {
Review Comment:
I don't think we need this, it would throw anyway if the queue has not been started yet
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] boneanxs commented on pull request #7582: [HUDI-5488] Make sure Disrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
boneanxs commented on PR #7582:
URL: https://github.com/apache/hudi/pull/7582#issuecomment-1383543941
The test failure should not relate to this, I can run it successfully in local side:
<img width="480" alt="Screenshot 2023-01-16 at 14 15 26" src="https://user-images.githubusercontent.com/10115332/212610479-d0234bca-4250-4aa5-a788-749362daaa3d.png">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7582: [HUDI-5488] Make sure Disrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7582:
URL: https://github.com/apache/hudi/pull/7582#issuecomment-1385988025
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14049",
"triggerID" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"triggerType" : "PUSH"
}, {
"hash" : "bb11dd9841a68f01289f371e3e6fc72ca84cbf25",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14269",
"triggerID" : "bb11dd9841a68f01289f371e3e6fc72ca84cbf25",
"triggerType" : "PUSH"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14316",
"triggerID" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"triggerType" : "PUSH"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14329",
"triggerID" : "1383184205",
"triggerType" : "MANUAL"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14330",
"triggerID" : "1383334669",
"triggerType" : "MANUAL"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14343",
"triggerID" : "1383864028",
"triggerType" : "MANUAL"
}, {
"hash" : "a94ec9cf09ce55b684fa059ce1ede73bead0e991",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14344",
"triggerID" : "a94ec9cf09ce55b684fa059ce1ede73bead0e991",
"triggerType" : "PUSH"
}, {
"hash" : "a94ec9cf09ce55b684fa059ce1ede73bead0e991",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14365",
"triggerID" : "1385170676",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* a94ec9cf09ce55b684fa059ce1ede73bead0e991 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14344) Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14365)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7582: [HUDI-5488] Make sure Disrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7582:
URL: https://github.com/apache/hudi/pull/7582#issuecomment-1396337898
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14049",
"triggerID" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"triggerType" : "PUSH"
}, {
"hash" : "bb11dd9841a68f01289f371e3e6fc72ca84cbf25",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14269",
"triggerID" : "bb11dd9841a68f01289f371e3e6fc72ca84cbf25",
"triggerType" : "PUSH"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14316",
"triggerID" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"triggerType" : "PUSH"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14329",
"triggerID" : "1383184205",
"triggerType" : "MANUAL"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14330",
"triggerID" : "1383334669",
"triggerType" : "MANUAL"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14343",
"triggerID" : "1383864028",
"triggerType" : "MANUAL"
}, {
"hash" : "a94ec9cf09ce55b684fa059ce1ede73bead0e991",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14344",
"triggerID" : "a94ec9cf09ce55b684fa059ce1ede73bead0e991",
"triggerType" : "PUSH"
}, {
"hash" : "a94ec9cf09ce55b684fa059ce1ede73bead0e991",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14365",
"triggerID" : "1385170676",
"triggerType" : "MANUAL"
}, {
"hash" : "80fedf433f5b2c83c28f2b6ec4d50a92b6911e06",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "80fedf433f5b2c83c28f2b6ec4d50a92b6911e06",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a94ec9cf09ce55b684fa059ce1ede73bead0e991 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14344) Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14365)
* 80fedf433f5b2c83c28f2b6ec4d50a92b6911e06 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7582: [HUDI-5488] Make sure Disrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7582:
URL: https://github.com/apache/hudi/pull/7582#issuecomment-1383337440
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14049",
"triggerID" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"triggerType" : "PUSH"
}, {
"hash" : "bb11dd9841a68f01289f371e3e6fc72ca84cbf25",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14269",
"triggerID" : "bb11dd9841a68f01289f371e3e6fc72ca84cbf25",
"triggerType" : "PUSH"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14316",
"triggerID" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"triggerType" : "PUSH"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14329",
"triggerID" : "1383184205",
"triggerType" : "MANUAL"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14330",
"triggerID" : "1383334669",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 0f230ae48d0179365c2658c5db3fe7819eaca252 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14316) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14329) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14330)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on pull request #7582: [HUDI-5488] Make sure Disrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on PR #7582:
URL: https://github.com/apache/hudi/pull/7582#issuecomment-1396402031
Landing this since there's no material changes since last revision, and CI passed previously
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin merged pull request #7582: [HUDI-5488] Make sure Disrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
alexeykudinkin merged PR #7582:
URL: https://github.com/apache/hudi/pull/7582
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on a diff in pull request #7582: [HUDI-5488]Make sure Discrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on code in PR #7582:
URL: https://github.com/apache/hudi/pull/7582#discussion_r1059638481
##########
hudi-common/src/main/java/org/apache/hudi/common/util/queue/DisruptorMessageQueue.java:
##########
@@ -105,6 +113,12 @@ protected void setHandlers(HoodieConsumer consumer) {
protected void start() {
queue.start();
+ synchronized (this) {
+ if (!isStart) {
+ isStart = true;
+ notifyAll();
+ }
+ }
Review Comment:
```suggestion
synchronized (this) {
if (!isStart) {
queue.start();
isStart = true;
notifyAll();
}
}
```
##########
hudi-common/src/main/java/org/apache/hudi/common/util/queue/DisruptorMessageQueue.java:
##########
@@ -60,6 +61,12 @@ public long size() {
@Override
public void insertRecord(I value) throws Exception {
+ if (!isStart) {
+ synchronized (this) {
+ wait();
+ }
+ }
Review Comment:
```suggestion
synchronized (this) {
If (!isStart) {
wait();
}
}
```
We need to let synchronized cover isStart check in case that `isStart ` is changed after `if (!isStart)` but before `wait();`
But in this way, we will do synchronized check for each record ingestion right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on pull request #7582: [HUDI-5488]Make sure Discrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on PR #7582:
URL: https://github.com/apache/hudi/pull/7582#issuecomment-1370629340
This PR may relate to https://issues.apache.org/jira/browse/HUDI-5369
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] boneanxs commented on pull request #7582: [HUDI-5488] Make sure Disrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
boneanxs commented on PR #7582:
URL: https://github.com/apache/hudi/pull/7582#issuecomment-1385170676
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7582: [HUDI-5488] Make sure Disrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7582:
URL: https://github.com/apache/hudi/pull/7582#issuecomment-1382655864
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14049",
"triggerID" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"triggerType" : "PUSH"
}, {
"hash" : "bb11dd9841a68f01289f371e3e6fc72ca84cbf25",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14269",
"triggerID" : "bb11dd9841a68f01289f371e3e6fc72ca84cbf25",
"triggerType" : "PUSH"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14316",
"triggerID" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* bb11dd9841a68f01289f371e3e6fc72ca84cbf25 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14269)
* 0f230ae48d0179365c2658c5db3fe7819eaca252 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14316)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7582: [HUDI-5488] Make sure Disrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7582:
URL: https://github.com/apache/hudi/pull/7582#issuecomment-1383195752
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14049",
"triggerID" : "424e4a25b477f8aab3f8b4e5590023d20cca98f3",
"triggerType" : "PUSH"
}, {
"hash" : "bb11dd9841a68f01289f371e3e6fc72ca84cbf25",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14269",
"triggerID" : "bb11dd9841a68f01289f371e3e6fc72ca84cbf25",
"triggerType" : "PUSH"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14316",
"triggerID" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"triggerType" : "PUSH"
}, {
"hash" : "0f230ae48d0179365c2658c5db3fe7819eaca252",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14329",
"triggerID" : "1383184205",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 0f230ae48d0179365c2658c5db3fe7819eaca252 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14316) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14329)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] boneanxs commented on pull request #7582: [HUDI-5488] Make sure Disrupt queue start first, then insert records
Posted by GitBox <gi...@apache.org>.
boneanxs commented on PR #7582:
URL: https://github.com/apache/hudi/pull/7582#issuecomment-1383864028
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org