You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "kamronis (via GitHub)" <gi...@apache.org> on 2025/12/22 10:28:30 UTC
[PR] perf: optimize removeCommitMetadata method in HoodieCDCLogger [hudi]
kamronis opened a new pull request, #17669:
URL: https://github.com/apache/hudi/pull/17669
### Describe the issue this Pull Request addresses
Around 35-40% of put time in HoodieCDCLogger is removeCommitMetadata. This is because for each record schema comparison is called.
Before:
<img width="663" height="172" alt="Selection_074" src="https://github.com/user-attachments/assets/fd07c702-cedb-47be-a6c8-c67d53c620d3" />
After:
<img width="776" height="165" alt="Selection_076" src="https://github.com/user-attachments/assets/3ab7a0ca-731d-449d-804d-aeaf375837f4" />
### Summary and Changelog
Added the logic to construct record based on schema without heavy comparison.
### Impact
Performance improve for CDC.
### Risk Level
None
### Documentation Update
None
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Enough context is provided in the sections above
- [ ] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: optimize removeCommitMetadata method in HoodieCDCLogger [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #17669:
URL: https://github.com/apache/hudi/pull/17669#issuecomment-3681489083
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b0751150551c0d00d59284fb3f1a8a282b076a9",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10432",
"triggerID" : "4b0751150551c0d00d59284fb3f1a8a282b076a9",
"triggerType" : "PUSH"
}, {
"hash" : "a123e7938f95f52f0cc813ccab36ad33d7bdc4d3",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a123e7938f95f52f0cc813ccab36ad33d7bdc4d3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4b0751150551c0d00d59284fb3f1a8a282b076a9 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10432)
* a123e7938f95f52f0cc813ccab36ad33d7bdc4d3 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: optimize removeCommitMetadata method in HoodieCDCLogger [hudi]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun merged PR #17669:
URL: https://github.com/apache/hudi/pull/17669
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: optimize removeCommitMetadata method in HoodieCDCLogger [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #17669:
URL: https://github.com/apache/hudi/pull/17669#issuecomment-3682078374
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b0751150551c0d00d59284fb3f1a8a282b076a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10432",
"triggerID" : "4b0751150551c0d00d59284fb3f1a8a282b076a9",
"triggerType" : "PUSH"
}, {
"hash" : "a123e7938f95f52f0cc813ccab36ad33d7bdc4d3",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10433",
"triggerID" : "a123e7938f95f52f0cc813ccab36ad33d7bdc4d3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a123e7938f95f52f0cc813ccab36ad33d7bdc4d3 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10433)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: optimize removeCommitMetadata method in HoodieCDCLogger [hudi]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on PR #17669:
URL: https://github.com/apache/hudi/pull/17669#issuecomment-3685634723
> `HoodieAvroUtils` makes sense. We can add it in there first as it's a direct manipulation on a Avro GenericRecord. While we are actively removing it, `HoodieSchema` is still delegating to it. So, full removal will not happen so quickly.
I think, we can merge this PR as @kamronis proposed. And make refactoring under https://github.com/apache/hudi/issues/17679
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: optimize removeCommitMetadata method in HoodieCDCLogger [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #17669:
URL: https://github.com/apache/hudi/pull/17669#issuecomment-3681770078
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b0751150551c0d00d59284fb3f1a8a282b076a9",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10432",
"triggerID" : "4b0751150551c0d00d59284fb3f1a8a282b076a9",
"triggerType" : "PUSH"
}, {
"hash" : "a123e7938f95f52f0cc813ccab36ad33d7bdc4d3",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10433",
"triggerID" : "a123e7938f95f52f0cc813ccab36ad33d7bdc4d3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4b0751150551c0d00d59284fb3f1a8a282b076a9 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10432)
* a123e7938f95f52f0cc813ccab36ad33d7bdc4d3 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10433)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: optimize removeCommitMetadata method in HoodieCDCLogger [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #17669:
URL: https://github.com/apache/hudi/pull/17669#issuecomment-3685139787
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b0751150551c0d00d59284fb3f1a8a282b076a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10432",
"triggerID" : "4b0751150551c0d00d59284fb3f1a8a282b076a9",
"triggerType" : "PUSH"
}, {
"hash" : "a123e7938f95f52f0cc813ccab36ad33d7bdc4d3",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10433",
"triggerID" : "a123e7938f95f52f0cc813ccab36ad33d7bdc4d3",
"triggerType" : "PUSH"
}, {
"hash" : "a123e7938f95f52f0cc813ccab36ad33d7bdc4d3",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3685136689",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* a123e7938f95f52f0cc813ccab36ad33d7bdc4d3 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10433)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: optimize removeCommitMetadata method in HoodieCDCLogger [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #17669:
URL: https://github.com/apache/hudi/pull/17669#issuecomment-3681477726
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b0751150551c0d00d59284fb3f1a8a282b076a9",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10432",
"triggerID" : "4b0751150551c0d00d59284fb3f1a8a282b076a9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4b0751150551c0d00d59284fb3f1a8a282b076a9 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10432)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: optimize removeCommitMetadata method in HoodieCDCLogger [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on PR #17669:
URL: https://github.com/apache/hudi/pull/17669#issuecomment-3684906736
@yihua This PR should be different form the _O(1)_ comparison https://github.com/apache/hudi/pull/17672, which only affects workflows that involves `HoodieMetadataPayload`.
For this PR, the main optimization here is avoiding the costly recursive schema checks. To be specific, performance improvements here comes from replacing the highly generic, recursive, and safety-heavy utility method with a specialized, flat, and shallow implementation.
## Less recursion:
**Old way:**
It uses a recursive switch statement. For every field, it calls `rewriteRecordWithNewSchema` again. If you have a record with 20 fields, that’s 20+ method calls, stack pushes/pops, and type check.
**New Way:**
The new `getRecordWithoutMetadata` performs a single, flat for loop over the top-level fields of the schema. Since CDC data stripping usually only happens at the top level, this avoids the overhead of traversing the entire object tree.
## I believe it also **alleviates GC pressure**:
**Old Way:**
The utility creates several helper objects for every record processed:
- A `Deque<String> fieldNames` to track the breadcrumb path (mainly for error reporting IIUC).
- `Map<String, String> renameCols` (even if empty).
- Multiple string concatenations for `createNamePrefix` and `createFullName`
**New Way:**
- It creates exactly one `GenericData.Record`. No collections, no iterators, and no string builders are initialized per record.
CMIIW @kamronis, the performance optimization here should see the most increase for records that are deeply nested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: optimize removeCommitMetadata method in HoodieCDCLogger [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #17669:
URL: https://github.com/apache/hudi/pull/17669#discussion_r2641866950
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieCDCLogger.java:
##########
@@ -275,7 +275,20 @@ private CDCTransformer getTransformer() {
}
private GenericRecord removeCommitMetadata(GenericRecord record) {
- return record == null ? null : HoodieAvroUtils.rewriteRecordWithNewSchema(record, dataSchema.toAvroSchema(), Collections.emptyMap());
+ return record == null ? null : getRecordWithoutMetadata(record);
+ }
+
+ private GenericRecord getRecordWithoutMetadata(GenericRecord record) {
Review Comment:
yes, we can do this because there is prerequisite that no fields reordering or renaming, maybe we should check all the usages of `HoodieAvroUtils.rewriteRecordWithNewSchema` and replace it with this more performant way if it is the similiar use case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: optimize removeCommitMetadata method in HoodieCDCLogger [hudi]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on PR #17669:
URL: https://github.com/apache/hudi/pull/17669#issuecomment-3685136689
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: optimize removeCommitMetadata method in HoodieCDCLogger [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on PR #17669:
URL: https://github.com/apache/hudi/pull/17669#issuecomment-3685598271
> Hi @voonhous! Thank you for reply. You are right, nested field will see much performance improve this way. And we can put this in utility class, but which one should I put it in? I thought HoodieAvroUtils, but I see you are working on its removal now.
`HoodieAvroUtils` makes sense. We can add it in there first as it's a direct manipulation on a Avro GenericRecord. While we are actively removing it, `HoodieSchema` is still delegating to it. So, full removal will not happen so quickly.
As of now, i feel the most apt and suitable area to place it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: optimize removeCommitMetadata method in HoodieCDCLogger [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #17669:
URL: https://github.com/apache/hudi/pull/17669#discussion_r2641870952
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieCDCLogger.java:
##########
@@ -275,7 +275,20 @@ private CDCTransformer getTransformer() {
}
private GenericRecord removeCommitMetadata(GenericRecord record) {
- return record == null ? null : HoodieAvroUtils.rewriteRecordWithNewSchema(record, dataSchema.toAvroSchema(), Collections.emptyMap());
+ return record == null ? null : getRecordWithoutMetadata(record);
+ }
+
+ private GenericRecord getRecordWithoutMetadata(GenericRecord record) {
Review Comment:
fire an issue to trace this: https://github.com/apache/hudi/issues/17679
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: optimize removeCommitMetadata method in HoodieCDCLogger [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #17669:
URL: https://github.com/apache/hudi/pull/17669#issuecomment-3685142753
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b0751150551c0d00d59284fb3f1a8a282b076a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10432",
"triggerID" : "4b0751150551c0d00d59284fb3f1a8a282b076a9",
"triggerType" : "PUSH"
}, {
"hash" : "a123e7938f95f52f0cc813ccab36ad33d7bdc4d3",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10433",
"triggerID" : "a123e7938f95f52f0cc813ccab36ad33d7bdc4d3",
"triggerType" : "PUSH"
}, {
"hash" : "a123e7938f95f52f0cc813ccab36ad33d7bdc4d3",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10433",
"triggerID" : "3685136689",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* a123e7938f95f52f0cc813ccab36ad33d7bdc4d3 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10433)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: optimize removeCommitMetadata method in HoodieCDCLogger [hudi]
Posted by "kamronis (via GitHub)" <gi...@apache.org>.
kamronis commented on code in PR #17669:
URL: https://github.com/apache/hudi/pull/17669#discussion_r2641892448
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieCDCLogger.java:
##########
@@ -275,7 +275,20 @@ private CDCTransformer getTransformer() {
}
private GenericRecord removeCommitMetadata(GenericRecord record) {
- return record == null ? null : HoodieAvroUtils.rewriteRecordWithNewSchema(record, dataSchema.toAvroSchema(), Collections.emptyMap());
+ return record == null ? null : getRecordWithoutMetadata(record);
+ }
+
+ private GenericRecord getRecordWithoutMetadata(GenericRecord record) {
Review Comment:
@danny0405 I can take this task. Please assign to me
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: optimize removeCommitMetadata method in HoodieCDCLogger [hudi]
Posted by "kamronis (via GitHub)" <gi...@apache.org>.
kamronis commented on PR #17669:
URL: https://github.com/apache/hudi/pull/17669#issuecomment-3685021549
Hi @voonhous! Thank you for reply.
You are right, nested field will see much performance improve this way.
And we can put this in utility class, but which one should I put it in?
I thought HoodieAvroUtils, but I see you are working on its removal now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: optimize removeCommitMetadata method in HoodieCDCLogger [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #17669:
URL: https://github.com/apache/hudi/pull/17669#issuecomment-3681468801
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b0751150551c0d00d59284fb3f1a8a282b076a9",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4b0751150551c0d00d59284fb3f1a8a282b076a9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4b0751150551c0d00d59284fb3f1a8a282b076a9 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: optimize removeCommitMetadata method in HoodieCDCLogger [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #17669:
URL: https://github.com/apache/hudi/pull/17669#issuecomment-3685327351
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b0751150551c0d00d59284fb3f1a8a282b076a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10432",
"triggerID" : "4b0751150551c0d00d59284fb3f1a8a282b076a9",
"triggerType" : "PUSH"
}, {
"hash" : "a123e7938f95f52f0cc813ccab36ad33d7bdc4d3",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10433",
"triggerID" : "a123e7938f95f52f0cc813ccab36ad33d7bdc4d3",
"triggerType" : "PUSH"
}, {
"hash" : "a123e7938f95f52f0cc813ccab36ad33d7bdc4d3",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10433",
"triggerID" : "3685136689",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* a123e7938f95f52f0cc813ccab36ad33d7bdc4d3 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10433)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: optimize removeCommitMetadata method in HoodieCDCLogger [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #17669:
URL: https://github.com/apache/hudi/pull/17669#issuecomment-3681672675
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b0751150551c0d00d59284fb3f1a8a282b076a9",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10432",
"triggerID" : "4b0751150551c0d00d59284fb3f1a8a282b076a9",
"triggerType" : "PUSH"
}, {
"hash" : "a123e7938f95f52f0cc813ccab36ad33d7bdc4d3",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10433",
"triggerID" : "a123e7938f95f52f0cc813ccab36ad33d7bdc4d3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4b0751150551c0d00d59284fb3f1a8a282b076a9 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10432)
* a123e7938f95f52f0cc813ccab36ad33d7bdc4d3 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=10433)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: optimize removeCommitMetadata method in HoodieCDCLogger [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on PR #17669:
URL: https://github.com/apache/hudi/pull/17669#issuecomment-3684641507
cc @voonhous
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf: optimize removeCommitMetadata method in HoodieCDCLogger [hudi]
Posted by "kamronis (via GitHub)" <gi...@apache.org>.
kamronis commented on PR #17669:
URL: https://github.com/apache/hudi/pull/17669#issuecomment-3684774614
> Could the perf issue be fixed by #17672 ?
This PR is about increasing performance of schema comparing. I suggest not to compare schema at all during metadata removal.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org