You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "Davis-Zhang-Onehouse (via GitHub)" <gi...@apache.org> on 2025/03/01 20:54:44 UTC
[PR] Hudi 9092 3 [hudi]
Davis-Zhang-Onehouse opened a new pull request, #12900:
URL: https://github.com/apache/hudi/pull/12900
### Change Logs
_Describe context and summary for this change. Highlight if any code was copied._
### Impact
_Describe any public API or user-facing feature change or any performance impact._
### Risk level (write none, low medium or high below)
_If medium or high, explain what verification was done to mitigate the risks._
### Documentation Update
_Describe any necessary documentation update if there is any new feature, config, or user-facing change. If not, put "none"._
- _The config description must be updated if new configs are added or the default value of the configs are changed_
- _Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the
ticket number here and follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make
changes to the website._
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2692540476
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3997",
"triggerID" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"triggerType" : "PUSH"
}, {
"hash" : "54eb05f956d8416c9f22578629592acc52eef93f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3998",
"triggerID" : "54eb05f956d8416c9f22578629592acc52eef93f",
"triggerType" : "PUSH"
}, {
"hash" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4001",
"triggerID" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"triggerType" : "PUSH"
}, {
"hash" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4002",
"triggerID" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ce47eecc9860b49515b510c4b1567f326e7e0c62 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4001)
* 25996d9bc7aef3436746bf334e99fe45289fc17d Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4002)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2692578569
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3997",
"triggerID" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"triggerType" : "PUSH"
}, {
"hash" : "54eb05f956d8416c9f22578629592acc52eef93f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3998",
"triggerID" : "54eb05f956d8416c9f22578629592acc52eef93f",
"triggerType" : "PUSH"
}, {
"hash" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4001",
"triggerID" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"triggerType" : "PUSH"
}, {
"hash" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4002",
"triggerID" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"triggerType" : "PUSH"
}, {
"hash" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 25996d9bc7aef3436746bf334e99fe45289fc17d Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4002)
* d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #12900:
URL: https://github.com/apache/hudi/pull/12900#discussion_r1978387695
##########
hudi-common/src/test/java/org/apache/hudi/common/testutils/Transformations.java:
##########
@@ -79,4 +83,14 @@ public static <T> List<T> randomSelect(List<T> items, int n) {
public static List<HoodieKey> randomSelectAsHoodieKeys(List<HoodieRecord> records, int n) {
return randomSelect(recordsToHoodieKeys(records), n);
}
+
+ public static byte[] writeInstantContentToBytes(HoodieInstantWriter writer) {
Review Comment:
Mark this as deprecated so that this is for retrofitting only.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2698154750
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "55cc2f7be57c6132c55c5f6d9884fbada80e9bbc",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4035",
"triggerID" : "55cc2f7be57c6132c55c5f6d9884fbada80e9bbc",
"triggerType" : "PUSH"
}, {
"hash" : "41abcc9cf7c328b801c93fb9f1edb150df879d94",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4046",
"triggerID" : "41abcc9cf7c328b801c93fb9f1edb150df879d94",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 55cc2f7be57c6132c55c5f6d9884fbada80e9bbc Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4035)
* 41abcc9cf7c328b801c93fb9f1edb150df879d94 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4046)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2696081543
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "82c37b34166c91a1c90dc1b501b3ffa12e898c2a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4033",
"triggerID" : "82c37b34166c91a1c90dc1b501b3ffa12e898c2a",
"triggerType" : "PUSH"
}, {
"hash" : "55cc2f7be57c6132c55c5f6d9884fbada80e9bbc",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "55cc2f7be57c6132c55c5f6d9884fbada80e9bbc",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 82c37b34166c91a1c90dc1b501b3ffa12e898c2a Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4033)
* 55cc2f7be57c6132c55c5f6d9884fbada80e9bbc UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #12900:
URL: https://github.com/apache/hudi/pull/12900#discussion_r1978353414
##########
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/versioning/v1/ActiveTimelineV1.java:
##########
@@ -138,25 +147,24 @@ public void createRequestedCommitWithReplaceMetadata(String instantTime, String
HoodieInstant instant = instantGenerator.createNewInstant(HoodieInstant.State.REQUESTED, actionType, instantTime);
LOG.info("Creating a new instant " + instant);
// Create the request replace file
- createFileInMetaPath(instantFileNameGenerator.getFileName(instant),
- TimelineMetadataUtils.serializeRequestedReplaceMetadata(new HoodieRequestedReplaceMetadata()), false);
- } catch (IOException e) {
- throw new HoodieIOException("Error create requested replace commit ", e);
+ createFileInMetaPath(instantFileNameGenerator.getFileName(instant), Option.of(new HoodieRequestedReplaceMetadata()), false);
+ } catch (HoodieIOException e) {
+ throw e;
}
Review Comment:
We could get rid of the try-catch.
##########
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/versioning/v2/ActiveTimelineV2.java:
##########
@@ -143,25 +152,24 @@ public void createRequestedCommitWithReplaceMetadata(String instantTime, String
HoodieInstant instant = instantGenerator.createNewInstant(HoodieInstant.State.REQUESTED, actionType, instantTime);
LOG.info("Creating a new instant " + instant);
// Create the request replace file
- createFileInMetaPath(instantFileNameGenerator.getFileName(instant),
- TimelineMetadataUtils.serializeRequestedReplaceMetadata(new HoodieRequestedReplaceMetadata()), false);
- } catch (IOException e) {
- throw new HoodieIOException("Error create requested replace commit ", e);
+ createFileInMetaPath(instantFileNameGenerator.getFileName(instant), Option.of(new HoodieRequestedReplaceMetadata()), false);
+ } catch (HoodieIOException e) {
+ throw e;
Review Comment:
We could get rid of the this try-catch?
##########
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/TimelineUtils.java:
##########
@@ -623,4 +624,15 @@ public static Option<InputStream> getInputStreamOptionLegacy(HoodieTimeline time
}
return Option.of(new ByteArrayInputStream(bytes.get()));
}
+
+ // TODO[HUDI-9094]: work around when caller needs to write byte array in raw. This method should be removed.
+ public static <T> Option<HoodieInstantWriter> getHoodieInstantWriterOption(HoodieTimeline timeline, Option<T> metadata) {
+ Option<HoodieInstantWriter> writerOption;
+ if (metadata.isPresent() && metadata.get() instanceof HoodieInstantWriter) {
+ writerOption = (Option<HoodieInstantWriter>) metadata;
+ } else {
+ writerOption = timeline.getInstantWriter(metadata);
+ }
+ return writerOption;
+ }
Review Comment:
This method seems not to use byte array.
##########
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/versioning/v1/ActiveTimelineV1.java:
##########
@@ -548,127 +551,101 @@ public void transitionRequestedToInflight(String commitType, String inFlightInst
}
@Override
- public void transitionRequestedToInflight(HoodieInstant requested, Option<byte[]> content) {
- transitionRequestedToInflight(requested, content, false);
+ public <T> void transitionRequestedToInflight(HoodieInstant requested, Option<T> metadata) {
Review Comment:
```suggestion
public <T> void transitionRequestedToInflight(HoodieInstant requested, Option<T> metadata) {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2695786766
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3997",
"triggerID" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"triggerType" : "PUSH"
}, {
"hash" : "54eb05f956d8416c9f22578629592acc52eef93f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3998",
"triggerID" : "54eb05f956d8416c9f22578629592acc52eef93f",
"triggerType" : "PUSH"
}, {
"hash" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4001",
"triggerID" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"triggerType" : "PUSH"
}, {
"hash" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4002",
"triggerID" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"triggerType" : "PUSH"
}, {
"hash" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4003",
"triggerID" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"triggerType" : "PUSH"
}, {
"hash" : "7577553bfb6d14dda0c4a538ad315a914690ea2a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4004",
"triggerID" : "7577553bfb6d14dda0c4a538ad315a914690ea2a",
"triggerType" : "PUSH"
}, {
"hash" : "5889b122c1ad7a076bf9d61cf6edd3f310d804de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4006",
"triggerID" : "5889b122c1ad7a076bf9d61cf6edd3f310d804de",
"triggerType" : "PUSH"
}, {
"hash" : "876e8278fdd743e006a3a5cc65ff4811239f5af3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4014",
"triggerID" : "876e8278fdd743e006a3a5cc65ff4811239f5af3",
"triggerType" : "PUSH"
}, {
"hash" : "3186fc1fb5b83d310e86fd37bdf12c5c4b81b178",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4019",
"triggerID" : "3186fc1fb5b83d310e86fd37bdf12c5c4b81b178",
"triggerType" : "PUSH"
}, {
"hash" : "f0623433a2605c782bbb755c516ffc03d4dada29",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4024",
"triggerID" : "f0623433a2605c782bbb755c516ffc03d4dada29",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* f0623433a2605c782bbb755c516ffc03d4dada29 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4024)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2695174205
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3997",
"triggerID" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"triggerType" : "PUSH"
}, {
"hash" : "54eb05f956d8416c9f22578629592acc52eef93f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3998",
"triggerID" : "54eb05f956d8416c9f22578629592acc52eef93f",
"triggerType" : "PUSH"
}, {
"hash" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4001",
"triggerID" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"triggerType" : "PUSH"
}, {
"hash" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4002",
"triggerID" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"triggerType" : "PUSH"
}, {
"hash" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4003",
"triggerID" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"triggerType" : "PUSH"
}, {
"hash" : "7577553bfb6d14dda0c4a538ad315a914690ea2a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4004",
"triggerID" : "7577553bfb6d14dda0c4a538ad315a914690ea2a",
"triggerType" : "PUSH"
}, {
"hash" : "5889b122c1ad7a076bf9d61cf6edd3f310d804de",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4006",
"triggerID" : "5889b122c1ad7a076bf9d61cf6edd3f310d804de",
"triggerType" : "PUSH"
}, {
"hash" : "876e8278fdd743e006a3a5cc65ff4811239f5af3",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "876e8278fdd743e006a3a5cc65ff4811239f5af3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5889b122c1ad7a076bf9d61cf6edd3f310d804de Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4006)
* 876e8278fdd743e006a3a5cc65ff4811239f5af3 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2692464236
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3997",
"triggerID" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"triggerType" : "PUSH"
}, {
"hash" : "54eb05f956d8416c9f22578629592acc52eef93f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3998",
"triggerID" : "54eb05f956d8416c9f22578629592acc52eef93f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c633f1a54ffbb6660dea21b15ef59abe9ec1307a Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3997)
* 54eb05f956d8416c9f22578629592acc52eef93f Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3998)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #12900:
URL: https://github.com/apache/hudi/pull/12900#discussion_r1977983664
##########
hudi-io/src/main/java/org/apache/hudi/common/util/FileIOUtils.java:
##########
@@ -191,8 +191,8 @@ public static void createFileInPath(HoodieStorage storage,
}
}
- public static void createFileInPath(HoodieStorage storage, StoragePath fullPath, Option<byte[]> content) {
- createFileInPath(storage, fullPath, content.map(bytes -> (outputStream) -> outputStream.write(bytes)), false);
+ public static void createFileInPath(HoodieStorage fileSystem, StoragePath fullPath, Option<HoodieInstantWriter> contentWriter) {
Review Comment:
```suggestion
public static void createFileInPath(HoodieStorage storage, StoragePath fullPath, Option<HoodieInstantWriter> contentWriter) {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2695623761
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3997",
"triggerID" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"triggerType" : "PUSH"
}, {
"hash" : "54eb05f956d8416c9f22578629592acc52eef93f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3998",
"triggerID" : "54eb05f956d8416c9f22578629592acc52eef93f",
"triggerType" : "PUSH"
}, {
"hash" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4001",
"triggerID" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"triggerType" : "PUSH"
}, {
"hash" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4002",
"triggerID" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"triggerType" : "PUSH"
}, {
"hash" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4003",
"triggerID" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"triggerType" : "PUSH"
}, {
"hash" : "7577553bfb6d14dda0c4a538ad315a914690ea2a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4004",
"triggerID" : "7577553bfb6d14dda0c4a538ad315a914690ea2a",
"triggerType" : "PUSH"
}, {
"hash" : "5889b122c1ad7a076bf9d61cf6edd3f310d804de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4006",
"triggerID" : "5889b122c1ad7a076bf9d61cf6edd3f310d804de",
"triggerType" : "PUSH"
}, {
"hash" : "876e8278fdd743e006a3a5cc65ff4811239f5af3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4014",
"triggerID" : "876e8278fdd743e006a3a5cc65ff4811239f5af3",
"triggerType" : "PUSH"
}, {
"hash" : "3186fc1fb5b83d310e86fd37bdf12c5c4b81b178",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4019",
"triggerID" : "3186fc1fb5b83d310e86fd37bdf12c5c4b81b178",
"triggerType" : "PUSH"
}, {
"hash" : "f0623433a2605c782bbb755c516ffc03d4dada29",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4024",
"triggerID" : "f0623433a2605c782bbb755c516ffc03d4dada29",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3186fc1fb5b83d310e86fd37bdf12c5c4b81b178 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4019)
* f0623433a2605c782bbb755c516ffc03d4dada29 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4024)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #12900:
URL: https://github.com/apache/hudi/pull/12900#discussion_r1978384504
##########
hudi-common/src/test/java/org/apache/hudi/common/testutils/FileCreateUtilsLegacy.java:
##########
@@ -149,30 +152,26 @@ public static void createInflightDeltaCommit(String basePath, String instantTime
public static void createReplaceCommit(CommitMetadataSerDe commitMetadataSerDe, String basePath,
String instantTime, HoodieReplaceCommitMetadata metadata) throws IOException {
- createMetaFile(basePath, instantTime, HoodieTimeline.REPLACE_COMMIT_EXTENSION,
- serializeCommitMetadata(commitMetadataSerDe, metadata).get());
+ createMetaFile(basePath, instantTime, HoodieTimeline.REPLACE_COMMIT_EXTENSION, Option.of(metadata));
}
public static void createRequestedClusterCommit(String basePath, String instantTime,
- HoodieRequestedReplaceMetadata requestedReplaceMetadata)
- throws IOException {
- createMetaFile(basePath, instantTime, HoodieTimeline.REQUESTED_CLUSTERING_COMMIT_EXTENSION,
- serializeRequestedReplaceMetadata(requestedReplaceMetadata).get());
+ HoodieRequestedReplaceMetadata requestedReplaceMetadata) throws IOException {
+ createMetaFile(basePath, instantTime, HoodieTimeline.REQUESTED_CLUSTERING_COMMIT_EXTENSION, Option.of(requestedReplaceMetadata));
}
public static void createRequestedCompactionCommit(String basePath, String instantTime,
- HoodieCompactionPlan requestedCompactionPlan)
- throws IOException {
- createMetaFile(basePath, instantTime, HoodieTimeline.REQUESTED_COMPACTION_EXTENSION,
- serializeCompactionPlan(requestedCompactionPlan).get());
+ HoodieCompactionPlan requestedCompactionPlan) throws IOException {
+ createMetaFile(basePath, instantTime, HoodieTimeline.REQUESTED_COMPACTION_EXTENSION, Option.of(requestedCompactionPlan));
}
public static void createRequestedRollbackFile(String basePath, String instantTime, HoodieRollbackPlan plan) throws IOException {
- createMetaFile(basePath, instantTime, HoodieTimeline.REQUESTED_ROLLBACK_EXTENSION, serializeRollbackPlan(plan).get());
+ createMetaFile(basePath, instantTime, HoodieTimeline.REQUESTED_ROLLBACK_EXTENSION, Option.of(plan));
}
public static void createRequestedRollbackFile(String basePath, String instantTime, byte[] content) throws IOException {
- createMetaFile(basePath, instantTime, HoodieTimeline.REQUESTED_ROLLBACK_EXTENSION, content);
+ createMetaFile(basePath, instantTime, HoodieTimeline.REQUESTED_ROLLBACK_EXTENSION,
Review Comment:
Would be good to create a follow-up JIRA ticket to track the refactoring of test code using byte array for handling metadata.
##########
hudi-common/src/test/java/org/apache/hudi/common/testutils/HoodieTestUtils.java:
##########
@@ -392,6 +393,10 @@ public static StoragePath getCompleteInstantPath(HoodieStorage storage, StorageP
return getCompleteInstantFileInfo(storage, parent, instantTime, action).getPath();
}
+ public static <T> byte[] convertMetadataToBytArray(T metadata) {
Review Comment:
```suggestion
public static <T> byte[] convertMetadataToByteArray(T metadata) {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2692879742
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3997",
"triggerID" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"triggerType" : "PUSH"
}, {
"hash" : "54eb05f956d8416c9f22578629592acc52eef93f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3998",
"triggerID" : "54eb05f956d8416c9f22578629592acc52eef93f",
"triggerType" : "PUSH"
}, {
"hash" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4001",
"triggerID" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"triggerType" : "PUSH"
}, {
"hash" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4002",
"triggerID" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"triggerType" : "PUSH"
}, {
"hash" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4003",
"triggerID" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"triggerType" : "PUSH"
}, {
"hash" : "7577553bfb6d14dda0c4a538ad315a914690ea2a",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4004",
"triggerID" : "7577553bfb6d14dda0c4a538ad315a914690ea2a",
"triggerType" : "PUSH"
}, {
"hash" : "5889b122c1ad7a076bf9d61cf6edd3f310d804de",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4006",
"triggerID" : "5889b122c1ad7a076bf9d61cf6edd3f310d804de",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 7577553bfb6d14dda0c4a538ad315a914690ea2a Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4004)
* 5889b122c1ad7a076bf9d61cf6edd3f310d804de Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4006)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2696405751
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "55cc2f7be57c6132c55c5f6d9884fbada80e9bbc",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "55cc2f7be57c6132c55c5f6d9884fbada80e9bbc",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 55cc2f7be57c6132c55c5f6d9884fbada80e9bbc UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Hudi 9092 3 [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2692442465
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3997",
"triggerID" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c633f1a54ffbb6660dea21b15ef59abe9ec1307a Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3997)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2692539929
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3997",
"triggerID" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"triggerType" : "PUSH"
}, {
"hash" : "54eb05f956d8416c9f22578629592acc52eef93f",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3998",
"triggerID" : "54eb05f956d8416c9f22578629592acc52eef93f",
"triggerType" : "PUSH"
}, {
"hash" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4001",
"triggerID" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"triggerType" : "PUSH"
}, {
"hash" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 54eb05f956d8416c9f22578629592acc52eef93f Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3998)
* ce47eecc9860b49515b510c4b1567f326e7e0c62 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4001)
* 25996d9bc7aef3436746bf334e99fe45289fc17d UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #12900:
URL: https://github.com/apache/hudi/pull/12900#discussion_r1978404070
##########
hudi-hadoop-common/src/test/java/org/apache/hudi/common/table/view/TestHoodieTableFileSystemView.java:
##########
@@ -1895,18 +1889,18 @@ public void testPendingCompactionWithDuplicateFileIdsAcrossPartitions(boolean pr
assertTrue(fileIdsInCompaction.contains(fileId));
}
- private void saveAsComplete(HoodieActiveTimeline timeline, HoodieInstant inflight, Option<byte[]> data) {
+ private void saveAsComplete(HoodieActiveTimeline timeline, HoodieInstant inflight, HoodieCommitMetadata metadata) {
if (inflight.getAction().equals(HoodieTimeline.COMPACTION_ACTION)) {
- timeline.transitionCompactionInflightToComplete(true, inflight, data);
+ timeline.transitionCompactionInflightToComplete(true, inflight, metadata);
} else {
HoodieInstant requested = INSTANT_GENERATOR.createNewInstant(State.REQUESTED, inflight.getAction(), inflight.requestedTime());
timeline.createNewInstant(requested);
timeline.transitionRequestedToInflight(requested, Option.empty());
- timeline.saveAsComplete(inflight, data);
+ timeline.saveAsComplete(inflight, Option.of(metadata));
}
}
- private void saveAsCompleteCluster(HoodieActiveTimeline timeline, HoodieInstant inflight, Option<byte[]> data) {
+ private <T> void saveAsCompleteCluster(HoodieActiveTimeline timeline, HoodieInstant inflight, HoodieCommitMetadata metadata) {
Review Comment:
```suggestion
private void saveAsCompleteCluster(HoodieActiveTimeline timeline, HoodieInstant inflight, HoodieCommitMetadata metadata) {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2698496950
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "55cc2f7be57c6132c55c5f6d9884fbada80e9bbc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4035",
"triggerID" : "55cc2f7be57c6132c55c5f6d9884fbada80e9bbc",
"triggerType" : "PUSH"
}, {
"hash" : "41abcc9cf7c328b801c93fb9f1edb150df879d94",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4046",
"triggerID" : "41abcc9cf7c328b801c93fb9f1edb150df879d94",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 41abcc9cf7c328b801c93fb9f1edb150df879d94 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4046)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2692879118
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3997",
"triggerID" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"triggerType" : "PUSH"
}, {
"hash" : "54eb05f956d8416c9f22578629592acc52eef93f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3998",
"triggerID" : "54eb05f956d8416c9f22578629592acc52eef93f",
"triggerType" : "PUSH"
}, {
"hash" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4001",
"triggerID" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"triggerType" : "PUSH"
}, {
"hash" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4002",
"triggerID" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"triggerType" : "PUSH"
}, {
"hash" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4003",
"triggerID" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"triggerType" : "PUSH"
}, {
"hash" : "7577553bfb6d14dda0c4a538ad315a914690ea2a",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4004",
"triggerID" : "7577553bfb6d14dda0c4a538ad315a914690ea2a",
"triggerType" : "PUSH"
}, {
"hash" : "5889b122c1ad7a076bf9d61cf6edd3f310d804de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5889b122c1ad7a076bf9d61cf6edd3f310d804de",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 7577553bfb6d14dda0c4a538ad315a914690ea2a Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4004)
* 5889b122c1ad7a076bf9d61cf6edd3f310d804de UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2692595641
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3997",
"triggerID" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"triggerType" : "PUSH"
}, {
"hash" : "54eb05f956d8416c9f22578629592acc52eef93f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3998",
"triggerID" : "54eb05f956d8416c9f22578629592acc52eef93f",
"triggerType" : "PUSH"
}, {
"hash" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4001",
"triggerID" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"triggerType" : "PUSH"
}, {
"hash" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4002",
"triggerID" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"triggerType" : "PUSH"
}, {
"hash" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4003",
"triggerID" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"triggerType" : "PUSH"
}, {
"hash" : "7577553bfb6d14dda0c4a538ad315a914690ea2a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4004",
"triggerID" : "7577553bfb6d14dda0c4a538ad315a914690ea2a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4003)
* 7577553bfb6d14dda0c4a538ad315a914690ea2a Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4004)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #12900:
URL: https://github.com/apache/hudi/pull/12900#discussion_r1978411566
##########
hudi-platform-service/hudi-metaserver/hudi-metaserver-client/src/main/java/org/apache/hudi/common/table/timeline/HoodieMetaserverBasedTimeline.java:
##########
@@ -56,23 +60,26 @@ protected void deleteInstantFile(HoodieInstant instant) {
}
@Override
- protected void transitionStateToComplete(boolean shouldLock, HoodieInstant fromInstant, HoodieInstant toInstant, Option<byte[]> data) {
+ protected <T> void transitionStateToComplete(boolean shouldLock, HoodieInstant fromInstant, HoodieInstant toInstant, Option<T> metadata) {
ValidationUtils.checkArgument(fromInstant.requestedTime().equals(toInstant.requestedTime()));
- metaserverClient.transitionInstantState(databaseName, tableName, fromInstant, toInstant, data);
+ metaserverClient.transitionInstantState(databaseName, tableName, fromInstant, toInstant,
+ metadata.map(m -> convertMetadataToBytArray(m, metadataSerDeV2)));
Review Comment:
```suggestion
metadata.map(m -> convertMetadataToByteArray(m, metadataSerDeV2)));
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2696067042
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "82c37b34166c91a1c90dc1b501b3ffa12e898c2a",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "82c37b34166c91a1c90dc1b501b3ffa12e898c2a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 82c37b34166c91a1c90dc1b501b3ffa12e898c2a UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2696410543
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "55cc2f7be57c6132c55c5f6d9884fbada80e9bbc",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4035",
"triggerID" : "55cc2f7be57c6132c55c5f6d9884fbada80e9bbc",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 55cc2f7be57c6132c55c5f6d9884fbada80e9bbc Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4035)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2696063891
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3997",
"triggerID" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"triggerType" : "PUSH"
}, {
"hash" : "54eb05f956d8416c9f22578629592acc52eef93f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3998",
"triggerID" : "54eb05f956d8416c9f22578629592acc52eef93f",
"triggerType" : "PUSH"
}, {
"hash" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4001",
"triggerID" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"triggerType" : "PUSH"
}, {
"hash" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4002",
"triggerID" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"triggerType" : "PUSH"
}, {
"hash" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4003",
"triggerID" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"triggerType" : "PUSH"
}, {
"hash" : "7577553bfb6d14dda0c4a538ad315a914690ea2a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4004",
"triggerID" : "7577553bfb6d14dda0c4a538ad315a914690ea2a",
"triggerType" : "PUSH"
}, {
"hash" : "5889b122c1ad7a076bf9d61cf6edd3f310d804de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4006",
"triggerID" : "5889b122c1ad7a076bf9d61cf6edd3f310d804de",
"triggerType" : "PUSH"
}, {
"hash" : "876e8278fdd743e006a3a5cc65ff4811239f5af3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4014",
"triggerID" : "876e8278fdd743e006a3a5cc65ff4811239f5af3",
"triggerType" : "PUSH"
}, {
"hash" : "3186fc1fb5b83d310e86fd37bdf12c5c4b81b178",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4019",
"triggerID" : "3186fc1fb5b83d310e86fd37bdf12c5c4b81b178",
"triggerType" : "PUSH"
}, {
"hash" : "f0623433a2605c782bbb755c516ffc03d4dada29",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4024",
"triggerID" : "f0623433a2605c782bbb755c516ffc03d4dada29",
"triggerType" : "PUSH"
}, {
"hash" : "82c37b34166c91a1c90dc1b501b3ffa12e898c2a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4033",
"triggerID" : "82c37b34166c91a1c90dc1b501b3ffa12e898c2a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 82c37b34166c91a1c90dc1b501b3ffa12e898c2a Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4033)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2698698912
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "41abcc9cf7c328b801c93fb9f1edb150df879d94",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4046",
"triggerID" : "41abcc9cf7c328b801c93fb9f1edb150df879d94",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 41abcc9cf7c328b801c93fb9f1edb150df879d94 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4046)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2692628282
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3997",
"triggerID" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"triggerType" : "PUSH"
}, {
"hash" : "54eb05f956d8416c9f22578629592acc52eef93f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3998",
"triggerID" : "54eb05f956d8416c9f22578629592acc52eef93f",
"triggerType" : "PUSH"
}, {
"hash" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4001",
"triggerID" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"triggerType" : "PUSH"
}, {
"hash" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4002",
"triggerID" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"triggerType" : "PUSH"
}, {
"hash" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4003",
"triggerID" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"triggerType" : "PUSH"
}, {
"hash" : "7577553bfb6d14dda0c4a538ad315a914690ea2a",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4004",
"triggerID" : "7577553bfb6d14dda0c4a538ad315a914690ea2a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 7577553bfb6d14dda0c4a538ad315a914690ea2a Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4004)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2695493900
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3997",
"triggerID" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"triggerType" : "PUSH"
}, {
"hash" : "54eb05f956d8416c9f22578629592acc52eef93f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3998",
"triggerID" : "54eb05f956d8416c9f22578629592acc52eef93f",
"triggerType" : "PUSH"
}, {
"hash" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4001",
"triggerID" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"triggerType" : "PUSH"
}, {
"hash" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4002",
"triggerID" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"triggerType" : "PUSH"
}, {
"hash" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4003",
"triggerID" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"triggerType" : "PUSH"
}, {
"hash" : "7577553bfb6d14dda0c4a538ad315a914690ea2a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4004",
"triggerID" : "7577553bfb6d14dda0c4a538ad315a914690ea2a",
"triggerType" : "PUSH"
}, {
"hash" : "5889b122c1ad7a076bf9d61cf6edd3f310d804de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4006",
"triggerID" : "5889b122c1ad7a076bf9d61cf6edd3f310d804de",
"triggerType" : "PUSH"
}, {
"hash" : "876e8278fdd743e006a3a5cc65ff4811239f5af3",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4014",
"triggerID" : "876e8278fdd743e006a3a5cc65ff4811239f5af3",
"triggerType" : "PUSH"
}, {
"hash" : "3186fc1fb5b83d310e86fd37bdf12c5c4b81b178",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4019",
"triggerID" : "3186fc1fb5b83d310e86fd37bdf12c5c4b81b178",
"triggerType" : "PUSH"
}, {
"hash" : "f0623433a2605c782bbb755c516ffc03d4dada29",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "f0623433a2605c782bbb755c516ffc03d4dada29",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 876e8278fdd743e006a3a5cc65ff4811239f5af3 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4014)
* 3186fc1fb5b83d310e86fd37bdf12c5c4b81b178 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4019)
* f0623433a2605c782bbb755c516ffc03d4dada29 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2695485259
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3997",
"triggerID" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"triggerType" : "PUSH"
}, {
"hash" : "54eb05f956d8416c9f22578629592acc52eef93f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3998",
"triggerID" : "54eb05f956d8416c9f22578629592acc52eef93f",
"triggerType" : "PUSH"
}, {
"hash" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4001",
"triggerID" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"triggerType" : "PUSH"
}, {
"hash" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4002",
"triggerID" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"triggerType" : "PUSH"
}, {
"hash" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4003",
"triggerID" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"triggerType" : "PUSH"
}, {
"hash" : "7577553bfb6d14dda0c4a538ad315a914690ea2a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4004",
"triggerID" : "7577553bfb6d14dda0c4a538ad315a914690ea2a",
"triggerType" : "PUSH"
}, {
"hash" : "5889b122c1ad7a076bf9d61cf6edd3f310d804de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4006",
"triggerID" : "5889b122c1ad7a076bf9d61cf6edd3f310d804de",
"triggerType" : "PUSH"
}, {
"hash" : "876e8278fdd743e006a3a5cc65ff4811239f5af3",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4014",
"triggerID" : "876e8278fdd743e006a3a5cc65ff4811239f5af3",
"triggerType" : "PUSH"
}, {
"hash" : "3186fc1fb5b83d310e86fd37bdf12c5c4b81b178",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3186fc1fb5b83d310e86fd37bdf12c5c4b81b178",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 876e8278fdd743e006a3a5cc65ff4811239f5af3 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4014)
* 3186fc1fb5b83d310e86fd37bdf12c5c4b81b178 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2695987661
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3997",
"triggerID" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"triggerType" : "PUSH"
}, {
"hash" : "54eb05f956d8416c9f22578629592acc52eef93f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3998",
"triggerID" : "54eb05f956d8416c9f22578629592acc52eef93f",
"triggerType" : "PUSH"
}, {
"hash" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4001",
"triggerID" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"triggerType" : "PUSH"
}, {
"hash" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4002",
"triggerID" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"triggerType" : "PUSH"
}, {
"hash" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4003",
"triggerID" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"triggerType" : "PUSH"
}, {
"hash" : "7577553bfb6d14dda0c4a538ad315a914690ea2a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4004",
"triggerID" : "7577553bfb6d14dda0c4a538ad315a914690ea2a",
"triggerType" : "PUSH"
}, {
"hash" : "5889b122c1ad7a076bf9d61cf6edd3f310d804de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4006",
"triggerID" : "5889b122c1ad7a076bf9d61cf6edd3f310d804de",
"triggerType" : "PUSH"
}, {
"hash" : "876e8278fdd743e006a3a5cc65ff4811239f5af3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4014",
"triggerID" : "876e8278fdd743e006a3a5cc65ff4811239f5af3",
"triggerType" : "PUSH"
}, {
"hash" : "3186fc1fb5b83d310e86fd37bdf12c5c4b81b178",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4019",
"triggerID" : "3186fc1fb5b83d310e86fd37bdf12c5c4b81b178",
"triggerType" : "PUSH"
}, {
"hash" : "f0623433a2605c782bbb755c516ffc03d4dada29",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4024",
"triggerID" : "f0623433a2605c782bbb755c516ffc03d4dada29",
"triggerType" : "PUSH"
}, {
"hash" : "82c37b34166c91a1c90dc1b501b3ffa12e898c2a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4033",
"triggerID" : "82c37b34166c91a1c90dc1b501b3ffa12e898c2a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* f0623433a2605c782bbb755c516ffc03d4dada29 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4024)
* 82c37b34166c91a1c90dc1b501b3ffa12e898c2a Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4033)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua merged PR #12900:
URL: https://github.com/apache/hudi/pull/12900
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2692914198
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3997",
"triggerID" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"triggerType" : "PUSH"
}, {
"hash" : "54eb05f956d8416c9f22578629592acc52eef93f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3998",
"triggerID" : "54eb05f956d8416c9f22578629592acc52eef93f",
"triggerType" : "PUSH"
}, {
"hash" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4001",
"triggerID" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"triggerType" : "PUSH"
}, {
"hash" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4002",
"triggerID" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"triggerType" : "PUSH"
}, {
"hash" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4003",
"triggerID" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"triggerType" : "PUSH"
}, {
"hash" : "7577553bfb6d14dda0c4a538ad315a914690ea2a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4004",
"triggerID" : "7577553bfb6d14dda0c4a538ad315a914690ea2a",
"triggerType" : "PUSH"
}, {
"hash" : "5889b122c1ad7a076bf9d61cf6edd3f310d804de",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4006",
"triggerID" : "5889b122c1ad7a076bf9d61cf6edd3f310d804de",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5889b122c1ad7a076bf9d61cf6edd3f310d804de Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4006)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2695357088
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3997",
"triggerID" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"triggerType" : "PUSH"
}, {
"hash" : "54eb05f956d8416c9f22578629592acc52eef93f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3998",
"triggerID" : "54eb05f956d8416c9f22578629592acc52eef93f",
"triggerType" : "PUSH"
}, {
"hash" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4001",
"triggerID" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"triggerType" : "PUSH"
}, {
"hash" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4002",
"triggerID" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"triggerType" : "PUSH"
}, {
"hash" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4003",
"triggerID" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"triggerType" : "PUSH"
}, {
"hash" : "7577553bfb6d14dda0c4a538ad315a914690ea2a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4004",
"triggerID" : "7577553bfb6d14dda0c4a538ad315a914690ea2a",
"triggerType" : "PUSH"
}, {
"hash" : "5889b122c1ad7a076bf9d61cf6edd3f310d804de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4006",
"triggerID" : "5889b122c1ad7a076bf9d61cf6edd3f310d804de",
"triggerType" : "PUSH"
}, {
"hash" : "876e8278fdd743e006a3a5cc65ff4811239f5af3",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4014",
"triggerID" : "876e8278fdd743e006a3a5cc65ff4811239f5af3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 876e8278fdd743e006a3a5cc65ff4811239f5af3 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4014)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2692595151
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3997",
"triggerID" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"triggerType" : "PUSH"
}, {
"hash" : "54eb05f956d8416c9f22578629592acc52eef93f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3998",
"triggerID" : "54eb05f956d8416c9f22578629592acc52eef93f",
"triggerType" : "PUSH"
}, {
"hash" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4001",
"triggerID" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"triggerType" : "PUSH"
}, {
"hash" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4002",
"triggerID" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"triggerType" : "PUSH"
}, {
"hash" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4003",
"triggerID" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"triggerType" : "PUSH"
}, {
"hash" : "7577553bfb6d14dda0c4a538ad315a914690ea2a",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "7577553bfb6d14dda0c4a538ad315a914690ea2a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 25996d9bc7aef3436746bf334e99fe45289fc17d Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4002)
* d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4003)
* 7577553bfb6d14dda0c4a538ad315a914690ea2a UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #12900:
URL: https://github.com/apache/hudi/pull/12900#discussion_r1978307100
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/clean/CleanActionExecutor.java:
##########
@@ -232,12 +230,9 @@ private HoodieCleanMetadata runClean(HoodieTable<T, I, K, O> table, HoodieInstan
this.txnManager.beginTransaction(Option.of(inflightInstant), Option.empty());
}
writeTableMetadata(metadata, inflightInstant.requestedTime());
- table.getActiveTimeline().transitionCleanInflightToComplete(false,
- inflightInstant, TimelineMetadataUtils.serializeCleanMetadata(metadata));
+ table.getActiveTimeline().transitionCleanInflightToComplete(false, inflightInstant, Option.of(metadata));
LOG.info("Marked clean started on " + inflightInstant.requestedTime() + " as complete");
return metadata;
- } catch (IOException e) {
- throw new HoodieIOException("Failed to clean up after commit", e);
Review Comment:
Should we still catch `HoodieIOException` here (which could be thrown by `transitionStateToComplete`)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #12900:
URL: https://github.com/apache/hudi/pull/12900#discussion_r1978331668
##########
hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/table/functional/TestCleanActionExecutor.java:
##########
@@ -138,7 +138,7 @@ void testPartialCleanFailure(CleanFailureType failureType) throws IOException {
when(activeTimeline.getCleanerTimeline()).thenReturn(cleanTimeline);
when(cleanTimeline.getInstants()).thenReturn(Collections.singletonList(cleanInstant));
when(activeTimeline.readCleanerPlan(cleanInstant)).thenReturn(cleanerPlan);
- when(activeTimeline.readCleanerInfoAsBytes(cleanInstant)).thenReturn(TimelineMetadataUtils.serializeCleanerPlan(cleanerPlan));
+ when(activeTimeline.readCleanerInfoAsBytes(cleanInstant)).thenReturn(Option.of(convertMetadataToBytArray(cleanerPlan)));
Review Comment:
Is this method `activeTimeline.readCleanerInfoAsBytes` still used? Let's create a follow-up JIRA ticket to clean up the usage on byte array related handling in the `HoodieTimeline` and subclasses.
##########
hudi-common/pom.xml:
##########
@@ -354,6 +354,12 @@
<artifactId>tally-core</artifactId>
<version>${tally.version}</version>
</dependency>
+ <dependency>
+ <groupId>commons-io</groupId>
+ <artifactId>commons-io</artifactId>
+ <version>2.18.0</version>
+ <scope>test</scope>
+ </dependency>
Review Comment:
Is this still needed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #12900:
URL: https://github.com/apache/hudi/pull/12900#discussion_r1978370376
##########
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/versioning/v2/CommitMetadataSerDeV2.java:
##########
@@ -93,23 +91,10 @@ public <T> T deserialize(HoodieInstant instant, InputStream inputStream, Boolean
}
@Override
- public Option<byte[]> serialize(org.apache.hudi.common.model.HoodieCommitMetadata commitMetadata) throws IOException {
- if (commitMetadata instanceof org.apache.hudi.common.model.HoodieReplaceCommitMetadata) {
- return serializeAvroMetadata(MetadataConversionUtils.convertCommitMetadata(commitMetadata), HoodieReplaceCommitMetadata.class);
+ public <T> Option<HoodieInstantWriter> getInstantWriter(T metadata) {
+ if (metadata instanceof org.apache.hudi.common.model.HoodieCommitMetadata) {
+ return TimelineMetadataUtils.getInstantWriter(Option.of(MetadataConversionUtils.convertCommitMetadataToAvro((HoodieCommitMetadata) metadata)));
Review Comment:
I see `TimelineMetadataUtils.getInstantWriter` is only used in `CommitMetadatSerDe` classes. Should the actually logic be moved here so we don't maintain the util method outside?
##########
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/versioning/v2/CommitMetadataSerDeV2.java:
##########
@@ -93,23 +91,10 @@ public <T> T deserialize(HoodieInstant instant, InputStream inputStream, Boolean
}
@Override
- public Option<byte[]> serialize(org.apache.hudi.common.model.HoodieCommitMetadata commitMetadata) throws IOException {
- if (commitMetadata instanceof org.apache.hudi.common.model.HoodieReplaceCommitMetadata) {
- return serializeAvroMetadata(MetadataConversionUtils.convertCommitMetadata(commitMetadata), HoodieReplaceCommitMetadata.class);
+ public <T> Option<HoodieInstantWriter> getInstantWriter(T metadata) {
+ if (metadata instanceof org.apache.hudi.common.model.HoodieCommitMetadata) {
Review Comment:
Does this handle `HoodieReplaceCommitMetadata`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #12900:
URL: https://github.com/apache/hudi/pull/12900#discussion_r1978540573
##########
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/TimelineUtils.java:
##########
@@ -623,4 +624,15 @@ public static Option<InputStream> getInputStreamOptionLegacy(HoodieTimeline time
}
return Option.of(new ByteArrayInputStream(bytes.get()));
}
+
+ // TODO[HUDI-9094]: work around when caller needs to write byte array in raw. This method should be removed.
+ public static <T> Option<HoodieInstantWriter> getHoodieInstantWriterOption(HoodieTimeline timeline, Option<T> metadata) {
+ Option<HoodieInstantWriter> writerOption;
+ if (metadata.isPresent() && metadata.get() instanceof HoodieInstantWriter) {
+ writerOption = (Option<HoodieInstantWriter>) metadata;
+ } else {
+ writerOption = timeline.getInstantWriter(metadata);
+ }
+ return writerOption;
+ }
Review Comment:
We discussed that the if branch `writerOption = (Option<HoodieInstantWriter>) metadata;` can still use byte array to retrofit old logic before such logic is fully cleaned up.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2696086128
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "82c37b34166c91a1c90dc1b501b3ffa12e898c2a",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4033",
"triggerID" : "82c37b34166c91a1c90dc1b501b3ffa12e898c2a",
"triggerType" : "PUSH"
}, {
"hash" : "55cc2f7be57c6132c55c5f6d9884fbada80e9bbc",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "55cc2f7be57c6132c55c5f6d9884fbada80e9bbc",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 82c37b34166c91a1c90dc1b501b3ffa12e898c2a Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4033)
* 55cc2f7be57c6132c55c5f6d9884fbada80e9bbc UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2696421654
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "55cc2f7be57c6132c55c5f6d9884fbada80e9bbc",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4035",
"triggerID" : "55cc2f7be57c6132c55c5f6d9884fbada80e9bbc",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 55cc2f7be57c6132c55c5f6d9884fbada80e9bbc Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4035)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2692520200
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3997",
"triggerID" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"triggerType" : "PUSH"
}, {
"hash" : "54eb05f956d8416c9f22578629592acc52eef93f",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3998",
"triggerID" : "54eb05f956d8416c9f22578629592acc52eef93f",
"triggerType" : "PUSH"
}, {
"hash" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 54eb05f956d8416c9f22578629592acc52eef93f Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3998)
* ce47eecc9860b49515b510c4b1567f326e7e0c62 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2692561614
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3997",
"triggerID" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"triggerType" : "PUSH"
}, {
"hash" : "54eb05f956d8416c9f22578629592acc52eef93f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3998",
"triggerID" : "54eb05f956d8416c9f22578629592acc52eef93f",
"triggerType" : "PUSH"
}, {
"hash" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4001",
"triggerID" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"triggerType" : "PUSH"
}, {
"hash" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4002",
"triggerID" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 25996d9bc7aef3436746bf334e99fe45289fc17d Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4002)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2698147634
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "55cc2f7be57c6132c55c5f6d9884fbada80e9bbc",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4035",
"triggerID" : "55cc2f7be57c6132c55c5f6d9884fbada80e9bbc",
"triggerType" : "PUSH"
}, {
"hash" : "41abcc9cf7c328b801c93fb9f1edb150df879d94",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "41abcc9cf7c328b801c93fb9f1edb150df879d94",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 55cc2f7be57c6132c55c5f6d9884fbada80e9bbc Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4035)
* 41abcc9cf7c328b801c93fb9f1edb150df879d94 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2698593573
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "41abcc9cf7c328b801c93fb9f1edb150df879d94",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "41abcc9cf7c328b801c93fb9f1edb150df879d94",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 41abcc9cf7c328b801c93fb9f1edb150df879d94 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2698596517
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "41abcc9cf7c328b801c93fb9f1edb150df879d94",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4046",
"triggerID" : "41abcc9cf7c328b801c93fb9f1edb150df879d94",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 41abcc9cf7c328b801c93fb9f1edb150df879d94 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4046)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2692494335
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3997",
"triggerID" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"triggerType" : "PUSH"
}, {
"hash" : "54eb05f956d8416c9f22578629592acc52eef93f",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3998",
"triggerID" : "54eb05f956d8416c9f22578629592acc52eef93f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 54eb05f956d8416c9f22578629592acc52eef93f Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3998)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2695488250
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3997",
"triggerID" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"triggerType" : "PUSH"
}, {
"hash" : "54eb05f956d8416c9f22578629592acc52eef93f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3998",
"triggerID" : "54eb05f956d8416c9f22578629592acc52eef93f",
"triggerType" : "PUSH"
}, {
"hash" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4001",
"triggerID" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"triggerType" : "PUSH"
}, {
"hash" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4002",
"triggerID" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"triggerType" : "PUSH"
}, {
"hash" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4003",
"triggerID" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"triggerType" : "PUSH"
}, {
"hash" : "7577553bfb6d14dda0c4a538ad315a914690ea2a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4004",
"triggerID" : "7577553bfb6d14dda0c4a538ad315a914690ea2a",
"triggerType" : "PUSH"
}, {
"hash" : "5889b122c1ad7a076bf9d61cf6edd3f310d804de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4006",
"triggerID" : "5889b122c1ad7a076bf9d61cf6edd3f310d804de",
"triggerType" : "PUSH"
}, {
"hash" : "876e8278fdd743e006a3a5cc65ff4811239f5af3",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4014",
"triggerID" : "876e8278fdd743e006a3a5cc65ff4811239f5af3",
"triggerType" : "PUSH"
}, {
"hash" : "3186fc1fb5b83d310e86fd37bdf12c5c4b81b178",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4019",
"triggerID" : "3186fc1fb5b83d310e86fd37bdf12c5c4b81b178",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 876e8278fdd743e006a3a5cc65ff4811239f5af3 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4014)
* 3186fc1fb5b83d310e86fd37bdf12c5c4b81b178 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4019)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2696106791
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "82c37b34166c91a1c90dc1b501b3ffa12e898c2a",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4033",
"triggerID" : "82c37b34166c91a1c90dc1b501b3ffa12e898c2a",
"triggerType" : "PUSH"
}, {
"hash" : "55cc2f7be57c6132c55c5f6d9884fbada80e9bbc",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4035",
"triggerID" : "55cc2f7be57c6132c55c5f6d9884fbada80e9bbc",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 82c37b34166c91a1c90dc1b501b3ffa12e898c2a Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4033)
* 55cc2f7be57c6132c55c5f6d9884fbada80e9bbc Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4035)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2692578960
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3997",
"triggerID" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"triggerType" : "PUSH"
}, {
"hash" : "54eb05f956d8416c9f22578629592acc52eef93f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3998",
"triggerID" : "54eb05f956d8416c9f22578629592acc52eef93f",
"triggerType" : "PUSH"
}, {
"hash" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4001",
"triggerID" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"triggerType" : "PUSH"
}, {
"hash" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4002",
"triggerID" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"triggerType" : "PUSH"
}, {
"hash" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4003",
"triggerID" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 25996d9bc7aef3436746bf334e99fe45289fc17d Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4002)
* d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4003)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2695177447
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3997",
"triggerID" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"triggerType" : "PUSH"
}, {
"hash" : "54eb05f956d8416c9f22578629592acc52eef93f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3998",
"triggerID" : "54eb05f956d8416c9f22578629592acc52eef93f",
"triggerType" : "PUSH"
}, {
"hash" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4001",
"triggerID" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"triggerType" : "PUSH"
}, {
"hash" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4002",
"triggerID" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"triggerType" : "PUSH"
}, {
"hash" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4003",
"triggerID" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"triggerType" : "PUSH"
}, {
"hash" : "7577553bfb6d14dda0c4a538ad315a914690ea2a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4004",
"triggerID" : "7577553bfb6d14dda0c4a538ad315a914690ea2a",
"triggerType" : "PUSH"
}, {
"hash" : "5889b122c1ad7a076bf9d61cf6edd3f310d804de",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4006",
"triggerID" : "5889b122c1ad7a076bf9d61cf6edd3f310d804de",
"triggerType" : "PUSH"
}, {
"hash" : "876e8278fdd743e006a3a5cc65ff4811239f5af3",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4014",
"triggerID" : "876e8278fdd743e006a3a5cc65ff4811239f5af3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5889b122c1ad7a076bf9d61cf6edd3f310d804de Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4006)
* 876e8278fdd743e006a3a5cc65ff4811239f5af3 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4014)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #12900:
URL: https://github.com/apache/hudi/pull/12900#discussion_r1978312697
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/clean/CleanActionExecutor.java:
##########
@@ -232,12 +230,9 @@ private HoodieCleanMetadata runClean(HoodieTable<T, I, K, O> table, HoodieInstan
this.txnManager.beginTransaction(Option.of(inflightInstant), Option.empty());
}
writeTableMetadata(metadata, inflightInstant.requestedTime());
- table.getActiveTimeline().transitionCleanInflightToComplete(false,
- inflightInstant, TimelineMetadataUtils.serializeCleanMetadata(metadata));
+ table.getActiveTimeline().transitionCleanInflightToComplete(false, inflightInstant, Option.of(metadata));
LOG.info("Marked clean started on " + inflightInstant.requestedTime() + " as complete");
return metadata;
- } catch (IOException e) {
- throw new HoodieIOException("Failed to clean up after commit", e);
Review Comment:
After checking this again, I think you removed it because `HoodieIOException` is already thrown when `IOException` is thrown in `transitionStateToComplete`, correct? If so, it should be good and there is no further change required.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2692463736
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3997",
"triggerID" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"triggerType" : "PUSH"
}, {
"hash" : "54eb05f956d8416c9f22578629592acc52eef93f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "54eb05f956d8416c9f22578629592acc52eef93f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c633f1a54ffbb6660dea21b15ef59abe9ec1307a Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3997)
* 54eb05f956d8416c9f22578629592acc52eef93f UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #12900:
URL: https://github.com/apache/hudi/pull/12900#discussion_r1977974006
##########
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieActiveTimeline.java:
##########
@@ -204,31 +202,29 @@ HoodieInstant transitionLogCompactionInflightToComplete(boolean shouldLock,
*
* @param shouldLock Whether to hold the lock when performing transition
* @param inflightInstant Inflight instant
- * @param data Extra Metadata
+ * @param metadata metadata to write into the instant file
* @return commit instant
*/
- HoodieInstant transitionCleanInflightToComplete(boolean shouldLock, HoodieInstant inflightInstant,
- Option<byte[]> data);
+ <T> HoodieInstant transitionCleanInflightToComplete(boolean shouldLock, HoodieInstant inflightInstant, Option<T> metadata);
/**
* Transition Clean State from requested to inflight.
*
* @param requestedInstant requested instant
- * @param data Optional data to be stored
+ * @param metadata metadata to write into the instant file
* @return commit instant
*/
- HoodieInstant transitionCleanRequestedToInflight(HoodieInstant requestedInstant, Option<byte[]> data);
+ <T> HoodieInstant transitionCleanRequestedToInflight(HoodieInstant requestedInstant, Option<T> metadata);
/**
* Transition Rollback State from inflight to Committed.
*
* @param shouldLock Whether to hold the lock when performing transition
* @param inflightInstant Inflight instant
- * @param data Extra Metadata
+ * @param metadata metadata to write into the instant file
* @return commit instant
*/
- HoodieInstant transitionRollbackInflightToComplete(boolean shouldLock,
- HoodieInstant inflightInstant, Option<byte[]> data);
+ <T> HoodieInstant transitionRollbackInflightToComplete(boolean shouldLock, HoodieInstant inflightInstant, Option<T> metadata);
Review Comment:
For all the APIs on a particular action, e.g., rollback, should they take a particular metadata class and avoid `Option` if not expecting `Opiton.empty()`, e.g., `HoodieRollbackMetadata` in this case?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2695496764
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3997",
"triggerID" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"triggerType" : "PUSH"
}, {
"hash" : "54eb05f956d8416c9f22578629592acc52eef93f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3998",
"triggerID" : "54eb05f956d8416c9f22578629592acc52eef93f",
"triggerType" : "PUSH"
}, {
"hash" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4001",
"triggerID" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"triggerType" : "PUSH"
}, {
"hash" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4002",
"triggerID" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"triggerType" : "PUSH"
}, {
"hash" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4003",
"triggerID" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"triggerType" : "PUSH"
}, {
"hash" : "7577553bfb6d14dda0c4a538ad315a914690ea2a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4004",
"triggerID" : "7577553bfb6d14dda0c4a538ad315a914690ea2a",
"triggerType" : "PUSH"
}, {
"hash" : "5889b122c1ad7a076bf9d61cf6edd3f310d804de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4006",
"triggerID" : "5889b122c1ad7a076bf9d61cf6edd3f310d804de",
"triggerType" : "PUSH"
}, {
"hash" : "876e8278fdd743e006a3a5cc65ff4811239f5af3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4014",
"triggerID" : "876e8278fdd743e006a3a5cc65ff4811239f5af3",
"triggerType" : "PUSH"
}, {
"hash" : "3186fc1fb5b83d310e86fd37bdf12c5c4b81b178",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4019",
"triggerID" : "3186fc1fb5b83d310e86fd37bdf12c5c4b81b178",
"triggerType" : "PUSH"
}, {
"hash" : "f0623433a2605c782bbb755c516ffc03d4dada29",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "f0623433a2605c782bbb755c516ffc03d4dada29",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3186fc1fb5b83d310e86fd37bdf12c5c4b81b178 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4019)
* f0623433a2605c782bbb755c516ffc03d4dada29 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Hudi 9092 3 [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2692416129
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3997",
"triggerID" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c633f1a54ffbb6660dea21b15ef59abe9ec1307a Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3997)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Hudi 9092 3 [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2692415595
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c633f1a54ffbb6660dea21b15ef59abe9ec1307a UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #12900:
URL: https://github.com/apache/hudi/pull/12900#discussion_r1978342991
##########
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/TimelineMetadataUtils.java:
##########
@@ -63,6 +58,19 @@
public class TimelineMetadataUtils {
private static final Integer DEFAULT_VERSION = 1;
+ private static final Map<Class<?>, DatumWriter<?>> DATUM_WRITERS = new ConcurrentHashMap<>();
+
+ // Legacy method to handle cases where byte [] is still used as opposed to leveraging HoodieInstantWriter.
+ @Deprecated
+ public static <T> byte[] convertMetadataToBytArray(T metadata, CommitMetadataSerDe serDe) {
Review Comment:
```suggestion
public static <T> byte[] convertMetadataToByteArray(T metadata, CommitMetadataSerDe serDe) {
```
##########
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/TimelineMetadataUtils.java:
##########
@@ -181,4 +140,17 @@ public static <T extends SpecificRecordBase> T deserializeAvroMetadata(InputStre
return fileReader.next();
}
}
+
+ public static <T extends SpecificRecordBase> Option<HoodieInstantWriter> getInstantWriter(Option<T> metadata) {
+ if (metadata.isEmpty()) {
+ return Option.empty();
+ }
+ return Option.of(outputStream -> {
+ DatumWriter<T> datumWriter = (DatumWriter<T>) DATUM_WRITERS.computeIfAbsent(metadata.get().getClass(), SpecificDatumWriter::new);
Review Comment:
Is `SpecificDatumWriter` thread-safe, i.e., using the same `SpecificDatumWriter` instance to serialize metadata concurrently does not incur any correctness issue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2696208726
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "82c37b34166c91a1c90dc1b501b3ffa12e898c2a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4033",
"triggerID" : "82c37b34166c91a1c90dc1b501b3ffa12e898c2a",
"triggerType" : "PUSH"
}, {
"hash" : "55cc2f7be57c6132c55c5f6d9884fbada80e9bbc",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4035",
"triggerID" : "55cc2f7be57c6132c55c5f6d9884fbada80e9bbc",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 55cc2f7be57c6132c55c5f6d9884fbada80e9bbc Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4035)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2698766054
All tests pass in Azure CI but Jacoco reporting failed due to `EOFException`. Merging this PR now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2692520520
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3997",
"triggerID" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"triggerType" : "PUSH"
}, {
"hash" : "54eb05f956d8416c9f22578629592acc52eef93f",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3998",
"triggerID" : "54eb05f956d8416c9f22578629592acc52eef93f",
"triggerType" : "PUSH"
}, {
"hash" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4001",
"triggerID" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 54eb05f956d8416c9f22578629592acc52eef93f Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3998)
* ce47eecc9860b49515b510c4b1567f326e7e0c62 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4001)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #12900:
URL: https://github.com/apache/hudi/pull/12900#discussion_r1978381987
##########
hudi-common/src/test/java/org/apache/hudi/common/testutils/FileCreateUtilsLegacy.java:
##########
@@ -83,11 +81,18 @@ private static void createMetaFile(String basePath, String instantTime, String s
}
private static void createMetaFile(String basePath, String instantTime, String suffix) throws IOException {
- createMetaFile(basePath, instantTime, suffix, getUTF8Bytes(""));
+ createMetaFile(basePath, instantTime, suffix, Option.empty());
}
- private static void createMetaFile(String basePath, String instantTime, String suffix, byte[] content) throws IOException {
- createMetaFile(getTimelinePath(new StoragePath(basePath)).toUri().getPath(), instantTime, InProcessTimeGenerator::createNewInstantTime, suffix, content);
+ private static <T> void createMetaFile(String basePath, String instantTime, String suffix, Option<T> metadata) throws IOException {
+ createMetaFile(getTimelinePath(new StoragePath(basePath)).toUri().getPath(), instantTime,
+ InProcessTimeGenerator::createNewInstantTime, suffix,
+ metadata.isEmpty() ? Option.empty() : COMMIT_METADATA_SER_DE.getInstantWriter(metadata.get()));
Review Comment:
nit: use `metadata.map()`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #12900:
URL: https://github.com/apache/hudi/pull/12900#discussion_r1978379964
##########
hudi-common/src/test/java/org/apache/hudi/common/testutils/FileCreateUtils.java:
##########
@@ -185,29 +173,28 @@ public static void createInflightReplaceCommit(HoodieTableMetaClient metaClient,
public static void createReplaceCommit(HoodieTableMetaClient metaClient, CommitMetadataSerDe commitMetadataSerDe,
String instantTime, HoodieReplaceCommitMetadata metadata) throws IOException {
createMetaFile(metaClient, instantTime, HoodieTimeline.REPLACE_COMMIT_EXTENSION,
- serializeCommitMetadata(commitMetadataSerDe, metadata).get());
+ Option.of(metadata));
}
public static void createReplaceCommit(HoodieTableMetaClient metaClient, CommitMetadataSerDe commitMetadataSerDe,
String instantTime, String completionTime, HoodieReplaceCommitMetadata metadata) throws IOException {
createMetaFileInTimelinePath(
metaClient, instantTime, () -> completionTime, HoodieTimeline.REPLACE_COMMIT_EXTENSION,
- serializeCommitMetadata(commitMetadataSerDe, metadata).get());
+ metaClient.getCommitMetadataSerDe().getInstantWriter(metadata));
}
public static void createRequestedClusterCommit(HoodieTableMetaClient metaClient, String instantTime,
HoodieRequestedReplaceMetadata requestedReplaceMetadata)
throws IOException {
createMetaFile(metaClient, instantTime, HoodieTimeline.REQUESTED_CLUSTERING_COMMIT_EXTENSION,
- serializeRequestedReplaceMetadata(requestedReplaceMetadata).get());
+ Option.of(requestedReplaceMetadata));
}
- public static void createInflightClusterCommit(HoodieTableMetaClient metaClient, CommitMetadataSerDe commitMetadataSerDe,
- String instantTime, Option<HoodieCommitMetadata> inflightReplaceMetadata)
+ public static <T> void createInflightClusterCommit(HoodieTableMetaClient metaClient, CommitMetadataSerDe commitMetadataSerDe,
+ String instantTime, Option<T> inflightReplaceMetadata)
Review Comment:
nit: can still use `HoodieCommitMetadata`?
##########
hudi-common/src/test/java/org/apache/hudi/common/model/TestHoodieCommitMetadata.java:
##########
@@ -180,7 +181,7 @@ public void testCommitMetadataSerde() throws Exception {
// Case: Reading 0.x written commit metadata
HoodieInstant legacyInstant = INSTANT_GENERATOR.createNewInstant(HoodieInstant.State.COMPLETED, "commit", "1", "1", true);
CommitMetadataSerDe v1SerDe = new CommitMetadataSerDeV1();
- byte[] v1Bytes = v1SerDe.serialize(commitMetadata1).get();
+ byte[] v1Bytes = convertMetadataToBytArray(commitMetadata1, v1SerDe);
Review Comment:
```suggestion
byte[] v1Bytes = convertMetadataToByteArray(commitMetadata1, v1SerDe);
```
##########
hudi-common/src/test/java/org/apache/hudi/common/table/timeline/versioning/v2/TestCommitMetadataSerDeV2.java:
##########
@@ -65,9 +64,8 @@ public void testLegacyInstant() throws Exception {
// Serialize and deserialize
- Option<byte[]> serialized = serDeV1.serialize(metadata);
- assertTrue(serialized.isPresent());
- HoodieCommitMetadata deserialized = getSerDe().deserialize(instant, new ByteArrayInputStream(serialized.get()), () -> false, HoodieCommitMetadata.class);
+ byte[] serialized = convertMetadataToBytArray(metadata, serDeV1);
Review Comment:
```suggestion
byte[] serialized = convertMetadataToByteArray(metadata, serDeV1);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "Davis-Zhang-Onehouse (via GitHub)" <gi...@apache.org>.
Davis-Zhang-Onehouse commented on code in PR #12900:
URL: https://github.com/apache/hudi/pull/12900#discussion_r1978487089
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/clean/CleanActionExecutor.java:
##########
@@ -232,12 +230,9 @@ private HoodieCleanMetadata runClean(HoodieTable<T, I, K, O> table, HoodieInstan
this.txnManager.beginTransaction(Option.of(inflightInstant), Option.empty());
}
writeTableMetadata(metadata, inflightInstant.requestedTime());
- table.getActiveTimeline().transitionCleanInflightToComplete(false,
- inflightInstant, TimelineMetadataUtils.serializeCleanMetadata(metadata));
+ table.getActiveTimeline().transitionCleanInflightToComplete(false, inflightInstant, Option.of(metadata));
LOG.info("Marked clean started on " + inflightInstant.requestedTime() + " as complete");
return metadata;
- } catch (IOException e) {
- throw new HoodieIOException("Failed to clean up after commit", e);
Review Comment:
yes
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/clean/CleanActionExecutor.java:
##########
@@ -232,12 +230,9 @@ private HoodieCleanMetadata runClean(HoodieTable<T, I, K, O> table, HoodieInstan
this.txnManager.beginTransaction(Option.of(inflightInstant), Option.empty());
}
writeTableMetadata(metadata, inflightInstant.requestedTime());
- table.getActiveTimeline().transitionCleanInflightToComplete(false,
- inflightInstant, TimelineMetadataUtils.serializeCleanMetadata(metadata));
+ table.getActiveTimeline().transitionCleanInflightToComplete(false, inflightInstant, Option.of(metadata));
LOG.info("Marked clean started on " + inflightInstant.requestedTime() + " as complete");
return metadata;
- } catch (IOException e) {
- throw new HoodieIOException("Failed to clean up after commit", e);
Review Comment:
yes it already throw the exception with the right type so no need to catch and convert.
##########
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/TimelineMetadataUtils.java:
##########
@@ -63,6 +58,19 @@
public class TimelineMetadataUtils {
private static final Integer DEFAULT_VERSION = 1;
+ private static final Map<Class<?>, DatumWriter<?>> DATUM_WRITERS = new ConcurrentHashMap<>();
+
+ // Legacy method to handle cases where byte [] is still used as opposed to leveraging HoodieInstantWriter.
+ @Deprecated
+ public static <T> byte[] convertMetadataToBytArray(T metadata, CommitMetadataSerDe serDe) {
Review Comment:
done
##########
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/versioning/v2/CommitMetadataSerDeV2.java:
##########
@@ -93,23 +91,10 @@ public <T> T deserialize(HoodieInstant instant, InputStream inputStream, Boolean
}
@Override
- public Option<byte[]> serialize(org.apache.hudi.common.model.HoodieCommitMetadata commitMetadata) throws IOException {
- if (commitMetadata instanceof org.apache.hudi.common.model.HoodieReplaceCommitMetadata) {
- return serializeAvroMetadata(MetadataConversionUtils.convertCommitMetadata(commitMetadata), HoodieReplaceCommitMetadata.class);
+ public <T> Option<HoodieInstantWriter> getInstantWriter(T metadata) {
+ if (metadata instanceof org.apache.hudi.common.model.HoodieCommitMetadata) {
Review Comment:
yes
##########
hudi-hadoop-common/src/test/java/org/apache/hudi/common/table/view/TestHoodieTableFileSystemView.java:
##########
@@ -1895,18 +1889,18 @@ public void testPendingCompactionWithDuplicateFileIdsAcrossPartitions(boolean pr
assertTrue(fileIdsInCompaction.contains(fileId));
}
- private void saveAsComplete(HoodieActiveTimeline timeline, HoodieInstant inflight, Option<byte[]> data) {
+ private void saveAsComplete(HoodieActiveTimeline timeline, HoodieInstant inflight, HoodieCommitMetadata metadata) {
if (inflight.getAction().equals(HoodieTimeline.COMPACTION_ACTION)) {
- timeline.transitionCompactionInflightToComplete(true, inflight, data);
+ timeline.transitionCompactionInflightToComplete(true, inflight, metadata);
} else {
HoodieInstant requested = INSTANT_GENERATOR.createNewInstant(State.REQUESTED, inflight.getAction(), inflight.requestedTime());
timeline.createNewInstant(requested);
timeline.transitionRequestedToInflight(requested, Option.empty());
- timeline.saveAsComplete(inflight, data);
+ timeline.saveAsComplete(inflight, Option.of(metadata));
}
}
- private void saveAsCompleteCluster(HoodieActiveTimeline timeline, HoodieInstant inflight, Option<byte[]> data) {
+ private <T> void saveAsCompleteCluster(HoodieActiveTimeline timeline, HoodieInstant inflight, HoodieCommitMetadata metadata) {
Review Comment:
done
##########
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/versioning/v1/ActiveTimelineV1.java:
##########
@@ -138,25 +147,24 @@ public void createRequestedCommitWithReplaceMetadata(String instantTime, String
HoodieInstant instant = instantGenerator.createNewInstant(HoodieInstant.State.REQUESTED, actionType, instantTime);
LOG.info("Creating a new instant " + instant);
// Create the request replace file
- createFileInMetaPath(instantFileNameGenerator.getFileName(instant),
- TimelineMetadataUtils.serializeRequestedReplaceMetadata(new HoodieRequestedReplaceMetadata()), false);
- } catch (IOException e) {
- throw new HoodieIOException("Error create requested replace commit ", e);
+ createFileInMetaPath(instantFileNameGenerator.getFileName(instant), Option.of(new HoodieRequestedReplaceMetadata()), false);
+ } catch (HoodieIOException e) {
+ throw e;
}
Review Comment:
done
##########
hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/table/functional/TestCleanActionExecutor.java:
##########
@@ -138,7 +138,7 @@ void testPartialCleanFailure(CleanFailureType failureType) throws IOException {
when(activeTimeline.getCleanerTimeline()).thenReturn(cleanTimeline);
when(cleanTimeline.getInstants()).thenReturn(Collections.singletonList(cleanInstant));
when(activeTimeline.readCleanerPlan(cleanInstant)).thenReturn(cleanerPlan);
- when(activeTimeline.readCleanerInfoAsBytes(cleanInstant)).thenReturn(TimelineMetadataUtils.serializeCleanerPlan(cleanerPlan));
+ when(activeTimeline.readCleanerInfoAsBytes(cleanInstant)).thenReturn(Option.of(convertMetadataToBytArray(cleanerPlan)));
Review Comment:
https://issues.apache.org/jira/browse/HUDI-9099 tracking jira
##########
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/TimelineMetadataUtils.java:
##########
@@ -181,4 +140,17 @@ public static <T extends SpecificRecordBase> T deserializeAvroMetadata(InputStre
return fileReader.next();
}
}
+
+ public static <T extends SpecificRecordBase> Option<HoodieInstantWriter> getInstantWriter(Option<T> metadata) {
+ if (metadata.isEmpty()) {
+ return Option.empty();
+ }
+ return Option.of(outputStream -> {
+ DatumWriter<T> datumWriter = (DatumWriter<T>) DATUM_WRITERS.computeIfAbsent(metadata.get().getClass(), SpecificDatumWriter::new);
Review Comment:
fixed
##########
hudi-common/src/test/java/org/apache/hudi/common/testutils/FileCreateUtilsLegacy.java:
##########
@@ -83,11 +81,18 @@ private static void createMetaFile(String basePath, String instantTime, String s
}
private static void createMetaFile(String basePath, String instantTime, String suffix) throws IOException {
- createMetaFile(basePath, instantTime, suffix, getUTF8Bytes(""));
+ createMetaFile(basePath, instantTime, suffix, Option.empty());
}
- private static void createMetaFile(String basePath, String instantTime, String suffix, byte[] content) throws IOException {
- createMetaFile(getTimelinePath(new StoragePath(basePath)).toUri().getPath(), instantTime, InProcessTimeGenerator::createNewInstantTime, suffix, content);
+ private static <T> void createMetaFile(String basePath, String instantTime, String suffix, Option<T> metadata) throws IOException {
+ createMetaFile(getTimelinePath(new StoragePath(basePath)).toUri().getPath(), instantTime,
+ InProcessTimeGenerator::createNewInstantTime, suffix,
+ metadata.isEmpty() ? Option.empty() : COMMIT_METADATA_SER_DE.getInstantWriter(metadata.get()));
Review Comment:
done
##########
hudi-common/pom.xml:
##########
@@ -354,6 +354,12 @@
<artifactId>tally-core</artifactId>
<version>${tally.version}</version>
</dependency>
+ <dependency>
+ <groupId>commons-io</groupId>
+ <artifactId>commons-io</artifactId>
+ <version>2.18.0</version>
+ <scope>test</scope>
+ </dependency>
Review Comment:
removed as we already declared the dependency in the same pom file
##########
hudi-client/hudi-client-common/src/test/java/org/apache/hudi/common/table/timeline/TestArchivedTimelineV2.java:
##########
@@ -100,7 +101,7 @@ private void writeArchivedTimeline(int batchSize, long startTs) throws Exception
String completionTime = String.valueOf(instantTimeTs + 10);
HoodieInstant instant = INSTANT_GENERATOR.createNewInstant(HoodieInstant.State.COMPLETED, "commit", instantTime, completionTime);
HoodieCommitMetadata metadata = testTable.createCommitMetadata(instantTime, WriteOperationType.INSERT, Arrays.asList("par1", "par2"), 10, false);
- byte[] serializedMetadata = TimelineMetadataUtils.serializeCommitMetadata(metaClient.getCommitMetadataSerDe(), metadata).get();
+ byte[] serializedMetadata = convertMetadataToBytArray(metadata);
Review Comment:
this is for test usage, if they want to be explicit on v1 or v2, use the same underlying method as convertMetadataToBytArray
fixed the typo
##########
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/versioning/v2/ActiveTimelineV2.java:
##########
@@ -143,25 +152,24 @@ public void createRequestedCommitWithReplaceMetadata(String instantTime, String
HoodieInstant instant = instantGenerator.createNewInstant(HoodieInstant.State.REQUESTED, actionType, instantTime);
LOG.info("Creating a new instant " + instant);
// Create the request replace file
- createFileInMetaPath(instantFileNameGenerator.getFileName(instant),
- TimelineMetadataUtils.serializeRequestedReplaceMetadata(new HoodieRequestedReplaceMetadata()), false);
- } catch (IOException e) {
- throw new HoodieIOException("Error create requested replace commit ", e);
+ createFileInMetaPath(instantFileNameGenerator.getFileName(instant), Option.of(new HoodieRequestedReplaceMetadata()), false);
+ } catch (HoodieIOException e) {
+ throw e;
Review Comment:
done
##########
hudi-common/src/test/java/org/apache/hudi/common/testutils/Transformations.java:
##########
@@ -79,4 +83,14 @@ public static <T> List<T> randomSelect(List<T> items, int n) {
public static List<HoodieKey> randomSelectAsHoodieKeys(List<HoodieRecord> records, int n) {
return randomSelect(recordsToHoodieKeys(records), n);
}
+
+ public static byte[] writeInstantContentToBytes(HoodieInstantWriter writer) {
Review Comment:
done and tracked in https://issues.apache.org/jira/browse/HUDI-9099
##########
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/versioning/v2/CommitMetadataSerDeV2.java:
##########
@@ -93,23 +91,10 @@ public <T> T deserialize(HoodieInstant instant, InputStream inputStream, Boolean
}
@Override
- public Option<byte[]> serialize(org.apache.hudi.common.model.HoodieCommitMetadata commitMetadata) throws IOException {
- if (commitMetadata instanceof org.apache.hudi.common.model.HoodieReplaceCommitMetadata) {
- return serializeAvroMetadata(MetadataConversionUtils.convertCommitMetadata(commitMetadata), HoodieReplaceCommitMetadata.class);
+ public <T> Option<HoodieInstantWriter> getInstantWriter(T metadata) {
+ if (metadata instanceof org.apache.hudi.common.model.HoodieCommitMetadata) {
+ return TimelineMetadataUtils.getInstantWriter(Option.of(MetadataConversionUtils.convertCommitMetadataToAvro((HoodieCommitMetadata) metadata)));
Review Comment:
moved
##########
hudi-common/src/test/java/org/apache/hudi/common/table/timeline/versioning/v2/TestCommitMetadataSerDeV2.java:
##########
@@ -65,9 +64,8 @@ public void testLegacyInstant() throws Exception {
// Serialize and deserialize
- Option<byte[]> serialized = serDeV1.serialize(metadata);
- assertTrue(serialized.isPresent());
- HoodieCommitMetadata deserialized = getSerDe().deserialize(instant, new ByteArrayInputStream(serialized.get()), () -> false, HoodieCommitMetadata.class);
+ byte[] serialized = convertMetadataToBytArray(metadata, serDeV1);
Review Comment:
done
##########
hudi-common/src/test/java/org/apache/hudi/common/model/TestHoodieCommitMetadata.java:
##########
@@ -180,7 +181,7 @@ public void testCommitMetadataSerde() throws Exception {
// Case: Reading 0.x written commit metadata
HoodieInstant legacyInstant = INSTANT_GENERATOR.createNewInstant(HoodieInstant.State.COMPLETED, "commit", "1", "1", true);
CommitMetadataSerDe v1SerDe = new CommitMetadataSerDeV1();
- byte[] v1Bytes = v1SerDe.serialize(commitMetadata1).get();
+ byte[] v1Bytes = convertMetadataToBytArray(commitMetadata1, v1SerDe);
Review Comment:
done
##########
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/TimelineUtils.java:
##########
@@ -623,4 +624,15 @@ public static Option<InputStream> getInputStreamOptionLegacy(HoodieTimeline time
}
return Option.of(new ByteArrayInputStream(bytes.get()));
}
+
+ // TODO[HUDI-9094]: work around when caller needs to write byte array in raw. This method should be removed.
+ public static <T> Option<HoodieInstantWriter> getHoodieInstantWriterOption(HoodieTimeline timeline, Option<T> metadata) {
+ Option<HoodieInstantWriter> writerOption;
+ if (metadata.isPresent() && metadata.get() instanceof HoodieInstantWriter) {
+ writerOption = (Option<HoodieInstantWriter>) metadata;
+ } else {
+ writerOption = timeline.getInstantWriter(metadata);
+ }
+ return writerOption;
+ }
Review Comment:
create jira
##########
hudi-common/src/test/java/org/apache/hudi/common/testutils/HoodieTestUtils.java:
##########
@@ -392,6 +393,10 @@ public static StoragePath getCompleteInstantPath(HoodieStorage storage, StorageP
return getCompleteInstantFileInfo(storage, parent, instantTime, action).getPath();
}
+ public static <T> byte[] convertMetadataToBytArray(T metadata) {
Review Comment:
done
##########
hudi-common/src/test/java/org/apache/hudi/common/testutils/FileCreateUtilsLegacy.java:
##########
@@ -149,30 +152,26 @@ public static void createInflightDeltaCommit(String basePath, String instantTime
public static void createReplaceCommit(CommitMetadataSerDe commitMetadataSerDe, String basePath,
String instantTime, HoodieReplaceCommitMetadata metadata) throws IOException {
- createMetaFile(basePath, instantTime, HoodieTimeline.REPLACE_COMMIT_EXTENSION,
- serializeCommitMetadata(commitMetadataSerDe, metadata).get());
+ createMetaFile(basePath, instantTime, HoodieTimeline.REPLACE_COMMIT_EXTENSION, Option.of(metadata));
}
public static void createRequestedClusterCommit(String basePath, String instantTime,
- HoodieRequestedReplaceMetadata requestedReplaceMetadata)
- throws IOException {
- createMetaFile(basePath, instantTime, HoodieTimeline.REQUESTED_CLUSTERING_COMMIT_EXTENSION,
- serializeRequestedReplaceMetadata(requestedReplaceMetadata).get());
+ HoodieRequestedReplaceMetadata requestedReplaceMetadata) throws IOException {
+ createMetaFile(basePath, instantTime, HoodieTimeline.REQUESTED_CLUSTERING_COMMIT_EXTENSION, Option.of(requestedReplaceMetadata));
}
public static void createRequestedCompactionCommit(String basePath, String instantTime,
- HoodieCompactionPlan requestedCompactionPlan)
- throws IOException {
- createMetaFile(basePath, instantTime, HoodieTimeline.REQUESTED_COMPACTION_EXTENSION,
- serializeCompactionPlan(requestedCompactionPlan).get());
+ HoodieCompactionPlan requestedCompactionPlan) throws IOException {
+ createMetaFile(basePath, instantTime, HoodieTimeline.REQUESTED_COMPACTION_EXTENSION, Option.of(requestedCompactionPlan));
}
public static void createRequestedRollbackFile(String basePath, String instantTime, HoodieRollbackPlan plan) throws IOException {
- createMetaFile(basePath, instantTime, HoodieTimeline.REQUESTED_ROLLBACK_EXTENSION, serializeRollbackPlan(plan).get());
+ createMetaFile(basePath, instantTime, HoodieTimeline.REQUESTED_ROLLBACK_EXTENSION, Option.of(plan));
}
public static void createRequestedRollbackFile(String basePath, String instantTime, byte[] content) throws IOException {
- createMetaFile(basePath, instantTime, HoodieTimeline.REQUESTED_ROLLBACK_EXTENSION, content);
+ createMetaFile(basePath, instantTime, HoodieTimeline.REQUESTED_ROLLBACK_EXTENSION,
Review Comment:
https://issues.apache.org/jira/browse/HUDI-9099
##########
hudi-common/src/test/java/org/apache/hudi/common/testutils/FileCreateUtils.java:
##########
@@ -185,29 +173,28 @@ public static void createInflightReplaceCommit(HoodieTableMetaClient metaClient,
public static void createReplaceCommit(HoodieTableMetaClient metaClient, CommitMetadataSerDe commitMetadataSerDe,
String instantTime, HoodieReplaceCommitMetadata metadata) throws IOException {
createMetaFile(metaClient, instantTime, HoodieTimeline.REPLACE_COMMIT_EXTENSION,
- serializeCommitMetadata(commitMetadataSerDe, metadata).get());
+ Option.of(metadata));
}
public static void createReplaceCommit(HoodieTableMetaClient metaClient, CommitMetadataSerDe commitMetadataSerDe,
String instantTime, String completionTime, HoodieReplaceCommitMetadata metadata) throws IOException {
createMetaFileInTimelinePath(
metaClient, instantTime, () -> completionTime, HoodieTimeline.REPLACE_COMMIT_EXTENSION,
- serializeCommitMetadata(commitMetadataSerDe, metadata).get());
+ metaClient.getCommitMetadataSerDe().getInstantWriter(metadata));
}
public static void createRequestedClusterCommit(HoodieTableMetaClient metaClient, String instantTime,
HoodieRequestedReplaceMetadata requestedReplaceMetadata)
throws IOException {
createMetaFile(metaClient, instantTime, HoodieTimeline.REQUESTED_CLUSTERING_COMMIT_EXTENSION,
- serializeRequestedReplaceMetadata(requestedReplaceMetadata).get());
+ Option.of(requestedReplaceMetadata));
}
- public static void createInflightClusterCommit(HoodieTableMetaClient metaClient, CommitMetadataSerDe commitMetadataSerDe,
- String instantTime, Option<HoodieCommitMetadata> inflightReplaceMetadata)
+ public static <T> void createInflightClusterCommit(HoodieTableMetaClient metaClient, CommitMetadataSerDe commitMetadataSerDe,
+ String instantTime, Option<T> inflightReplaceMetadata)
Review Comment:
revised.
##########
hudi-platform-service/hudi-metaserver/hudi-metaserver-client/src/main/java/org/apache/hudi/common/table/timeline/HoodieMetaserverBasedTimeline.java:
##########
@@ -56,23 +60,26 @@ protected void deleteInstantFile(HoodieInstant instant) {
}
@Override
- protected void transitionStateToComplete(boolean shouldLock, HoodieInstant fromInstant, HoodieInstant toInstant, Option<byte[]> data) {
+ protected <T> void transitionStateToComplete(boolean shouldLock, HoodieInstant fromInstant, HoodieInstant toInstant, Option<T> metadata) {
ValidationUtils.checkArgument(fromInstant.requestedTime().equals(toInstant.requestedTime()));
- metaserverClient.transitionInstantState(databaseName, tableName, fromInstant, toInstant, data);
+ metaserverClient.transitionInstantState(databaseName, tableName, fromInstant, toInstant,
+ metadata.map(m -> convertMetadataToBytArray(m, metadataSerDeV2)));
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2695985953
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3997",
"triggerID" : "c633f1a54ffbb6660dea21b15ef59abe9ec1307a",
"triggerType" : "PUSH"
}, {
"hash" : "54eb05f956d8416c9f22578629592acc52eef93f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=3998",
"triggerID" : "54eb05f956d8416c9f22578629592acc52eef93f",
"triggerType" : "PUSH"
}, {
"hash" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4001",
"triggerID" : "ce47eecc9860b49515b510c4b1567f326e7e0c62",
"triggerType" : "PUSH"
}, {
"hash" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4002",
"triggerID" : "25996d9bc7aef3436746bf334e99fe45289fc17d",
"triggerType" : "PUSH"
}, {
"hash" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4003",
"triggerID" : "d0aecd48540a65464cdf9dd6062d2a10f6a8a7e0",
"triggerType" : "PUSH"
}, {
"hash" : "7577553bfb6d14dda0c4a538ad315a914690ea2a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4004",
"triggerID" : "7577553bfb6d14dda0c4a538ad315a914690ea2a",
"triggerType" : "PUSH"
}, {
"hash" : "5889b122c1ad7a076bf9d61cf6edd3f310d804de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4006",
"triggerID" : "5889b122c1ad7a076bf9d61cf6edd3f310d804de",
"triggerType" : "PUSH"
}, {
"hash" : "876e8278fdd743e006a3a5cc65ff4811239f5af3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4014",
"triggerID" : "876e8278fdd743e006a3a5cc65ff4811239f5af3",
"triggerType" : "PUSH"
}, {
"hash" : "3186fc1fb5b83d310e86fd37bdf12c5c4b81b178",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4019",
"triggerID" : "3186fc1fb5b83d310e86fd37bdf12c5c4b81b178",
"triggerType" : "PUSH"
}, {
"hash" : "f0623433a2605c782bbb755c516ffc03d4dada29",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4024",
"triggerID" : "f0623433a2605c782bbb755c516ffc03d4dada29",
"triggerType" : "PUSH"
}, {
"hash" : "82c37b34166c91a1c90dc1b501b3ffa12e898c2a",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "82c37b34166c91a1c90dc1b501b3ffa12e898c2a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* f0623433a2605c782bbb755c516ffc03d4dada29 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4024)
* 82c37b34166c91a1c90dc1b501b3ffa12e898c2a UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12900:
URL: https://github.com/apache/hudi/pull/12900#issuecomment-2696068604
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "82c37b34166c91a1c90dc1b501b3ffa12e898c2a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4033",
"triggerID" : "82c37b34166c91a1c90dc1b501b3ffa12e898c2a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 82c37b34166c91a1c90dc1b501b3ffa12e898c2a Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4033)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "Davis-Zhang-Onehouse (via GitHub)" <gi...@apache.org>.
Davis-Zhang-Onehouse commented on code in PR #12900:
URL: https://github.com/apache/hudi/pull/12900#discussion_r1978169045
##########
hudi-io/src/main/java/org/apache/hudi/common/util/FileIOUtils.java:
##########
@@ -191,8 +191,8 @@ public static void createFileInPath(HoodieStorage storage,
}
}
- public static void createFileInPath(HoodieStorage storage, StoragePath fullPath, Option<byte[]> content) {
- createFileInPath(storage, fullPath, content.map(bytes -> (outputStream) -> outputStream.write(bytes)), false);
+ public static void createFileInPath(HoodieStorage fileSystem, StoragePath fullPath, Option<HoodieInstantWriter> contentWriter) {
Review Comment:
done
##########
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieActiveTimeline.java:
##########
@@ -204,31 +202,29 @@ HoodieInstant transitionLogCompactionInflightToComplete(boolean shouldLock,
*
* @param shouldLock Whether to hold the lock when performing transition
* @param inflightInstant Inflight instant
- * @param data Extra Metadata
+ * @param metadata metadata to write into the instant file
* @return commit instant
*/
- HoodieInstant transitionCleanInflightToComplete(boolean shouldLock, HoodieInstant inflightInstant,
- Option<byte[]> data);
+ <T> HoodieInstant transitionCleanInflightToComplete(boolean shouldLock, HoodieInstant inflightInstant, Option<T> metadata);
/**
* Transition Clean State from requested to inflight.
*
* @param requestedInstant requested instant
- * @param data Optional data to be stored
+ * @param metadata metadata to write into the instant file
* @return commit instant
*/
- HoodieInstant transitionCleanRequestedToInflight(HoodieInstant requestedInstant, Option<byte[]> data);
+ <T> HoodieInstant transitionCleanRequestedToInflight(HoodieInstant requestedInstant, Option<T> metadata);
/**
* Transition Rollback State from inflight to Committed.
*
* @param shouldLock Whether to hold the lock when performing transition
* @param inflightInstant Inflight instant
- * @param data Extra Metadata
+ * @param metadata metadata to write into the instant file
* @return commit instant
*/
- HoodieInstant transitionRollbackInflightToComplete(boolean shouldLock,
- HoodieInstant inflightInstant, Option<byte[]> data);
+ <T> HoodieInstant transitionRollbackInflightToComplete(boolean shouldLock, HoodieInstant inflightInstant, Option<T> metadata);
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9092] Deprecate byte array of serialization of instants [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #12900:
URL: https://github.com/apache/hudi/pull/12900#discussion_r1978322059
##########
hudi-client/hudi-client-common/src/test/java/org/apache/hudi/common/table/timeline/TestArchivedTimelineV2.java:
##########
@@ -100,7 +101,7 @@ private void writeArchivedTimeline(int batchSize, long startTs) throws Exception
String completionTime = String.valueOf(instantTimeTs + 10);
HoodieInstant instant = INSTANT_GENERATOR.createNewInstant(HoodieInstant.State.COMPLETED, "commit", instantTime, completionTime);
HoodieCommitMetadata metadata = testTable.createCommitMetadata(instantTime, WriteOperationType.INSERT, Arrays.asList("par1", "par2"), 10, false);
- byte[] serializedMetadata = TimelineMetadataUtils.serializeCommitMetadata(metaClient.getCommitMetadataSerDe(), metadata).get();
+ byte[] serializedMetadata = convertMetadataToBytArray(metadata);
Review Comment:
Should this util method still take a meta client instance as an argument to consider different serialization (i.e., JSON vs Avro between V1 and V2 for commit metadata)?
##########
hudi-client/hudi-client-common/src/test/java/org/apache/hudi/common/table/timeline/TestArchivedTimelineV2.java:
##########
@@ -100,7 +101,7 @@ private void writeArchivedTimeline(int batchSize, long startTs) throws Exception
String completionTime = String.valueOf(instantTimeTs + 10);
HoodieInstant instant = INSTANT_GENERATOR.createNewInstant(HoodieInstant.State.COMPLETED, "commit", instantTime, completionTime);
HoodieCommitMetadata metadata = testTable.createCommitMetadata(instantTime, WriteOperationType.INSERT, Arrays.asList("par1", "par2"), 10, false);
- byte[] serializedMetadata = TimelineMetadataUtils.serializeCommitMetadata(metaClient.getCommitMetadataSerDe(), metadata).get();
+ byte[] serializedMetadata = convertMetadataToBytArray(metadata);
Review Comment:
```suggestion
byte[] serializedMetadata = convertMetadataToByteArray(metadata);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org