You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "yihua (via GitHub)" <gi...@apache.org> on 2025/09/16 02:27:08 UTC
[PR] fix: Fix HFile reader compatibility [hudi]
yihua opened a new pull request, #13897:
URL: https://github.com/apache/hudi/pull/13897
### Change Logs
_Describe context and summary for this change. Highlight if any code was copied._
### Impact
_Describe any public API or user-facing feature change or any performance impact._
### Risk level (write none, low medium or high below)
_If medium or high, explain what verification was done to mitigate the risks._
### Documentation Update
_Describe any necessary documentation update if there is any new feature, config, or user-facing change. If not, put "none"._
- _The config description must be updated if new configs are added or the default value of the configs are changed_
- _Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the
ticket number here and follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make
changes to the website._
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(storage-format)!: Fix HFile writer to properly encode checksum for compatibility [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13897:
URL: https://github.com/apache/hudi/pull/13897#issuecomment-3313892694
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "98f44f95f3575ceb457769a5a2d640eaf6f63c36",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8549",
"triggerID" : "98f44f95f3575ceb457769a5a2d640eaf6f63c36",
"triggerType" : "PUSH"
}, {
"hash" : "9cd9e935e2f998b1425f74c3b6ac13eb419cfd0b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8645",
"triggerID" : "9cd9e935e2f998b1425f74c3b6ac13eb419cfd0b",
"triggerType" : "PUSH"
}, {
"hash" : "b4e138800e44033d622248034e7825d8eb6b7f77",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8647",
"triggerID" : "b4e138800e44033d622248034e7825d8eb6b7f77",
"triggerType" : "PUSH"
}, {
"hash" : "35e49f9dcf4f845042ecf2c2e5276b4a829ee71a",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "35e49f9dcf4f845042ecf2c2e5276b4a829ee71a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b4e138800e44033d622248034e7825d8eb6b7f77 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8647)
* 35e49f9dcf4f845042ecf2c2e5276b4a829ee71a UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(storage-format)!: Fix HFile writer to properly encode checksum for compatibility [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13897:
URL: https://github.com/apache/hudi/pull/13897#issuecomment-3310532536
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "98f44f95f3575ceb457769a5a2d640eaf6f63c36",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8549",
"triggerID" : "98f44f95f3575ceb457769a5a2d640eaf6f63c36",
"triggerType" : "PUSH"
}, {
"hash" : "9cd9e935e2f998b1425f74c3b6ac13eb419cfd0b",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8645",
"triggerID" : "9cd9e935e2f998b1425f74c3b6ac13eb419cfd0b",
"triggerType" : "PUSH"
}, {
"hash" : "b4e138800e44033d622248034e7825d8eb6b7f77",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8647",
"triggerID" : "b4e138800e44033d622248034e7825d8eb6b7f77",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9cd9e935e2f998b1425f74c3b6ac13eb419cfd0b Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8645)
* b4e138800e44033d622248034e7825d8eb6b7f77 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8647)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(storage-format)!: Fix HFile writer to properly encode checksum for compatibility [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #13897:
URL: https://github.com/apache/hudi/pull/13897#discussion_r2364566741
##########
hudi-io/src/main/java/org/apache/hudi/io/hfile/HFileBlock.java:
##########
@@ -323,9 +321,9 @@ protected long getStartOffsetInBuffForWrite() {
* Returns checksum bytes if checksum type is not NULL.
* Note that current HFileReaderImpl does not support non-NULL checksum.
*/
- private byte[] generateChecksumBytes(ChecksumType type) {
+ private byte[] generateChecksumBytes(ChecksumType type, int numChecksumBytes) {
if (type == ChecksumType.NULL) {
- return EMPTY_BYTE_ARRAY;
+ return new byte[numChecksumBytes];
Review Comment:
Right, HBase reader does not check the checksum if the checksum type is NULL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(storage-format)!: Fix HFile writer to properly encode checksum for compatibility [hudi]
Posted by "linliu-code (via GitHub)" <gi...@apache.org>.
linliu-code commented on code in PR #13897:
URL: https://github.com/apache/hudi/pull/13897#discussion_r2363971748
##########
hudi-io/src/main/java/org/apache/hudi/io/hfile/HFileBlock.java:
##########
@@ -323,9 +321,9 @@ protected long getStartOffsetInBuffForWrite() {
* Returns checksum bytes if checksum type is not NULL.
* Note that current HFileReaderImpl does not support non-NULL checksum.
*/
- private byte[] generateChecksumBytes(ChecksumType type) {
+ private byte[] generateChecksumBytes(ChecksumType type, int numChecksumBytes) {
if (type == ChecksumType.NULL) {
- return EMPTY_BYTE_ARRAY;
+ return new byte[numChecksumBytes];
Review Comment:
Does it means hbase reader will not check the content of these bytes since the Checksum type is null?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: Fix HFile reader compatibility [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13897:
URL: https://github.com/apache/hudi/pull/13897#issuecomment-3294640930
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "98f44f95f3575ceb457769a5a2d640eaf6f63c36",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8549",
"triggerID" : "98f44f95f3575ceb457769a5a2d640eaf6f63c36",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 98f44f95f3575ceb457769a5a2d640eaf6f63c36 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8549)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(storage-format)!: Fix HFile writer to properly encode checksum for compatibility [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13897:
URL: https://github.com/apache/hudi/pull/13897#issuecomment-3310379547
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "98f44f95f3575ceb457769a5a2d640eaf6f63c36",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8549",
"triggerID" : "98f44f95f3575ceb457769a5a2d640eaf6f63c36",
"triggerType" : "PUSH"
}, {
"hash" : "9cd9e935e2f998b1425f74c3b6ac13eb419cfd0b",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8645",
"triggerID" : "9cd9e935e2f998b1425f74c3b6ac13eb419cfd0b",
"triggerType" : "PUSH"
}, {
"hash" : "b4e138800e44033d622248034e7825d8eb6b7f77",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b4e138800e44033d622248034e7825d8eb6b7f77",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 98f44f95f3575ceb457769a5a2d640eaf6f63c36 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8549)
* 9cd9e935e2f998b1425f74c3b6ac13eb419cfd0b Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8645)
* b4e138800e44033d622248034e7825d8eb6b7f77 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(storage-format)!: Fix HFile writer to properly encode checksum for compatibility [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13897:
URL: https://github.com/apache/hudi/pull/13897#issuecomment-3310365403
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "98f44f95f3575ceb457769a5a2d640eaf6f63c36",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8549",
"triggerID" : "98f44f95f3575ceb457769a5a2d640eaf6f63c36",
"triggerType" : "PUSH"
}, {
"hash" : "9cd9e935e2f998b1425f74c3b6ac13eb419cfd0b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9cd9e935e2f998b1425f74c3b6ac13eb419cfd0b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 98f44f95f3575ceb457769a5a2d640eaf6f63c36 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8549)
* 9cd9e935e2f998b1425f74c3b6ac13eb419cfd0b UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: Fix HFile reader compatibility [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13897:
URL: https://github.com/apache/hudi/pull/13897#issuecomment-3294624225
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "98f44f95f3575ceb457769a5a2d640eaf6f63c36",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8549",
"triggerID" : "98f44f95f3575ceb457769a5a2d640eaf6f63c36",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 98f44f95f3575ceb457769a5a2d640eaf6f63c36 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8549)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(storage-format)!: Fix HFile writer to properly encode checksum for compatibility [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan merged PR #13897:
URL: https://github.com/apache/hudi/pull/13897
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(storage-format)!: Fix HFile writer to properly encode checksum for compatibility [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #13897:
URL: https://github.com/apache/hudi/pull/13897#discussion_r2364581763
##########
hudi-io/src/main/java/org/apache/hudi/io/hfile/HFileBlock.java:
##########
@@ -290,15 +290,13 @@ public ByteBuffer serialize() throws IOException {
// onDiskSizeWithoutHeader = uncompressedSizeWithoutHeader + Checksum.
// In order to pass this check, either we make isUseHBaseChecksum false in HFileContext (hbase),
// or we set this value to zero.
- buf.putInt(0);
+ buf.putInt(DEFAULT_BYTES_PER_CHECKSUM);
Review Comment:
Good catch. The comment is updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(storage-format)!: Fix HFile writer to properly encode checksum for compatibility [hudi]
Posted by "linliu-code (via GitHub)" <gi...@apache.org>.
linliu-code commented on PR #13897:
URL: https://github.com/apache/hudi/pull/13897#issuecomment-3312881122
LGTM.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(storage-format)!: Fix HFile writer to properly encode checksum for compatibility [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #13897:
URL: https://github.com/apache/hudi/pull/13897#discussion_r2364566741
##########
hudi-io/src/main/java/org/apache/hudi/io/hfile/HFileBlock.java:
##########
@@ -323,9 +321,9 @@ protected long getStartOffsetInBuffForWrite() {
* Returns checksum bytes if checksum type is not NULL.
* Note that current HFileReaderImpl does not support non-NULL checksum.
*/
- private byte[] generateChecksumBytes(ChecksumType type) {
+ private byte[] generateChecksumBytes(ChecksumType type, int numChecksumBytes) {
if (type == ChecksumType.NULL) {
- return EMPTY_BYTE_ARRAY;
+ return new byte[numChecksumBytes];
Review Comment:
Right, HBase reader does not check the checksum if the checksum type is NULL. However it still checks the checksum length.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(storage-format)!: Fix HFile writer to properly encode checksum for compatibility [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13897:
URL: https://github.com/apache/hudi/pull/13897#issuecomment-3313896254
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "98f44f95f3575ceb457769a5a2d640eaf6f63c36",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8549",
"triggerID" : "98f44f95f3575ceb457769a5a2d640eaf6f63c36",
"triggerType" : "PUSH"
}, {
"hash" : "9cd9e935e2f998b1425f74c3b6ac13eb419cfd0b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8645",
"triggerID" : "9cd9e935e2f998b1425f74c3b6ac13eb419cfd0b",
"triggerType" : "PUSH"
}, {
"hash" : "b4e138800e44033d622248034e7825d8eb6b7f77",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8647",
"triggerID" : "b4e138800e44033d622248034e7825d8eb6b7f77",
"triggerType" : "PUSH"
}, {
"hash" : "35e49f9dcf4f845042ecf2c2e5276b4a829ee71a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8658",
"triggerID" : "35e49f9dcf4f845042ecf2c2e5276b4a829ee71a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b4e138800e44033d622248034e7825d8eb6b7f77 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8647)
* 35e49f9dcf4f845042ecf2c2e5276b4a829ee71a Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8658)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(storage-format)!: Fix HFile writer to properly encode checksum for compatibility [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13897:
URL: https://github.com/apache/hudi/pull/13897#issuecomment-3314068759
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "98f44f95f3575ceb457769a5a2d640eaf6f63c36",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8549",
"triggerID" : "98f44f95f3575ceb457769a5a2d640eaf6f63c36",
"triggerType" : "PUSH"
}, {
"hash" : "9cd9e935e2f998b1425f74c3b6ac13eb419cfd0b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8645",
"triggerID" : "9cd9e935e2f998b1425f74c3b6ac13eb419cfd0b",
"triggerType" : "PUSH"
}, {
"hash" : "b4e138800e44033d622248034e7825d8eb6b7f77",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8647",
"triggerID" : "b4e138800e44033d622248034e7825d8eb6b7f77",
"triggerType" : "PUSH"
}, {
"hash" : "35e49f9dcf4f845042ecf2c2e5276b4a829ee71a",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8658",
"triggerID" : "35e49f9dcf4f845042ecf2c2e5276b4a829ee71a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 35e49f9dcf4f845042ecf2c2e5276b4a829ee71a Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8658)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(storage-format)!: Fix HFile writer to properly encode checksum for compatibility [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13897:
URL: https://github.com/apache/hudi/pull/13897#issuecomment-3310367697
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "98f44f95f3575ceb457769a5a2d640eaf6f63c36",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8549",
"triggerID" : "98f44f95f3575ceb457769a5a2d640eaf6f63c36",
"triggerType" : "PUSH"
}, {
"hash" : "9cd9e935e2f998b1425f74c3b6ac13eb419cfd0b",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8645",
"triggerID" : "9cd9e935e2f998b1425f74c3b6ac13eb419cfd0b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 98f44f95f3575ceb457769a5a2d640eaf6f63c36 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8549)
* 9cd9e935e2f998b1425f74c3b6ac13eb419cfd0b Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8645)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(storage-format)!: Fix HFile writer to properly encode checksum for compatibility [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13897:
URL: https://github.com/apache/hudi/pull/13897#issuecomment-3310469330
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "98f44f95f3575ceb457769a5a2d640eaf6f63c36",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8549",
"triggerID" : "98f44f95f3575ceb457769a5a2d640eaf6f63c36",
"triggerType" : "PUSH"
}, {
"hash" : "9cd9e935e2f998b1425f74c3b6ac13eb419cfd0b",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8645",
"triggerID" : "9cd9e935e2f998b1425f74c3b6ac13eb419cfd0b",
"triggerType" : "PUSH"
}, {
"hash" : "b4e138800e44033d622248034e7825d8eb6b7f77",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8647",
"triggerID" : "b4e138800e44033d622248034e7825d8eb6b7f77",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 98f44f95f3575ceb457769a5a2d640eaf6f63c36 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8549)
* 9cd9e935e2f998b1425f74c3b6ac13eb419cfd0b Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8645)
* b4e138800e44033d622248034e7825d8eb6b7f77 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8647)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(storage-format)!: Fix HFile writer to properly encode checksum for compatibility [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13897:
URL: https://github.com/apache/hudi/pull/13897#issuecomment-3310704765
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "98f44f95f3575ceb457769a5a2d640eaf6f63c36",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8549",
"triggerID" : "98f44f95f3575ceb457769a5a2d640eaf6f63c36",
"triggerType" : "PUSH"
}, {
"hash" : "9cd9e935e2f998b1425f74c3b6ac13eb419cfd0b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8645",
"triggerID" : "9cd9e935e2f998b1425f74c3b6ac13eb419cfd0b",
"triggerType" : "PUSH"
}, {
"hash" : "b4e138800e44033d622248034e7825d8eb6b7f77",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8647",
"triggerID" : "b4e138800e44033d622248034e7825d8eb6b7f77",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b4e138800e44033d622248034e7825d8eb6b7f77 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8647)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: Fix HFile reader compatibility [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13897:
URL: https://github.com/apache/hudi/pull/13897#issuecomment-3294621460
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "98f44f95f3575ceb457769a5a2d640eaf6f63c36",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "98f44f95f3575ceb457769a5a2d640eaf6f63c36",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 98f44f95f3575ceb457769a5a2d640eaf6f63c36 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(storage-format)!: Fix HFile writer to properly encode checksum for compatibility [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on code in PR #13897:
URL: https://github.com/apache/hudi/pull/13897#discussion_r2364416308
##########
hudi-io/src/main/java/org/apache/hudi/io/hfile/HFileBlock.java:
##########
@@ -290,15 +290,13 @@ public ByteBuffer serialize() throws IOException {
// onDiskSizeWithoutHeader = uncompressedSizeWithoutHeader + Checksum.
// In order to pass this check, either we make isUseHBaseChecksum false in HFileContext (hbase),
// or we set this value to zero.
- buf.putInt(0);
+ buf.putInt(DEFAULT_BYTES_PER_CHECKSUM);
Review Comment:
should we adjust the comment in L290 ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org