You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Siyao Meng (Jira)" <ji...@apache.org> on 2024/03/06 19:18:00 UTC
[jira] [Comment Edited] (HDDS-10471) [hsync] MockDatanodeStorage.writeChunk should make a copy of byte string
[ https://issues.apache.org/jira/browse/HDDS-10471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824135#comment-17824135 ]
Siyao Meng edited comment on HDDS-10471 at 3/6/24 7:17 PM:
-----------------------------------------------------------
On a related note, this bug can be repro'ed on a modified master branch just by not copying the byte[]:
https://github.com/smengcl/hadoop-ozone/commit/e2ca388e1fc2199c860023ddfc5d0624595d9ef8
CI:
https://github.com/smengcl/hadoop-ozone/actions/runs/8167296693/job/22327879409
The fact that this crashes JVM on Linux but not on macOS might indicate some differences in memory management / gc / platform-specific bug. Also this only crashes when using direct buffer. Non-direct buffer (before HDDS-9843 is merged) is fine.
was (Author: smeng):
On a related note, this bug can be repro'ed on a modified master branch just by not copying the byte[]:
https://github.com/smengcl/hadoop-ozone/commit/e2ca388e1fc2199c860023ddfc5d0624595d9ef8
CI:
https://github.com/smengcl/hadoop-ozone/actions/runs/8167296693/job/22327879409
The fact that this crashes JVM on Linux but not on macOS might indicate some differences in memory management / gc / platform-specific bug. Also this only crashes with using direct buffer. Non-direct buffer (before HDDS-9843 is merged) is fine.
> [hsync] MockDatanodeStorage.writeChunk should make a copy of byte string
> ------------------------------------------------------------------------
>
> Key: HDDS-10471
> URL: https://issues.apache.org/jira/browse/HDDS-10471
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Wei-Chiu Chuang
> Assignee: Wei-Chiu Chuang
> Priority: Major
> Labels: pull-request-available
> Fix For: HDDS-7593
>
>
> MockDatanodeStorage.writeChunk simply adds the bytestring chunk info into its internal map, and after HDDS-9843 which made chunk buffer direct buffer, it causes some sort of memory corruption as described in HDDS-10431. Suspect it's a JVM bug.
> Solution is simple: make a copy of the byte buffer.
> Thanks [~smeng] for narrowing down the culprit.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org