You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Stephen O'Donnell (Jira)" <ji...@apache.org> on 2022/02/03 13:03:00 UTC
[jira] [Created] (HDDS-6256) EC: Parity blocks are padded incorrectly with zeros to the chunk size
Stephen O'Donnell created HDDS-6256:
---------------------------------------
Summary: EC: Parity blocks are padded incorrectly with zeros to the chunk size
Key: HDDS-6256
URL: https://issues.apache.org/jira/browse/HDDS-6256
Project: Apache Ozone
Issue Type: Sub-task
Reporter: Stephen O'Donnell
I write a tiny key into a 3-2 bucket. Container group is still open - this looks good 3 containers as expected:
Replicas: [State: OPEN; ReplicaIndex: 4; Origin: 3fc2c737-83b1-4485-91ac-b771e41fa1dd; Location: 3fc2c737-83b1-4485-91ac-b771e41fa1dd/ozone_datanode_1.ozone_default,
State: OPEN; ReplicaIndex: 5; Origin: 1be82529-d5f2-4c6b-8573-0bb75863ded1; Location: 1be82529-d5f2-4c6b-8573-0bb75863ded1/ozone_datanode_4.ozone_default,
State: OPEN; ReplicaIndex: 1; Origin: e1d6ca08-d5eb-4e8a-b42a-bf1257e9f673; Location: e1d6ca08-d5eb-4e8a-b42a-bf1257e9f673/ozone_datanode_2.ozone_default]
In the JSON:
"bytesUsed" : 177,
"replicaIndex" : 1
...
"bytesUsed" : 1048576,
"replicaIndex" : 5
...
"bytesUsed" : 1048576,
"replicaIndex" : 4
So the parity should only have 177 bytes too, but it appears to have 1MB. Checking the block file on disk, it is indeed 1MB:
bash-4.2$ cd /data/hdds/hdds/CID-137befd9-f6ec-430b-b0e0-ee683ac1d987/current/containerDir0/3/
bash-4.2$ ls
chunks metadata
bash-4.2$ cd chunks/
bash-4.2$ ls
109611004723200004.block
bash-4.2$ ls -al
total 1032
drwxr-xr-x 2 hadoop users 4096 Feb 2 22:08 .
drwxr-xr-x 4 hadoop users 4096 Feb 2 22:08 ..
-rw-r--r-- 1 hadoop users 1048576 Feb 2 22:08 109611004723200004.block
So something is not correct with the write path here. Checking the block contents, its all padded with zeros:
bash-4.2$ hexdump -C 109611004723200004.block
00000000 e4 e5 e6 1a 10 1a 10 1a e4 07 24 25 21 d4 24 d3 |..........$%!.$.|
00000010 25 da 2c 06 16 16 e4 07 24 25 21 d4 24 d3 25 da |%.,.....$%!.$.%.|
00000020 2c eb 27 db 12 1b 24 25 21 d4 24 d3 25 da 2c eb |,.'...$%!.$.%.,.|
00000030 27 db 12 1b 24 25 25 db d5 d4 21 d2 06 22 23 10 |'...$%%...!.."#.|
00000040 10 16 16 10 07 27 db 12 1b 24 25 21 d4 24 d1 23 |.....'...$%!.$.#|
00000050 2c 06 22 22 10 10 16 16 10 07 27 db 12 1b d0 21 |,.""......'....!|
00000060 d4 da 2c db 2e 23 22 27 28 06 22 22 10 e5 16 16 |..,..#"'(.""....|
00000070 e4 07 27 db 12 1b d4 24 24 d1 25 d7 23 da 06 22 |..'....$$.%.#.."|
00000080 22 10 e5 16 16 e5 07 27 db 12 1b d4 24 24 2e 25 |"......'....$$.%|
00000090 d8 2c 23 2e da 06 e4 17 e5 1a e4 12 e3 1a e4 12 |.,#.............|
000000a0 10 1a e5 07 e6 e4 e7 11 d5 13 13 10 10 e3 d7 e3 |................|
000000b0 06 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00100000
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org