[jira] [Created] (HDDS-16874) FilePerChunk overwrite check misses chunks after the first in a block

["Chi-Hsuan Huang (Jira)" <ji...@apache.org>]

Chi-Hsuan Huang created HDDS-16874:
--------------------------------------

             Summary: FilePerChunk overwrite check misses chunks after the first in a block
                 Key: HDDS-16874
                 URL: https://issues.apache.org/jira/browse/HDDS-16874
             Project: Apache Ozone
          Issue Type: Bug
          Components: Ozone Datanode
            Reporter: Chi-Hsuan Huang


{{FilePerChunkStrategy.writeChunk}} decides whether a write is an overwrite with {{info.getOffset\(\) < chunkFile.length\(\)}}. Since HDDS\-2717 the chunk offset is relative to the block, but in the FILE\_PER\_CHUNK layout each chunk has its own file. For any chunk after the first, the offset is at least one chunk length, so a rewrite of an existing chunk is not detected.

I found two effects with unit tests on {{TestFilePerChunkStrategy}} \(second chunk, offset = len\):h1. A COMBINED retry counts the chunk twice in container {{bytesUsed}} \(expected 20, actual 40\).
h1. A repeated COMMIT\_DATA, when the chunk file exists and the tmp file is gone, fails with {{NoSuchFileException}}.
The first chunk \(offset 0\) behaves correctly in both cases.

On datanode restart, {{ContainerStateMachine.applyTransaction}} replays WriteChunk entries after the last snapshot as COMMIT\_DATA, and {{HddsDispatcher}} does not skip already applied entries. So effect 2 could fail the apply and trigger a pipeline close. I have not reproduced this end to end.

_Expected behavior_
A rewrite of any existing chunk in a FILE\_PER\_CHUNK container should be treated as an overwrite.

_Notes_
FILE\_PER\_CHUNK is deprecated for new containers \(HDDS\-11753\), so this only affects existing containers in that layout. Found while reviewing HDDS\-16844, which keeps the current behavior.





--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org