You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Ethan Rose (Jira)" <ji...@apache.org> on 2022/01/27 23:26:00 UTC
[jira] [Updated] (HDDS-6235) Empty KeyValueContainers are replicated without chunks directory
[ https://issues.apache.org/jira/browse/HDDS-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ethan Rose updated HDDS-6235:
-----------------------------
Summary: Empty KeyValueContainers are replicated without chunks directory (was: Empty KeyValueContainers cannot be replicated)
> Empty KeyValueContainers are replicated without chunks directory
> ----------------------------------------------------------------
>
> Key: HDDS-6235
> URL: https://issues.apache.org/jira/browse/HDDS-6235
> Project: Apache Ozone
> Issue Type: Bug
> Components: Ozone Datanode
> Affects Versions: 1.0.0, 1.1.0, 1.2.0
> Reporter: Ethan Rose
> Assignee: Ethan Rose
> Priority: Major
>
> An empty KeyValueContainer will have an empty chunks directory. TarContainerPacker#pack recurses into directories adding files into containers, but if the chunks directory is empty, it will not be included in the tar. The receiver will unpack the tar successfully, but the resulting container will not have a chunks directory. After this, the container will not be able to replicated further, as the tar packing step requires all container pieces to be present on disk. This issue is more likely to occur due to HDDS-5359, which causes many empty containers to be tracked by SCM indefinitely.
> Since the issue only affects empty containers, there does not appear to be any data loss risk, even though the container scanner may detect it as "corruption". The issue may manifest as a container continuously attempting to be replicated and failing, or the container being marked unhealthy by the background container scanner (if it is enabled).
> This Jira will fix the issue with the tar packer, and also add a repair step on datanode startup to create the chunks directory for containers that do not have one. This step should be a quick addition to datanode startup that already iterates all the containers, and should not impact startup time.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org