[jira] [Created] (HDDS-6203) CleanUp gz files failed to be fully written during Container move

["Xu Shao Hong (Jira)" <ji...@apache.org>]

Xu Shao Hong created HDDS-6203:
----------------------------------

             Summary: CleanUp gz files failed to be fully written during Container move
                 Key: HDDS-6203
                 URL: https://issues.apache.org/jira/browse/HDDS-6203
             Project: Apache Ozone
          Issue Type: Bug
            Reporter: Xu Shao Hong
            Assignee: Xu Shao Hong


Right now, the container re-replication will be sent with GRPC as gz files to the temporary dir. If the temporary dir is small, there will be a contest for space left that concurrent threads downloading the containers will compete to write the downloaded byte buffer to the actual files with FileOutputStream. 

Once the thread fails to write the buffer to file, the current logic will not clean up the failed and incomplete file and just complete exceptionally as the code shows.

 
{code:java}
GrpcReplicationClient

@Override
public void onNext(CopyContainerResponseProto chunk) {
  try {
    chunk.getData().writeTo(stream);
  } catch (IOException e) {
    response.completeExceptionally(e);
  }
} {code}
the exception will be caught at ```getContainerDataFromReplicas``` and only will be logged as an error.

Thus it is necessary to clean up the possible incomplete files which failed in such case.

(I have tested manually to mimic such a contest case and proved the incomplete files remained, the example could be seen in attachment)

 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org