You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Sumit Agrawal (Jira)" <ji...@apache.org> on 2023/08/16 14:51:00 UTC
[jira] [Commented] (HDDS-8977) Ratis crash if a lot of directories deleted at once
[ https://issues.apache.org/jira/browse/HDDS-8977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17755109#comment-17755109 ]
Sumit Agrawal commented on HDDS-8977:
-------------------------------------
Hi [~izlenko] ,
Can you please share more details, like count of directory and files as might be present in log,
> Ratis crash if a lot of directories deleted at once
> ---------------------------------------------------
>
> Key: HDDS-8977
> URL: https://issues.apache.org/jira/browse/HDDS-8977
> Project: Apache Ozone
> Issue Type: Bug
> Reporter: Ivan Zlenko
> Assignee: Sumit Agrawal
> Priority: Major
>
> DirectoryDeletingService periodically scans for deleted directories and sends a message containing all deleted directories.
> There is no checks for this message before it will be sent. So if we delete enough directory we can force Ratis to crash because message with list of directories would be to large to be handled by Ratis:
> {code:java}
> 2023-06-30 16:49:25,822 [DirectoryDeletingService#0] ERROR org.apache.hadoop.ozone.om.ratis.OzoneManagerRatisServer: Unknown error occurs org.apache.ratis.server.raftlog.RaftLogIOException: Log entry size 41464041 exceeds the max buffer limit of 33554432 at org.apache.ratis.server.raftlog.RaftLogBase.appendImpl(RaftLogBase.java:184) at org.apache.ratis.server.raftlog.RaftLogBase.lambda$append$2(RaftLogBase.java:161) at org.apache.ratis.server.raftlog.RaftLogSequentialOps$Runner.runSequentially(RaftLogSequentialOps.java:69) at org.apache.ratis.server.raftlog.RaftLogBase.append(RaftLogBase.java:161) at org.apache.ratis.server.impl.ServerState.appendLog(ServerState.java:366) at org.apache.ratis.server.impl.RaftServerImpl.appendTransaction(RaftServerImpl.java:770) at org.apache.ratis.server.impl.RaftServerImpl.submitClientRequestAsync(RaftServerImpl.java:878) at org.apache.ratis.server.impl.RaftServerImpl.lambda$null$12(RaftServerImpl.java:815) at org.apache.ratis.util.JavaUtils.callAsUnchecked(JavaUtils.java:117) at org.apache.ratis.server.impl.RaftServerImpl.lambda$executeSubmitClientRequestAsync$13(RaftServerImpl.java:815) at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750){code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org