You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Bharat Viswanadham (Jira)" <ji...@apache.org> on 2022/01/07 21:52:00 UTC
[jira] [Assigned] (HDDS-3032) Cannot write 32MB chunks
[ https://issues.apache.org/jira/browse/HDDS-3032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bharat Viswanadham reassigned HDDS-3032:
----------------------------------------
Assignee: (was: Bharat Viswanadham)
> Cannot write 32MB chunks
> ------------------------
>
> Key: HDDS-3032
> URL: https://issues.apache.org/jira/browse/HDDS-3032
> Project: Apache Ozone
> Issue Type: Bug
> Components: Ozone Datanode
> Reporter: Attila Doroszlai
> Priority: Major
> Labels: TriagePending
>
> Writing 32MB chunks fails with various errors.
> {code:title=steps to reproduce}
> ozone freon dcg -t 1 -n 1 -s 33554432
> {code}
> 1. With Ratis 0.5.0-90cd474-SNAPSHOT (used by current Ozone master):
> {code}
> org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException: RESOURCE_EXHAUSTED: gRPC message exceeds maximum size 33554432: 33554686
> at org.apache.ratis.thirdparty.io.grpc.Status.asRuntimeException(Status.java:524)
> at org.apache.ratis.thirdparty.io.grpc.internal.MessageDeframer.processHeader(MessageDeframer.java:387)
> {code}
> Which is strange, because [Datanode attempts to set max. message size|https://github.com/apache/hadoop-ozone/blob/4ba1932dab4692a9cc1bcfb8903ef650e32ec7ba/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/XceiverServerRatis.java#L204-L206] to 32MB + 16KB.
> 2. With Ratis built locally from current Ratis master (46f255cb):
> {code}
> Caused by: org.apache.ratis.protocol.StateMachineException: org.apache.ratis.server.raftlog.RaftLogIOException from Server ccb25fbf-9bd1-4094-a632-00f4168213bb@group-B1FA90A78F31: Log entry size 33554666 exceeds the max buffer limit of 33554432
> at org.apache.ratis.server.raftlog.RaftLog.appendImpl(RaftLog.java:178)
> at org.apache.ratis.server.raftlog.RaftLog.lambda$append$2(RaftLog.java:157)
> at org.apache.ratis.server.raftlog.RaftLogSequentialOps$Runner.runSequentially(RaftLogSequentialOps.java:68)
> at org.apache.ratis.server.raftlog.RaftLog.append(RaftLog.java:157)
> at org.apache.ratis.server.impl.ServerState.appendLog(ServerState.java:282)
> at org.apache.ratis.server.impl.RaftServerImpl.appendTransaction(RaftServerImpl.java:518)
> at org.apache.ratis.server.impl.RaftServerImpl.submitClientRequestAsync(RaftServerImpl.java:604)
> {code}
> With {{ozone.scm.chunk.size=32MB}} setting, {{ozone freon ockg -n 1 -t 1 -s 33554432}} also fails, but without apparent errors in the datanode log.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org