You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Ethan Rose (Jira)" <ji...@apache.org> on 2021/10/20 20:35:11 UTC
[jira] [Updated] (HDDS-3032) Cannot write 32MB chunks
[ https://issues.apache.org/jira/browse/HDDS-3032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ethan Rose updated HDDS-3032:
-----------------------------
Target Version/s: 1.3.0 (was: 1.2.0)
I am managing the 1.2.0 release and we currently have more than 600 issues targeted for 1.2.0. I am moving the target field to 1.3.0.
If you are actively working on this jira and believe this should be targeted for the 1.2.0 release, Please reach out to me via Apache email or Slack.
> Cannot write 32MB chunks
> ------------------------
>
> Key: HDDS-3032
> URL: https://issues.apache.org/jira/browse/HDDS-3032
> Project: Apache Ozone
> Issue Type: Bug
> Components: Ozone Datanode
> Reporter: Attila Doroszlai
> Assignee: Bharat Viswanadham
> Priority: Major
> Labels: TriagePending
>
> Writing 32MB chunks fails with various errors.
> {code:title=steps to reproduce}
> ozone freon dcg -t 1 -n 1 -s 33554432
> {code}
> 1. With Ratis 0.5.0-90cd474-SNAPSHOT (used by current Ozone master):
> {code}
> org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException: RESOURCE_EXHAUSTED: gRPC message exceeds maximum size 33554432: 33554686
> at org.apache.ratis.thirdparty.io.grpc.Status.asRuntimeException(Status.java:524)
> at org.apache.ratis.thirdparty.io.grpc.internal.MessageDeframer.processHeader(MessageDeframer.java:387)
> {code}
> Which is strange, because [Datanode attempts to set max. message size|https://github.com/apache/hadoop-ozone/blob/4ba1932dab4692a9cc1bcfb8903ef650e32ec7ba/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/XceiverServerRatis.java#L204-L206] to 32MB + 16KB.
> 2. With Ratis built locally from current Ratis master (46f255cb):
> {code}
> Caused by: org.apache.ratis.protocol.StateMachineException: org.apache.ratis.server.raftlog.RaftLogIOException from Server ccb25fbf-9bd1-4094-a632-00f4168213bb@group-B1FA90A78F31: Log entry size 33554666 exceeds the max buffer limit of 33554432
> at org.apache.ratis.server.raftlog.RaftLog.appendImpl(RaftLog.java:178)
> at org.apache.ratis.server.raftlog.RaftLog.lambda$append$2(RaftLog.java:157)
> at org.apache.ratis.server.raftlog.RaftLogSequentialOps$Runner.runSequentially(RaftLogSequentialOps.java:68)
> at org.apache.ratis.server.raftlog.RaftLog.append(RaftLog.java:157)
> at org.apache.ratis.server.impl.ServerState.appendLog(ServerState.java:282)
> at org.apache.ratis.server.impl.RaftServerImpl.appendTransaction(RaftServerImpl.java:518)
> at org.apache.ratis.server.impl.RaftServerImpl.submitClientRequestAsync(RaftServerImpl.java:604)
> {code}
> With {{ozone.scm.chunk.size=32MB}} setting, {{ozone freon ockg -n 1 -t 1 -s 33554432}} also fails, but without apparent errors in the datanode log.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org