You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by "jsduncan2 (via GitHub)" <gi...@apache.org> on 2023/08/28 20:34:43 UTC

[GitHub] [accumulo] jsduncan2 commented on issue #3731: TabletServerBatchReaderIterator gets into infinite loop

jsduncan2 commented on issue #3731:
URL: https://github.com/apache/accumulo/issues/3731#issuecomment-1696374252

   We are also seeing this with the BatchWriter. If one large mutation (> 90 * 1024 * 1024 bytes) or several mutations that add up to that same size or greater are sent, we enter an infinite loop. The BatchWriter client receives a “java.io.IOException: org.apache.thrift.transport.TTransportException: Socket is closed by peer.” here: https://github.com/apache/accumulo/blob/main/core/src/main/java/org/apache/accumulo/core/clientImpl/TabletServerBatchWriter.java#L897
   
   For what it’s worth, Thrift 0.14 seems to have added this max message size check (different from the max frame size check): https://issues.apache.org/jira/browse/THRIFT-5021
   
   A possible additional improvement could be to have the BatchWriter break its set of mutations into chunks that are smaller than Thrift’s max message size (while also respecting all other constraints).
   
   This doesn’t address the case where a single mutations exceeds the max message size limit. I’m not sure the best solution here. Possibly throwing a `MutationsRejectedException`?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org