[jira] [Commented] (THRIFT-5108) Memory allocation error in TBufferTransports

["Mario Emmenlauer (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/THRIFT-5108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043077#comment-17043077 ] 

Mario Emmenlauer commented on THRIFT-5108:
------------------------------------------

This is actually not a bug as I first thought. The code in the relevant section could be more readable and I will submit a PR soon. However the actual problem is that I do not understand how the transport is meant to be used.

I have implemented my own C++ thrift server on top of boost::beast websockets. I use a {{TMemoryBuffer}} as the output buffer of the {{process()}} method, then {{borrow()}} the pointer to send the data through the websocket and finally I {{consume()}} the data.

But the buffer becomes fuller and fuller until it reaches 2GB and throws. Does {{consume()}} not free the write-buffer for subsequent writes? How should one "clear" the buffer for the next write? I could destroy and re-create the Transport and the Protocol after every {{process()}}, but that must be inefficient?

> Memory allocation error in TBufferTransports
> --------------------------------------------
>
>                 Key: THRIFT-5108
>                 URL: https://issues.apache.org/jira/browse/THRIFT-5108
>             Project: Thrift
>          Issue Type: Improvement
>          Components: C++ - Library
>    Affects Versions: 0.13.0
>            Reporter: Mario Emmenlauer
>            Assignee: Mario Emmenlauer
>            Priority: Major
>
> When using the memory buffered transport I randomly get {{TTransportException}} in {{TBufferTransports.cpp}} method
>  
>  {{void TMemoryBuffer::ensureCanWrite(uint32_t len) {}}
>  {{    ...}}
>  {{    // Grow the buffer as necessary.}}
> {{    uint64_t new_size = bufferSize_;}}
> {{    while (len > avail) {}}
> {{        new_size = new_size > 0 ? new_size * 2 : 1;}}
> {{        if (new_size > maxBufferSize_) {}}
> {{            throwTTransportException(TTransportException::BAD_ARGS,}}
> {{                "Internal buffer size overflow when requesting a buffer of size " + std::to_string(new_size));}}
> {{        }}}
> {{        avail = available_write() + (static_cast<uint32_t>(new_size) - bufferSize_);}}
>         }



--
This message was sent by Atlassian Jira
(v8.3.4#803005)