[jira] [Resolved] (THRIFT-6434) Release exhausted input buffers in Ruby transports

["Dmytro Shteflyuk (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/THRIFT-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dmytro Shteflyuk resolved THRIFT-6434.
--------------------------------------
    Fix Version/s: 0.26.0
       Resolution: Fixed

> Release exhausted input buffers in Ruby transports
> --------------------------------------------------
>
>                 Key: THRIFT-6434
>                 URL: https://issues.apache.org/jira/browse/THRIFT-6434
>             Project: Thrift
>          Issue Type: Bug
>          Components: Ruby - Library
>            Reporter: Dmytro Shteflyuk
>            Assignee: Dmytro Shteflyuk
>            Priority: Major
>             Fix For: 0.26.0
>
>          Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> h3. Problem
> Ruby's BufferedTransport, FramedTransport, HeaderTransport and HTTPClientTransport keep references to consumed input after the caller has finished reading it. Reads advance a cursor, but the backing String remains owned until a later refill or response replaces it. FramedTransport and HeaderTransport also leave that storage referenced after close.
> h3. Client impact
> An idle, long-lived transport can retain its previous large frame or HTTP response even after the application has dropped the returned data. BufferedTransport retains its last read buffer, which can also be large when the caller requests a large read. This is retention of the last buffer, rather than accumulation of every previous response.
> h3. Reproduction
> Save the following as repro.rb and run it from the repository root with:
> {code:bash}
> ruby -Ilib/rb/lib repro.rb
> {code}
> The example selects the pure Ruby implementation so it does not require a compiled extension.
> {code:ruby}
> $LOADED_FEATURES << 'thrift_native.rb'
> require 'thrift'
> def retained_input_bytes(reader)
>   if reader.is_a?(Thrift::FramedTransport)
>     reader.instance_variable_get(:@rbuf).bytesize
>   else
>     reader.instance_variable_get(:@read_buffer).string.bytesize
>   end
> end
> size = 8 * 1024 * 1024
> [Thrift::FramedTransport, Thrift::HeaderTransport].each do |klass|
>   reader = Thread.new do
>     wire = Thrift::MemoryBufferTransport.new
>     writer = klass.new(wire)
>     writer.write('x'.b * size)
>     writer.flush
>     transport = klass.new(wire)
>     (size / 16_384).times do
>       raise 'incorrect data' unless transport.read_all(16_384) == 'x'.b * 16_384
>     end
>     transport
>   end.value
>   GC.start
>   puts "#{klass}: consumed=#{size}, retained_input=#{retained_input_bytes(reader)}"
>   reader.close
>   puts "#{klass}: retained_after_close=#{retained_input_bytes(reader)}"
> end
> {code}
> Testing on master commit {{926dcc971c97aa935f00bc98848352e905c4c755}} produces:
> {code}
> Thrift::FramedTransport: consumed=8388608, retained_input=8388608
> Thrift::FramedTransport: retained_after_close=8388608
> Thrift::HeaderTransport: consumed=8388608, retained_input=8388608
> Thrift::HeaderTransport: retained_after_close=8388608
> {code}
> All payload bytes are verified before the ownership check. The producer and consumer stack has terminated, and the transport remains alive during GC.
> h3. Expected behavior
> Release transport-owned input storage once it is fully consumed, without waiting for another request or response. Preserve unread and pipelined bytes, returned data, Header reply metadata, and existing EOF/error behavior. Closing a transport or encountering an error must not discard input that remains readable under the existing contract.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)