[jira] [Created] (PROTON-2736) TLS OpenSSL library: hang with large application data frames

["Clifford Jansen (Jira)" <ji...@apache.org>]

Clifford Jansen created PROTON-2736:
---------------------------------------

             Summary: TLS OpenSSL library: hang with large application data frames
                 Key: PROTON-2736
                 URL: https://issues.apache.org/jira/browse/PROTON-2736
             Project: Qpid Proton
          Issue Type: Bug
          Components: proton-c
    Affects Versions: proton-c-0.38.0
            Reporter: Clifford Jansen
            Assignee: Clifford Jansen


OpenSSL maintains a buffer large enough for the largest possible TLS protocol record + 1K.  The Proton TLS decrypt loop is unaware of record boundaries and repeatedly adds encrypted bytes at one end and takes out decrypted bytes at the other, stopping when there is no more to decrypt or no more application buffer space to move decrypted content into.

It also tests if there are remaining decrypted bytes available should the application provide additional buffers.  This test can fail in the case that the OpenSSL buffer is completely filled with:

 handshake record > 1K followed by
 partial max sized application data record

The SSL_peek operation will not see any application data and Proton "remembers" the full buffer without allowing that the handshake record has been processed and the buffer is no longer full.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org