You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@thrift.apache.org by "Arthur Meyer (Created) (JIRA)" <ji...@apache.org> on 2011/12/09 09:38:43 UTC
[jira] [Created] (THRIFT-1457) Capacity of TframedTransport write
buffer is never reset
Capacity of TframedTransport write buffer is never reset
--------------------------------------------------------
Key: THRIFT-1457
URL: https://issues.apache.org/jira/browse/THRIFT-1457
Project: Thrift
Issue Type: Improvement
Components: Java - Library
Affects Versions: 0.6.1
Reporter: Arthur Meyer
The writeBuffer_ instance variable of TframedTransport can only grow and is never reset to its original capacity. This causes memory issues for clients using a connection pool.
The size of this buffer grows to 16MB in our application. With a connection pool of 50 connections, this leads to a loss up to 800MB.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1457) Capacity of TframedTransport write
buffer is never reset
Posted by "Bryan Duxbury (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13168579#comment-13168579 ]
Bryan Duxbury commented on THRIFT-1457:
---------------------------------------
Because allocating write buffers takes time, there are a lot of situations where it's desirable for a buffer to never shrink just to avoid the possibility of having to reallocate it later. Your situation is the opposite, where you'd actually prefer to save the memory at the expense of time. I think for us to accept a patch for this issue, we would have to support both strategies.
> Capacity of TframedTransport write buffer is never reset
> --------------------------------------------------------
>
> Key: THRIFT-1457
> URL: https://issues.apache.org/jira/browse/THRIFT-1457
> Project: Thrift
> Issue Type: Improvement
> Components: Java - Library
> Affects Versions: 0.6.1
> Reporter: Arthur Meyer
> Attachments: THRIFT-1457.patch
>
>
> The writeBuffer_ instance variable of TframedTransport can only grow and is never reset to its original capacity. This causes memory issues for clients using a connection pool.
> The size of this buffer grows to 16MB in our application. With a connection pool of 50 connections, this leads to a loss up to 800MB.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (THRIFT-1457) Capacity of TframedTransport write
buffer is never reset
Posted by "Arthur Meyer (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arthur Meyer updated THRIFT-1457:
---------------------------------
Attachment: THRIFT-1457.patch
attached possible patch
> Capacity of TframedTransport write buffer is never reset
> --------------------------------------------------------
>
> Key: THRIFT-1457
> URL: https://issues.apache.org/jira/browse/THRIFT-1457
> Project: Thrift
> Issue Type: Improvement
> Components: Java - Library
> Affects Versions: 0.6.1
> Reporter: Arthur Meyer
> Attachments: THRIFT-1457.patch
>
>
> The writeBuffer_ instance variable of TframedTransport can only grow and is never reset to its original capacity. This causes memory issues for clients using a connection pool.
> The size of this buffer grows to 16MB in our application. With a connection pool of 50 connections, this leads to a loss up to 800MB.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1457) Capacity of TframedTransport write
buffer is never reset
Posted by "Bryan Duxbury (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13168729#comment-13168729 ]
Bryan Duxbury commented on THRIFT-1457:
---------------------------------------
You could just implement another framed transport that discards buffers instead of reuses them...
> Capacity of TframedTransport write buffer is never reset
> --------------------------------------------------------
>
> Key: THRIFT-1457
> URL: https://issues.apache.org/jira/browse/THRIFT-1457
> Project: Thrift
> Issue Type: Improvement
> Components: Java - Library
> Affects Versions: 0.6.1
> Reporter: Arthur Meyer
> Attachments: THRIFT-1457.patch
>
>
> The writeBuffer_ instance variable of TframedTransport can only grow and is never reset to its original capacity. This causes memory issues for clients using a connection pool.
> The size of this buffer grows to 16MB in our application. With a connection pool of 50 connections, this leads to a loss up to 800MB.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1457) Capacity of TframedTransport write
buffer is never reset
Posted by "Arthur Meyer (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13168711#comment-13168711 ]
Arthur Meyer commented on THRIFT-1457:
--------------------------------------
I understand your concerns. How do you propose to support both use strategies?
I could add an extra constructor with an extra parameter to specify the strategy to use.
Based on the parameter the buffer will be either reset or reused.
I'm not sure how to support both strategies in the code that uses TByteArrayOutputStreams.
Could you give me some pointers?
> Capacity of TframedTransport write buffer is never reset
> --------------------------------------------------------
>
> Key: THRIFT-1457
> URL: https://issues.apache.org/jira/browse/THRIFT-1457
> Project: Thrift
> Issue Type: Improvement
> Components: Java - Library
> Affects Versions: 0.6.1
> Reporter: Arthur Meyer
> Attachments: THRIFT-1457.patch
>
>
> The writeBuffer_ instance variable of TframedTransport can only grow and is never reset to its original capacity. This causes memory issues for clients using a connection pool.
> The size of this buffer grows to 16MB in our application. With a connection pool of 50 connections, this leads to a loss up to 800MB.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira