You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@thrift.apache.org by Brian Bloniarz <bb...@trulia.com> on 2011/12/17 01:47:55 UTC
RFC patch: Java isset space optimization
Hi, I have a small patch for Java to optimize the heapspace footprint
of (for example):
struct OptIntPair {
1: i32 a
2: i32 b
}
Each instance will have an __isset_bit_vector which points to a BitSet,
which points to a long[], adding around 50 bytes of overhead to this object.
The patch changes this by storing a raw byte and doing direct bitfield operations, like:
byte __isset_bitfield;
public void unsetB() {
__isset_bitfield = EncodingUtils.clearBit(__isset_bitfield, __B_ISSET_ID);
}
A little nasty, but a big space win: on my machine, this brings down the total
size of an OptIntPair from 85 bytes to 25 bytes. A BitSet gets used as a fallback
when more than 64 __isset entries are needed.
Is this something that could be considered? The patch is attached. Other people have
mentioned this shortcoming before, see slide 22 of:
http://www.slideshare.net/aszegedi/everything-i-ever-learned-about-jvm-performance-tuning-twitter
Thanks for such useful software,
Brian Bloniarz
Re: RFC patch: Java isset space optimization
Posted by Brian Bloniarz <bb...@trulia.com>.
Cool, thanks for the help Bryan and Jake. I added unit tests
and created: https://issues.apache.org/jira/browse/THRIFT-1469
Thanks,
-Brian
On 12/17/2011 09:04 AM, Bryan Duxbury wrote:
> This patch is pretty cool. I think I also would like to see it in a ticket,
> and I'd like to make sure we have test cases that cover all of this new
> variety of possible implementations.
>
> -Bryan
>
> On Fri, Dec 16, 2011 at 4:47 PM, Brian Bloniarz <bb...@trulia.com>wrote:
>
>> Hi, I have a small patch for Java to optimize the heapspace footprint
>> of (for example):
>>
>> struct OptIntPair {
>> 1: i32 a
>> 2: i32 b
>> }
>>
>> Each instance will have an __isset_bit_vector which points to a BitSet,
>> which points to a long[], adding around 50 bytes of overhead to this
>> object.
>>
>> The patch changes this by storing a raw byte and doing direct bitfield
>> operations, like:
>> byte __isset_bitfield;
>>
>> public void unsetB() {
>> __isset_bitfield = EncodingUtils.clearBit(__isset_bitfield,
>> __B_ISSET_ID);
>> }
>> A little nasty, but a big space win: on my machine, this brings down the
>> total
>> size of an OptIntPair from 85 bytes to 25 bytes. A BitSet gets used as a
>> fallback
>> when more than 64 __isset entries are needed.
>>
>> Is this something that could be considered? The patch is attached. Other
>> people have
>> mentioned this shortcoming before, see slide 22 of:
>>
>> http://www.slideshare.net/aszegedi/everything-i-ever-learned-about-jvm-performance-tuning-twitter
>>
>> Thanks for such useful software,
>> Brian Bloniarz
>>
>
Re: RFC patch: Java isset space optimization
Posted by Bryan Duxbury <br...@rapleaf.com>.
This patch is pretty cool. I think I also would like to see it in a ticket,
and I'd like to make sure we have test cases that cover all of this new
variety of possible implementations.
-Bryan
On Fri, Dec 16, 2011 at 4:47 PM, Brian Bloniarz <bb...@trulia.com>wrote:
> Hi, I have a small patch for Java to optimize the heapspace footprint
> of (for example):
>
> struct OptIntPair {
> 1: i32 a
> 2: i32 b
> }
>
> Each instance will have an __isset_bit_vector which points to a BitSet,
> which points to a long[], adding around 50 bytes of overhead to this
> object.
>
> The patch changes this by storing a raw byte and doing direct bitfield
> operations, like:
> byte __isset_bitfield;
>
> public void unsetB() {
> __isset_bitfield = EncodingUtils.clearBit(__isset_bitfield,
> __B_ISSET_ID);
> }
> A little nasty, but a big space win: on my machine, this brings down the
> total
> size of an OptIntPair from 85 bytes to 25 bytes. A BitSet gets used as a
> fallback
> when more than 64 __isset entries are needed.
>
> Is this something that could be considered? The patch is attached. Other
> people have
> mentioned this shortcoming before, see slide 22 of:
>
> http://www.slideshare.net/aszegedi/everything-i-ever-learned-about-jvm-performance-tuning-twitter
>
> Thanks for such useful software,
> Brian Bloniarz
>
Re: RFC patch: Java isset space optimization
Posted by Jake Farrell <JF...@onesite.com>.
Brian
if you would like to submit a patch for inclusion please see http://wiki.apache.org/thrift/HowToContribute
- Jake
On Dec 16, 2011, at 6:47 PM, Brian Bloniarz wrote:
> Hi, I have a small patch for Java to optimize the heapspace footprint
> of (for example):
>
> struct OptIntPair {
> 1: i32 a
> 2: i32 b
> }
>
> Each instance will have an __isset_bit_vector which points to a BitSet,
> which points to a long[], adding around 50 bytes of overhead to this object.
>
> The patch changes this by storing a raw byte and doing direct bitfield operations, like:
> byte __isset_bitfield;
>
> public void unsetB() {
> __isset_bitfield = EncodingUtils.clearBit(__isset_bitfield, __B_ISSET_ID);
> }
> A little nasty, but a big space win: on my machine, this brings down the total
> size of an OptIntPair from 85 bytes to 25 bytes. A BitSet gets used as a fallback
> when more than 64 __isset entries are needed.
>
> Is this something that could be considered? The patch is attached. Other people have
> mentioned this shortcoming before, see slide 22 of:
> http://www.slideshare.net/aszegedi/everything-i-ever-learned-about-jvm-performance-tuning-twitter
>
> Thanks for such useful software,
> Brian Bloniarz
> <patch.isset_bitfield>