Java sizeof!

[Ryan Rawson <ry...@gmail.com>]

Yes such a thing (kind of) exists:

http://java.sun.com/javase/6/docs/api/java/lang/instrument/package-summary.html

You merely have to create a java agent jar which then gets access to this
magical interface.  Here is a sourceforge project:

http://sourceforge.net/projects/sizeof

My measurements on java 1.6 64 bits, OS X:
- empty object = 16 bytes
- array of bytes: byte size + 24 bytes
- ByteBuffer = 56 bytes (storage separately)
- object, 1 int = 24 bytes
- object, 2 ints = 24 bytes
- object, 1 long = 24 bytes

It seems pretty obvious that java -d64 is using dword/qword (32/64 bit)
alignment.  Objects seem to be 64 bit aligned, but fields are not
necessairly.  This is from the 1 int vs 2 int = 24 byte test.

This aligns with the results of Erik's HeapSize values.

I dont know how much overhead a java agent has like this, so it might not be
worthwhile to use except as validation/verification of our HeapSize
assumptions.

With the use of ByteBuffer and 0-copy we can extract good memory
performance, well, except the ByteBuffer instance itself is not exactly
super light weight (56 bytes?) for what is essentially a few pointers (plus
control bytes, plus blah blah).  So keeping the number of extraneous
ByteBuffer instances low will help.

-ryan

Re: Java sizeof!

[Erik Holstad <er...@gmail.com>]

Hi Ryan!
Yeah that looks like the numbers that I've gotten too except for the
ByteBuffer
where I got 48 instead of 56

    hb
        class [B
    offset
        int
    isReadOnly
        boolean
    bigEndian
        boolean
    nativeByteOrder
        boolean
    mark
        int
    position
        int
    limit
        int
    capacity
        int
    address
        long

c[0] 31, c[1] 3, refSize 4, size 43
usage from ClassSize, BB 48

But maybe I'm missing something.

Like you said, not really sure if we can run this with HBase or just as an
extra tool
to verify the cost and size of all the objects that we are using, so we have
different
ways of figuring it out. I will put up my current test code as soon as I
cleaned it up
so if you could do that too, when you have a chance that would be cool.

Erik

Re: Java sizeof!

[stack <st...@duboce.net>]

Good find Ryan.

How about we write a little program to generate a java interface file that
has in it the SIZEOF all the types we are interested in?  Anyone signing up
for the heapSize contract could make use of the defines?  Are sizes very
different when 32vs64bit?  If so, should we generate two interfaces and on
startup toggle which to use sizing?

St.Ack

On Thu, Mar 5, 2009 at 1:55 AM, Ryan Rawson <ry...@gmail.com> wrote:

> Yes such a thing (kind of) exists:
>
>
> http://java.sun.com/javase/6/docs/api/java/lang/instrument/package-summary.html
>
> You merely have to create a java agent jar which then gets access to this
> magical interface.  Here is a sourceforge project:
>
> http://sourceforge.net/projects/sizeof
>
> My measurements on java 1.6 64 bits, OS X:
> - empty object = 16 bytes
> - array of bytes: byte size + 24 bytes
> - ByteBuffer = 56 bytes (storage separately)
> - object, 1 int = 24 bytes
> - object, 2 ints = 24 bytes
> - object, 1 long = 24 bytes
>
> It seems pretty obvious that java -d64 is using dword/qword (32/64 bit)
> alignment.  Objects seem to be 64 bit aligned, but fields are not
> necessairly.  This is from the 1 int vs 2 int = 24 byte test.
>
> This aligns with the results of Erik's HeapSize values.
>
> I dont know how much overhead a java agent has like this, so it might not
> be
> worthwhile to use except as validation/verification of our HeapSize
> assumptions.
>
> With the use of ByteBuffer and 0-copy we can extract good memory
> performance, well, except the ByteBuffer instance itself is not exactly
> super light weight (56 bytes?) for what is essentially a few pointers (plus
> control bytes, plus blah blah).  So keeping the number of extraneous
> ByteBuffer instances low will help.
>
> -ryan
>