You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@directmemory.apache.org by Jaromir Hamala <ja...@gmail.com> on 2013/02/05 17:54:16 UTC
Benchmarks available?
HI guys,
are there benchmarks for DirectMemory available? How well it performs with
10's GB of memory. Are there production deployments of DM? From what I can
see it holds key - Pointer(-Impl) map on a Java heap, isn't this a major
limitation in terms of scalability?
Thank for very much!
Oh, by the way: Thanks for your effort on this project, it looks very
interesting!
--
“Perfection is achieved, not when there is nothing more to add, but when
there is nothing left to take away.”
Antoine de Saint Exupéry
Re: Benchmarks available?
Posted by "Raffaele P. Guidi" <ra...@gmail.com>.
That would be great (both things)
Il giorno 07/feb/2013 11:21, "Jaromir Hamala" <ja...@gmail.com> ha
scritto:
> Hi Raffaele,
>
> thanks for the reply. Having a fresh benchmark would be good. It could
> serve for two purposes:
> -numbers to brag about! :)
> -regression test ensuring that new features / fixes do not decrease the
> performance
>
> I can try to design something during the upcoming weekend. I currently do
> not have an easy access to a machine with more than 16 GB RAM, but I can
> always start a High Memory EC2 instance.
>
> Regarding the keys in a heap: Currently there is Key-Pointer<K> pair being
> held in a memory. The Pointer itself is quite heavy. In theory there could
> be a "PointerLight" for some less-hot-data, but I understand it's not a
> priority as of now.
>
> Anyway, I have to say I really like to idea of the project - I would like
> to contribute. I was thinking I could take look at DIRECTMEMORY-31
> or DIRECTMEMORY-98 and send a patch.
>
> Cheers,
> Jaromir
>
>
> On Thu, Feb 7, 2013 at 7:39 AM, Raffaele P. Guidi <
> raffaele.p.guidi@gmail.com> wrote:
>
>> Well last benchmarks are a bit (quite) outdated and can be found on my
>> blog - actually they were performed on a totally different code base. This
>> is probably the most relevant form of contribution that we need, should you
>> be interested, because they are hard to do (I don't have 10gb, in example).
>>
>> No production deployments reported, yet (you could be first! ;)
>>
>> Yes keys are in a heap based map (this cannot be avoided) that's why dm's
>> usage is more suitable when payloads are significantly larger then keys. We
>> use efficient maps (from guava), in any case.
>>
>> Thanks for your interest.
>>
>> Ciao,
>> R
>> Il giorno 05/feb/2013 17:55, "Jaromir Hamala" <ja...@gmail.com>
>> ha scritto:
>>
>> HI guys,
>>>
>>> are there benchmarks for DirectMemory available? How well it performs
>>> with 10's GB of memory. Are there production deployments of DM? From what I
>>> can see it holds key - Pointer(-Impl) map on a Java heap, isn't this a
>>> major limitation in terms of scalability?
>>>
>>> Thank for very much!
>>> Oh, by the way: Thanks for your effort on this project, it looks very
>>> interesting!
>>>
>>> --
>>> “Perfection is achieved, not when there is nothing more to add, but when
>>> there is nothing left to take away.”
>>> Antoine de Saint Exupéry
>>>
>>
>
>
> --
> “Perfection is achieved, not when there is nothing more to add, but when
> there is nothing left to take away.”
> Antoine de Saint Exupéry
>
Re: Benchmarks available?
Posted by Jaromir Hamala <ja...@gmail.com>.
Hi Raffaele,
thanks for the reply. Having a fresh benchmark would be good. It could
serve for two purposes:
-numbers to brag about! :)
-regression test ensuring that new features / fixes do not decrease the
performance
I can try to design something during the upcoming weekend. I currently do
not have an easy access to a machine with more than 16 GB RAM, but I can
always start a High Memory EC2 instance.
Regarding the keys in a heap: Currently there is Key-Pointer<K> pair being
held in a memory. The Pointer itself is quite heavy. In theory there could
be a "PointerLight" for some less-hot-data, but I understand it's not a
priority as of now.
Anyway, I have to say I really like to idea of the project - I would like
to contribute. I was thinking I could take look at DIRECTMEMORY-31
or DIRECTMEMORY-98 and send a patch.
Cheers,
Jaromir
On Thu, Feb 7, 2013 at 7:39 AM, Raffaele P. Guidi <
raffaele.p.guidi@gmail.com> wrote:
> Well last benchmarks are a bit (quite) outdated and can be found on my
> blog - actually they were performed on a totally different code base. This
> is probably the most relevant form of contribution that we need, should you
> be interested, because they are hard to do (I don't have 10gb, in example).
>
> No production deployments reported, yet (you could be first! ;)
>
> Yes keys are in a heap based map (this cannot be avoided) that's why dm's
> usage is more suitable when payloads are significantly larger then keys. We
> use efficient maps (from guava), in any case.
>
> Thanks for your interest.
>
> Ciao,
> R
> Il giorno 05/feb/2013 17:55, "Jaromir Hamala" <ja...@gmail.com>
> ha scritto:
>
> HI guys,
>>
>> are there benchmarks for DirectMemory available? How well it performs
>> with 10's GB of memory. Are there production deployments of DM? From what I
>> can see it holds key - Pointer(-Impl) map on a Java heap, isn't this a
>> major limitation in terms of scalability?
>>
>> Thank for very much!
>> Oh, by the way: Thanks for your effort on this project, it looks very
>> interesting!
>>
>> --
>> “Perfection is achieved, not when there is nothing more to add, but when
>> there is nothing left to take away.”
>> Antoine de Saint Exupéry
>>
>
--
“Perfection is achieved, not when there is nothing more to add, but when
there is nothing left to take away.”
Antoine de Saint Exupéry
Re: Benchmarks available?
Posted by Olivier Lamy <ol...@apache.org>.
if some scripts or maven goals are available I can try to find a
machine to do that.
Lemme know.
2013/2/7 Raffaele P. Guidi <ra...@gmail.com>:
> Well last benchmarks are a bit (quite) outdated and can be found on my blog
> - actually they were performed on a totally different code base. This is
> probably the most relevant form of contribution that we need, should you be
> interested, because they are hard to do (I don't have 10gb, in example).
>
> No production deployments reported, yet (you could be first! ;)
>
> Yes keys are in a heap based map (this cannot be avoided) that's why dm's
> usage is more suitable when payloads are significantly larger then keys. We
> use efficient maps (from guava), in any case.
>
> Thanks for your interest.
>
> Ciao,
> R
>
> Il giorno 05/feb/2013 17:55, "Jaromir Hamala" <ja...@gmail.com> ha
> scritto:
>
>> HI guys,
>>
>> are there benchmarks for DirectMemory available? How well it performs with
>> 10's GB of memory. Are there production deployments of DM? From what I can
>> see it holds key - Pointer(-Impl) map on a Java heap, isn't this a major
>> limitation in terms of scalability?
>>
>> Thank for very much!
>> Oh, by the way: Thanks for your effort on this project, it looks very
>> interesting!
>>
>> --
>> “Perfection is achieved, not when there is nothing more to add, but when
>> there is nothing left to take away.”
>> Antoine de Saint Exupéry
--
Olivier Lamy
Talend: http://coders.talend.com
http://twitter.com/olamy | http://linkedin.com/in/olamy
Re: Benchmarks available?
Posted by "Raffaele P. Guidi" <ra...@gmail.com>.
Well last benchmarks are a bit (quite) outdated and can be found on my blog
- actually they were performed on a totally different code base. This is
probably the most relevant form of contribution that we need, should you be
interested, because they are hard to do (I don't have 10gb, in example).
No production deployments reported, yet (you could be first! ;)
Yes keys are in a heap based map (this cannot be avoided) that's why dm's
usage is more suitable when payloads are significantly larger then keys. We
use efficient maps (from guava), in any case.
Thanks for your interest.
Ciao,
R
Il giorno 05/feb/2013 17:55, "Jaromir Hamala" <ja...@gmail.com> ha
scritto:
> HI guys,
>
> are there benchmarks for DirectMemory available? How well it performs with
> 10's GB of memory. Are there production deployments of DM? From what I can
> see it holds key - Pointer(-Impl) map on a Java heap, isn't this a major
> limitation in terms of scalability?
>
> Thank for very much!
> Oh, by the way: Thanks for your effort on this project, it looks very
> interesting!
>
> --
> “Perfection is achieved, not when there is nothing more to add, but when
> there is nothing left to take away.”
> Antoine de Saint Exupéry
>