Columnar layout for Mnemonic objects

[Johnu George <jo...@apache.org>]

Hey everyone,
      I had a great opportunity to talk to many Apache developers during
the Apache conference held in FL.  Regarding the Mnemonic project, some
were concerned with the cache efficiency of the generic object model
supported in Mnemonic.  Most of the applications prefer columnar layout for
cache efficiency and bulk load operations.  It can be currently supported
in Mnemonic by generating the layout on demand before the real computing.
However, the better solution is to move this logic to the storage layer
where data layout is provided by the user.( We can provide one extra
annotation field for every durable object?)

Any comments?

Regards,
Johnu

Re: Columnar layout for Mnemonic objects

["Gang(Gary) Wang" <ga...@apache.org>]

Hi Johnu I appreciate that you share the feedback of ApacheCon with us and
your conceptual design.

Yes, in some sense, the runtime layout is in favor of arranging a set of
same type data on a continuous memory space because this way could bring
many benefits from the cache line, SIMD, transferring, space saving.

As you mentioned, the better solution could be of embedding the columnar
layout mechanism to the infrastructure of durable object model with added
annotations, it could take most of benefits from columnar data as well as
providing the same view as object graphs.

The concerns are about the operation cost of columnar data deleting,
continuous memory space extension to accommodate more items and sync. if
index changed. but I believe that could be possible to resolve them all
very well, Thanks!

+1 from me.

Very truly yours.
Gary


On Mon, May 22, 2017 at 12:41 PM, Johnu George <jo...@apache.org> wrote:

> Hey everyone,
>       I had a great opportunity to talk to many Apache developers during
> the Apache conference held in FL.  Regarding the Mnemonic project, some
> were concerned with the cache efficiency of the generic object model
> supported in Mnemonic.  Most of the applications prefer columnar layout for
> cache efficiency and bulk load operations.  It can be currently supported
> in Mnemonic by generating the layout on demand before the real computing.
> However, the better solution is to move this logic to the storage layer
> where data layout is provided by the user.( We can provide one extra
> annotation field for every durable object?)
>
> Any comments?
>
> Regards,
> Johnu
>