You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@druid.apache.org by Hao Luo <hl...@twitter.com.INVALID> on 2019/07/14 07:01:45 UTC

When is GenericIndexed V2 constructed?

Hi,
    I have a few questions regarding the V2.

   1. When is it constructed from values? The fromArray() or fromIterable()
   seems always create version 1 and createGenericIndexedVersionTwo only loads
   existing V2 GenericIndexed.
   2. What is the purpose of GenericIndexed V2? When is it used?

Thanks.
Hao Luo

Re: When is GenericIndexed V2 constructed?

Posted by Gian Merlino <gi...@apache.org>.

Hey Hao,

GenericIndexed V2 is used when a column is large enough that it must be
spread over multiple files (each file has a 2GB limit, since that's the
most that can be mapped into a single ByteBuffer). Check out
LargeColumnSupportedComplexColumnSerializer for an entry point. It's used
today for most of the builtin complex columns.

On Sun, Jul 14, 2019 at 12:02 AM Hao Luo <hl...@twitter.com.invalid> wrote:

> Hi,
>     I have a few questions regarding the V2.
>
>    1. When is it constructed from values? The fromArray() or fromIterable()
>    seems always create version 1 and createGenericIndexedVersionTwo only
> loads
>    existing V2 GenericIndexed.
>    2. What is the purpose of GenericIndexed V2? When is it used?
>
> Thanks.
> Hao Luo
>