You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@drill.apache.org by Stefán Baxter <st...@activitystream.com> on 2015/11/10 22:59:59 UTC

Parquet and dictionary based encoding in Drill 1.3

Hi,

Is it safe switch on store.parquet.enable_dictionary_encoding and is the
scanning of dictionary based columns optimized?

Regards,
  -Stefán

Re: Parquet and dictionary based encoding in Drill 1.3

Posted by Julien Le Dem <ju...@dremio.com>.

Note that parquet files written by parquet-mr will use dictionary encoding
by default.

On Mon, Nov 30, 2015 at 5:52 PM, Parth Chandra <pa...@apache.org> wrote:

> Hi Stefan
>
>   Looks like no one answered this for you. There has not been too much
> testing around dictionary encoded columns which is why it is not enabled by
> default. However, I have been running my dev instances with dictionary
> enabled for many months now and see no ill effects. And yes, the vectorized
> reader does handle dictionary encoded data.
>
>   If you do try enabling dictionary encoding, will you also post your
> experience? I know there is some plan around testing Parquet more
> comprehensively.
>
> Parth
>
> On Tue, Nov 10, 2015 at 1:59 PM, Stefán Baxter <st...@activitystream.com>
> wrote:
>
> > Hi,
> >
> > Is it safe switch on store.parquet.enable_dictionary_encoding and is the
> > scanning of dictionary based columns optimized?
> >
> > Regards,
> >   -Stefán
> >
>



-- 
Julien

Re: Parquet and dictionary based encoding in Drill 1.3

Posted by Parth Chandra <pa...@apache.org>.

Hi Stefan

  Looks like no one answered this for you. There has not been too much
testing around dictionary encoded columns which is why it is not enabled by
default. However, I have been running my dev instances with dictionary
enabled for many months now and see no ill effects. And yes, the vectorized
reader does handle dictionary encoded data.

  If you do try enabling dictionary encoding, will you also post your
experience? I know there is some plan around testing Parquet more
comprehensively.

Parth

On Tue, Nov 10, 2015 at 1:59 PM, Stefán Baxter <st...@activitystream.com>
wrote:

> Hi,
>
> Is it safe switch on store.parquet.enable_dictionary_encoding and is the
> scanning of dictionary based columns optimized?
>
> Regards,
>   -Stefán
>