How are Vectorization and Columnar execution implemented in Drill

[Yijie Shen <he...@gmail.com>]

Hi folks,

I'm new to Drill and find
"Drill also provides an execution layer that performs SQL processing
directly on columnar data without row materialization" and "vectorization
in Drill allows the CPU to operate on vectors" on
https://drill.apache.org/architecture/ page.

I'm really curious about how these are implemented in Drill, after some
search in the current code base, I could only find parquet files could be
read in as vectors, how are these vectors handled then in `vectorized`
manner? Could someone nicely point out where I can find more
information/code on this?

Thanks in advance.

Yijie

Re: How are Vectorization and Columnar execution implemented in Drill

[Hanifi GUNES <ha...@gmail.com>]

Not sure if you have a more specific question but Drill basically relies on
an abstraction that we call Value Vectors. Take a look at [1]. The whole
execution relies on these columnar structs so no row-wise materialization
is done.


1: https://drill.apache.org/docs/value-vectors/

-Hanifi

2016-05-18 20:34 GMT-07:00 Yijie Shen <he...@gmail.com>:

> Hi folks,
>
> I'm new to Drill and find
> "Drill also provides an execution layer that performs SQL processing
> directly on columnar data without row materialization" and "vectorization
> in Drill allows the CPU to operate on vectors" on
> https://drill.apache.org/architecture/ page.
>
> I'm really curious about how these are implemented in Drill, after some
> search in the current code base, I could only find parquet files could be
> read in as vectors, how are these vectors handled then in `vectorized`
> manner? Could someone nicely point out where I can find more
> information/code on this?
>
> Thanks in advance.
>
> Yijie
>