Parquet release

[Julien Le Dem <ju...@dremio.com>]

Now that the ByteBuffer branch has been merged, I'd like to make a Parquet
release so that Drill can get off its Parquet fork.
Let me know if there's anything specific you'd want to do before the
release.

-- 
Julien

Re: Parquet release

[Julien Le Dem <ju...@dremio.com>]

There had been improvements in Drill at the time this was done (numbers I
don't have unfortunately), however since it was already in Drill, merging
it in Parquet did not change the perf from Drill's perspective.
We should do some evaluation from the Parquet side.
Some of the performance shows up only if you use off-heap memory (which
parquet-mr does not by default) because the direct decompressor works on
off-heap memory as well.
Some of the perf should be in parquet-mr as well as we remove some buffer
copies.


On Thu, Nov 5, 2015 at 5:44 PM, Dmitriy Ryaboy <dv...@gmail.com> wrote:

> Release early and often.
>
> I'm just curious: did the bytebuffer work generate any interesting
> benchmarks?
>
> On Thu, Nov 5, 2015 at 5:39 PM, Julien Le Dem <ju...@dremio.com> wrote:
>
> > Now that the ByteBuffer branch has been merged, I'd like to make a
> Parquet
> > release so that Drill can get off its Parquet fork.
> > Let me know if there's anything specific you'd want to do before the
> > release.
> >
> > --
> > Julien
> >
>



-- 
Julien

Re: Parquet release

[Dmitriy Ryaboy <dv...@gmail.com>]

Release early and often.

I'm just curious: did the bytebuffer work generate any interesting
benchmarks?

On Thu, Nov 5, 2015 at 5:39 PM, Julien Le Dem <ju...@dremio.com> wrote:

> Now that the ByteBuffer branch has been merged, I'd like to make a Parquet
> release so that Drill can get off its Parquet fork.
> Let me know if there's anything specific you'd want to do before the
> release.
>
> --
> Julien
>