You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@iceberg.apache.org by Ryan Blue <rb...@netflix.com.INVALID> on 2020/12/05 01:13:57 UTC

Sync notes for 2 December 2020

Hi everyone,

I just wrote up my notes for the sync on Wednesday. Feel free to comment if
I’ve missed anything or didn’t remember clearly.

Here’s a quick summary of the main points:

   - Highlights: Spark stored procedures are available, Glue and Nessie
   catalogs are committed, and Hive now supports DDL and projection pushdown!
   Thanks to Anton, Jack, Ryan M., Peter, Christine, Marton, Adrian, and
   Adrien!
   - We’re going to aim for an 0.11.0 release in the next couple weeks to
   avoid waiting until January
      - It would be nice to get the Hive config changes in if possible
      - Will try to get MERGE INTO done in Spark (DELETE FROM was merged
      today!)
      - Will also aim to get in Flink CDC writes, as well as Flink
      streaming reads
      - Please speak up if you need something specific!
   - Support for relative paths is a good idea, will probably add them to
   the spec

The full notes are here:
https://docs.google.com/document/d/1YuGhUdukLP5gGiqCbk0A5_Wifqe2CZWgOd3TbhY3UQg/edit#heading=h.r951w2dduwmy
-- 
Ryan Blue
Software Engineer
Netflix

Re: Sync notes for 2 December 2020

Posted by Ryan Blue <rb...@netflix.com.INVALID>.

Thanks for the note, Peter. No problem if it doesn't get done, that was
just one that I thought we may want to include. Since we were unable to ask
anyone actually working on it, we knew it was aspirational. We also know it
could be a stretch to get the release out this year as well. Have a great
holiday!

On Mon, Dec 7, 2020 at 8:11 AM Peter Vary <pv...@cloudera.com> wrote:

> Hi Team,
>
> I will be OOO starting from the end of this week. If someone wants to pick
> up Hive config changes, feel free to grab it. I am afraid I will not have
> enough time to do it this year.
>
> Thanks,
> Peter
>
> On Dec 5, 2020, at 02:13, Ryan Blue <rb...@netflix.com.INVALID> wrote:
>
> Hi everyone,
>
> I just wrote up my notes for the sync on Wednesday. Feel free to comment
> if I’ve missed anything or didn’t remember clearly.
>
> Here’s a quick summary of the main points:
>
>    - Highlights: Spark stored procedures are available, Glue and Nessie
>    catalogs are committed, and Hive now supports DDL and projection pushdown!
>    Thanks to Anton, Jack, Ryan M., Peter, Christine, Marton, Adrian, and
>    Adrien!
>    - We’re going to aim for an 0.11.0 release in the next couple weeks to
>    avoid waiting until January
>       - It would be nice to get the Hive config changes in if possible
>       - Will try to get MERGE INTO done in Spark (DELETE FROM was merged
>       today!)
>       - Will also aim to get in Flink CDC writes, as well as Flink
>       streaming reads
>       - Please speak up if you need something specific!
>    - Support for relative paths is a good idea, will probably add them to
>    the spec
>
> The full notes are here:
> https://docs.google.com/document/d/1YuGhUdukLP5gGiqCbk0A5_Wifqe2CZWgOd3TbhY3UQg/edit#heading=h.r951w2dduwmy
> --
> Ryan Blue
> Software Engineer
> Netflix
>
>
>

-- 
Ryan Blue
Software Engineer
Netflix

Re: Sync notes for 2 December 2020

Posted by Peter Vary <pv...@cloudera.com.INVALID>.

Hi Team,

I will be OOO starting from the end of this week. If someone wants to pick up Hive config changes, feel free to grab it. I am afraid I will not have enough time to do it this year.

Thanks,
Peter

> On Dec 5, 2020, at 02:13, Ryan Blue <rb...@netflix.com.INVALID> wrote:
> 
> Hi everyone,
> 
> I just wrote up my notes for the sync on Wednesday. Feel free to comment if I’ve missed anything or didn’t remember clearly.
> 
> Here’s a quick summary of the main points:
> 
> Highlights: Spark stored procedures are available, Glue and Nessie catalogs are committed, and Hive now supports DDL and projection pushdown! Thanks to Anton, Jack, Ryan M., Peter, Christine, Marton, Adrian, and Adrien!
> We’re going to aim for an 0.11.0 release in the next couple weeks to avoid waiting until January
> It would be nice to get the Hive config changes in if possible
> Will try to get MERGE INTO done in Spark (DELETE FROM was merged today!)
> Will also aim to get in Flink CDC writes, as well as Flink streaming reads
> Please speak up if you need something specific!
> Support for relative paths is a good idea, will probably add them to the spec
> The full notes are here: https://docs.google.com/document/d/1YuGhUdukLP5gGiqCbk0A5_Wifqe2CZWgOd3TbhY3UQg/edit#heading=h.r951w2dduwmy <https://docs.google.com/document/d/1YuGhUdukLP5gGiqCbk0A5_Wifqe2CZWgOd3TbhY3UQg/edit#heading=h.r951w2dduwmy>
> -- 
> Ryan Blue
> Software Engineer
> Netflix