You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@parquet.apache.org by Xinli shang <sh...@uber.com.INVALID> on 2019/01/08 17:08:01 UTC
Re: [VOTE] Modular Encryption design sign-off
Hi all,
Can we have one more binding votes to sign off the encryption design to let
us move on to implementation? Or, if you think 2 current binding votes can
also be considered as sign off in this particular case, given this design
has been extensively discussed in a year and we have tested it in our
staging environment with privates, please let us know.
Once we are unblocked, we will work on the implementation and integration
with our security ecosystem. Maybe someday, we can show the community how
Parquet encryption can work in large scale production. I would be really
appreciated if we can be unblocked.
Xinli
On Wed, Dec 19, 2018 at 9:58 AM Gidon Gershinsky <gg...@gmail.com> wrote:
> Hi all,
>
> This is to remind that we have 2 binding votes - and need one more in order
> to sign off the encryption.
>
> Cheers, Gidon.
>
> On Sat, Dec 15, 2018 at 10:13 AM Zoltan Ivanfi <zi...@cloudera.com.invalid>
> wrote:
>
> > +1 (binding)
> >
> > Thanks for your efforts, Gidon!
> >
> > On Fri, Dec 14, 2018 at 6:55 PM Ryan Blue <rb...@netflix.com.invalid>
> > wrote:
> >
> > > +1 (binding)
> > >
> > > Thanks for all your work on this, Gidon!
> > >
> > > On Fri, Dec 14, 2018 at 9:48 AM Gidon Gershinsky <gg...@gmail.com>
> > wrote:
> > >
> > > > Hello Parquet developers,
> > > >
> > > > After a productive round of discussions, we have a new enhanced
> version
> > > of
> > > > the design document. You can find it here:
> > > >
> > > >
> > >
> >
> https://github.com/ggershinsky/parquet-format/blob/p1450-crypto-specification/Encryption.md
> > > >
> > > > This is a call for vote (either new, or resumption of the previous
> one.
> > > If
> > > > you have voted +1 before, feel free to do that again :).
> > > >
> > > > Cheers, Gidon
> > > >
> > > > ---------- Forwarded message ---------
> > > > From: Gidon Gershinsky <gg...@gmail.com>
> > > > Date: Tue, Oct 16, 2018 at 11:11 AM
> > > > Subject: [VOTE] Modular Encryption design sign-off
> > > > To: <de...@parquet.apache.org>
> > > >
> > > >
> > > > Hello Parquet developers,
> > > >
> > > > Per the last sync discussion, it is time to call for a vote on the
> > > Parquet
> > > > Modular Encryption design sign-off. The design doc can be found at
> the
> > > > encryption branch of the parquet-format repository,
> > > >
> https://github.com/apache/parquet-format/blob/encryption/Encryption.md
> > .
> > > >
> > > > The design is stable by now. This work had started 10 months ago, has
> > > been
> > > > extensively reviewed - and implemented (in Java, partially in C++),
> > by a
> > > > number of folks from different companies. To continue with the
> > > > implementation pull requests, we need the design to be formally
> signed
> > > off
> > > > by the community.
> > > >
> > > > Cheers, Gidon
> > > >
> > >
> > >
> > > --
> > > Ryan Blue
> > > Software Engineer
> > > Netflix
> > >
> >
>
--
Xinli Shang
Re: [VOTE] Modular Encryption design sign-off
Posted by Gidon Gershinsky <gg...@gmail.com>.
Ryan, Zoltan, Wes,
Thank you for reviewing the encryption spec and voting for it. The last
round of design updates had been complete in early December, so all 3 votes
refer to the current version.
Now that the spec has been signed off, we are getting back to the pull
requests. While the implementation of this mechanism is indeed a
significant amount of work, fortunately we don't start from the scratch.
Much of the Java code is already implemented, we'll need to clean it up,
add column index encryption and update the pull requests. The C++ code is
also in a good shape, though its implementation is based on an older design
version, with a larger gap to close.
The first step would be to merge the parquet-format pull requests - #114
(the spec) and #124 (the parquet.thrift updates for encryption). Once
this is done, we'll proceed to parquet-mr and arrow/cpp pull requests.
Cheers, Gidon.
On Wed, Jan 16, 2019, 09:39 Ryan Blue <rblue@netflix.com.invalid wrote:
> This vote passes, with at least 3 binding +1 votes.
>
> Some votes are from before we made significant changes, so I'm not sure
> which ones to count. Regardless, thank you to everyone that took the time
> to review the proposal.
>
> And thanks to Gidon for taking the lead and getting it done!
>
> rb
>
> On Wed, Jan 16, 2019 at 9:15 AM Wes McKinney <we...@gmail.com> wrote:
>
> > +1 (binding). Sorry for the delay in reviewing. There are a lot of
> > moving parts here and different configurations (same key for footer
> > and columns, different keys, plaintext footer, etc.) and so the
> > implementations are justifiably going to be a lot of work.
> >
> > @Tham, per your comment
> >
> > > We are going to release our product with encrypted parquet soon
> >
> > I haven't seen the C++ encryption PR updated in a while. Have you been
> > doing development on this somewhere else?
> >
> > On Tue, Jan 15, 2019 at 8:37 PM tham@emotiv.com <th...@emotiv.com> wrote:
> > >
> > > Hi all,
> > >
> > > I’m working on C++ parquet encryption module and my project at Emotiv
> is
> > currently using it in development phase. We are going to release our
> > product with encrypted parquet soon. So I’ll appreciate a lot if we can
> > process voting procedure on crypto specification as soon as possible,
> then
> > we can continue with implementation and Java-C++ interop testing.
> > >
> > > Many thanks,
> > > Tham
> > >
> > > On 2019/01/08 17:08:01, Xinli shang <sh...@uber.com.INVALID> wrote:
> > > > Hi all,
> > > >
> > > > Can we have one more binding votes to sign off the encryption design
> > to let
> > > > us move on to implementation? Or, if you think 2 current binding
> > votes can
> > > > also be considered as sign off in this particular case, given this
> > design
> > > > has been extensively discussed in a year and we have tested it in our
> > > > staging environment with privates, please let us know.
> > > >
> > > > Once we are unblocked, we will work on the implementation and
> > integration
> > > > with our security ecosystem. Maybe someday, we can show the community
> > how
> > > > Parquet encryption can work in large scale production. I would be
> > really
> > > > appreciated if we can be unblocked.
> > > >
> > > > Xinli
> > > >
> > > > On Wed, Dec 19, 2018 at 9:58 AM Gidon Gershinsky <gg...@gmail.com>
> > wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > This is to remind that we have 2 binding votes - and need one more
> > in order
> > > > > to sign off the encryption.
> > > > >
> > > > > Cheers, Gidon.
> > > > >
> > > > > On Sat, Dec 15, 2018 at 10:13 AM Zoltan Ivanfi
> > <zi...@cloudera.com.invalid>
> > > > > wrote:
> > > > >
> > > > > > +1 (binding)
> > > > > >
> > > > > > Thanks for your efforts, Gidon!
> > > > > >
> > > > > > On Fri, Dec 14, 2018 at 6:55 PM Ryan Blue
> > <rb...@netflix.com.invalid>
> > > > > > wrote:
> > > > > >
> > > > > > > +1 (binding)
> > > > > > >
> > > > > > > Thanks for all your work on this, Gidon!
> > > > > > >
> > > > > > > On Fri, Dec 14, 2018 at 9:48 AM Gidon Gershinsky <
> > gg5070@gmail.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > > Hello Parquet developers,
> > > > > > > >
> > > > > > > > After a productive round of discussions, we have a new
> enhanced
> > > > > version
> > > > > > > of
> > > > > > > > the design document. You can find it here:
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> >
> https://github.com/ggershinsky/parquet-format/blob/p1450-crypto-specification/Encryption.md
> > > > > > > >
> > > > > > > > This is a call for vote (either new, or resumption of the
> > previous
> > > > > one.
> > > > > > > If
> > > > > > > > you have voted +1 before, feel free to do that again :).
> > > > > > > >
> > > > > > > > Cheers, Gidon
> > > > > > > >
> > > > > > > > ---------- Forwarded message ---------
> > > > > > > > From: Gidon Gershinsky <gg...@gmail.com>
> > > > > > > > Date: Tue, Oct 16, 2018 at 11:11 AM
> > > > > > > > Subject: [VOTE] Modular Encryption design sign-off
> > > > > > > > To: <de...@parquet.apache.org>
> > > > > > > >
> > > > > > > >
> > > > > > > > Hello Parquet developers,
> > > > > > > >
> > > > > > > > Per the last sync discussion, it is time to call for a vote
> on
> > the
> > > > > > > Parquet
> > > > > > > > Modular Encryption design sign-off. The design doc can be
> > found at
> > > > > the
> > > > > > > > encryption branch of the parquet-format repository,
> > > > > > > >
> > > > >
> > https://github.com/apache/parquet-format/blob/encryption/Encryption.md
> > > > > > .
> > > > > > > >
> > > > > > > > The design is stable by now. This work had started 10 months
> > ago, has
> > > > > > > been
> > > > > > > > extensively reviewed - and implemented (in Java, partially
> in
> > C++),
> > > > > > by a
> > > > > > > > number of folks from different companies. To continue with
> the
> > > > > > > > implementation pull requests, we need the design to be
> formally
> > > > > signed
> > > > > > > off
> > > > > > > > by the community.
> > > > > > > >
> > > > > > > > Cheers, Gidon
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Ryan Blue
> > > > > > > Software Engineer
> > > > > > > Netflix
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Xinli Shang
> > > >
> >
>
>
> --
> Ryan Blue
> Software Engineer
> Netflix
>
Re: [VOTE] Modular Encryption design sign-off
Posted by Ryan Blue <rb...@netflix.com.INVALID>.
This vote passes, with at least 3 binding +1 votes.
Some votes are from before we made significant changes, so I'm not sure
which ones to count. Regardless, thank you to everyone that took the time
to review the proposal.
And thanks to Gidon for taking the lead and getting it done!
rb
On Wed, Jan 16, 2019 at 9:15 AM Wes McKinney <we...@gmail.com> wrote:
> +1 (binding). Sorry for the delay in reviewing. There are a lot of
> moving parts here and different configurations (same key for footer
> and columns, different keys, plaintext footer, etc.) and so the
> implementations are justifiably going to be a lot of work.
>
> @Tham, per your comment
>
> > We are going to release our product with encrypted parquet soon
>
> I haven't seen the C++ encryption PR updated in a while. Have you been
> doing development on this somewhere else?
>
> On Tue, Jan 15, 2019 at 8:37 PM tham@emotiv.com <th...@emotiv.com> wrote:
> >
> > Hi all,
> >
> > I’m working on C++ parquet encryption module and my project at Emotiv is
> currently using it in development phase. We are going to release our
> product with encrypted parquet soon. So I’ll appreciate a lot if we can
> process voting procedure on crypto specification as soon as possible, then
> we can continue with implementation and Java-C++ interop testing.
> >
> > Many thanks,
> > Tham
> >
> > On 2019/01/08 17:08:01, Xinli shang <sh...@uber.com.INVALID> wrote:
> > > Hi all,
> > >
> > > Can we have one more binding votes to sign off the encryption design
> to let
> > > us move on to implementation? Or, if you think 2 current binding
> votes can
> > > also be considered as sign off in this particular case, given this
> design
> > > has been extensively discussed in a year and we have tested it in our
> > > staging environment with privates, please let us know.
> > >
> > > Once we are unblocked, we will work on the implementation and
> integration
> > > with our security ecosystem. Maybe someday, we can show the community
> how
> > > Parquet encryption can work in large scale production. I would be
> really
> > > appreciated if we can be unblocked.
> > >
> > > Xinli
> > >
> > > On Wed, Dec 19, 2018 at 9:58 AM Gidon Gershinsky <gg...@gmail.com>
> wrote:
> > >
> > > > Hi all,
> > > >
> > > > This is to remind that we have 2 binding votes - and need one more
> in order
> > > > to sign off the encryption.
> > > >
> > > > Cheers, Gidon.
> > > >
> > > > On Sat, Dec 15, 2018 at 10:13 AM Zoltan Ivanfi
> <zi...@cloudera.com.invalid>
> > > > wrote:
> > > >
> > > > > +1 (binding)
> > > > >
> > > > > Thanks for your efforts, Gidon!
> > > > >
> > > > > On Fri, Dec 14, 2018 at 6:55 PM Ryan Blue
> <rb...@netflix.com.invalid>
> > > > > wrote:
> > > > >
> > > > > > +1 (binding)
> > > > > >
> > > > > > Thanks for all your work on this, Gidon!
> > > > > >
> > > > > > On Fri, Dec 14, 2018 at 9:48 AM Gidon Gershinsky <
> gg5070@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > > Hello Parquet developers,
> > > > > > >
> > > > > > > After a productive round of discussions, we have a new enhanced
> > > > version
> > > > > > of
> > > > > > > the design document. You can find it here:
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> https://github.com/ggershinsky/parquet-format/blob/p1450-crypto-specification/Encryption.md
> > > > > > >
> > > > > > > This is a call for vote (either new, or resumption of the
> previous
> > > > one.
> > > > > > If
> > > > > > > you have voted +1 before, feel free to do that again :).
> > > > > > >
> > > > > > > Cheers, Gidon
> > > > > > >
> > > > > > > ---------- Forwarded message ---------
> > > > > > > From: Gidon Gershinsky <gg...@gmail.com>
> > > > > > > Date: Tue, Oct 16, 2018 at 11:11 AM
> > > > > > > Subject: [VOTE] Modular Encryption design sign-off
> > > > > > > To: <de...@parquet.apache.org>
> > > > > > >
> > > > > > >
> > > > > > > Hello Parquet developers,
> > > > > > >
> > > > > > > Per the last sync discussion, it is time to call for a vote on
> the
> > > > > > Parquet
> > > > > > > Modular Encryption design sign-off. The design doc can be
> found at
> > > > the
> > > > > > > encryption branch of the parquet-format repository,
> > > > > > >
> > > >
> https://github.com/apache/parquet-format/blob/encryption/Encryption.md
> > > > > .
> > > > > > >
> > > > > > > The design is stable by now. This work had started 10 months
> ago, has
> > > > > > been
> > > > > > > extensively reviewed - and implemented (in Java, partially in
> C++),
> > > > > by a
> > > > > > > number of folks from different companies. To continue with the
> > > > > > > implementation pull requests, we need the design to be formally
> > > > signed
> > > > > > off
> > > > > > > by the community.
> > > > > > >
> > > > > > > Cheers, Gidon
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Ryan Blue
> > > > > > Software Engineer
> > > > > > Netflix
> > > > > >
> > > > >
> > > >
> > >
> > >
> > > --
> > > Xinli Shang
> > >
>
--
Ryan Blue
Software Engineer
Netflix
Re: [VOTE] Modular Encryption design sign-off
Posted by Wes McKinney <we...@gmail.com>.
hi Tham -- I created https://issues.apache.org/jira/browse/ARROW-4302
about adding OpenSSL to the build toolchain. Let's discuss there
On Wed, Jan 16, 2019 at 8:33 PM tham <th...@emotiv.com> wrote:
>
> Hi all,
>
> Thank you all for very quick process. I really appriciated that.
>
> > I haven't seen the C++ encryption PR updated in a while. Have you been
> doing development on this somewhere else?
> Sorry, I’ve been working silently in our own repo, but still some small fixes. I’m waiting for review process, then Gidon and I will discuss about detailed implementation for the update.
>
> Moreover, I have some difficulties in update CMakeList about adding flexible OpenSSL thirdparty. It will be great if someone who has experiences with Cmake can help.
>
> Thanks,
> Tham
>
> From: Wes McKinney
> Sent: Thursday, January 17, 2019 12:15 AM
> To: Parquet Dev
> Subject: Re: [VOTE] Modular Encryption design sign-off
>
> +1 (binding). Sorry for the delay in reviewing. There are a lot of
> moving parts here and different configurations (same key for footer
> and columns, different keys, plaintext footer, etc.) and so the
> implementations are justifiably going to be a lot of work.
>
> @Tham, per your comment
>
> > We are going to release our product with encrypted parquet soon
>
> I haven't seen the C++ encryption PR updated in a while. Have you been
> doing development on this somewhere else?
>
> On Tue, Jan 15, 2019 at 8:37 PM tham@emotiv.com <th...@emotiv.com> wrote:
> >
> > Hi all,
> >
> > I’m working on C++ parquet encryption module and my project at Emotiv is currently using it in development phase. We are going to release our product with encrypted parquet soon. So I’ll appreciate a lot if we can process voting procedure on crypto specification as soon as possible, then we can continue with implementation and Java-C++ interop testing.
> >
> > Many thanks,
> > Tham
> >
> > On 2019/01/08 17:08:01, Xinli shang <sh...@uber.com.INVALID> wrote:
> > > Hi all,
> > >
> > > Can we have one more binding votes to sign off the encryption design to let
> > > us move on to implementation? Or, if you think 2 current binding votes can
> > > also be considered as sign off in this particular case, given this design
> > > has been extensively discussed in a year and we have tested it in our
> > > staging environment with privates, please let us know.
> > >
> > > Once we are unblocked, we will work on the implementation and integration
> > > with our security ecosystem. Maybe someday, we can show the community how
> > > Parquet encryption can work in large scale production. I would be really
> > > appreciated if we can be unblocked.
> > >
> > > Xinli
> > >
> > > On Wed, Dec 19, 2018 at 9:58 AM Gidon Gershinsky <gg...@gmail.com> wrote:
> > >
> > > > Hi all,
> > > >
> > > > This is to remind that we have 2 binding votes - and need one more in order
> > > > to sign off the encryption.
> > > >
> > > > Cheers, Gidon.
> > > >
> > > > On Sat, Dec 15, 2018 at 10:13 AM Zoltan Ivanfi <zi...@cloudera.com.invalid>
> > > > wrote:
> > > >
> > > > > +1 (binding)
> > > > >
> > > > > Thanks for your efforts, Gidon!
> > > > >
> > > > > On Fri, Dec 14, 2018 at 6:55 PM Ryan Blue <rb...@netflix.com.invalid>
> > > > > wrote:
> > > > >
> > > > > > +1 (binding)
> > > > > >
> > > > > > Thanks for all your work on this, Gidon!
> > > > > >
> > > > > > On Fri, Dec 14, 2018 at 9:48 AM Gidon Gershinsky <gg...@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > > Hello Parquet developers,
> > > > > > >
> > > > > > > After a productive round of discussions, we have a new enhanced
> > > > version
> > > > > > of
> > > > > > > the design document. You can find it here:
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > https://github.com/ggershinsky/parquet-format/blob/p1450-crypto-specification/Encryption.md
> > > > > > >
> > > > > > > This is a call for vote (either new, or resumption of the previous
> > > > one.
> > > > > > If
> > > > > > > you have voted +1 before, feel free to do that again :).
> > > > > > >
> > > > > > > Cheers, Gidon
> > > > > > >
> > > > > > > ---------- Forwarded message ---------
> > > > > > > From: Gidon Gershinsky <gg...@gmail.com>
> > > > > > > Date: Tue, Oct 16, 2018 at 11:11 AM
> > > > > > > Subject: [VOTE] Modular Encryption design sign-off
> > > > > > > To: <de...@parquet.apache.org>
> > > > > > >
> > > > > > >
> > > > > > > Hello Parquet developers,
> > > > > > >
> > > > > > > Per the last sync discussion, it is time to call for a vote on the
> > > > > > Parquet
> > > > > > > Modular Encryption design sign-off. The design doc can be found at
> > > > the
> > > > > > > encryption branch of the parquet-format repository,
> > > > > > >
> > > > https://github.com/apache/parquet-format/blob/encryption/Encryption.md
> > > > > .
> > > > > > >
> > > > > > > The design is stable by now. This work had started 10 months ago, has
> > > > > > been
> > > > > > > extensively reviewed - and implemented (in Java, partially in C++),
> > > > > by a
> > > > > > > number of folks from different companies. To continue with the
> > > > > > > implementation pull requests, we need the design to be formally
> > > > signed
> > > > > > off
> > > > > > > by the community.
> > > > > > >
> > > > > > > Cheers, Gidon
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Ryan Blue
> > > > > > Software Engineer
> > > > > > Netflix
> > > > > >
> > > > >
> > > >
> > >
> > >
> > > --
> > > Xinli Shang
> > >
>
RE: [VOTE] Modular Encryption design sign-off
Posted by tham <th...@emotiv.com>.
Hi all,
Thank you all for very quick process. I really appriciated that.
> I haven't seen the C++ encryption PR updated in a while. Have you been
doing development on this somewhere else?
Sorry, I’ve been working silently in our own repo, but still some small fixes. I’m waiting for review process, then Gidon and I will discuss about detailed implementation for the update.
Moreover, I have some difficulties in update CMakeList about adding flexible OpenSSL thirdparty. It will be great if someone who has experiences with Cmake can help.
Thanks,
Tham
From: Wes McKinney
Sent: Thursday, January 17, 2019 12:15 AM
To: Parquet Dev
Subject: Re: [VOTE] Modular Encryption design sign-off
+1 (binding). Sorry for the delay in reviewing. There are a lot of
moving parts here and different configurations (same key for footer
and columns, different keys, plaintext footer, etc.) and so the
implementations are justifiably going to be a lot of work.
@Tham, per your comment
> We are going to release our product with encrypted parquet soon
I haven't seen the C++ encryption PR updated in a while. Have you been
doing development on this somewhere else?
On Tue, Jan 15, 2019 at 8:37 PM tham@emotiv.com <th...@emotiv.com> wrote:
>
> Hi all,
>
> I’m working on C++ parquet encryption module and my project at Emotiv is currently using it in development phase. We are going to release our product with encrypted parquet soon. So I’ll appreciate a lot if we can process voting procedure on crypto specification as soon as possible, then we can continue with implementation and Java-C++ interop testing.
>
> Many thanks,
> Tham
>
> On 2019/01/08 17:08:01, Xinli shang <sh...@uber.com.INVALID> wrote:
> > Hi all,
> >
> > Can we have one more binding votes to sign off the encryption design to let
> > us move on to implementation? Or, if you think 2 current binding votes can
> > also be considered as sign off in this particular case, given this design
> > has been extensively discussed in a year and we have tested it in our
> > staging environment with privates, please let us know.
> >
> > Once we are unblocked, we will work on the implementation and integration
> > with our security ecosystem. Maybe someday, we can show the community how
> > Parquet encryption can work in large scale production. I would be really
> > appreciated if we can be unblocked.
> >
> > Xinli
> >
> > On Wed, Dec 19, 2018 at 9:58 AM Gidon Gershinsky <gg...@gmail.com> wrote:
> >
> > > Hi all,
> > >
> > > This is to remind that we have 2 binding votes - and need one more in order
> > > to sign off the encryption.
> > >
> > > Cheers, Gidon.
> > >
> > > On Sat, Dec 15, 2018 at 10:13 AM Zoltan Ivanfi <zi...@cloudera.com.invalid>
> > > wrote:
> > >
> > > > +1 (binding)
> > > >
> > > > Thanks for your efforts, Gidon!
> > > >
> > > > On Fri, Dec 14, 2018 at 6:55 PM Ryan Blue <rb...@netflix.com.invalid>
> > > > wrote:
> > > >
> > > > > +1 (binding)
> > > > >
> > > > > Thanks for all your work on this, Gidon!
> > > > >
> > > > > On Fri, Dec 14, 2018 at 9:48 AM Gidon Gershinsky <gg...@gmail.com>
> > > > wrote:
> > > > >
> > > > > > Hello Parquet developers,
> > > > > >
> > > > > > After a productive round of discussions, we have a new enhanced
> > > version
> > > > > of
> > > > > > the design document. You can find it here:
> > > > > >
> > > > > >
> > > > >
> > > >
> > > https://github.com/ggershinsky/parquet-format/blob/p1450-crypto-specification/Encryption.md
> > > > > >
> > > > > > This is a call for vote (either new, or resumption of the previous
> > > one.
> > > > > If
> > > > > > you have voted +1 before, feel free to do that again :).
> > > > > >
> > > > > > Cheers, Gidon
> > > > > >
> > > > > > ---------- Forwarded message ---------
> > > > > > From: Gidon Gershinsky <gg...@gmail.com>
> > > > > > Date: Tue, Oct 16, 2018 at 11:11 AM
> > > > > > Subject: [VOTE] Modular Encryption design sign-off
> > > > > > To: <de...@parquet.apache.org>
> > > > > >
> > > > > >
> > > > > > Hello Parquet developers,
> > > > > >
> > > > > > Per the last sync discussion, it is time to call for a vote on the
> > > > > Parquet
> > > > > > Modular Encryption design sign-off. The design doc can be found at
> > > the
> > > > > > encryption branch of the parquet-format repository,
> > > > > >
> > > https://github.com/apache/parquet-format/blob/encryption/Encryption.md
> > > > .
> > > > > >
> > > > > > The design is stable by now. This work had started 10 months ago, has
> > > > > been
> > > > > > extensively reviewed - and implemented (in Java, partially in C++),
> > > > by a
> > > > > > number of folks from different companies. To continue with the
> > > > > > implementation pull requests, we need the design to be formally
> > > signed
> > > > > off
> > > > > > by the community.
> > > > > >
> > > > > > Cheers, Gidon
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Ryan Blue
> > > > > Software Engineer
> > > > > Netflix
> > > > >
> > > >
> > >
> >
> >
> > --
> > Xinli Shang
> >
Re: [VOTE] Modular Encryption design sign-off
Posted by Wes McKinney <we...@gmail.com>.
+1 (binding). Sorry for the delay in reviewing. There are a lot of
moving parts here and different configurations (same key for footer
and columns, different keys, plaintext footer, etc.) and so the
implementations are justifiably going to be a lot of work.
@Tham, per your comment
> We are going to release our product with encrypted parquet soon
I haven't seen the C++ encryption PR updated in a while. Have you been
doing development on this somewhere else?
On Tue, Jan 15, 2019 at 8:37 PM tham@emotiv.com <th...@emotiv.com> wrote:
>
> Hi all,
>
> I’m working on C++ parquet encryption module and my project at Emotiv is currently using it in development phase. We are going to release our product with encrypted parquet soon. So I’ll appreciate a lot if we can process voting procedure on crypto specification as soon as possible, then we can continue with implementation and Java-C++ interop testing.
>
> Many thanks,
> Tham
>
> On 2019/01/08 17:08:01, Xinli shang <sh...@uber.com.INVALID> wrote:
> > Hi all,
> >
> > Can we have one more binding votes to sign off the encryption design to let
> > us move on to implementation? Or, if you think 2 current binding votes can
> > also be considered as sign off in this particular case, given this design
> > has been extensively discussed in a year and we have tested it in our
> > staging environment with privates, please let us know.
> >
> > Once we are unblocked, we will work on the implementation and integration
> > with our security ecosystem. Maybe someday, we can show the community how
> > Parquet encryption can work in large scale production. I would be really
> > appreciated if we can be unblocked.
> >
> > Xinli
> >
> > On Wed, Dec 19, 2018 at 9:58 AM Gidon Gershinsky <gg...@gmail.com> wrote:
> >
> > > Hi all,
> > >
> > > This is to remind that we have 2 binding votes - and need one more in order
> > > to sign off the encryption.
> > >
> > > Cheers, Gidon.
> > >
> > > On Sat, Dec 15, 2018 at 10:13 AM Zoltan Ivanfi <zi...@cloudera.com.invalid>
> > > wrote:
> > >
> > > > +1 (binding)
> > > >
> > > > Thanks for your efforts, Gidon!
> > > >
> > > > On Fri, Dec 14, 2018 at 6:55 PM Ryan Blue <rb...@netflix.com.invalid>
> > > > wrote:
> > > >
> > > > > +1 (binding)
> > > > >
> > > > > Thanks for all your work on this, Gidon!
> > > > >
> > > > > On Fri, Dec 14, 2018 at 9:48 AM Gidon Gershinsky <gg...@gmail.com>
> > > > wrote:
> > > > >
> > > > > > Hello Parquet developers,
> > > > > >
> > > > > > After a productive round of discussions, we have a new enhanced
> > > version
> > > > > of
> > > > > > the design document. You can find it here:
> > > > > >
> > > > > >
> > > > >
> > > >
> > > https://github.com/ggershinsky/parquet-format/blob/p1450-crypto-specification/Encryption.md
> > > > > >
> > > > > > This is a call for vote (either new, or resumption of the previous
> > > one.
> > > > > If
> > > > > > you have voted +1 before, feel free to do that again :).
> > > > > >
> > > > > > Cheers, Gidon
> > > > > >
> > > > > > ---------- Forwarded message ---------
> > > > > > From: Gidon Gershinsky <gg...@gmail.com>
> > > > > > Date: Tue, Oct 16, 2018 at 11:11 AM
> > > > > > Subject: [VOTE] Modular Encryption design sign-off
> > > > > > To: <de...@parquet.apache.org>
> > > > > >
> > > > > >
> > > > > > Hello Parquet developers,
> > > > > >
> > > > > > Per the last sync discussion, it is time to call for a vote on the
> > > > > Parquet
> > > > > > Modular Encryption design sign-off. The design doc can be found at
> > > the
> > > > > > encryption branch of the parquet-format repository,
> > > > > >
> > > https://github.com/apache/parquet-format/blob/encryption/Encryption.md
> > > > .
> > > > > >
> > > > > > The design is stable by now. This work had started 10 months ago, has
> > > > > been
> > > > > > extensively reviewed - and implemented (in Java, partially in C++),
> > > > by a
> > > > > > number of folks from different companies. To continue with the
> > > > > > implementation pull requests, we need the design to be formally
> > > signed
> > > > > off
> > > > > > by the community.
> > > > > >
> > > > > > Cheers, Gidon
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Ryan Blue
> > > > > Software Engineer
> > > > > Netflix
> > > > >
> > > >
> > >
> >
> >
> > --
> > Xinli Shang
> >
Re: [VOTE] Modular Encryption design sign-off
Posted by .
Hi all,
I’m working on C++ parquet encryption module and my project at Emotiv is currently using it in development phase. We are going to release our product with encrypted parquet soon. So I’ll appreciate a lot if we can process voting procedure on crypto specification as soon as possible, then we can continue with implementation and Java-C++ interop testing.
Many thanks,
Tham
On 2019/01/08 17:08:01, Xinli shang <sh...@uber.com.INVALID> wrote:
> Hi all,
>
> Can we have one more binding votes to sign off the encryption design to let
> us move on to implementation? Or, if you think 2 current binding votes can
> also be considered as sign off in this particular case, given this design
> has been extensively discussed in a year and we have tested it in our
> staging environment with privates, please let us know.
>
> Once we are unblocked, we will work on the implementation and integration
> with our security ecosystem. Maybe someday, we can show the community how
> Parquet encryption can work in large scale production. I would be really
> appreciated if we can be unblocked.
>
> Xinli
>
> On Wed, Dec 19, 2018 at 9:58 AM Gidon Gershinsky <gg...@gmail.com> wrote:
>
> > Hi all,
> >
> > This is to remind that we have 2 binding votes - and need one more in order
> > to sign off the encryption.
> >
> > Cheers, Gidon.
> >
> > On Sat, Dec 15, 2018 at 10:13 AM Zoltan Ivanfi <zi...@cloudera.com.invalid>
> > wrote:
> >
> > > +1 (binding)
> > >
> > > Thanks for your efforts, Gidon!
> > >
> > > On Fri, Dec 14, 2018 at 6:55 PM Ryan Blue <rb...@netflix.com.invalid>
> > > wrote:
> > >
> > > > +1 (binding)
> > > >
> > > > Thanks for all your work on this, Gidon!
> > > >
> > > > On Fri, Dec 14, 2018 at 9:48 AM Gidon Gershinsky <gg...@gmail.com>
> > > wrote:
> > > >
> > > > > Hello Parquet developers,
> > > > >
> > > > > After a productive round of discussions, we have a new enhanced
> > version
> > > > of
> > > > > the design document. You can find it here:
> > > > >
> > > > >
> > > >
> > >
> > https://github.com/ggershinsky/parquet-format/blob/p1450-crypto-specification/Encryption.md
> > > > >
> > > > > This is a call for vote (either new, or resumption of the previous
> > one.
> > > > If
> > > > > you have voted +1 before, feel free to do that again :).
> > > > >
> > > > > Cheers, Gidon
> > > > >
> > > > > ---------- Forwarded message ---------
> > > > > From: Gidon Gershinsky <gg...@gmail.com>
> > > > > Date: Tue, Oct 16, 2018 at 11:11 AM
> > > > > Subject: [VOTE] Modular Encryption design sign-off
> > > > > To: <de...@parquet.apache.org>
> > > > >
> > > > >
> > > > > Hello Parquet developers,
> > > > >
> > > > > Per the last sync discussion, it is time to call for a vote on the
> > > > Parquet
> > > > > Modular Encryption design sign-off. The design doc can be found at
> > the
> > > > > encryption branch of the parquet-format repository,
> > > > >
> > https://github.com/apache/parquet-format/blob/encryption/Encryption.md
> > > .
> > > > >
> > > > > The design is stable by now. This work had started 10 months ago, has
> > > > been
> > > > > extensively reviewed - and implemented (in Java, partially in C++),
> > > by a
> > > > > number of folks from different companies. To continue with the
> > > > > implementation pull requests, we need the design to be formally
> > signed
> > > > off
> > > > > by the community.
> > > > >
> > > > > Cheers, Gidon
> > > > >
> > > >
> > > >
> > > > --
> > > > Ryan Blue
> > > > Software Engineer
> > > > Netflix
> > > >
> > >
> >
>
>
> --
> Xinli Shang
>