You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Jorge Cardoso Leitão <jo...@gmail.com> on 2021/06/27 06:40:15 UTC
[VOTE] Donation of rust arrow2 and parquet2
Hi,
I would like to bring to this mailing list a proposal to donate the source
code of arrow2 [1] and parquet2 [2] as experimental repositories [3] within
Apache Arrow, conditional on IP clearance.
The specific PRs are:
* https://github.com/apache/arrow-experimental-rs-arrow2/pull/1
* https://github.com/apache/arrow-experimental-rs-parquet2/pull/1
The source code contains rewrites of the arrow and parquet crates with
safety and security in mind. In particular,
* no buffer transmutes
* no unsafe APIs marked as safe
* parquet's implementation is unsafe free
There are many other important features, such as big endian support and IPC
2.0 support. There is one regression over latest: support nested types in
parquet read and write. I observe no negative impact on performance.
See a longer discussion in [4] over the reasons why the current rust
implementation is susceptible to safety violations. In particular, many
core APIs of the crate are considered security vulnerabilities under
RustSec's [5] definitions, and are difficult to address on its current
design.
I validated that it is possible to migrate DataFusion [6] and Polars [7]
without further code changes.
The vote will be open for at least 72 hours.
[ ] +1 Accept the code donation as experimental repos.
[ ] +0
[ ] -1 Do not accept the code donation as experimental repos because...
[1]
https://github.com/apache/arrow/blob/master/docs/source/developers/experimental_repos.rst
[2] https://github.com/jorgecarleitao/arrow2
[3] https://github.com/jorgecarleitao/parquet2
[4] https://github.com/jorgecarleitao/arrow2#faq
[5] https://rustsec.org/
[6] https://github.com/apache/arrow-datafusion/pull/68
[7] https://github.com/pola-rs/polars
Re: [VOTE] Donation of rust arrow2 and parquet2
Posted by Andy Grove <an...@gmail.com>.
+1 (binding)
On Sun, Jun 27, 2021 at 7:05 AM Fernando Herrera <
fernando.j.herrera@gmail.com> wrote:
> +1
>
> On Sun, 27 Jun 2021, 07:40 Jorge Cardoso Leitão, <jorgecarleitao@gmail.com
> >
> wrote:
>
> > Hi,
> >
> > I would like to bring to this mailing list a proposal to donate the
> source
> > code of arrow2 [1] and parquet2 [2] as experimental repositories [3]
> within
> > Apache Arrow, conditional on IP clearance.
> >
> > The specific PRs are:
> >
> > * https://github.com/apache/arrow-experimental-rs-arrow2/pull/1
> > * https://github.com/apache/arrow-experimental-rs-parquet2/pull/1
> >
> > The source code contains rewrites of the arrow and parquet crates with
> > safety and security in mind. In particular,
> >
> > * no buffer transmutes
> > * no unsafe APIs marked as safe
> > * parquet's implementation is unsafe free
> >
> > There are many other important features, such as big endian support and
> IPC
> > 2.0 support. There is one regression over latest: support nested types in
> > parquet read and write. I observe no negative impact on performance.
> >
> > See a longer discussion in [4] over the reasons why the current rust
> > implementation is susceptible to safety violations. In particular, many
> > core APIs of the crate are considered security vulnerabilities under
> > RustSec's [5] definitions, and are difficult to address on its current
> > design.
> >
> > I validated that it is possible to migrate DataFusion [6] and Polars [7]
> > without further code changes.
> >
> > The vote will be open for at least 72 hours.
> >
> > [ ] +1 Accept the code donation as experimental repos.
> > [ ] +0
> > [ ] -1 Do not accept the code donation as experimental repos because...
> >
> > [1]
> >
> >
> https://github.com/apache/arrow/blob/master/docs/source/developers/experimental_repos.rst
> > [2] https://github.com/jorgecarleitao/arrow2
> > [3] https://github.com/jorgecarleitao/parquet2
> > [4] https://github.com/jorgecarleitao/arrow2#faq
> > [5] https://rustsec.org/
> > [6] https://github.com/apache/arrow-datafusion/pull/68
> > [7] https://github.com/pola-rs/polars
> >
>
Re: [VOTE] Donation of rust arrow2 and parquet2
Posted by Fernando Herrera <fe...@gmail.com>.
+1
On Sun, 27 Jun 2021, 07:40 Jorge Cardoso Leitão, <jo...@gmail.com>
wrote:
> Hi,
>
> I would like to bring to this mailing list a proposal to donate the source
> code of arrow2 [1] and parquet2 [2] as experimental repositories [3] within
> Apache Arrow, conditional on IP clearance.
>
> The specific PRs are:
>
> * https://github.com/apache/arrow-experimental-rs-arrow2/pull/1
> * https://github.com/apache/arrow-experimental-rs-parquet2/pull/1
>
> The source code contains rewrites of the arrow and parquet crates with
> safety and security in mind. In particular,
>
> * no buffer transmutes
> * no unsafe APIs marked as safe
> * parquet's implementation is unsafe free
>
> There are many other important features, such as big endian support and IPC
> 2.0 support. There is one regression over latest: support nested types in
> parquet read and write. I observe no negative impact on performance.
>
> See a longer discussion in [4] over the reasons why the current rust
> implementation is susceptible to safety violations. In particular, many
> core APIs of the crate are considered security vulnerabilities under
> RustSec's [5] definitions, and are difficult to address on its current
> design.
>
> I validated that it is possible to migrate DataFusion [6] and Polars [7]
> without further code changes.
>
> The vote will be open for at least 72 hours.
>
> [ ] +1 Accept the code donation as experimental repos.
> [ ] +0
> [ ] -1 Do not accept the code donation as experimental repos because...
>
> [1]
>
> https://github.com/apache/arrow/blob/master/docs/source/developers/experimental_repos.rst
> [2] https://github.com/jorgecarleitao/arrow2
> [3] https://github.com/jorgecarleitao/parquet2
> [4] https://github.com/jorgecarleitao/arrow2#faq
> [5] https://rustsec.org/
> [6] https://github.com/apache/arrow-datafusion/pull/68
> [7] https://github.com/pola-rs/polars
>
Re: [VOTE] Donation of rust arrow2 and parquet2
Posted by Sutou Kouhei <ko...@clear-code.com>.
+1
In <CA...@mail.gmail.com>
"[VOTE] Donation of rust arrow2 and parquet2" on Sun, 27 Jun 2021 08:40:15 +0200,
Jorge Cardoso Leitão <jo...@gmail.com> wrote:
> Hi,
>
> I would like to bring to this mailing list a proposal to donate the source
> code of arrow2 [1] and parquet2 [2] as experimental repositories [3] within
> Apache Arrow, conditional on IP clearance.
>
> The specific PRs are:
>
> * https://github.com/apache/arrow-experimental-rs-arrow2/pull/1
> * https://github.com/apache/arrow-experimental-rs-parquet2/pull/1
>
> The source code contains rewrites of the arrow and parquet crates with
> safety and security in mind. In particular,
>
> * no buffer transmutes
> * no unsafe APIs marked as safe
> * parquet's implementation is unsafe free
>
> There are many other important features, such as big endian support and IPC
> 2.0 support. There is one regression over latest: support nested types in
> parquet read and write. I observe no negative impact on performance.
>
> See a longer discussion in [4] over the reasons why the current rust
> implementation is susceptible to safety violations. In particular, many
> core APIs of the crate are considered security vulnerabilities under
> RustSec's [5] definitions, and are difficult to address on its current
> design.
>
> I validated that it is possible to migrate DataFusion [6] and Polars [7]
> without further code changes.
>
> The vote will be open for at least 72 hours.
>
> [ ] +1 Accept the code donation as experimental repos.
> [ ] +0
> [ ] -1 Do not accept the code donation as experimental repos because...
>
> [1]
> https://github.com/apache/arrow/blob/master/docs/source/developers/experimental_repos.rst
> [2] https://github.com/jorgecarleitao/arrow2
> [3] https://github.com/jorgecarleitao/parquet2
> [4] https://github.com/jorgecarleitao/arrow2#faq
> [5] https://rustsec.org/
> [6] https://github.com/apache/arrow-datafusion/pull/68
> [7] https://github.com/pola-rs/polars
Re: [VOTE] Donation of rust arrow2 and parquet2
Posted by Krisztián Szűcs <sz...@gmail.com>.
+1 (binding)
On Mon, Jun 28, 2021 at 10:57 PM Neal Richardson
<ne...@gmail.com> wrote:
>
> +1
>
> On Mon, Jun 28, 2021 at 1:29 PM Andrew Lamb <al...@influxdata.com> wrote:
>
> > +1
> >
> > On Mon, Jun 28, 2021 at 1:13 PM QP Hou <qp...@scribd.com.invalid> wrote:
> >
> > > +1 (non binding)
> > >
> > > Really exciting stuff, amazing work Jorge.
> > >
> > > On Mon, Jun 28, 2021 at 8:32 AM Antoine Pitrou <an...@python.org>
> > wrote:
> > > >
> > > > +1 as well (binding)
> > > >
> > > >
> > > > Le 28/06/2021 à 17:28, Ben Kietzman a écrit :
> > > > > +1 (binding)
> > > > >
> > > > > On Mon, Jun 28, 2021 at 5:35 AM Wes McKinney <we...@gmail.com>
> > > wrote:
> > > > >
> > > > >> +1 (binding)
> > > > >>
> > > > >> On Mon, Jun 28, 2021 at 11:08 AM Daniël Heres <
> > danielheres@gmail.com>
> > > > >> wrote:
> > > > >>>
> > > > >>> +1 (non binding)
> > > > >>>
> > > > >>> Great work Jorge!
> > > > >>>
> > > > >>> On Mon, Jun 28, 2021, 10:26 Weston Steimel <
> > weston.steimel@gmail.com
> > > >
> > > > >> wrote:
> > > > >>>
> > > > >>>> +1
> > > > >>>>
> > > > >>>> On Sun, 27 Jun 2021, 07:41 Jorge Cardoso Leitão, <
> > > > >> jorgecarleitao@gmail.com
> > > > >>>>>
> > > > >>>> wrote:
> > > > >>>>
> > > > >>>>> Hi,
> > > > >>>>>
> > > > >>>>> I would like to bring to this mailing list a proposal to donate
> > the
> > > > >>>> source
> > > > >>>>> code of arrow2 [1] and parquet2 [2] as experimental repositories
> > > [3]
> > > > >>>> within
> > > > >>>>> Apache Arrow, conditional on IP clearance.
> > > > >>>>>
> > > > >>>>> The specific PRs are:
> > > > >>>>>
> > > > >>>>> * https://github.com/apache/arrow-experimental-rs-arrow2/pull/1
> > > > >>>>> *
> > https://github.com/apache/arrow-experimental-rs-parquet2/pull/1
> > > > >>>>>
> > > > >>>>> The source code contains rewrites of the arrow and parquet crates
> > > > >> with
> > > > >>>>> safety and security in mind. In particular,
> > > > >>>>>
> > > > >>>>> * no buffer transmutes
> > > > >>>>> * no unsafe APIs marked as safe
> > > > >>>>> * parquet's implementation is unsafe free
> > > > >>>>>
> > > > >>>>> There are many other important features, such as big endian
> > support
> > > > >> and
> > > > >>>> IPC
> > > > >>>>> 2.0 support. There is one regression over latest: support nested
> > > > >> types in
> > > > >>>>> parquet read and write. I observe no negative impact on
> > > performance.
> > > > >>>>>
> > > > >>>>> See a longer discussion in [4] over the reasons why the current
> > > rust
> > > > >>>>> implementation is susceptible to safety violations. In
> > particular,
> > > > >> many
> > > > >>>>> core APIs of the crate are considered security vulnerabilities
> > > under
> > > > >>>>> RustSec's [5] definitions, and are difficult to address on its
> > > > >> current
> > > > >>>>> design.
> > > > >>>>>
> > > > >>>>> I validated that it is possible to migrate DataFusion [6] and
> > > Polars
> > > > >> [7]
> > > > >>>>> without further code changes.
> > > > >>>>>
> > > > >>>>> The vote will be open for at least 72 hours.
> > > > >>>>>
> > > > >>>>> [ ] +1 Accept the code donation as experimental repos.
> > > > >>>>> [ ] +0
> > > > >>>>> [ ] -1 Do not accept the code donation as experimental repos
> > > > >> because...
> > > > >>>>>
> > > > >>>>> [1]
> > > > >>>>>
> > > > >>>>>
> > > > >>>>
> > > > >>
> > >
> > https://github.com/apache/arrow/blob/master/docs/source/developers/experimental_repos.rst
> > > > >>>>> [2] https://github.com/jorgecarleitao/arrow2
> > > > >>>>> [3] https://github.com/jorgecarleitao/parquet2
> > > > >>>>> [4] https://github.com/jorgecarleitao/arrow2#faq
> > > > >>>>> [5] https://rustsec.org/
> > > > >>>>> [6] https://github.com/apache/arrow-datafusion/pull/68
> > > > >>>>> [7] https://github.com/pola-rs/polars
> > > > >>>>>
> > > > >>>>
> > > > >>
> > > > >
> > >
> >
Re: [VOTE] Donation of rust arrow2 and parquet2
Posted by Neal Richardson <ne...@gmail.com>.
+1
On Mon, Jun 28, 2021 at 1:29 PM Andrew Lamb <al...@influxdata.com> wrote:
> +1
>
> On Mon, Jun 28, 2021 at 1:13 PM QP Hou <qp...@scribd.com.invalid> wrote:
>
> > +1 (non binding)
> >
> > Really exciting stuff, amazing work Jorge.
> >
> > On Mon, Jun 28, 2021 at 8:32 AM Antoine Pitrou <an...@python.org>
> wrote:
> > >
> > > +1 as well (binding)
> > >
> > >
> > > Le 28/06/2021 à 17:28, Ben Kietzman a écrit :
> > > > +1 (binding)
> > > >
> > > > On Mon, Jun 28, 2021 at 5:35 AM Wes McKinney <we...@gmail.com>
> > wrote:
> > > >
> > > >> +1 (binding)
> > > >>
> > > >> On Mon, Jun 28, 2021 at 11:08 AM Daniël Heres <
> danielheres@gmail.com>
> > > >> wrote:
> > > >>>
> > > >>> +1 (non binding)
> > > >>>
> > > >>> Great work Jorge!
> > > >>>
> > > >>> On Mon, Jun 28, 2021, 10:26 Weston Steimel <
> weston.steimel@gmail.com
> > >
> > > >> wrote:
> > > >>>
> > > >>>> +1
> > > >>>>
> > > >>>> On Sun, 27 Jun 2021, 07:41 Jorge Cardoso Leitão, <
> > > >> jorgecarleitao@gmail.com
> > > >>>>>
> > > >>>> wrote:
> > > >>>>
> > > >>>>> Hi,
> > > >>>>>
> > > >>>>> I would like to bring to this mailing list a proposal to donate
> the
> > > >>>> source
> > > >>>>> code of arrow2 [1] and parquet2 [2] as experimental repositories
> > [3]
> > > >>>> within
> > > >>>>> Apache Arrow, conditional on IP clearance.
> > > >>>>>
> > > >>>>> The specific PRs are:
> > > >>>>>
> > > >>>>> * https://github.com/apache/arrow-experimental-rs-arrow2/pull/1
> > > >>>>> *
> https://github.com/apache/arrow-experimental-rs-parquet2/pull/1
> > > >>>>>
> > > >>>>> The source code contains rewrites of the arrow and parquet crates
> > > >> with
> > > >>>>> safety and security in mind. In particular,
> > > >>>>>
> > > >>>>> * no buffer transmutes
> > > >>>>> * no unsafe APIs marked as safe
> > > >>>>> * parquet's implementation is unsafe free
> > > >>>>>
> > > >>>>> There are many other important features, such as big endian
> support
> > > >> and
> > > >>>> IPC
> > > >>>>> 2.0 support. There is one regression over latest: support nested
> > > >> types in
> > > >>>>> parquet read and write. I observe no negative impact on
> > performance.
> > > >>>>>
> > > >>>>> See a longer discussion in [4] over the reasons why the current
> > rust
> > > >>>>> implementation is susceptible to safety violations. In
> particular,
> > > >> many
> > > >>>>> core APIs of the crate are considered security vulnerabilities
> > under
> > > >>>>> RustSec's [5] definitions, and are difficult to address on its
> > > >> current
> > > >>>>> design.
> > > >>>>>
> > > >>>>> I validated that it is possible to migrate DataFusion [6] and
> > Polars
> > > >> [7]
> > > >>>>> without further code changes.
> > > >>>>>
> > > >>>>> The vote will be open for at least 72 hours.
> > > >>>>>
> > > >>>>> [ ] +1 Accept the code donation as experimental repos.
> > > >>>>> [ ] +0
> > > >>>>> [ ] -1 Do not accept the code donation as experimental repos
> > > >> because...
> > > >>>>>
> > > >>>>> [1]
> > > >>>>>
> > > >>>>>
> > > >>>>
> > > >>
> >
> https://github.com/apache/arrow/blob/master/docs/source/developers/experimental_repos.rst
> > > >>>>> [2] https://github.com/jorgecarleitao/arrow2
> > > >>>>> [3] https://github.com/jorgecarleitao/parquet2
> > > >>>>> [4] https://github.com/jorgecarleitao/arrow2#faq
> > > >>>>> [5] https://rustsec.org/
> > > >>>>> [6] https://github.com/apache/arrow-datafusion/pull/68
> > > >>>>> [7] https://github.com/pola-rs/polars
> > > >>>>>
> > > >>>>
> > > >>
> > > >
> >
>
Re: [VOTE] Donation of rust arrow2 and parquet2
Posted by Andrew Lamb <al...@influxdata.com>.
+1
On Mon, Jun 28, 2021 at 1:13 PM QP Hou <qp...@scribd.com.invalid> wrote:
> +1 (non binding)
>
> Really exciting stuff, amazing work Jorge.
>
> On Mon, Jun 28, 2021 at 8:32 AM Antoine Pitrou <an...@python.org> wrote:
> >
> > +1 as well (binding)
> >
> >
> > Le 28/06/2021 à 17:28, Ben Kietzman a écrit :
> > > +1 (binding)
> > >
> > > On Mon, Jun 28, 2021 at 5:35 AM Wes McKinney <we...@gmail.com>
> wrote:
> > >
> > >> +1 (binding)
> > >>
> > >> On Mon, Jun 28, 2021 at 11:08 AM Daniël Heres <da...@gmail.com>
> > >> wrote:
> > >>>
> > >>> +1 (non binding)
> > >>>
> > >>> Great work Jorge!
> > >>>
> > >>> On Mon, Jun 28, 2021, 10:26 Weston Steimel <weston.steimel@gmail.com
> >
> > >> wrote:
> > >>>
> > >>>> +1
> > >>>>
> > >>>> On Sun, 27 Jun 2021, 07:41 Jorge Cardoso Leitão, <
> > >> jorgecarleitao@gmail.com
> > >>>>>
> > >>>> wrote:
> > >>>>
> > >>>>> Hi,
> > >>>>>
> > >>>>> I would like to bring to this mailing list a proposal to donate the
> > >>>> source
> > >>>>> code of arrow2 [1] and parquet2 [2] as experimental repositories
> [3]
> > >>>> within
> > >>>>> Apache Arrow, conditional on IP clearance.
> > >>>>>
> > >>>>> The specific PRs are:
> > >>>>>
> > >>>>> * https://github.com/apache/arrow-experimental-rs-arrow2/pull/1
> > >>>>> * https://github.com/apache/arrow-experimental-rs-parquet2/pull/1
> > >>>>>
> > >>>>> The source code contains rewrites of the arrow and parquet crates
> > >> with
> > >>>>> safety and security in mind. In particular,
> > >>>>>
> > >>>>> * no buffer transmutes
> > >>>>> * no unsafe APIs marked as safe
> > >>>>> * parquet's implementation is unsafe free
> > >>>>>
> > >>>>> There are many other important features, such as big endian support
> > >> and
> > >>>> IPC
> > >>>>> 2.0 support. There is one regression over latest: support nested
> > >> types in
> > >>>>> parquet read and write. I observe no negative impact on
> performance.
> > >>>>>
> > >>>>> See a longer discussion in [4] over the reasons why the current
> rust
> > >>>>> implementation is susceptible to safety violations. In particular,
> > >> many
> > >>>>> core APIs of the crate are considered security vulnerabilities
> under
> > >>>>> RustSec's [5] definitions, and are difficult to address on its
> > >> current
> > >>>>> design.
> > >>>>>
> > >>>>> I validated that it is possible to migrate DataFusion [6] and
> Polars
> > >> [7]
> > >>>>> without further code changes.
> > >>>>>
> > >>>>> The vote will be open for at least 72 hours.
> > >>>>>
> > >>>>> [ ] +1 Accept the code donation as experimental repos.
> > >>>>> [ ] +0
> > >>>>> [ ] -1 Do not accept the code donation as experimental repos
> > >> because...
> > >>>>>
> > >>>>> [1]
> > >>>>>
> > >>>>>
> > >>>>
> > >>
> https://github.com/apache/arrow/blob/master/docs/source/developers/experimental_repos.rst
> > >>>>> [2] https://github.com/jorgecarleitao/arrow2
> > >>>>> [3] https://github.com/jorgecarleitao/parquet2
> > >>>>> [4] https://github.com/jorgecarleitao/arrow2#faq
> > >>>>> [5] https://rustsec.org/
> > >>>>> [6] https://github.com/apache/arrow-datafusion/pull/68
> > >>>>> [7] https://github.com/pola-rs/polars
> > >>>>>
> > >>>>
> > >>
> > >
>
Re: [VOTE] Donation of rust arrow2 and parquet2
Posted by QP Hou <qp...@scribd.com.INVALID>.
+1 (non binding)
Really exciting stuff, amazing work Jorge.
On Mon, Jun 28, 2021 at 8:32 AM Antoine Pitrou <an...@python.org> wrote:
>
> +1 as well (binding)
>
>
> Le 28/06/2021 à 17:28, Ben Kietzman a écrit :
> > +1 (binding)
> >
> > On Mon, Jun 28, 2021 at 5:35 AM Wes McKinney <we...@gmail.com> wrote:
> >
> >> +1 (binding)
> >>
> >> On Mon, Jun 28, 2021 at 11:08 AM Daniël Heres <da...@gmail.com>
> >> wrote:
> >>>
> >>> +1 (non binding)
> >>>
> >>> Great work Jorge!
> >>>
> >>> On Mon, Jun 28, 2021, 10:26 Weston Steimel <we...@gmail.com>
> >> wrote:
> >>>
> >>>> +1
> >>>>
> >>>> On Sun, 27 Jun 2021, 07:41 Jorge Cardoso Leitão, <
> >> jorgecarleitao@gmail.com
> >>>>>
> >>>> wrote:
> >>>>
> >>>>> Hi,
> >>>>>
> >>>>> I would like to bring to this mailing list a proposal to donate the
> >>>> source
> >>>>> code of arrow2 [1] and parquet2 [2] as experimental repositories [3]
> >>>> within
> >>>>> Apache Arrow, conditional on IP clearance.
> >>>>>
> >>>>> The specific PRs are:
> >>>>>
> >>>>> * https://github.com/apache/arrow-experimental-rs-arrow2/pull/1
> >>>>> * https://github.com/apache/arrow-experimental-rs-parquet2/pull/1
> >>>>>
> >>>>> The source code contains rewrites of the arrow and parquet crates
> >> with
> >>>>> safety and security in mind. In particular,
> >>>>>
> >>>>> * no buffer transmutes
> >>>>> * no unsafe APIs marked as safe
> >>>>> * parquet's implementation is unsafe free
> >>>>>
> >>>>> There are many other important features, such as big endian support
> >> and
> >>>> IPC
> >>>>> 2.0 support. There is one regression over latest: support nested
> >> types in
> >>>>> parquet read and write. I observe no negative impact on performance.
> >>>>>
> >>>>> See a longer discussion in [4] over the reasons why the current rust
> >>>>> implementation is susceptible to safety violations. In particular,
> >> many
> >>>>> core APIs of the crate are considered security vulnerabilities under
> >>>>> RustSec's [5] definitions, and are difficult to address on its
> >> current
> >>>>> design.
> >>>>>
> >>>>> I validated that it is possible to migrate DataFusion [6] and Polars
> >> [7]
> >>>>> without further code changes.
> >>>>>
> >>>>> The vote will be open for at least 72 hours.
> >>>>>
> >>>>> [ ] +1 Accept the code donation as experimental repos.
> >>>>> [ ] +0
> >>>>> [ ] -1 Do not accept the code donation as experimental repos
> >> because...
> >>>>>
> >>>>> [1]
> >>>>>
> >>>>>
> >>>>
> >> https://github.com/apache/arrow/blob/master/docs/source/developers/experimental_repos.rst
> >>>>> [2] https://github.com/jorgecarleitao/arrow2
> >>>>> [3] https://github.com/jorgecarleitao/parquet2
> >>>>> [4] https://github.com/jorgecarleitao/arrow2#faq
> >>>>> [5] https://rustsec.org/
> >>>>> [6] https://github.com/apache/arrow-datafusion/pull/68
> >>>>> [7] https://github.com/pola-rs/polars
> >>>>>
> >>>>
> >>
> >
Re: [VOTE] Donation of rust arrow2 and parquet2
Posted by Antoine Pitrou <an...@python.org>.
+1 as well (binding)
Le 28/06/2021 à 17:28, Ben Kietzman a écrit :
> +1 (binding)
>
> On Mon, Jun 28, 2021 at 5:35 AM Wes McKinney <we...@gmail.com> wrote:
>
>> +1 (binding)
>>
>> On Mon, Jun 28, 2021 at 11:08 AM Daniël Heres <da...@gmail.com>
>> wrote:
>>>
>>> +1 (non binding)
>>>
>>> Great work Jorge!
>>>
>>> On Mon, Jun 28, 2021, 10:26 Weston Steimel <we...@gmail.com>
>> wrote:
>>>
>>>> +1
>>>>
>>>> On Sun, 27 Jun 2021, 07:41 Jorge Cardoso Leitão, <
>> jorgecarleitao@gmail.com
>>>>>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I would like to bring to this mailing list a proposal to donate the
>>>> source
>>>>> code of arrow2 [1] and parquet2 [2] as experimental repositories [3]
>>>> within
>>>>> Apache Arrow, conditional on IP clearance.
>>>>>
>>>>> The specific PRs are:
>>>>>
>>>>> * https://github.com/apache/arrow-experimental-rs-arrow2/pull/1
>>>>> * https://github.com/apache/arrow-experimental-rs-parquet2/pull/1
>>>>>
>>>>> The source code contains rewrites of the arrow and parquet crates
>> with
>>>>> safety and security in mind. In particular,
>>>>>
>>>>> * no buffer transmutes
>>>>> * no unsafe APIs marked as safe
>>>>> * parquet's implementation is unsafe free
>>>>>
>>>>> There are many other important features, such as big endian support
>> and
>>>> IPC
>>>>> 2.0 support. There is one regression over latest: support nested
>> types in
>>>>> parquet read and write. I observe no negative impact on performance.
>>>>>
>>>>> See a longer discussion in [4] over the reasons why the current rust
>>>>> implementation is susceptible to safety violations. In particular,
>> many
>>>>> core APIs of the crate are considered security vulnerabilities under
>>>>> RustSec's [5] definitions, and are difficult to address on its
>> current
>>>>> design.
>>>>>
>>>>> I validated that it is possible to migrate DataFusion [6] and Polars
>> [7]
>>>>> without further code changes.
>>>>>
>>>>> The vote will be open for at least 72 hours.
>>>>>
>>>>> [ ] +1 Accept the code donation as experimental repos.
>>>>> [ ] +0
>>>>> [ ] -1 Do not accept the code donation as experimental repos
>> because...
>>>>>
>>>>> [1]
>>>>>
>>>>>
>>>>
>> https://github.com/apache/arrow/blob/master/docs/source/developers/experimental_repos.rst
>>>>> [2] https://github.com/jorgecarleitao/arrow2
>>>>> [3] https://github.com/jorgecarleitao/parquet2
>>>>> [4] https://github.com/jorgecarleitao/arrow2#faq
>>>>> [5] https://rustsec.org/
>>>>> [6] https://github.com/apache/arrow-datafusion/pull/68
>>>>> [7] https://github.com/pola-rs/polars
>>>>>
>>>>
>>
>
Re: [VOTE] Donation of rust arrow2 and parquet2
Posted by Ben Kietzman <be...@ursacomputing.com>.
+1 (binding)
On Mon, Jun 28, 2021 at 5:35 AM Wes McKinney <we...@gmail.com> wrote:
> +1 (binding)
>
> On Mon, Jun 28, 2021 at 11:08 AM Daniël Heres <da...@gmail.com>
> wrote:
> >
> > +1 (non binding)
> >
> > Great work Jorge!
> >
> > On Mon, Jun 28, 2021, 10:26 Weston Steimel <we...@gmail.com>
> wrote:
> >
> > > +1
> > >
> > > On Sun, 27 Jun 2021, 07:41 Jorge Cardoso Leitão, <
> jorgecarleitao@gmail.com
> > > >
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > I would like to bring to this mailing list a proposal to donate the
> > > source
> > > > code of arrow2 [1] and parquet2 [2] as experimental repositories [3]
> > > within
> > > > Apache Arrow, conditional on IP clearance.
> > > >
> > > > The specific PRs are:
> > > >
> > > > * https://github.com/apache/arrow-experimental-rs-arrow2/pull/1
> > > > * https://github.com/apache/arrow-experimental-rs-parquet2/pull/1
> > > >
> > > > The source code contains rewrites of the arrow and parquet crates
> with
> > > > safety and security in mind. In particular,
> > > >
> > > > * no buffer transmutes
> > > > * no unsafe APIs marked as safe
> > > > * parquet's implementation is unsafe free
> > > >
> > > > There are many other important features, such as big endian support
> and
> > > IPC
> > > > 2.0 support. There is one regression over latest: support nested
> types in
> > > > parquet read and write. I observe no negative impact on performance.
> > > >
> > > > See a longer discussion in [4] over the reasons why the current rust
> > > > implementation is susceptible to safety violations. In particular,
> many
> > > > core APIs of the crate are considered security vulnerabilities under
> > > > RustSec's [5] definitions, and are difficult to address on its
> current
> > > > design.
> > > >
> > > > I validated that it is possible to migrate DataFusion [6] and Polars
> [7]
> > > > without further code changes.
> > > >
> > > > The vote will be open for at least 72 hours.
> > > >
> > > > [ ] +1 Accept the code donation as experimental repos.
> > > > [ ] +0
> > > > [ ] -1 Do not accept the code donation as experimental repos
> because...
> > > >
> > > > [1]
> > > >
> > > >
> > >
> https://github.com/apache/arrow/blob/master/docs/source/developers/experimental_repos.rst
> > > > [2] https://github.com/jorgecarleitao/arrow2
> > > > [3] https://github.com/jorgecarleitao/parquet2
> > > > [4] https://github.com/jorgecarleitao/arrow2#faq
> > > > [5] https://rustsec.org/
> > > > [6] https://github.com/apache/arrow-datafusion/pull/68
> > > > [7] https://github.com/pola-rs/polars
> > > >
> > >
>
Re: [VOTE] Donation of rust arrow2 and parquet2
Posted by Wes McKinney <we...@gmail.com>.
+1 (binding)
On Mon, Jun 28, 2021 at 11:08 AM Daniël Heres <da...@gmail.com> wrote:
>
> +1 (non binding)
>
> Great work Jorge!
>
> On Mon, Jun 28, 2021, 10:26 Weston Steimel <we...@gmail.com> wrote:
>
> > +1
> >
> > On Sun, 27 Jun 2021, 07:41 Jorge Cardoso Leitão, <jorgecarleitao@gmail.com
> > >
> > wrote:
> >
> > > Hi,
> > >
> > > I would like to bring to this mailing list a proposal to donate the
> > source
> > > code of arrow2 [1] and parquet2 [2] as experimental repositories [3]
> > within
> > > Apache Arrow, conditional on IP clearance.
> > >
> > > The specific PRs are:
> > >
> > > * https://github.com/apache/arrow-experimental-rs-arrow2/pull/1
> > > * https://github.com/apache/arrow-experimental-rs-parquet2/pull/1
> > >
> > > The source code contains rewrites of the arrow and parquet crates with
> > > safety and security in mind. In particular,
> > >
> > > * no buffer transmutes
> > > * no unsafe APIs marked as safe
> > > * parquet's implementation is unsafe free
> > >
> > > There are many other important features, such as big endian support and
> > IPC
> > > 2.0 support. There is one regression over latest: support nested types in
> > > parquet read and write. I observe no negative impact on performance.
> > >
> > > See a longer discussion in [4] over the reasons why the current rust
> > > implementation is susceptible to safety violations. In particular, many
> > > core APIs of the crate are considered security vulnerabilities under
> > > RustSec's [5] definitions, and are difficult to address on its current
> > > design.
> > >
> > > I validated that it is possible to migrate DataFusion [6] and Polars [7]
> > > without further code changes.
> > >
> > > The vote will be open for at least 72 hours.
> > >
> > > [ ] +1 Accept the code donation as experimental repos.
> > > [ ] +0
> > > [ ] -1 Do not accept the code donation as experimental repos because...
> > >
> > > [1]
> > >
> > >
> > https://github.com/apache/arrow/blob/master/docs/source/developers/experimental_repos.rst
> > > [2] https://github.com/jorgecarleitao/arrow2
> > > [3] https://github.com/jorgecarleitao/parquet2
> > > [4] https://github.com/jorgecarleitao/arrow2#faq
> > > [5] https://rustsec.org/
> > > [6] https://github.com/apache/arrow-datafusion/pull/68
> > > [7] https://github.com/pola-rs/polars
> > >
> >
Re: [VOTE] Donation of rust arrow2 and parquet2
Posted by Daniël Heres <da...@gmail.com>.
+1 (non binding)
Great work Jorge!
On Mon, Jun 28, 2021, 10:26 Weston Steimel <we...@gmail.com> wrote:
> +1
>
> On Sun, 27 Jun 2021, 07:41 Jorge Cardoso Leitão, <jorgecarleitao@gmail.com
> >
> wrote:
>
> > Hi,
> >
> > I would like to bring to this mailing list a proposal to donate the
> source
> > code of arrow2 [1] and parquet2 [2] as experimental repositories [3]
> within
> > Apache Arrow, conditional on IP clearance.
> >
> > The specific PRs are:
> >
> > * https://github.com/apache/arrow-experimental-rs-arrow2/pull/1
> > * https://github.com/apache/arrow-experimental-rs-parquet2/pull/1
> >
> > The source code contains rewrites of the arrow and parquet crates with
> > safety and security in mind. In particular,
> >
> > * no buffer transmutes
> > * no unsafe APIs marked as safe
> > * parquet's implementation is unsafe free
> >
> > There are many other important features, such as big endian support and
> IPC
> > 2.0 support. There is one regression over latest: support nested types in
> > parquet read and write. I observe no negative impact on performance.
> >
> > See a longer discussion in [4] over the reasons why the current rust
> > implementation is susceptible to safety violations. In particular, many
> > core APIs of the crate are considered security vulnerabilities under
> > RustSec's [5] definitions, and are difficult to address on its current
> > design.
> >
> > I validated that it is possible to migrate DataFusion [6] and Polars [7]
> > without further code changes.
> >
> > The vote will be open for at least 72 hours.
> >
> > [ ] +1 Accept the code donation as experimental repos.
> > [ ] +0
> > [ ] -1 Do not accept the code donation as experimental repos because...
> >
> > [1]
> >
> >
> https://github.com/apache/arrow/blob/master/docs/source/developers/experimental_repos.rst
> > [2] https://github.com/jorgecarleitao/arrow2
> > [3] https://github.com/jorgecarleitao/parquet2
> > [4] https://github.com/jorgecarleitao/arrow2#faq
> > [5] https://rustsec.org/
> > [6] https://github.com/apache/arrow-datafusion/pull/68
> > [7] https://github.com/pola-rs/polars
> >
>
Re: [VOTE] Donation of rust arrow2 and parquet2
Posted by Weston Steimel <we...@gmail.com>.
+1
On Sun, 27 Jun 2021, 07:41 Jorge Cardoso Leitão, <jo...@gmail.com>
wrote:
> Hi,
>
> I would like to bring to this mailing list a proposal to donate the source
> code of arrow2 [1] and parquet2 [2] as experimental repositories [3] within
> Apache Arrow, conditional on IP clearance.
>
> The specific PRs are:
>
> * https://github.com/apache/arrow-experimental-rs-arrow2/pull/1
> * https://github.com/apache/arrow-experimental-rs-parquet2/pull/1
>
> The source code contains rewrites of the arrow and parquet crates with
> safety and security in mind. In particular,
>
> * no buffer transmutes
> * no unsafe APIs marked as safe
> * parquet's implementation is unsafe free
>
> There are many other important features, such as big endian support and IPC
> 2.0 support. There is one regression over latest: support nested types in
> parquet read and write. I observe no negative impact on performance.
>
> See a longer discussion in [4] over the reasons why the current rust
> implementation is susceptible to safety violations. In particular, many
> core APIs of the crate are considered security vulnerabilities under
> RustSec's [5] definitions, and are difficult to address on its current
> design.
>
> I validated that it is possible to migrate DataFusion [6] and Polars [7]
> without further code changes.
>
> The vote will be open for at least 72 hours.
>
> [ ] +1 Accept the code donation as experimental repos.
> [ ] +0
> [ ] -1 Do not accept the code donation as experimental repos because...
>
> [1]
>
> https://github.com/apache/arrow/blob/master/docs/source/developers/experimental_repos.rst
> [2] https://github.com/jorgecarleitao/arrow2
> [3] https://github.com/jorgecarleitao/parquet2
> [4] https://github.com/jorgecarleitao/arrow2#faq
> [5] https://rustsec.org/
> [6] https://github.com/apache/arrow-datafusion/pull/68
> [7] https://github.com/pola-rs/polars
>