You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Wes McKinney <we...@gmail.com> on 2017/07/20 18:14:58 UTC
[VOTE] Accept contribution of Plasma Object Store
Dear all,
The Plasma Object Store provides a server process, reference C++ client, and
Python binding for managing a collection of binary "objects" in POSIX shared
memory. Applications use a lightweight messaging protocol to create and delete
memory blocks in the object store, evict objects to make room for new objects,
and increment and decrement reference counts to indicate shared ownership of
memory. It also provides for subscribing to notifications about object
activity. The system helps simplify ownership transfer and memory lifetime of
shared memory blocks, which can be much more complicated in a peer-to-peer
architecture.
The object store has been used in conjunction with the Apache Arrow libraries
to provide for zero-copy access to collections of large objects stored in
shared memory. Incorporating this project into Apache Arrow will help the
community continue to develop and innovate technology for low-overhead sharing
of complex datasets across multiple processes.
Plasma Object Store was developed by the Ray project with the UC Berkeley
RISELab. There have been 8 contributors, with about 3.1 KLOC of C++ code and an
additional 5.1 KLOC of thirdparty C and C++ code which we have reviewed for
compatibility with the Apache Software Foundation's policies on license
compatibility.
This code was split off from the Ray project from commit id
`b94b4a35e04d8d2c0af4420518a4e9a94c1c9b9f` [1] and modified by the authors for
inclusion in Apache Arrow in a GitHub pull request [2]. This code has been
staged in a separate repository for review by the community and ASF IP
Clearance:
- https://github.com/ray-project/arrow-plasma-object-store/tree/11795753b0850cf5ad50d640067a8517ad8629a2#diff-69e56fcedf1b794992b790684902dcd4
This vote is to determine whether the Arrow PMC is in favor of accepting the
code contribution. If the vote passes, the PMC and the authors of the code will
work together to complete the ASF IP Clearance process and import the Plasma
Object Store into Apache Arrow for inclusion in a future release:
[ ] +1 : Accept contribution of Plasma Object Store
[ ] 0 : No opinion
[ ] -1 : Reject contribution because...
The vote is open for 72 hours and will close at 18:15 UTC on Sunday 23 July
2017 and the results will be announced on this list.
Thanks,
Wes
[1]: https://github.com/ray-project/ray/commit/b94b4a35e04d8d2c0af4420518a4e9a94c1c9b9f
[2]: https://github.com/apache/arrow/pull/742
Re: [VOTE] Accept contribution of Plasma Object Store
Posted by "Gang(Gary) Wang" <ga...@apache.org>.
+1, Thanks!
Gary.
On Thu, Jul 20, 2017 at 11:16 AM, Wes McKinney <we...@gmail.com> wrote:
> +1
>
> On Thu, Jul 20, 2017 at 2:14 PM, Wes McKinney <we...@gmail.com> wrote:
> > Dear all,
> >
> > The Plasma Object Store provides a server process, reference C++ client,
> and
> > Python binding for managing a collection of binary "objects" in POSIX
> shared
> > memory. Applications use a lightweight messaging protocol to create and
> delete
> > memory blocks in the object store, evict objects to make room for new
> objects,
> > and increment and decrement reference counts to indicate shared
> ownership of
> > memory. It also provides for subscribing to notifications about object
> > activity. The system helps simplify ownership transfer and memory
> lifetime of
> > shared memory blocks, which can be much more complicated in a
> peer-to-peer
> > architecture.
> >
> > The object store has been used in conjunction with the Apache Arrow
> libraries
> > to provide for zero-copy access to collections of large objects stored in
> > shared memory. Incorporating this project into Apache Arrow will help the
> > community continue to develop and innovate technology for low-overhead
> sharing
> > of complex datasets across multiple processes.
> >
> > Plasma Object Store was developed by the Ray project with the UC Berkeley
> > RISELab. There have been 8 contributors, with about 3.1 KLOC of C++ code
> and an
> > additional 5.1 KLOC of thirdparty C and C++ code which we have reviewed
> for
> > compatibility with the Apache Software Foundation's policies on license
> > compatibility.
> >
> > This code was split off from the Ray project from commit id
> > `b94b4a35e04d8d2c0af4420518a4e9a94c1c9b9f` [1] and modified by the
> authors for
> > inclusion in Apache Arrow in a GitHub pull request [2]. This code has
> been
> > staged in a separate repository for review by the community and ASF IP
> > Clearance:
> >
> > - https://github.com/ray-project/arrow-plasma-object-store/tree/
> 11795753b0850cf5ad50d640067a8517ad8629a2#diff-
> 69e56fcedf1b794992b790684902dcd4
> >
> > This vote is to determine whether the Arrow PMC is in favor of accepting
> the
> > code contribution. If the vote passes, the PMC and the authors of the
> code will
> > work together to complete the ASF IP Clearance process and import the
> Plasma
> > Object Store into Apache Arrow for inclusion in a future release:
> >
> > [ ] +1 : Accept contribution of Plasma Object Store
> > [ ] 0 : No opinion
> > [ ] -1 : Reject contribution because...
> >
> > The vote is open for 72 hours and will close at 18:15 UTC on Sunday 23
> July
> > 2017 and the results will be announced on this list.
> >
> > Thanks,
> > Wes
> >
> > [1]: https://github.com/ray-project/ray/commit/
> b94b4a35e04d8d2c0af4420518a4e9a94c1c9b9f
> > [2]: https://github.com/apache/arrow/pull/742
>
Re: [VOTE] Accept contribution of Plasma Object Store
Posted by Wes McKinney <we...@gmail.com>.
+1
On Thu, Jul 20, 2017 at 2:14 PM, Wes McKinney <we...@gmail.com> wrote:
> Dear all,
>
> The Plasma Object Store provides a server process, reference C++ client, and
> Python binding for managing a collection of binary "objects" in POSIX shared
> memory. Applications use a lightweight messaging protocol to create and delete
> memory blocks in the object store, evict objects to make room for new objects,
> and increment and decrement reference counts to indicate shared ownership of
> memory. It also provides for subscribing to notifications about object
> activity. The system helps simplify ownership transfer and memory lifetime of
> shared memory blocks, which can be much more complicated in a peer-to-peer
> architecture.
>
> The object store has been used in conjunction with the Apache Arrow libraries
> to provide for zero-copy access to collections of large objects stored in
> shared memory. Incorporating this project into Apache Arrow will help the
> community continue to develop and innovate technology for low-overhead sharing
> of complex datasets across multiple processes.
>
> Plasma Object Store was developed by the Ray project with the UC Berkeley
> RISELab. There have been 8 contributors, with about 3.1 KLOC of C++ code and an
> additional 5.1 KLOC of thirdparty C and C++ code which we have reviewed for
> compatibility with the Apache Software Foundation's policies on license
> compatibility.
>
> This code was split off from the Ray project from commit id
> `b94b4a35e04d8d2c0af4420518a4e9a94c1c9b9f` [1] and modified by the authors for
> inclusion in Apache Arrow in a GitHub pull request [2]. This code has been
> staged in a separate repository for review by the community and ASF IP
> Clearance:
>
> - https://github.com/ray-project/arrow-plasma-object-store/tree/11795753b0850cf5ad50d640067a8517ad8629a2#diff-69e56fcedf1b794992b790684902dcd4
>
> This vote is to determine whether the Arrow PMC is in favor of accepting the
> code contribution. If the vote passes, the PMC and the authors of the code will
> work together to complete the ASF IP Clearance process and import the Plasma
> Object Store into Apache Arrow for inclusion in a future release:
>
> [ ] +1 : Accept contribution of Plasma Object Store
> [ ] 0 : No opinion
> [ ] -1 : Reject contribution because...
>
> The vote is open for 72 hours and will close at 18:15 UTC on Sunday 23 July
> 2017 and the results will be announced on this list.
>
> Thanks,
> Wes
>
> [1]: https://github.com/ray-project/ray/commit/b94b4a35e04d8d2c0af4420518a4e9a94c1c9b9f
> [2]: https://github.com/apache/arrow/pull/742
Re: [VOTE] Accept contribution of Plasma Object Store
Posted by Julien Le Dem <ju...@gmail.com>.
+1
On Sun, Jul 23, 2017 at 8:00 AM, Arun K. Subramaniyan <sarunkarthi@gmail.com
> wrote:
> +1
>
> On Sun, Jul 23, 2017 at 1:16 AM Uwe L. Korn <uw...@xhochy.com> wrote:
>
> > +1
> >
> > On Fri, Jul 21, 2017, at 01:37 AM, Julian Hyde wrote:
> > > +1
> > >
> > > > On Jul 20, 2017, at 3:07 PM, Bryan Cutler <cu...@gmail.com> wrote:
> > > >
> > > > +1 sounds great!
> > > >
> > > > On Thu, Jul 20, 2017 at 11:14 AM, Wes McKinney <we...@gmail.com>
> > wrote:
> > > >
> > > >> Dear all,
> > > >>
> > > >> The Plasma Object Store provides a server process, reference C++
> > client,
> > > >> and
> > > >> Python binding for managing a collection of binary "objects" in
> POSIX
> > > >> shared
> > > >> memory. Applications use a lightweight messaging protocol to create
> > and
> > > >> delete
> > > >> memory blocks in the object store, evict objects to make room for
> new
> > > >> objects,
> > > >> and increment and decrement reference counts to indicate shared
> > ownership
> > > >> of
> > > >> memory. It also provides for subscribing to notifications about
> object
> > > >> activity. The system helps simplify ownership transfer and memory
> > lifetime
> > > >> of
> > > >> shared memory blocks, which can be much more complicated in a
> > peer-to-peer
> > > >> architecture.
> > > >>
> > > >> The object store has been used in conjunction with the Apache Arrow
> > > >> libraries
> > > >> to provide for zero-copy access to collections of large objects
> > stored in
> > > >> shared memory. Incorporating this project into Apache Arrow will
> help
> > the
> > > >> community continue to develop and innovate technology for
> low-overhead
> > > >> sharing
> > > >> of complex datasets across multiple processes.
> > > >>
> > > >> Plasma Object Store was developed by the Ray project with the UC
> > Berkeley
> > > >> RISELab. There have been 8 contributors, with about 3.1 KLOC of C++
> > code
> > > >> and an
> > > >> additional 5.1 KLOC of thirdparty C and C++ code which we have
> > reviewed for
> > > >> compatibility with the Apache Software Foundation's policies on
> > license
> > > >> compatibility.
> > > >>
> > > >> This code was split off from the Ray project from commit id
> > > >> `b94b4a35e04d8d2c0af4420518a4e9a94c1c9b9f` [1] and modified by the
> > > >> authors for
> > > >> inclusion in Apache Arrow in a GitHub pull request [2]. This code
> has
> > been
> > > >> staged in a separate repository for review by the community and ASF
> IP
> > > >> Clearance:
> > > >>
> > > >> - https://github.com/ray-project/arrow-plasma-object-store/tree/
> > > >> 11795753b0850cf5ad50d640067a8517ad8629a2#diff-
> > > >> 69e56fcedf1b794992b790684902dcd4
> > > >>
> > > >> This vote is to determine whether the Arrow PMC is in favor of
> > accepting
> > > >> the
> > > >> code contribution. If the vote passes, the PMC and the authors of
> the
> > code
> > > >> will
> > > >> work together to complete the ASF IP Clearance process and import
> the
> > > >> Plasma
> > > >> Object Store into Apache Arrow for inclusion in a future release:
> > > >>
> > > >> [ ] +1 : Accept contribution of Plasma Object Store
> > > >> [ ] 0 : No opinion
> > > >> [ ] -1 : Reject contribution because...
> > > >>
> > > >> The vote is open for 72 hours and will close at 18:15 UTC on Sunday
> > 23 July
> > > >> 2017 and the results will be announced on this list.
> > > >>
> > > >> Thanks,
> > > >> Wes
> > > >>
> > > >> [1]: https://github.com/ray-project/ray/commit/
> > > >> b94b4a35e04d8d2c0af4420518a4e9a94c1c9b9f
> > > >> [2]: https://github.com/apache/arrow/pull/742
> > > >>
> > >
> >
>
Re: [VOTE] Accept contribution of Plasma Object Store
Posted by "Arun K. Subramaniyan" <sa...@gmail.com>.
+1
On Sun, Jul 23, 2017 at 1:16 AM Uwe L. Korn <uw...@xhochy.com> wrote:
> +1
>
> On Fri, Jul 21, 2017, at 01:37 AM, Julian Hyde wrote:
> > +1
> >
> > > On Jul 20, 2017, at 3:07 PM, Bryan Cutler <cu...@gmail.com> wrote:
> > >
> > > +1 sounds great!
> > >
> > > On Thu, Jul 20, 2017 at 11:14 AM, Wes McKinney <we...@gmail.com>
> wrote:
> > >
> > >> Dear all,
> > >>
> > >> The Plasma Object Store provides a server process, reference C++
> client,
> > >> and
> > >> Python binding for managing a collection of binary "objects" in POSIX
> > >> shared
> > >> memory. Applications use a lightweight messaging protocol to create
> and
> > >> delete
> > >> memory blocks in the object store, evict objects to make room for new
> > >> objects,
> > >> and increment and decrement reference counts to indicate shared
> ownership
> > >> of
> > >> memory. It also provides for subscribing to notifications about object
> > >> activity. The system helps simplify ownership transfer and memory
> lifetime
> > >> of
> > >> shared memory blocks, which can be much more complicated in a
> peer-to-peer
> > >> architecture.
> > >>
> > >> The object store has been used in conjunction with the Apache Arrow
> > >> libraries
> > >> to provide for zero-copy access to collections of large objects
> stored in
> > >> shared memory. Incorporating this project into Apache Arrow will help
> the
> > >> community continue to develop and innovate technology for low-overhead
> > >> sharing
> > >> of complex datasets across multiple processes.
> > >>
> > >> Plasma Object Store was developed by the Ray project with the UC
> Berkeley
> > >> RISELab. There have been 8 contributors, with about 3.1 KLOC of C++
> code
> > >> and an
> > >> additional 5.1 KLOC of thirdparty C and C++ code which we have
> reviewed for
> > >> compatibility with the Apache Software Foundation's policies on
> license
> > >> compatibility.
> > >>
> > >> This code was split off from the Ray project from commit id
> > >> `b94b4a35e04d8d2c0af4420518a4e9a94c1c9b9f` [1] and modified by the
> > >> authors for
> > >> inclusion in Apache Arrow in a GitHub pull request [2]. This code has
> been
> > >> staged in a separate repository for review by the community and ASF IP
> > >> Clearance:
> > >>
> > >> - https://github.com/ray-project/arrow-plasma-object-store/tree/
> > >> 11795753b0850cf5ad50d640067a8517ad8629a2#diff-
> > >> 69e56fcedf1b794992b790684902dcd4
> > >>
> > >> This vote is to determine whether the Arrow PMC is in favor of
> accepting
> > >> the
> > >> code contribution. If the vote passes, the PMC and the authors of the
> code
> > >> will
> > >> work together to complete the ASF IP Clearance process and import the
> > >> Plasma
> > >> Object Store into Apache Arrow for inclusion in a future release:
> > >>
> > >> [ ] +1 : Accept contribution of Plasma Object Store
> > >> [ ] 0 : No opinion
> > >> [ ] -1 : Reject contribution because...
> > >>
> > >> The vote is open for 72 hours and will close at 18:15 UTC on Sunday
> 23 July
> > >> 2017 and the results will be announced on this list.
> > >>
> > >> Thanks,
> > >> Wes
> > >>
> > >> [1]: https://github.com/ray-project/ray/commit/
> > >> b94b4a35e04d8d2c0af4420518a4e9a94c1c9b9f
> > >> [2]: https://github.com/apache/arrow/pull/742
> > >>
> >
>
Re: [VOTE] Accept contribution of Plasma Object Store
Posted by "Uwe L. Korn" <uw...@xhochy.com>.
+1
On Fri, Jul 21, 2017, at 01:37 AM, Julian Hyde wrote:
> +1
>
> > On Jul 20, 2017, at 3:07 PM, Bryan Cutler <cu...@gmail.com> wrote:
> >
> > +1 sounds great!
> >
> > On Thu, Jul 20, 2017 at 11:14 AM, Wes McKinney <we...@gmail.com> wrote:
> >
> >> Dear all,
> >>
> >> The Plasma Object Store provides a server process, reference C++ client,
> >> and
> >> Python binding for managing a collection of binary "objects" in POSIX
> >> shared
> >> memory. Applications use a lightweight messaging protocol to create and
> >> delete
> >> memory blocks in the object store, evict objects to make room for new
> >> objects,
> >> and increment and decrement reference counts to indicate shared ownership
> >> of
> >> memory. It also provides for subscribing to notifications about object
> >> activity. The system helps simplify ownership transfer and memory lifetime
> >> of
> >> shared memory blocks, which can be much more complicated in a peer-to-peer
> >> architecture.
> >>
> >> The object store has been used in conjunction with the Apache Arrow
> >> libraries
> >> to provide for zero-copy access to collections of large objects stored in
> >> shared memory. Incorporating this project into Apache Arrow will help the
> >> community continue to develop and innovate technology for low-overhead
> >> sharing
> >> of complex datasets across multiple processes.
> >>
> >> Plasma Object Store was developed by the Ray project with the UC Berkeley
> >> RISELab. There have been 8 contributors, with about 3.1 KLOC of C++ code
> >> and an
> >> additional 5.1 KLOC of thirdparty C and C++ code which we have reviewed for
> >> compatibility with the Apache Software Foundation's policies on license
> >> compatibility.
> >>
> >> This code was split off from the Ray project from commit id
> >> `b94b4a35e04d8d2c0af4420518a4e9a94c1c9b9f` [1] and modified by the
> >> authors for
> >> inclusion in Apache Arrow in a GitHub pull request [2]. This code has been
> >> staged in a separate repository for review by the community and ASF IP
> >> Clearance:
> >>
> >> - https://github.com/ray-project/arrow-plasma-object-store/tree/
> >> 11795753b0850cf5ad50d640067a8517ad8629a2#diff-
> >> 69e56fcedf1b794992b790684902dcd4
> >>
> >> This vote is to determine whether the Arrow PMC is in favor of accepting
> >> the
> >> code contribution. If the vote passes, the PMC and the authors of the code
> >> will
> >> work together to complete the ASF IP Clearance process and import the
> >> Plasma
> >> Object Store into Apache Arrow for inclusion in a future release:
> >>
> >> [ ] +1 : Accept contribution of Plasma Object Store
> >> [ ] 0 : No opinion
> >> [ ] -1 : Reject contribution because...
> >>
> >> The vote is open for 72 hours and will close at 18:15 UTC on Sunday 23 July
> >> 2017 and the results will be announced on this list.
> >>
> >> Thanks,
> >> Wes
> >>
> >> [1]: https://github.com/ray-project/ray/commit/
> >> b94b4a35e04d8d2c0af4420518a4e9a94c1c9b9f
> >> [2]: https://github.com/apache/arrow/pull/742
> >>
>
Re: [VOTE] Accept contribution of Plasma Object Store
Posted by Julian Hyde <jh...@apache.org>.
+1
> On Jul 20, 2017, at 3:07 PM, Bryan Cutler <cu...@gmail.com> wrote:
>
> +1 sounds great!
>
> On Thu, Jul 20, 2017 at 11:14 AM, Wes McKinney <we...@gmail.com> wrote:
>
>> Dear all,
>>
>> The Plasma Object Store provides a server process, reference C++ client,
>> and
>> Python binding for managing a collection of binary "objects" in POSIX
>> shared
>> memory. Applications use a lightweight messaging protocol to create and
>> delete
>> memory blocks in the object store, evict objects to make room for new
>> objects,
>> and increment and decrement reference counts to indicate shared ownership
>> of
>> memory. It also provides for subscribing to notifications about object
>> activity. The system helps simplify ownership transfer and memory lifetime
>> of
>> shared memory blocks, which can be much more complicated in a peer-to-peer
>> architecture.
>>
>> The object store has been used in conjunction with the Apache Arrow
>> libraries
>> to provide for zero-copy access to collections of large objects stored in
>> shared memory. Incorporating this project into Apache Arrow will help the
>> community continue to develop and innovate technology for low-overhead
>> sharing
>> of complex datasets across multiple processes.
>>
>> Plasma Object Store was developed by the Ray project with the UC Berkeley
>> RISELab. There have been 8 contributors, with about 3.1 KLOC of C++ code
>> and an
>> additional 5.1 KLOC of thirdparty C and C++ code which we have reviewed for
>> compatibility with the Apache Software Foundation's policies on license
>> compatibility.
>>
>> This code was split off from the Ray project from commit id
>> `b94b4a35e04d8d2c0af4420518a4e9a94c1c9b9f` [1] and modified by the
>> authors for
>> inclusion in Apache Arrow in a GitHub pull request [2]. This code has been
>> staged in a separate repository for review by the community and ASF IP
>> Clearance:
>>
>> - https://github.com/ray-project/arrow-plasma-object-store/tree/
>> 11795753b0850cf5ad50d640067a8517ad8629a2#diff-
>> 69e56fcedf1b794992b790684902dcd4
>>
>> This vote is to determine whether the Arrow PMC is in favor of accepting
>> the
>> code contribution. If the vote passes, the PMC and the authors of the code
>> will
>> work together to complete the ASF IP Clearance process and import the
>> Plasma
>> Object Store into Apache Arrow for inclusion in a future release:
>>
>> [ ] +1 : Accept contribution of Plasma Object Store
>> [ ] 0 : No opinion
>> [ ] -1 : Reject contribution because...
>>
>> The vote is open for 72 hours and will close at 18:15 UTC on Sunday 23 July
>> 2017 and the results will be announced on this list.
>>
>> Thanks,
>> Wes
>>
>> [1]: https://github.com/ray-project/ray/commit/
>> b94b4a35e04d8d2c0af4420518a4e9a94c1c9b9f
>> [2]: https://github.com/apache/arrow/pull/742
>>
Re: [VOTE] Accept contribution of Plasma Object Store
Posted by Bryan Cutler <cu...@gmail.com>.
+1 sounds great!
On Thu, Jul 20, 2017 at 11:14 AM, Wes McKinney <we...@gmail.com> wrote:
> Dear all,
>
> The Plasma Object Store provides a server process, reference C++ client,
> and
> Python binding for managing a collection of binary "objects" in POSIX
> shared
> memory. Applications use a lightweight messaging protocol to create and
> delete
> memory blocks in the object store, evict objects to make room for new
> objects,
> and increment and decrement reference counts to indicate shared ownership
> of
> memory. It also provides for subscribing to notifications about object
> activity. The system helps simplify ownership transfer and memory lifetime
> of
> shared memory blocks, which can be much more complicated in a peer-to-peer
> architecture.
>
> The object store has been used in conjunction with the Apache Arrow
> libraries
> to provide for zero-copy access to collections of large objects stored in
> shared memory. Incorporating this project into Apache Arrow will help the
> community continue to develop and innovate technology for low-overhead
> sharing
> of complex datasets across multiple processes.
>
> Plasma Object Store was developed by the Ray project with the UC Berkeley
> RISELab. There have been 8 contributors, with about 3.1 KLOC of C++ code
> and an
> additional 5.1 KLOC of thirdparty C and C++ code which we have reviewed for
> compatibility with the Apache Software Foundation's policies on license
> compatibility.
>
> This code was split off from the Ray project from commit id
> `b94b4a35e04d8d2c0af4420518a4e9a94c1c9b9f` [1] and modified by the
> authors for
> inclusion in Apache Arrow in a GitHub pull request [2]. This code has been
> staged in a separate repository for review by the community and ASF IP
> Clearance:
>
> - https://github.com/ray-project/arrow-plasma-object-store/tree/
> 11795753b0850cf5ad50d640067a8517ad8629a2#diff-
> 69e56fcedf1b794992b790684902dcd4
>
> This vote is to determine whether the Arrow PMC is in favor of accepting
> the
> code contribution. If the vote passes, the PMC and the authors of the code
> will
> work together to complete the ASF IP Clearance process and import the
> Plasma
> Object Store into Apache Arrow for inclusion in a future release:
>
> [ ] +1 : Accept contribution of Plasma Object Store
> [ ] 0 : No opinion
> [ ] -1 : Reject contribution because...
>
> The vote is open for 72 hours and will close at 18:15 UTC on Sunday 23 July
> 2017 and the results will be announced on this list.
>
> Thanks,
> Wes
>
> [1]: https://github.com/ray-project/ray/commit/
> b94b4a35e04d8d2c0af4420518a4e9a94c1c9b9f
> [2]: https://github.com/apache/arrow/pull/742
>