You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@toree.apache.org by Gino Bustelo <gi...@bustelos.com> on 2016/04/29 15:44:26 UTC
May Update
Last month can be found http://wiki.apache.org/incubator/April2016.
Here is the draft for this month. Please give feedback...
--------------------
Toree
Toree provides applications with a mechanism to interactively and remotely
access Apache Spark. It enables interactive workloads between applications
and a Spark cluster. As a Jupyter Notebook extension, it provides the user
with a preconfigured environment for interacting with Spark using Scala,
Python, R or SQL.
Toree has been incubating since 2015-12-02.
Three most important issues to address in the move towards graduation:
1. Removal of LGPL dependency
2. Ease of installation
3. Documentation of APIs
Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
aware of?
TOREE-262 - Progress on removal of LGPL dependency
How has the community developed since the last report?
1. Active communication in mailing list and gitter with early adopters
2. Still working on transitioning users from Spark Kernel project into
Toree.
3. More external contributions being made
How has the project developed since the last report?
1. Contrinue working with JeroMQ community to further their transition into
MPL v2 and away from LGPL. 3 committers to go.
2. Substantial effort in our build system to make Apache compliant releases.
3. Addressing issues opened by community
Date of last release:
None since incubation.
When were the last committers or PMC members elected?
No new additions since incubation
Re: May Update
Posted by Gino Bustelo <gi...@bustelos.com>.
Sourav... I think you are replying in the wrong thread.
On Thu, May 5, 2016 at 7:38 AM, Sourav Mazumder <sourav.mazumder00@gmail.com
> wrote:
> Hi Gino,
>
> It does not solve the problem of running a Spark job (on Yarn) remotely
> from a Jupyter notebook which is running on say in a laptop/some machine.
>
> The issue is in yarn-client mode the laptop needs to get access to all the
> slave nodes where the executors would be running. In a typical security
> scenario of an organization the slave nodes are behind firewall and cannot
> be accessed from any random machine outside.
>
> Regards,
> Sourav
>
> On Wed, May 4, 2016 at 11:47 AM, Gino Bustelo <gi...@bustelos.com> wrote:
>
> > Wiki has been updated for May 2016
> >
> > On Fri, Apr 29, 2016 at 11:32 AM, Gino Bustelo <gi...@bustelos.com>
> wrote:
> >
> > > My bad... copy paste error. @Luciano, I agree... my intention was to
> > leave
> > > that section the same
> > >
> > > 1. Resolve LGPL dependency: This is in progress, and hopefully we will
> > have the dependency license changed to a license that is compatible
> > with AL2. 2. Make a release: nearing completion 3. Grow a diverse
> > community: We should put some emphasis on growing the community and
> > making it diverse (the rule is at least three independent
> contributors)
> > >
> > >
> > > On Fri, Apr 29, 2016 at 9:45 AM, Luciano Resende <luckbr1975@gmail.com
> >
> > > wrote:
> > >
> > >> I still think the important issues toward graduation are similar to
> what
> > >> was reported in the previous report:
> > >>
> > >> Three most important issues to address in the move towards graduation:
> > >> 1. Resolve LGPL dependency: This is in progress, and hopefully we
> > >> will have the dependency license changed to a license that is
> > >> compatible with AL2. 2. Make a release: nearing completion 3.
> > >> Grow a diverse community: We should put some emphasis on growing the
> > >> community and making it diverse (the rule is at least three
> > >> independent contributors)
> > >>
> > >>
> > >>
> > >> On Fri, Apr 29, 2016 at 6:44 AM, Gino Bustelo <gi...@bustelos.com>
> > wrote:
> > >>
> > >> > Last month can be found http://wiki.apache.org/incubator/April2016.
> > >> > Here is the draft for this month. Please give feedback...
> > >> >
> > >> > --------------------
> > >> > Toree
> > >> >
> > >> > Toree provides applications with a mechanism to interactively and
> > >> remotely
> > >> > access Apache Spark. It enables interactive workloads between
> > >> applications
> > >> > and a Spark cluster. As a Jupyter Notebook extension, it provides
> the
> > >> user
> > >> > with a preconfigured environment for interacting with Spark using
> > Scala,
> > >> > Python, R or SQL.
> > >> >
> > >> > Toree has been incubating since 2015-12-02.
> > >> >
> > >> > Three most important issues to address in the move towards
> graduation:
> > >> >
> > >> > 1. Removal of LGPL dependency
> > >> > 2. Ease of installation
> > >> > 3. Documentation of APIs
> > >> >
> > >> > Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to
> be
> > >> > aware of?
> > >> >
> > >> > TOREE-262 - Progress on removal of LGPL dependency
> > >> >
> > >> > How has the community developed since the last report?
> > >> >
> > >> > 1. Active communication in mailing list and gitter with early
> > adopters
> > >> > 2. Still working on transitioning users from Spark Kernel project
> > into
> > >> > Toree.
> > >> > 3. More external contributions being made
> > >> >
> > >> > How has the project developed since the last report?
> > >> >
> > >> > 1. Contrinue working with JeroMQ community to further their
> > transition
> > >> > into
> > >> > MPL v2 and away from LGPL. 3 committers to go.
> > >> > 2. Substantial effort in our build system to make Apache compliant
> > >> > releases.
> > >> > 3. Addressing issues opened by community
> > >> >
> > >> > Date of last release:
> > >> >
> > >> > None since incubation.
> > >> >
> > >> > When were the last committers or PMC members elected?
> > >> >
> > >> > No new additions since incubation
> > >> >
> > >>
> > >>
> > >>
> > >> --
> > >> Luciano Resende
> > >> http://twitter.com/lresende1975
> > >> http://lresende.blogspot.com/
> > >>
> > >
> > >
> >
>
Re: May Update
Posted by Sourav Mazumder <so...@gmail.com>.
Hi Gino,
It does not solve the problem of running a Spark job (on Yarn) remotely
from a Jupyter notebook which is running on say in a laptop/some machine.
The issue is in yarn-client mode the laptop needs to get access to all the
slave nodes where the executors would be running. In a typical security
scenario of an organization the slave nodes are behind firewall and cannot
be accessed from any random machine outside.
Regards,
Sourav
On Wed, May 4, 2016 at 11:47 AM, Gino Bustelo <gi...@bustelos.com> wrote:
> Wiki has been updated for May 2016
>
> On Fri, Apr 29, 2016 at 11:32 AM, Gino Bustelo <gi...@bustelos.com> wrote:
>
> > My bad... copy paste error. @Luciano, I agree... my intention was to
> leave
> > that section the same
> >
> > 1. Resolve LGPL dependency: This is in progress, and hopefully we will
> have the dependency license changed to a license that is compatible
> with AL2. 2. Make a release: nearing completion 3. Grow a diverse
> community: We should put some emphasis on growing the community and
> making it diverse (the rule is at least three independent contributors)
> >
> >
> > On Fri, Apr 29, 2016 at 9:45 AM, Luciano Resende <lu...@gmail.com>
> > wrote:
> >
> >> I still think the important issues toward graduation are similar to what
> >> was reported in the previous report:
> >>
> >> Three most important issues to address in the move towards graduation:
> >> 1. Resolve LGPL dependency: This is in progress, and hopefully we
> >> will have the dependency license changed to a license that is
> >> compatible with AL2. 2. Make a release: nearing completion 3.
> >> Grow a diverse community: We should put some emphasis on growing the
> >> community and making it diverse (the rule is at least three
> >> independent contributors)
> >>
> >>
> >>
> >> On Fri, Apr 29, 2016 at 6:44 AM, Gino Bustelo <gi...@bustelos.com>
> wrote:
> >>
> >> > Last month can be found http://wiki.apache.org/incubator/April2016.
> >> > Here is the draft for this month. Please give feedback...
> >> >
> >> > --------------------
> >> > Toree
> >> >
> >> > Toree provides applications with a mechanism to interactively and
> >> remotely
> >> > access Apache Spark. It enables interactive workloads between
> >> applications
> >> > and a Spark cluster. As a Jupyter Notebook extension, it provides the
> >> user
> >> > with a preconfigured environment for interacting with Spark using
> Scala,
> >> > Python, R or SQL.
> >> >
> >> > Toree has been incubating since 2015-12-02.
> >> >
> >> > Three most important issues to address in the move towards graduation:
> >> >
> >> > 1. Removal of LGPL dependency
> >> > 2. Ease of installation
> >> > 3. Documentation of APIs
> >> >
> >> > Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
> >> > aware of?
> >> >
> >> > TOREE-262 - Progress on removal of LGPL dependency
> >> >
> >> > How has the community developed since the last report?
> >> >
> >> > 1. Active communication in mailing list and gitter with early
> adopters
> >> > 2. Still working on transitioning users from Spark Kernel project
> into
> >> > Toree.
> >> > 3. More external contributions being made
> >> >
> >> > How has the project developed since the last report?
> >> >
> >> > 1. Contrinue working with JeroMQ community to further their
> transition
> >> > into
> >> > MPL v2 and away from LGPL. 3 committers to go.
> >> > 2. Substantial effort in our build system to make Apache compliant
> >> > releases.
> >> > 3. Addressing issues opened by community
> >> >
> >> > Date of last release:
> >> >
> >> > None since incubation.
> >> >
> >> > When were the last committers or PMC members elected?
> >> >
> >> > No new additions since incubation
> >> >
> >>
> >>
> >>
> >> --
> >> Luciano Resende
> >> http://twitter.com/lresende1975
> >> http://lresende.blogspot.com/
> >>
> >
> >
>
Re: May Update
Posted by Gino Bustelo <gi...@bustelos.com>.
Wiki has been updated for May 2016
On Fri, Apr 29, 2016 at 11:32 AM, Gino Bustelo <gi...@bustelos.com> wrote:
> My bad... copy paste error. @Luciano, I agree... my intention was to leave
> that section the same
>
> 1. Resolve LGPL dependency: This is in progress, and hopefully we will have the dependency license changed to a license that is compatible with AL2. 2. Make a release: nearing completion 3. Grow a diverse community: We should put some emphasis on growing the community and making it diverse (the rule is at least three independent contributors)
>
>
> On Fri, Apr 29, 2016 at 9:45 AM, Luciano Resende <lu...@gmail.com>
> wrote:
>
>> I still think the important issues toward graduation are similar to what
>> was reported in the previous report:
>>
>> Three most important issues to address in the move towards graduation:
>> 1. Resolve LGPL dependency: This is in progress, and hopefully we
>> will have the dependency license changed to a license that is
>> compatible with AL2. 2. Make a release: nearing completion 3.
>> Grow a diverse community: We should put some emphasis on growing the
>> community and making it diverse (the rule is at least three
>> independent contributors)
>>
>>
>>
>> On Fri, Apr 29, 2016 at 6:44 AM, Gino Bustelo <gi...@bustelos.com> wrote:
>>
>> > Last month can be found http://wiki.apache.org/incubator/April2016.
>> > Here is the draft for this month. Please give feedback...
>> >
>> > --------------------
>> > Toree
>> >
>> > Toree provides applications with a mechanism to interactively and
>> remotely
>> > access Apache Spark. It enables interactive workloads between
>> applications
>> > and a Spark cluster. As a Jupyter Notebook extension, it provides the
>> user
>> > with a preconfigured environment for interacting with Spark using Scala,
>> > Python, R or SQL.
>> >
>> > Toree has been incubating since 2015-12-02.
>> >
>> > Three most important issues to address in the move towards graduation:
>> >
>> > 1. Removal of LGPL dependency
>> > 2. Ease of installation
>> > 3. Documentation of APIs
>> >
>> > Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
>> > aware of?
>> >
>> > TOREE-262 - Progress on removal of LGPL dependency
>> >
>> > How has the community developed since the last report?
>> >
>> > 1. Active communication in mailing list and gitter with early adopters
>> > 2. Still working on transitioning users from Spark Kernel project into
>> > Toree.
>> > 3. More external contributions being made
>> >
>> > How has the project developed since the last report?
>> >
>> > 1. Contrinue working with JeroMQ community to further their transition
>> > into
>> > MPL v2 and away from LGPL. 3 committers to go.
>> > 2. Substantial effort in our build system to make Apache compliant
>> > releases.
>> > 3. Addressing issues opened by community
>> >
>> > Date of last release:
>> >
>> > None since incubation.
>> >
>> > When were the last committers or PMC members elected?
>> >
>> > No new additions since incubation
>> >
>>
>>
>>
>> --
>> Luciano Resende
>> http://twitter.com/lresende1975
>> http://lresende.blogspot.com/
>>
>
>
Re: May Update
Posted by Gino Bustelo <gi...@bustelos.com>.
My bad... copy paste error. @Luciano, I agree... my intention was to leave
that section the same
1. Resolve LGPL dependency: This is in progress, and hopefully we will
have the dependency license changed to a license that is
compatible with AL2. 2. Make a release: nearing completion 3.
Grow a diverse community: We should put some emphasis on growing the
community and making it diverse (the rule is at least three
independent contributors)
On Fri, Apr 29, 2016 at 9:45 AM, Luciano Resende <lu...@gmail.com>
wrote:
> I still think the important issues toward graduation are similar to what
> was reported in the previous report:
>
> Three most important issues to address in the move towards graduation:
> 1. Resolve LGPL dependency: This is in progress, and hopefully we
> will have the dependency license changed to a license that is
> compatible with AL2. 2. Make a release: nearing completion 3.
> Grow a diverse community: We should put some emphasis on growing the
> community and making it diverse (the rule is at least three
> independent contributors)
>
>
>
> On Fri, Apr 29, 2016 at 6:44 AM, Gino Bustelo <gi...@bustelos.com> wrote:
>
> > Last month can be found http://wiki.apache.org/incubator/April2016.
> > Here is the draft for this month. Please give feedback...
> >
> > --------------------
> > Toree
> >
> > Toree provides applications with a mechanism to interactively and
> remotely
> > access Apache Spark. It enables interactive workloads between
> applications
> > and a Spark cluster. As a Jupyter Notebook extension, it provides the
> user
> > with a preconfigured environment for interacting with Spark using Scala,
> > Python, R or SQL.
> >
> > Toree has been incubating since 2015-12-02.
> >
> > Three most important issues to address in the move towards graduation:
> >
> > 1. Removal of LGPL dependency
> > 2. Ease of installation
> > 3. Documentation of APIs
> >
> > Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
> > aware of?
> >
> > TOREE-262 - Progress on removal of LGPL dependency
> >
> > How has the community developed since the last report?
> >
> > 1. Active communication in mailing list and gitter with early adopters
> > 2. Still working on transitioning users from Spark Kernel project into
> > Toree.
> > 3. More external contributions being made
> >
> > How has the project developed since the last report?
> >
> > 1. Contrinue working with JeroMQ community to further their transition
> > into
> > MPL v2 and away from LGPL. 3 committers to go.
> > 2. Substantial effort in our build system to make Apache compliant
> > releases.
> > 3. Addressing issues opened by community
> >
> > Date of last release:
> >
> > None since incubation.
> >
> > When were the last committers or PMC members elected?
> >
> > No new additions since incubation
> >
>
>
>
> --
> Luciano Resende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>
Re: May Update
Posted by Luciano Resende <lu...@gmail.com>.
I still think the important issues toward graduation are similar to what
was reported in the previous report:
Three most important issues to address in the move towards graduation:
1. Resolve LGPL dependency: This is in progress, and hopefully we
will have the dependency license changed to a license that is
compatible with AL2. 2. Make a release: nearing completion 3.
Grow a diverse community: We should put some emphasis on growing the
community and making it diverse (the rule is at least three
independent contributors)
On Fri, Apr 29, 2016 at 6:44 AM, Gino Bustelo <gi...@bustelos.com> wrote:
> Last month can be found http://wiki.apache.org/incubator/April2016.
> Here is the draft for this month. Please give feedback...
>
> --------------------
> Toree
>
> Toree provides applications with a mechanism to interactively and remotely
> access Apache Spark. It enables interactive workloads between applications
> and a Spark cluster. As a Jupyter Notebook extension, it provides the user
> with a preconfigured environment for interacting with Spark using Scala,
> Python, R or SQL.
>
> Toree has been incubating since 2015-12-02.
>
> Three most important issues to address in the move towards graduation:
>
> 1. Removal of LGPL dependency
> 2. Ease of installation
> 3. Documentation of APIs
>
> Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
> aware of?
>
> TOREE-262 - Progress on removal of LGPL dependency
>
> How has the community developed since the last report?
>
> 1. Active communication in mailing list and gitter with early adopters
> 2. Still working on transitioning users from Spark Kernel project into
> Toree.
> 3. More external contributions being made
>
> How has the project developed since the last report?
>
> 1. Contrinue working with JeroMQ community to further their transition
> into
> MPL v2 and away from LGPL. 3 committers to go.
> 2. Substantial effort in our build system to make Apache compliant
> releases.
> 3. Addressing issues opened by community
>
> Date of last release:
>
> None since incubation.
>
> When were the last committers or PMC members elected?
>
> No new additions since incubation
>
--
Luciano Resende
http://twitter.com/lresende1975
http://lresende.blogspot.com/