Request for vision review

[Lucas Bonatto Miguel <lu...@apache.org>]

Hi,

As the current version of Marvin is not getting enough traction in both
usage and development, some action needs to be taken. After discussing with
a few stakeholders, we believe that the scope of the problem Marvin is
trying to solve should be simplified. One of the ideas that I would like to
propose is for us to remove some parts of the code, related to training and
inference execution in production, and instead generate packages that can
be executed by other de-facto platforms out there, argo, kfp, tecton, tfx,
etc.

I have no doubts that this would simplify the codebase immensely, and in
addition, it would make Marvin useful for more people, by taking advantage
of this new integration layer.

If enough people are interested in this change, I suggest we schedule a
meeting to kick off the ideas for this RFC.

Please respond with +1 or -1

Best,
Lucas

Re: Request for vision review

[Lucas Cardoso Silva <ca...@gmail.com>]

+1 I think it's a good plan to discuss these changes.

On Thu, Nov 4, 2021 at 6:03 PM Lucas Bonatto Miguel <lu...@apache.org>
wrote:

> Hi,
>
> As the current version of Marvin is not getting enough traction in both
> usage and development, some action needs to be taken. After discussing with
> a few stakeholders, we believe that the scope of the problem Marvin is
> trying to solve should be simplified. One of the ideas that I would like to
> propose is for us to remove some parts of the code, related to training and
> inference execution in production, and instead generate packages that can
> be executed by other de-facto platforms out there, argo, kfp, tecton, tfx,
> etc.
>
> I have no doubts that this would simplify the codebase immensely, and in
> addition, it would make Marvin useful for more people, by taking advantage
> of this new integration layer.
>
> If enough people are interested in this change, I suggest we schedule a
> meeting to kick off the ideas for this RFC.
>
> Please respond with +1 or -1
>
> Best,
> Lucas
>

Re: Request for vision review

[Lucas Bonatto Miguel <lu...@apache.org>]

Hi all,

We just finished the meeting, here are some notes and next steps:

 - There were 6 persons in the meeting
 - We all agree Marvin scope should be simplified, we want to give more
focus on things Marvin is already doing well and there are no similar
solutions in the community
 - We want to integrate with other projects for the things that would be
removed from our scope, mainly the engine-executor, pipeline execution and
the notebook plugin.
 - For the pipeline portion we plan to have an easy-to-use DSL that can
compile to several frameworks (KFP, Airflow, etc)
 - We will start a RFC detailing the necessary changes to be implemented
 - We will release the last 0.x.x version asap, including all the changes
from the past months
 - Once we start work on top of the RFC, a new major version will be
released, 1.x.x, so we can keep a clear distinction for users in the old
version
 - In order to allow current users to continue having support of the
community, we should write tutorials on helping them to migrate the part of
their applications that depend on the things we will be removing from
Marvin to other tools we recommend

Thx,
Lucas

On Thu, Nov 11, 2021 at 10:33 AM Daniel Lucredio <da...@ufscar.br>
wrote:

> +1
>
> Em sex., 5 de nov. de 2021 às 14:16, Zhang Yifei <yi...@gmail.com>
> escreveu:
>
> > +1
> >
> > Em sex., 5 de nov. de 2021 às 08:25, Mario de Sá Vera <
> desavera@gmail.com>
> > escreveu:
> >
> > > +1
> > >
> > > Em qui., 4 de nov. de 2021 às 21:03, Lucas Bonatto Miguel <
> > > lucasbm88@apache.org> escreveu:
> > >
> > > > Hi,
> > > >
> > > > As the current version of Marvin is not getting enough traction in
> both
> > > > usage and development, some action needs to be taken. After
> discussing
> > > with
> > > > a few stakeholders, we believe that the scope of the problem Marvin
> is
> > > > trying to solve should be simplified. One of the ideas that I would
> > like
> > > to
> > > > propose is for us to remove some parts of the code, related to
> training
> > > and
> > > > inference execution in production, and instead generate packages that
> > can
> > > > be executed by other de-facto platforms out there, argo, kfp, tecton,
> > > tfx,
> > > > etc.
> > > >
> > > > I have no doubts that this would simplify the codebase immensely, and
> > in
> > > > addition, it would make Marvin useful for more people, by taking
> > > advantage
> > > > of this new integration layer.
> > > >
> > > > If enough people are interested in this change, I suggest we
> schedule a
> > > > meeting to kick off the ideas for this RFC.
> > > >
> > > > Please respond with +1 or -1
> > > >
> > > > Best,
> > > > Lucas
> > > >
> > >
> >
> >
> > --
> > --------------------------------------------------------------
> > Zhang Yifei
> >
>

Re: Request for vision review

[Daniel Lucredio <da...@ufscar.br>]

+1

Em sex., 5 de nov. de 2021 às 14:16, Zhang Yifei <yi...@gmail.com>
escreveu:

> +1
>
> Em sex., 5 de nov. de 2021 às 08:25, Mario de Sá Vera <de...@gmail.com>
> escreveu:
>
> > +1
> >
> > Em qui., 4 de nov. de 2021 às 21:03, Lucas Bonatto Miguel <
> > lucasbm88@apache.org> escreveu:
> >
> > > Hi,
> > >
> > > As the current version of Marvin is not getting enough traction in both
> > > usage and development, some action needs to be taken. After discussing
> > with
> > > a few stakeholders, we believe that the scope of the problem Marvin is
> > > trying to solve should be simplified. One of the ideas that I would
> like
> > to
> > > propose is for us to remove some parts of the code, related to training
> > and
> > > inference execution in production, and instead generate packages that
> can
> > > be executed by other de-facto platforms out there, argo, kfp, tecton,
> > tfx,
> > > etc.
> > >
> > > I have no doubts that this would simplify the codebase immensely, and
> in
> > > addition, it would make Marvin useful for more people, by taking
> > advantage
> > > of this new integration layer.
> > >
> > > If enough people are interested in this change, I suggest we schedule a
> > > meeting to kick off the ideas for this RFC.
> > >
> > > Please respond with +1 or -1
> > >
> > > Best,
> > > Lucas
> > >
> >
>
>
> --
> --------------------------------------------------------------
> Zhang Yifei
>

Re: Request for vision review

[Zhang Yifei <yi...@gmail.com>]

+1

Em sex., 5 de nov. de 2021 às 08:25, Mario de Sá Vera <de...@gmail.com>
escreveu:

> +1
>
> Em qui., 4 de nov. de 2021 às 21:03, Lucas Bonatto Miguel <
> lucasbm88@apache.org> escreveu:
>
> > Hi,
> >
> > As the current version of Marvin is not getting enough traction in both
> > usage and development, some action needs to be taken. After discussing
> with
> > a few stakeholders, we believe that the scope of the problem Marvin is
> > trying to solve should be simplified. One of the ideas that I would like
> to
> > propose is for us to remove some parts of the code, related to training
> and
> > inference execution in production, and instead generate packages that can
> > be executed by other de-facto platforms out there, argo, kfp, tecton,
> tfx,
> > etc.
> >
> > I have no doubts that this would simplify the codebase immensely, and in
> > addition, it would make Marvin useful for more people, by taking
> advantage
> > of this new integration layer.
> >
> > If enough people are interested in this change, I suggest we schedule a
> > meeting to kick off the ideas for this RFC.
> >
> > Please respond with +1 or -1
> >
> > Best,
> > Lucas
> >
>


-- 
--------------------------------------------------------------
Zhang Yifei

Re: Request for vision review

[Mario de Sá Vera <de...@gmail.com>]

+1

Em qui., 4 de nov. de 2021 às 21:03, Lucas Bonatto Miguel <
lucasbm88@apache.org> escreveu:

> Hi,
>
> As the current version of Marvin is not getting enough traction in both
> usage and development, some action needs to be taken. After discussing with
> a few stakeholders, we believe that the scope of the problem Marvin is
> trying to solve should be simplified. One of the ideas that I would like to
> propose is for us to remove some parts of the code, related to training and
> inference execution in production, and instead generate packages that can
> be executed by other de-facto platforms out there, argo, kfp, tecton, tfx,
> etc.
>
> I have no doubts that this would simplify the codebase immensely, and in
> addition, it would make Marvin useful for more people, by taking advantage
> of this new integration layer.
>
> If enough people are interested in this change, I suggest we schedule a
> meeting to kick off the ideas for this RFC.
>
> Please respond with +1 or -1
>
> Best,
> Lucas
>