Goals for 2018

[Donald Szeto <do...@apache.org>]

Hi all,

I would like to propose some goals for the project to achieve in the year
of 2018. I am starting a list of things that I feel important. Please feel
free to chime in. It would be great if we can finalize on a short list of
things by end of this month.

I will start with 2 major categories of goals: PredictionIO functionality
and project operations.

Functionality:

- Keeping the core value of bringing machine learning to production through
template-based build-train-deploy
  - Templates are tested against new versions of PIO
  - More flexible building by supporting other build tools natively
  - Training and deployment using easy-to-understand packaging and
transparent tracking
  - Expose engine instance management better and make it easy to use
- Multiple template types support
  - Support existing DASE templates
  - Add support to Spark ML-based templates
- Decouple event server as a core requirement
- Evaluate Kappa architecture
- Evaluate multiple runtime support as time allows
- General housekeeping
  - Deprecate old Spark and Scala versions
  - Upgrade core dependencies

Project operations:

- Maintain a release cadence of at least one version every 2 months,
starting with a release at the end of this month. Releases can be as small
as patch versions, to get bug fixes into public's hands.
- Rotate release managers. Chan and I have cut releases. It would be nice
to have a couple more PMC members to have experience in releasing.

If you have any question please feel free to ask. Happy to add more details
to them.

Regards,
Donald

Re: Goals for 2018

[Donald Szeto <do...@apache.org>]

I think they would both fall into the item

> Training and deployment using easy-to-understand packaging and
transparent tracking

Training part: A REST API (something like
https://issues.apache.org/jira/browse/PIO-29) that would provide an
alternative to CLI and expose more tracking information for engine
instances.

Deployment part: Existing "pio deploy", plus options to deploy to AWS EMR
(and other similar products from other vendors), and generating Docker
containers.

On Mon, Jan 8, 2018 at 11:03 PM, Simon Chan <si...@salesforce.com> wrote:

> Great list, Donald. How about:
> - built-in support to EMR Spark
> - A REST API option to replace engine.json files and CLI
> ?
>
> Simon
>
> On Mon, Jan 8, 2018 at 11:00 PM Donald Szeto <do...@apache.org> wrote:
>
> > Hi all,
> >
> > I would like to propose some goals for the project to achieve in the year
> > of 2018. I am starting a list of things that I feel important. Please
> feel
> > free to chime in. It would be great if we can finalize on a short list of
> > things by end of this month.
> >
> > I will start with 2 major categories of goals: PredictionIO functionality
> > and project operations.
> >
> > Functionality:
> >
> > - Keeping the core value of bringing machine learning to production
> through
> > template-based build-train-deploy
> >   - Templates are tested against new versions of PIO
> >   - More flexible building by supporting other build tools natively
> >   - Training and deployment using easy-to-understand packaging and
> > transparent tracking
> >   - Expose engine instance management better and make it easy to use
> > - Multiple template types support
> >   - Support existing DASE templates
> >   - Add support to Spark ML-based templates
> > - Decouple event server as a core requirement
> > - Evaluate Kappa architecture
> > - Evaluate multiple runtime support as time allows
> > - General housekeeping
> >   - Deprecate old Spark and Scala versions
> >   - Upgrade core dependencies
> >
> > Project operations:
> >
> > - Maintain a release cadence of at least one version every 2 months,
> > starting with a release at the end of this month. Releases can be as
> small
> > as patch versions, to get bug fixes into public's hands.
> > - Rotate release managers. Chan and I have cut releases. It would be nice
> > to have a couple more PMC members to have experience in releasing.
> >
> > If you have any question please feel free to ask. Happy to add more
> details
> > to them.
> >
> > Regards,
> > Donald
> >
>

Re: Goals for 2018

[Simon Chan <si...@salesforce.com>]

Great list, Donald. How about:
- built-in support to EMR Spark
- A REST API option to replace engine.json files and CLI
?

Simon

On Mon, Jan 8, 2018 at 11:00 PM Donald Szeto <do...@apache.org> wrote:

> Hi all,
>
> I would like to propose some goals for the project to achieve in the year
> of 2018. I am starting a list of things that I feel important. Please feel
> free to chime in. It would be great if we can finalize on a short list of
> things by end of this month.
>
> I will start with 2 major categories of goals: PredictionIO functionality
> and project operations.
>
> Functionality:
>
> - Keeping the core value of bringing machine learning to production through
> template-based build-train-deploy
>   - Templates are tested against new versions of PIO
>   - More flexible building by supporting other build tools natively
>   - Training and deployment using easy-to-understand packaging and
> transparent tracking
>   - Expose engine instance management better and make it easy to use
> - Multiple template types support
>   - Support existing DASE templates
>   - Add support to Spark ML-based templates
> - Decouple event server as a core requirement
> - Evaluate Kappa architecture
> - Evaluate multiple runtime support as time allows
> - General housekeeping
>   - Deprecate old Spark and Scala versions
>   - Upgrade core dependencies
>
> Project operations:
>
> - Maintain a release cadence of at least one version every 2 months,
> starting with a release at the end of this month. Releases can be as small
> as patch versions, to get bug fixes into public's hands.
> - Rotate release managers. Chan and I have cut releases. It would be nice
> to have a couple more PMC members to have experience in releasing.
>
> If you have any question please feel free to ask. Happy to add more details
> to them.
>
> Regards,
> Donald
>