You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apex.apache.org by "Sanjay M Pujare (JIRA)" <ji...@apache.org> on 2016/11/28 19:54:58 UTC
[jira] [Comment Edited] (APEXCORE-576) Apex/Malhar Extensions
[ https://issues.apache.org/jira/browse/APEXCORE-576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15702868#comment-15702868 ]
Sanjay M Pujare edited comment on APEXCORE-576 at 11/28/16 7:54 PM:
--------------------------------------------------------------------
As far as I understand then the main enhancement here is the "registry" and a process to pick items from the registry to merge into the main repository and we will be using the current git process of pull-requests and merging across forks. Are there existing models we can look at for the registry?
A few things that need to be hammered out:
- registry access control, and format
- criteria for merging external contributions into the main repo (quality, unit tests, demand, licensing)
- level of automation available, or otherwise the manual process used to merge contributions
- dependency matrix management (e.g. an item depends on malhar 3.5.0 minimum)
- guidelines for adding items to the registry to be followed by contributors
was (Author: sanjaypujare):
As far as I understand then the main enhancement here is the "registry" and a process to pick items from the registry to merge into the main repository and we will be using the current git process of pull-requests and merging across forks. Are there existing models we can look at for the registry?
A few things that need to be hammered out:
- registry access control, and format
- criteria for merging external contributions into the main repo (quality, unit tests, demand, licensing)
- level of automation available, or otherwise the manual process used to merge contributions
- dependency matrix management (e.g. an item depends malhar 3.5.0 minimum)
- guidelines for adding items to the registry to be followed by contributors
> Apex/Malhar Extensions
> ----------------------
>
> Key: APEXCORE-576
> URL: https://issues.apache.org/jira/browse/APEXCORE-576
> Project: Apache Apex Core
> Issue Type: New Feature
> Components: Website
> Reporter: Chinmay Kolhatkar
>
> The purpose of this task is to provide a way to external contributors to make better contributions to Apache Apex project.
> The idea looks something like this:
> 1. One could have extension to apex core/malhar in their own repository and just register itself with Apache Apex.
> 2. As it matures and find more and more use we can consume that in mainstream releases.
> 3. Some possibilities of of Apex extensions are as follows:
> a. Operators - DataSources, DataDestinations, Parsers, Formatters, Processing etc.
> b. Apex Engine Plugables
> c. External Integrations
> d. Integration with other platform like Machine learning, Graph engines etc.
> e. Application which are ready to use.
> d. Apex related tools which can ease the development and usage of apex.
> The initial discussion about this has happened here:
> https://lists.apache.org/thread.html/3d6ca2b46c53df77f37f54d64e18607a623c5a54f439e1afebfbef35@%3Cdev.apex.apache.org%3E
> More concrete discussion/implementation proposal required for this task to progress.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: [jira] [Comment Edited] (APEXCORE-576) Apex/Malhar Extensions
Posted by Sanjay Pujare <sa...@datatorrent.com>.
Ananth: can you add these comments to the JIRA (https://issues.apache.org/jira/browse/APEXCORE-576) instead of following up on the email thread? Thanks
On 11/28/16, 2:08 PM, "Ananth G" <an...@gmail.com> wrote:
If we were to model something along the lines of "Spark packages", I am
afraid that the story of Apex being a "rich ecosystem" might be diluted.
Today I see Malhar as a strong reason to convince someone that we can
interconnect many things as part of the offering. The moment we put a
layered process, my only concern is that the review process would be very
very loose and hence lower quality creeping in as Apex committers need not
necessarily review such packages and hence provide inputs to the evolving
Apex core etc. There is a risk that things might diverge more than we want
them to ?
Thoughts ?
Regards,
Ananth
On Tue, Nov 29, 2016 at 6:54 AM, Sanjay M Pujare (JIRA) <ji...@apache.org>
wrote:
>
> [ https://issues.apache.org/jira/browse/APEXCORE-576?page=
> com.atlassian.jira.plugin.system.issuetabpanels:comment-
> tabpanel&focusedCommentId=15702868#comment-15702868 ]
>
> Sanjay M Pujare edited comment on APEXCORE-576 at 11/28/16 7:54 PM:
> --------------------------------------------------------------------
>
> As far as I understand then the main enhancement here is the "registry"
> and a process to pick items from the registry to merge into the main
> repository and we will be using the current git process of pull-requests
> and merging across forks. Are there existing models we can look at for the
> registry?
>
> A few things that need to be hammered out:
> - registry access control, and format
> - criteria for merging external contributions into the main repo (quality,
> unit tests, demand, licensing)
> - level of automation available, or otherwise the manual process used to
> merge contributions
> - dependency matrix management (e.g. an item depends on malhar 3.5.0
> minimum)
> - guidelines for adding items to the registry to be followed by
> contributors
>
>
> was (Author: sanjaypujare):
> As far as I understand then the main enhancement here is the "registry"
> and a process to pick items from the registry to merge into the main
> repository and we will be using the current git process of pull-requests
> and merging across forks. Are there existing models we can look at for the
> registry?
>
> A few things that need to be hammered out:
> - registry access control, and format
> - criteria for merging external contributions into the main repo (quality,
> unit tests, demand, licensing)
> - level of automation available, or otherwise the manual process used to
> merge contributions
> - dependency matrix management (e.g. an item depends malhar 3.5.0 minimum)
> - guidelines for adding items to the registry to be followed by
> contributors
>
> > Apex/Malhar Extensions
> > ----------------------
> >
> > Key: APEXCORE-576
> > URL: https://issues.apache.org/jira/browse/APEXCORE-576
> > Project: Apache Apex Core
> > Issue Type: New Feature
> > Components: Website
> > Reporter: Chinmay Kolhatkar
> >
> > The purpose of this task is to provide a way to external contributors to
> make better contributions to Apache Apex project.
> > The idea looks something like this:
> > 1. One could have extension to apex core/malhar in their own repository
> and just register itself with Apache Apex.
> > 2. As it matures and find more and more use we can consume that in
> mainstream releases.
> > 3. Some possibilities of of Apex extensions are as follows:
> > a. Operators - DataSources, DataDestinations, Parsers, Formatters,
> Processing etc.
> > b. Apex Engine Plugables
> > c. External Integrations
> > d. Integration with other platform like Machine learning, Graph
> engines etc.
> > e. Application which are ready to use.
> > d. Apex related tools which can ease the development and usage of
> apex.
> > The initial discussion about this has happened here:
> > https://lists.apache.org/thread.html/3d6ca2b46c53df77f37f54d64e1860
> 7a623c5a54f439e1afebfbef35@%3Cdev.apex.apache.org%3E
> > More concrete discussion/implementation proposal required for this task
> to progress.
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)
>
Re: [jira] [Comment Edited] (APEXCORE-576) Apex/Malhar Extensions
Posted by Ananth G <an...@gmail.com>.
If we were to model something along the lines of "Spark packages", I am
afraid that the story of Apex being a "rich ecosystem" might be diluted.
Today I see Malhar as a strong reason to convince someone that we can
interconnect many things as part of the offering. The moment we put a
layered process, my only concern is that the review process would be very
very loose and hence lower quality creeping in as Apex committers need not
necessarily review such packages and hence provide inputs to the evolving
Apex core etc. There is a risk that things might diverge more than we want
them to ?
Thoughts ?
Regards,
Ananth
On Tue, Nov 29, 2016 at 6:54 AM, Sanjay M Pujare (JIRA) <ji...@apache.org>
wrote:
>
> [ https://issues.apache.org/jira/browse/APEXCORE-576?page=
> com.atlassian.jira.plugin.system.issuetabpanels:comment-
> tabpanel&focusedCommentId=15702868#comment-15702868 ]
>
> Sanjay M Pujare edited comment on APEXCORE-576 at 11/28/16 7:54 PM:
> --------------------------------------------------------------------
>
> As far as I understand then the main enhancement here is the "registry"
> and a process to pick items from the registry to merge into the main
> repository and we will be using the current git process of pull-requests
> and merging across forks. Are there existing models we can look at for the
> registry?
>
> A few things that need to be hammered out:
> - registry access control, and format
> - criteria for merging external contributions into the main repo (quality,
> unit tests, demand, licensing)
> - level of automation available, or otherwise the manual process used to
> merge contributions
> - dependency matrix management (e.g. an item depends on malhar 3.5.0
> minimum)
> - guidelines for adding items to the registry to be followed by
> contributors
>
>
> was (Author: sanjaypujare):
> As far as I understand then the main enhancement here is the "registry"
> and a process to pick items from the registry to merge into the main
> repository and we will be using the current git process of pull-requests
> and merging across forks. Are there existing models we can look at for the
> registry?
>
> A few things that need to be hammered out:
> - registry access control, and format
> - criteria for merging external contributions into the main repo (quality,
> unit tests, demand, licensing)
> - level of automation available, or otherwise the manual process used to
> merge contributions
> - dependency matrix management (e.g. an item depends malhar 3.5.0 minimum)
> - guidelines for adding items to the registry to be followed by
> contributors
>
> > Apex/Malhar Extensions
> > ----------------------
> >
> > Key: APEXCORE-576
> > URL: https://issues.apache.org/jira/browse/APEXCORE-576
> > Project: Apache Apex Core
> > Issue Type: New Feature
> > Components: Website
> > Reporter: Chinmay Kolhatkar
> >
> > The purpose of this task is to provide a way to external contributors to
> make better contributions to Apache Apex project.
> > The idea looks something like this:
> > 1. One could have extension to apex core/malhar in their own repository
> and just register itself with Apache Apex.
> > 2. As it matures and find more and more use we can consume that in
> mainstream releases.
> > 3. Some possibilities of of Apex extensions are as follows:
> > a. Operators - DataSources, DataDestinations, Parsers, Formatters,
> Processing etc.
> > b. Apex Engine Plugables
> > c. External Integrations
> > d. Integration with other platform like Machine learning, Graph
> engines etc.
> > e. Application which are ready to use.
> > d. Apex related tools which can ease the development and usage of
> apex.
> > The initial discussion about this has happened here:
> > https://lists.apache.org/thread.html/3d6ca2b46c53df77f37f54d64e1860
> 7a623c5a54f439e1afebfbef35@%3Cdev.apex.apache.org%3E
> > More concrete discussion/implementation proposal required for this task
> to progress.
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)
>