Proposing Changes To Heron ECO

[Josh Fischer <jo...@joshfischer.io>]

I've recently discovered a bug while trying to create a custom grouping
example with ECO.

Historically with Flux (which ECO is based off of) you would define a
custom grouping like below.

- name: "bolt-1 --> bolt2"
    from: "bolt-1"
    to: "bolt-2"
    grouping:
      type: CUSTOM
      customClass:
        className: "org.apache.storm.testing.NGrouping"

I propose we move the nested "customClass" properties up a level.  This
will simplify the structure of the definition and will still allow us to
call config methods, set properties, and call constructors like any other
component defined in the topology file. This above definition is causing
snake yaml to throw an exception during instantiation of the customClass
objects.

A verbose example of the proposed custom grouping definition change is
below.  The major change is that "customClass" nesting has been removed.

- from: "wordSpout"
  to: "printBolt"
  grouping:
    type: CUSTOM
    className: "org.apache.heron.eco.starters.CustomGrouping"
    constructorArgs:
      - ref: "property-holder"
    configMethods:
       - name: "sampleConfigurationMethod"
         args:
           - "${ecoPropertyOne}"
           - MB
    properties:
      - name: "numberProperty"
        value: 11
      - name: "publicProperty"
        value: "This is public property"

Any comments and/or questions are appreciated!

Re: Proposing Changes To Heron ECO

[Karthik Ramasamy <ka...@streaml.io>]

Good proposal - uses existing code and adds more capabilities for custom
grouping.

+1

On Thu, Mar 29, 2018 at 8:16 PM Josh Fischer <jo...@joshfischer.io> wrote:

> I've recently discovered a bug while trying to create a custom grouping
> example with ECO.
>
> Historically with Flux (which ECO is based off of) you would define a
> custom grouping like below.
>
> - name: "bolt-1 --> bolt2"
>     from: "bolt-1"
>     to: "bolt-2"
>     grouping:
>       type: CUSTOM
>       customClass:
>         className: "org.apache.storm.testing.NGrouping"
>
> I propose we move the nested "customClass" properties up a level.  This
> will simplify the structure of the definition and will still allow us to
> call config methods, set properties, and call constructors like any other
> component defined in the topology file. This above definition is causing
> snake yaml to throw an exception during instantiation of the customClass
> objects.
>
> A verbose example of the proposed custom grouping definition change is
> below.  The major change is that "customClass" nesting has been removed.
>
> - from: "wordSpout"
>   to: "printBolt"
>   grouping:
>     type: CUSTOM
>     className: "org.apache.heron.eco.starters.CustomGrouping"
>     constructorArgs:
>       - ref: "property-holder"
>     configMethods:
>        - name: "sampleConfigurationMethod"
>          args:
>            - "${ecoPropertyOne}"
>            - MB
>     properties:
>       - name: "numberProperty"
>         value: 11
>       - name: "publicProperty"
>         value: "This is public property"
>
> Any comments and/or questions are appreciated!
>