You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@metron.apache.org by Otto Fowler <ot...@gmail.com> on 2017/12/18 19:52:36 UTC

[MEETING NOTES] 12/18/17 Developer Community Meeting

2017-12-18 Dev Community Meeting

Agenda

   -

   Call for reviewers, ideas how to get more involvement, what people can
   do to help (Otto Fowler)
   -

   Feature branches : we have two now, what are they and how are we going
   to work on them (Otto Fowler)
   -

   ES 5.6 upgrade (Michael Miklavcic)
   -

   Release Status (Michael Miklavcic)
   -

   Short Term Roadmap (Michael Miklavcic)
   -

   Release process WRT formalized upgrade and installation instructions to
   be included as a part of a release (Jon Zeolla)
   -

   Any concerns/questions with the secondary repo for bro. (Jon Zeolla)


Attendees

Jon Zeolla, James Sirota, Matt Foley, Otto Fowler, Hakan Akansel, Justin
Leet, Michael Miklavcic, Nick Allen, Ryan Merriman, Laurens Vets

Discussion

Call for reviewers, ideas how to get more involvement, what people can do
to help (Otto)

   -

   Nick Agrees, suggests potentially simplifying the review process.
   -

   Certain stellar functions that implement certain algos are difficult to
   review properly and rely heavily on the initial implementer.
   -

   Michael suggests heavy focus/scrutiny of the testing/documentation of PRs
   -

   Otto suggests that the bar to spin up Metron may be too high, and could
   simplify the full-dev/PoC spin up.  Justin agrees.
   -

   Three suggested DISCUSS threads
   -

      What’s a better way for us to document reviews and contributions in
      Metron?
      1.

         How to overcome developer inertia for spinning up new envs such as
         testing ansible changes or similar
         -

      How can we lower the barrier to entry for new users to Metron?
      -

   Need to keep multiple PRs and feature branches top of mind to simplify
   review.


Feature Branches (Otto)

   -

   METRON-777
   <https://issues.apache.org/jira/projects/METRON/issues/METRON-777?filter=allopenissues>
   and METRON-1344
   <https://issues.apache.org/jira/projects/METRON/issues/METRON-1344?filter=allissues>
   -

   Are we all comfortable with how we use FBs?
   -

      Ryan, how do we manage follow-on PRs for a FB?
      -

      Try to avoid bugfixes that would be useful to master being put solely
      in a FB.
      -

      All FB processes should align with our policies to commit/review
      against master, with a slightly higher tolerance for instability.
      -

         I.e. Some interim steps may create regressions, but we should
         consider being comfortable with this in order to simplify review
         -

      Still feeling this out, maybe a future DISCUSS on how to determine if
      you should be creating a FB.
      -

      Consider FB-specific documentation to identify what/where/why/how.
      Otto has an example here
      <https://cwiki.apache.org/confluence/display/METRON/Metron+Extension+System+and+Parser+Extensions>
      .


ES 5.6 upgrade (Michael)

   -

   Michael:  Should be ready for review, looking for testing, etc.  Could
   use help with a multinode instance, performance testing, etc..
   METRON-939
   <https://issues.apache.org/jira/projects/METRON/issues/METRON-939?filter=allopenissues>
   (#840 <https://github.com/apache/metron/pull/840>)
   -

   Otto:  Unclear on what versions of ES Metron should run on.
   -

      Michael:  Looking to support only ES 5.6.2, unable to currently
      support multiple versions of ES due to the complexity/testing reqs.
      -

   Otto:  Unclear on status of #619
   <https://github.com/apache/metron/pull/619>.
   -

      Michael:  This is a subset of the Xpack work, and that xpack support
      is currently planning to be a follow-on.  Still using the
transport client
      from ES under the hood, which is not recommended (should move to the REST
      API client).
      -

      Otto:  We should keep the people who put together #619
      <https://github.com/apache/metron/pull/619> involved (i.e. understand
      their wants and needs) with the more recent ES 5 changes, and
any follow-on
      PRs.


Release Status and Short Term Roadmap (Michael)

   -

   Matt:  Looking to do in the near term, RC2.
   -

   Otto:  Should we have a skip one branch for bigger changes, instead of
   cherry-picking?  May help get larger changes into a release.


Release process WRT formalized upgrade and installation instructions to be
included as a part of a release (Jon)

   -

   Justin and Jon think that we need to improve our process to do
   Upgrading.md, install guide, and upgrading guides should be
   -

   Discuss thread on How to get Upgrade testing feasible or better
   technically


Any concerns/questions with the secondary repo for bro (Jon)

   -

   Jon:  Looking to receive feedback on the split, address any concerns,
   etc.
   -

   Nick:  We can probably plan to continue to align the release process
   with metron, can revisit if this becomes an issue.
   -

   Otto:  Until metron and the bro plugin are managed separately (i.e. if
   you’d want to upgrade/manage the bro plugin separate from metron) this
   should not be an issue.
   -

      Jon:  Shouldn’t be an issue, can handle as it comes.  Many people who
      use bro packages probably will just pull from master and not a specific
      release, so pressure to do releases should be low.
      -

   Jon:  We rev versions for metron immediately after a release, but given
   the plugin may not change significantly or at all between metron releases,
   we should delay until this is necessary.




   There will be a few discussion threads that come out of this on

   * reviewing and contributing guides

   * barrier to entry for deployment

   * ease of deployment for testing and review

   * upgrading


   Thanks to everyone who attended.



   ottO

Re: [MEETING NOTES] 12/18/17 Developer Community Meeting

Posted by Otto Fowler <ot...@gmail.com>.
Please not:

1. Feel free to comment on anything here on the list
2. It is not Jon Zeolla’s fault that the formatting is wrong


On December 18, 2017 at 14:52:36, Otto Fowler (ottobackwards@gmail.com)
wrote:

2017-12-18 Dev Community Meeting

Agenda

   -

   Call for reviewers, ideas how to get more involvement, what people can
   do to help (Otto Fowler)
   -

   Feature branches : we have two now, what are they and how are we going
   to work on them (Otto Fowler)
   -

   ES 5.6 upgrade (Michael Miklavcic)
   -

   Release Status (Michael Miklavcic)
   -

   Short Term Roadmap (Michael Miklavcic)
   -

   Release process WRT formalized upgrade and installation instructions to
   be included as a part of a release (Jon Zeolla)
   -

   Any concerns/questions with the secondary repo for bro. (Jon Zeolla)


Attendees

Jon Zeolla, James Sirota, Matt Foley, Otto Fowler, Hakan Akansel, Justin
Leet, Michael Miklavcic, Nick Allen, Ryan Merriman, Laurens Vets

Discussion

Call for reviewers, ideas how to get more involvement, what people can do
to help (Otto)

   -

   Nick Agrees, suggests potentially simplifying the review process.
   -

   Certain stellar functions that implement certain algos are difficult to
   review properly and rely heavily on the initial implementer.
   -

   Michael suggests heavy focus/scrutiny of the testing/documentation of PRs
   -

   Otto suggests that the bar to spin up Metron may be too high, and could
   simplify the full-dev/PoC spin up.  Justin agrees.
   -

   Three suggested DISCUSS threads
   -
      -

      What’s a better way for us to document reviews and contributions in
      Metron?
      -
         1.

         How to overcome developer inertia for spinning up new envs such as
         testing ansible changes or similar
         -

      How can we lower the barrier to entry for new users to Metron?
      -

   Need to keep multiple PRs and feature branches top of mind to simplify
   review.


Feature Branches (Otto)

   -

   METRON-777
   <https://issues.apache.org/jira/projects/METRON/issues/METRON-777?filter=allopenissues>
   and METRON-1344
   <https://issues.apache.org/jira/projects/METRON/issues/METRON-1344?filter=allissues>
   -

   Are we all comfortable with how we use FBs?
   -
      -

      Ryan, how do we manage follow-on PRs for a FB?
      -

      Try to avoid bugfixes that would be useful to master being put solely
      in a FB.
      -

      All FB processes should align with our policies to commit/review
      against master, with a slightly higher tolerance for instability.
      -
         -

         I.e. Some interim steps may create regressions, but we should
         consider being comfortable with this in order to simplify review
         -

      Still feeling this out, maybe a future DISCUSS on how to determine if
      you should be creating a FB.
      -

      Consider FB-specific documentation to identify what/where/why/how.
       Otto has an example here
      <https://cwiki.apache.org/confluence/display/METRON/Metron+Extension+System+and+Parser+Extensions>
      .


ES 5.6 upgrade (Michael)

   -

   Michael:  Should be ready for review, looking for testing, etc.  Could
   use help with a multinode instance, performance testing, etc..
   METRON-939
   <https://issues.apache.org/jira/projects/METRON/issues/METRON-939?filter=allopenissues>
   (#840 <https://github.com/apache/metron/pull/840>)
   -

   Otto:  Unclear on what versions of ES Metron should run on.
   -
      -

      Michael:  Looking to support only ES 5.6.2, unable to currently support
      multiple versions of ES due to the complexity/testing reqs.
      -

   Otto:  Unclear on status of #619
   <https://github.com/apache/metron/pull/619>.
   -
      -

      Michael:  This is a subset of the Xpack work, and that xpack support
      is currently planning to be a follow-on.  Still using the
transport client
      from ES under the hood, which is not recommended (should move to the REST
      API client).
      -

      Otto:  We should keep the people who put together #619
      <https://github.com/apache/metron/pull/619> involved (i.e. understand
      their wants and needs) with the more recent ES 5 changes, and
any follow-on
      PRs.


Release Status and Short Term Roadmap (Michael)

   -

   Matt:  Looking to do in the near term, RC2.
   -

   Otto:  Should we have a skip one branch for bigger changes, instead of
   cherry-picking?  May help get larger changes into a release.


Release process WRT formalized upgrade and installation instructions to be
included as a part of a release (Jon)

   -

   Justin and Jon think that we need to improve our process to do
   Upgrading.md, install guide, and upgrading guides should be
   -

   Discuss thread on How to get Upgrade testing feasible or better
   technically


Any concerns/questions with the secondary repo for bro (Jon)

   -

   Jon:  Looking to receive feedback on the split, address any concerns,
   etc.
   -

   Nick:  We can probably plan to continue to align the release process
   with metron, can revisit if this becomes an issue.
   -

   Otto:  Until metron and the bro plugin are managed separately (i.e. if
   you’d want to upgrade/manage the bro plugin separate from metron) this
   should not be an issue.
   -
      -

      Jon:  Shouldn’t be an issue, can handle as it comes.  Many people who
      use bro packages probably will just pull from master and not a specific
      release, so pressure to do releases should be low.
      -

   Jon:  We rev versions for metron immediately after a release, but given
   the plugin may not change significantly or at all between metron releases,
   we should delay until this is necessary.




   There will be a few discussion threads that come out of this on

   * reviewing and contributing guides

   * barrier to entry for deployment

   * ease of deployment for testing and review

   * upgrading


   Thanks to everyone who attended.



   ottO