[DISCUSSION] Docker OS support

[Ning Wang <wa...@gmail.com>]

Hi,

We have talked about it a few times in different places. Let's make a
decision here. We don't have enough resources to support many docker OSes
(technically we only support Debian, Ubuntu, and CentOS but each of them
has multiple versions).

Things we have agreed on (my understanding)
- for binary release, we are going to choose one OS one version. I
remember that Debian 9 was the decision. An open question is: do we want to
switch to Debian 10 now? We haven't had a binary release yet, so there is
still time. As the main target, I think 1. the image should compile
successfully. and 2 the image should be tested: the tools and an example
topology.

- for source release, we haven't talked about it very much. Personally I
feel it is a reasonable expectation that, if we have the Docker file in the
core folders, it should at least compile. Otherwise, it's a failed build
hence a failed release. We are not going to test run the tools and examples
for each release though.

Finally, a question about OS versions. I believe at least two releases
should be supported because most people don't upgrade to the latest version
when it is out in more serious scenarios, hence IMO the second oldest could
often be more useful than the latest one from a convenience point of view.
On the other hand, this is too flexible, and we will spend time thinking
about it again. I think a clear guideline could be helpful to us. For
example:
- We only include only Debian (popular in the server world) and Ubuntu
(popular in servers and workstations).
- for the main OS (Debian) we choose the latest to compile and test for our
binary release.
- for a given OS, only two versions (most likely the most and the second
recent releases) are included in the core source code and they need to be
fixed ASAP if they don't compile.
- OSs and other versions could be included in a special folder and but
there is no guarantee that the code will compile. Fixes from the community
are welcome.


How do you guys think?
--ning

Re: [DISCUSSION] Docker OS support

[Ning Wang <wa...@gmail.com>]

A checklist of things to check could be great. We should create two, one
for source release and one for binary release.

It is agood point that Debian and Ubuntu are similar, so CentOS might give
us more coverage. Between Debian and Ubuntu, my personal vote might be
slightly towards Ubuntu because of the popularity in both desktop and
server world. But I am ok with either one.

Ubuntu16 is a bit tricky to me. If we drop it and go to 18+20, we are
supporting only the releases of the most recent 2 years which is really
short period of time for people to adopt.




On Sun, May 17, 2020 at 10:41 AM Nicholas Nezis <ni...@gmail.com>
wrote:

> Some thoughts I'd like to add to the discussion:
>
> Debian and Ubuntu are somewhat similar. Would it make sense to pick one of
> those and pair with Centos?
>
> I've run into issues with Ubuntu images having DNS issues in Kubernetes.
> https://github.com/kubernetes/kubeadm/issues/787
>
> Ubuntu 14.04 should be removed. It was end of life last year. For those of
> you using Heron in production, is anyone using Ubuntu 16? My vote would be
> to remove it also. For what it's worth, my use case is in K8s and we use
> the Centos7 image on a Centos 7 OS.
>
> We should do a test of the official image and helm chart in Kubernetes
> (maybe with Kind https://kind.sigs.k8s.io/ ) as part of our release
> process. This might be nice to add as an integration test.
>
>
>
>
>
>
>
> On Sun, May 17, 2020, 12:37 PM Ning Wang <wa...@gmail.com> wrote:
>
> > That is a good point. We need to adjust the OS version accordingly too.
> >
> > On Sun, May 17, 2020 at 2:28 AM Windham Wong <wi...@stormeye.io>
> > wrote:
> >
> > >
> > > debian 10 has no python 2.7 support. Ubuntu 16.04 has no python 3.7+
> dev
> > > library support. hope this helps
> > > --
> > > Sent from myMail for Android Sunday, 17 May 2020, 11:34AM +08:00 from
> > Ning
> > > Wang  wangninggm@gmail.com :
> > >
> > > >Hi,
> > > >
> > > >We have talked about it a few times in different places. Let's make a
> > > >decision here. We don't have enough resources to support many docker
> > OSes
> > > >(technically we only support Debian, Ubuntu, and CentOS but each of
> them
> > > >has multiple versions).
> > > >
> > > >Things we have agreed on (my understanding)
> > > >- for binary release, we are going to choose one OS one version. I
> > > >remember that Debian 9 was the decision. An open question is: do we
> want
> > > to
> > > >switch to Debian 10 now? We haven't had a binary release yet, so there
> > is
> > > >still time. As the main target, I think 1. the image should compile
> > > >successfully. and 2 the image should be tested: the tools and an
> example
> > > >topology.
> > > >
> > > >- for source release, we haven't talked about it very much.
> Personally I
> > > >feel it is a reasonable expectation that, if we have the Docker file
> in
> > > the
> > > >core folders, it should at least compile. Otherwise, it's a failed
> build
> > > >hence a failed release. We are not going to test run the tools and
> > > examples
> > > >for each release though.
> > > >
> > > >Finally, a question about OS versions. I believe at least two releases
> > > >should be supported because most people don't upgrade to the latest
> > > version
> > > >when it is out in more serious scenarios, hence IMO the second oldest
> > > could
> > > >often be more useful than the latest one from a convenience point of
> > view.
> > > >On the other hand, this is too flexible, and we will spend time
> thinking
> > > >about it again. I think a clear guideline could be helpful to us. For
> > > >example:
> > > >- We only include only Debian (popular in the server world) and Ubuntu
> > > >(popular in servers and workstations).
> > > >- for the main OS (Debian) we choose the latest to compile and test
> for
> > > our
> > > >binary release.
> > > >- for a given OS, only two versions (most likely the most and the
> second
> > > >recent releases) are included in the core source code and they need to
> > be
> > > >fixed ASAP if they don't compile.
> > > >- OSs and other versions could be included in a special folder and but
> > > >there is no guarantee that the code will compile. Fixes from the
> > community
> > > >are welcome.
> > > >
> > > >
> > > >How do you guys think?
> > > >--ning
> > >
> >
>

Re: [DISCUSSION] Docker OS support

[Ning Wang <wa...@gmail.com>]

Good to know! Thx.

On Sun, May 17, 2020 at 2:12 PM H W <hu...@gmail.com> wrote:

> We use Centos 7 as well.
>
> On Sun, May 17, 2020 at 10:41 AM Nicholas Nezis <ni...@gmail.com>
> wrote:
>
> > Some thoughts I'd like to add to the discussion:
> >
> > Debian and Ubuntu are somewhat similar. Would it make sense to pick one
> of
> > those and pair with Centos?
> >
> > I've run into issues with Ubuntu images having DNS issues in Kubernetes.
> > https://github.com/kubernetes/kubeadm/issues/787
> >
> > Ubuntu 14.04 should be removed. It was end of life last year. For those
> of
> > you using Heron in production, is anyone using Ubuntu 16? My vote would
> be
> > to remove it also. For what it's worth, my use case is in K8s and we use
> > the Centos7 image on a Centos 7 OS.
> >
> > We should do a test of the official image and helm chart in Kubernetes
> > (maybe with Kind https://kind.sigs.k8s.io/ ) as part of our release
> > process. This might be nice to add as an integration test.
> >
> >
> >
> >
> >
> >
> >
> > On Sun, May 17, 2020, 12:37 PM Ning Wang <wa...@gmail.com> wrote:
> >
> > > That is a good point. We need to adjust the OS version accordingly too.
> > >
> > > On Sun, May 17, 2020 at 2:28 AM Windham Wong <windham.wong@stormeye.io
> >
> > > wrote:
> > >
> > > >
> > > > debian 10 has no python 2.7 support. Ubuntu 16.04 has no python 3.7+
> > dev
> > > > library support. hope this helps
> > > > --
> > > > Sent from myMail for Android Sunday, 17 May 2020, 11:34AM +08:00 from
> > > Ning
> > > > Wang  wangninggm@gmail.com :
> > > >
> > > > >Hi,
> > > > >
> > > > >We have talked about it a few times in different places. Let's make
> a
> > > > >decision here. We don't have enough resources to support many docker
> > > OSes
> > > > >(technically we only support Debian, Ubuntu, and CentOS but each of
> > them
> > > > >has multiple versions).
> > > > >
> > > > >Things we have agreed on (my understanding)
> > > > >- for binary release, we are going to choose one OS one version. I
> > > > >remember that Debian 9 was the decision. An open question is: do we
> > want
> > > > to
> > > > >switch to Debian 10 now? We haven't had a binary release yet, so
> there
> > > is
> > > > >still time. As the main target, I think 1. the image should compile
> > > > >successfully. and 2 the image should be tested: the tools and an
> > example
> > > > >topology.
> > > > >
> > > > >- for source release, we haven't talked about it very much.
> > Personally I
> > > > >feel it is a reasonable expectation that, if we have the Docker file
> > in
> > > > the
> > > > >core folders, it should at least compile. Otherwise, it's a failed
> > build
> > > > >hence a failed release. We are not going to test run the tools and
> > > > examples
> > > > >for each release though.
> > > > >
> > > > >Finally, a question about OS versions. I believe at least two
> releases
> > > > >should be supported because most people don't upgrade to the latest
> > > > version
> > > > >when it is out in more serious scenarios, hence IMO the second
> oldest
> > > > could
> > > > >often be more useful than the latest one from a convenience point of
> > > view.
> > > > >On the other hand, this is too flexible, and we will spend time
> > thinking
> > > > >about it again. I think a clear guideline could be helpful to us.
> For
> > > > >example:
> > > > >- We only include only Debian (popular in the server world) and
> Ubuntu
> > > > >(popular in servers and workstations).
> > > > >- for the main OS (Debian) we choose the latest to compile and test
> > for
> > > > our
> > > > >binary release.
> > > > >- for a given OS, only two versions (most likely the most and the
> > second
> > > > >recent releases) are included in the core source code and they need
> to
> > > be
> > > > >fixed ASAP if they don't compile.
> > > > >- OSs and other versions could be included in a special folder and
> but
> > > > >there is no guarantee that the code will compile. Fixes from the
> > > community
> > > > >are welcome.
> > > > >
> > > > >
> > > > >How do you guys think?
> > > > >--ning
> > > >
> > >
> >
>

Re: [DISCUSSION] Docker OS support

[Josh Fischer <jo...@joshfischer.io>]

What is it?  I/someone else can give some help if needed.

On Fri, Jun 5, 2020 at 7:20 PM Ning Wang <wa...@gmail.com> wrote:

> That is the one I am blocked on. Will need to do another full tests before
> starting the vote.
>
> On Fri, Jun 5, 2020 at 2:22 PM Josh Fischer <jo...@joshfischer.io> wrote:
>
> > I just opened a PR that fixes javadoc generation for the website build.
> Do
> > we have anything else outstanding before we can start a vote on the next
> > release candidate?
> >
> > On Wed, May 27, 2020 at 11:31 AM Ning Wang <wa...@gmail.com> wrote:
> >
> > > Yeah. We have chosen Debian 10 for the binary release and clean up the
> > > license requirements for the first binary release.
> > >
> > > In the source level, we would also like to include a few
> > platforms/versions
> > > in the main source code to be "supported" and the others will be moved
> to
> > > like a "extra" folder and not actively supported/tested.
> > >
> > >
> > > On Wed, May 27, 2020 at 9:22 AM Dave Fisher <wa...@apache.org> wrote:
> > >
> > > > I thought that the purpose of this discussion was to find the one or
> > two
> > > > platforms for Heron to support in order to keep down the complexity
> of
> > > > making the first binary release for the community.
> > > >
> > > > Other platforms could be on a roadmap for future releases.
> > > >
> > > > Regards,
> > > > Dave
> > > >
> > > > > On May 26, 2020, at 12:26 PM, Ning Wang <wa...@gmail.com>
> > wrote:
> > > > >
> > > > > My concern about "latest" is that I feel most people hold it for
> some
> > > > time
> > > > > before migrating to the latest version (if it ever happens). If
> there
> > > are
> > > > > numbers, it could make sense to use the most popular version (plus
> > the
> > > > > other criterias).
> > > > >
> > > > > On Tue, May 26, 2020 at 12:14 PM Oliver Bristow <
> > > > oliver@oliverbristow.co.uk>
> > > > > wrote:
> > > > >
> > > > >> Sorry if I missed the rationale for Ubuntu 18.04 instead of 20.04,
> > but
> > > > >> would the latest LTS be a better choice? 18.04 was released before
> > > > Debian
> > > > >> 10, so that may be an argument for 20.04 re. compatibility if
> Debian
> > > 10
> > > > is
> > > > >> the standard.
> > > > >>
> > > > >> There's already a 20.04 Dockerfile + I It looks like the vagrant
> > setup
> > > > will
> > > > >> get an upgrade from 14.04 to 20.04 as a part of the python3
> upgrade,
> > > and
> > > > >> seems to run fine so far.
> > > > >>
> > > > >> On Tue, 26 May 2020 at 19:30, Josh Fischer <jo...@joshfischer.io>
> > > wrote:
> > > > >>
> > > > >>> Thanks for further clarifying. Yes, that is correct.
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>> On Tue, May 26, 2020 at 1:16 PM Ning Wang <wa...@gmail.com>
> > > > wrote:
> > > > >>>
> > > > >>>> +1.
> > > > >>>>
> > > > >>>> So my understanding is:
> > > > >>>> binary (in future release): Debian 10.
> > > > >>>> source: Debian 9/10 + ubuntu 18.04 + centos 7. version numbers
> > will
> > > be
> > > > >>>> updated in the future after more discussions/votes.
> > > > >>>>
> > > > >>>> others will be moved to a "extra" folder only for reference.
> > > > >>>>
> > > > >>>>
> > > > >>>>
> > > > >>>> On Tue, May 26, 2020 at 6:13 AM Josh Fischer <
> josh@joshfischer.io
> > >
> > > > >>> wrote:
> > > > >>>>
> > > > >>>>> Hi All,
> > > > >>>>>
> > > > >>>>> Based on what I've read on this thread a combination of
> > > CentOS/Debian
> > > > >>>> would
> > > > >>>>> be the better options for us to support multiple versions for
> the
> > > > >>>>> community.  We will also support one version of Ubuntu.
> > > > >>>>>
> > > > >>>>> So..... I will start a vote in 72 hours that calls on the vote
> on
> > > > >> this.
> > > > >>>>> I'm going to write down my understanding of what this thread
> > covers
> > > > >> to
> > > > >>>> get
> > > > >>>>> some actionable topics to vote on.  If I've written anything
> down
> > > > >>>>> incorrectly please speak up before we start a vote.
> > > > >>>>>
> > > > >>>>> Debian:
> > > > >>>>>    - Debian10 is the official docker image to support and be
> > > > >> published
> > > > >>>> to
> > > > >>>>> Docker Hub, we will also support Debian9.
> > > > >>>>> CentOS:
> > > > >>>>>    - We will support CentOS7.  Note, I do not think it has
> native
> > > > >>>>> Python3.7 support, but we are also not there yet.
> > > > >>>>> Ubuntu:
> > > > >>>>>    - We will still support one version. At this time the
> version
> > is
> > > > >>>> 18.04.
> > > > >>>>>
> > > > >>>>> We will upgrade supported versions of each OS as
> > > updates/deprecation
> > > > >> of
> > > > >>>>> support are needed.
> > > > >>>>>
> > > > >>>>> Again let's wait 72 hours before starting a vote.  Also please
> > > check
> > > > >>> that
> > > > >>>>> the statements I wrote down are what you understood!
> > > > >>>>>
> > > > >>>>> See below for notes on lazy consensus:
> > > > >>>>> https://community.apache.org/committers/lazyConsensus.html
> > > > >>>>>
> > > > >>>>> - Josh
> > > > >>>>>
> > > > >>>>>
> > > > >>>>> On Wed, May 20, 2020 at 9:41 PM Ning Wang <
> wangninggm@gmail.com>
> > > > >>> wrote:
> > > > >>>>>
> > > > >>>>>> +1
> > > > >>>>>>
> > > > >>>>>> On Wed, May 20, 2020 at 7:26 PM Nicholas Nezis <
> > > > >>>> nicholas.nezis@gmail.com
> > > > >>>>>>
> > > > >>>>>> wrote:
> > > > >>>>>>
> > > > >>>>>>> I like those options.
> > > > >>>>>>>
> > > > >>>>>>> On Wed, May 20, 2020 at 10:15 PM Josh Fischer <
> > > > >> josh@joshfischer.io
> > > > >>>>
> > > > >>>>>> wrote:
> > > > >>>>>>>
> > > > >>>>>>>> Let's wait 3 more days for anyone to give input.  Then we
> can
> > > > >>>> start a
> > > > >>>>>>> vote
> > > > >>>>>>>> to make a final decision on which containers for us to
> > support.
> > > > >>>> Keep
> > > > >>>>>> in
> > > > >>>>>>>> mind that even though we do need to plan for Python3 support
> > > > >>> (There
> > > > >>>>> is
> > > > >>>>>>> work
> > > > >>>>>>>> in progress now) we do not need it as of yet.
> > > > >>>>>>>>
> > > > >>>>>>>> - Josh
> > > > >>>>>>>>
> > > > >>>>>>>> On Wed, May 20, 2020 at 8:34 PM thinker0 <
> thinker0@gmail.com>
> > > > >>>> wrote:
> > > > >>>>>>>>
> > > > >>>>>>>>> My experience is mainly for CentOS 7, so I think it would
> be
> > > > >>> nice
> > > > >>>>> if
> > > > >>>>>> it
> > > > >>>>>>>> was
> > > > >>>>>>>>> included.
> > > > >>>>>>>>>
> > > > >>>>>>>>> 2020년 5월 21일 (목) 오전 9:56, Josh Fischer <
> josh@joshfischer.io
> > > > >>> 님이
> > > > >>>> 작성:
> > > > >>>>>>>>>
> > > > >>>>>>>>>> Well thought out Windham.  I like how you narrowed the
> > > > >>> criteria
> > > > >>>>>> down
> > > > >>>>>>> to
> > > > >>>>>>>>>> Debian10/CentOS7(depending on PY3 support) thinking about
> > > > >>>> overall
> > > > >>>>>>>> support
> > > > >>>>>>>>>> and what others would need/want.
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> +1
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> - Josh
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> On Mon, May 18, 2020 at 8:39 PM Windham Wong <
> > > > >>>>>>> windham.wong@stormeye.io
> > > > >>>>>>>>>
> > > > >>>>>>>>>> wrote:
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>> My experience is that, people tends to pick their
> > > > >> container
> > > > >>>>> image
> > > > >>>>>>> OS
> > > > >>>>>>>>>> based
> > > > >>>>>>>>>>> on few criterias only:
> > > > >>>>>>>>>>> 1) image size, alpine always win, then debian/centos and
> > > > >>> 3rd
> > > > >>>> is
> > > > >>>>>>>>>>> ubuntu/centos. alpine is always the smallest but not much
> > > > >>>>> library
> > > > >>>>>>>>>> provided
> > > > >>>>>>>>>>> and require to compile manually. debian is always good
> > > > >>>> because
> > > > >>>>> it
> > > > >>>>>>> has
> > > > >>>>>>>>>>> apt-get to provide loads of library. centos is cool as
> > > > >>> debian
> > > > >>>>>>> because
> > > > >>>>>>>>> it
> > > > >>>>>>>>>>> also has good yum repo. ubuntu is always the last because
> > > > >>> it
> > > > >>>> is
> > > > >>>>>>> fat,
> > > > >>>>>>>>> but
> > > > >>>>>>>>>> it
> > > > >>>>>>>>>>> provides very large range of libraries to use.
> > > > >>>>>>>>>>> 2) kernel feature, as mentioned last email, ubuntu has
> > > > >>> issue
> > > > >>>>> with
> > > > >>>>>>> DNS
> > > > >>>>>>>>> and
> > > > >>>>>>>>>>> some sort of network discover issues. I remember alpine
> > > > >> has
> > > > >>>>> some
> > > > >>>>>> as
> > > > >>>>>>>>> well,
> > > > >>>>>>>>>>> because it is very plain, requires many fine tunes be
> > > > >> done
> > > > >>>>>>> manually.
> > > > >>>>>>>> in
> > > > >>>>>>>>>>> this case, centos and debian should win.
> > > > >>>>>>>>>>> 3) linux stream, obviously, alpine, debian/ubuntu,
> > > > >> centos,
> > > > >>>> are
> > > > >>>>> 3
> > > > >>>>>>>>>> different
> > > > >>>>>>>>>>> streams that use different native tools. Poeple just pick
> > > > >>>> their
> > > > >>>>>>> base
> > > > >>>>>>>>>> image
> > > > >>>>>>>>>>> based on what they often use in their life. However, I
> > > > >> see
> > > > >>> a
> > > > >>>>> lot
> > > > >>>>>>>> debian
> > > > >>>>>>>>>> and
> > > > >>>>>>>>>>> alpine on docker hub because they are just smaller in
> > > > >> size.
> > > > >>>>>>>>>>> 4. library support. Python library is a very good example
> > > > >>> in
> > > > >>>>> this
> > > > >>>>>>>> case.
> > > > >>>>>>>>>> As
> > > > >>>>>>>>>>> Heron is turning to Python 3, which specifically Python
> > > > >>> 3.7+,
> > > > >>>>>> some
> > > > >>>>>>>>>>> discontinued based OS version does not provide the
> > > > >> support
> > > > >>>>>> anymore.
> > > > >>>>>>>> We
> > > > >>>>>>>>>> have
> > > > >>>>>>>>>>> tested out that, debian9 supports only up to Python3.5,
> > > > >>>> ubuntu
> > > > >>>>>>> 14.04
> > > > >>>>>>>>> does
> > > > >>>>>>>>>>> not support python3 and ubuntu 16.04 supports up to
> > > > >> Python
> > > > >>>> 3.5
> > > > >>>>>>> only.
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> I am not a big fan of using centos for docker. Concluding
> > > > >>>> all 4
> > > > >>>>>>>> points
> > > > >>>>>>>>>>> above, I believe keeping centos7 (if it supports
> > > > >> python37),
> > > > >>>>>>> debian10
> > > > >>>>>>>>> (not
> > > > >>>>>>>>>>> debian9), ubuntu 18.04 (supports python3.7+) would be the
> > > > >>>> best
> > > > >>>>>>>> solution
> > > > >>>>>>>>>> at
> > > > >>>>>>>>>>> the moment. We have to cut off the old images due to our
> > > > >>>>> library
> > > > >>>>>>>>> upgrade
> > > > >>>>>>>>>>> and this is unavoidable. Otherwise, we would have tons of
> > > > >>>>> issues
> > > > >>>>>>>>> sending
> > > > >>>>>>>>>> in
> > > > >>>>>>>>>>> about library support in the future and the workload
> > > > >> could
> > > > >>>> be a
> > > > >>>>>> lot
> > > > >>>>>>>>> more
> > > > >>>>>>>>>>> for just answering questions.
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> Windham Wong
> > > > >>>>>>>>>>> Co-Founder, Technical Director of
> > > > >>>>>>>>>>> Stormeye.io, Hong Kong Managed Security Operation Center
> > > > >>>>> Limited
> > > > >>>>>>>>>>> Email // windham.wong@stormeye.io (mailto:
> > > > >>>>>> windham.wong@stormeye.io
> > > > >>>>>>> )
> > > > >>>>>>>>>>> Phone // +852_3590_2212_|_+852_9832_0707
> > > > >> (tel:+85235902212)
> > > > >>>>>>>>>>> Fax // +852_3590_2202 (tel:+852_3590_2202)
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> On 5月 19 2020, at 6:09 早上, Josh Fischer <
> > > > >>> josh@joshfischer.io
> > > > >>>>>
> > > > >>>>>>> wrote:
> > > > >>>>>>>>>>>> Hi All, I think using CentOS and either Debian or
> > > > >> Ubuntu
> > > > >>>> is a
> > > > >>>>>>> good
> > > > >>>>>>>>>> idea.
> > > > >>>>>>>>>>> I would pick Debian over Ubuntu for the reason that Nick
> > > > >>>>> pointed
> > > > >>>>>>> out
> > > > >>>>>>>>> in a
> > > > >>>>>>>>>>> previous email of DNS issues in Kubernetes. I"ve copy and
> > > > >>>>> pasted
> > > > >>>>>> a
> > > > >>>>>>>>>> section
> > > > >>>>>>>>>>> of Nick's previous email. ### Start I've run into issues
> > > > >>> with
> > > > >>>>>>> Ubuntu
> > > > >>>>>>>>>> images
> > > > >>>>>>>>>>> having DNS issues in Kubernetes.
> > > > >>>>>>>>>>> https://github.com/kubernetes/kubeadm/issues/787 ### End
> > > > >>> As
> > > > >>>>> for
> > > > >>>>>>> what
> > > > >>>>>>>>>> ends
> > > > >>>>>>>>>>> up being the official Docker container I'm fine with
> > > > >> either
> > > > >>>>>> choice
> > > > >>>>>>>>> Debian
> > > > >>>>>>>>>>> or CentOS. On Mon, May 18, 2020 at 2:10 AM Ning Wang
> > > > >>> wrote: >
> > > > >>>>>> Yeah.
> > > > >>>>>>>> For
> > > > >>>>>>>>>>> binary release, the license is a huge factor. > > On Sun,
> > > > >>> May
> > > > >>>>> 17,
> > > > >>>>>>>> 2020
> > > > >>>>>>>>> at
> > > > >>>>>>>>>>> 2:32 PM Josh Fischer wrote: > > > Hi All, > > > > When it
> > > > >>>> comes
> > > > >>>>>> to
> > > > >>>>>>>>>> choosing
> > > > >>>>>>>>>>> the official Apache Docker image for Heron we > > have to
> > > > >>>> think
> > > > >>>>>> of
> > > > >>>>>>>> the
> > > > >>>>>>>>>>> licensing issues first. Everything that is > installed >
> > > > >>>
> > > > >>>> into
> > > > >>>>>> the
> > > > >>>>>>>>>>> container at build time with the exception of the "FROM"
> > > > >>>
> > > > >>>>
> > > > >>>>>>>> statement
> > > > >>>>>>>>> in
> > > > >>>>>>>>>>> the Dockerfile must be ALv2 compatible. OpenJDK is under
> > > > >>>
> > > > >>>>> GPLv2
> > > > >>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>>> which
> > > > >>>>>>>>>>> is not compatible with Apache. However we can get around
> > > > >>> this
> > > > >>>>> if
> > > > >>>>>>>
> > > > >>>>>>> we
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> bring the OpenJDK into the container via a FROM
> > > > >> statement.
> > > > >>>> I've
> > > > >>>>>>>> copied
> > > > >>>>>>>>> a
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>> reply from Dave in our dev list in a previous email.
> > > > >> See
> > > > >>>>> below.
> > > > >>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> ######## Start ######## > > Regarding OpenJDK and GPL2 -
> > > > >>> here
> > > > >>>>> is
> > > > >>>>>>> what
> > > > >>>>>>>>>> Roman
> > > > >>>>>>>>>>> the VP, Legal wrote when > > answering Beam’s questions.
> > > > >>>
> > > > >>>>
> > > > >>>>>
> > > > >>>>>>
> > > > >>>>>>>> Roman
> > > > >>>>>>>>>>> Shaposhnik commented on LEGAL-503: > >
> > > > >>>>>>>>>>> ---------------------------------------- > > > > Hey
> > > > >>> [~altay]
> > > > >>>>> if
> > > > >>>>>>> you
> > > > >>>>>>>>>> would
> > > > >>>>>>>>>>> like to continue linking to the Docker release > >
> > > > >> artifact
> > > > >>>>> from
> > > > >>>>>>> the
> > > > >>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>> https://beam.apache.org > > you will have: > > 1.
> > > > >>> Transition
> > > > >>>>> to
> > > > >>>>>>> the
> > > > >>>>>>>>>>> official ASF dockerhub org: > >
> > > > >>>>> https://hub.docker.com/u/apache
> > > > >>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>> 2.
> > > > >>>>>>>>>>> Start including that binary convenience artifact into
> > > > >> your
> > > > >>>> VOTE
> > > > >>>>>>>>
> > > > >>>>>>>>>> threads
> > > > >>>>>>>>>>> on Beam releases > > 3. Make sure that all Cat-X licenses
> > > > >>> are
> > > > >>>>>> ONLY
> > > > >>>>>>>>>> brought
> > > > >>>>>>>>>>> into your > > container via FROM statements > > ########
> > > > >>> End
> > > > >>>>>>>> ######## >
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>> So at the end of the day, we need to use a container
> > > > >> that
> > > > >>>> has
> > > > >>>>>>> Java
> > > > >>>>>>>>>
> > > > >>>>>>>>>>> already > > installed in it so that when we create our
> > > > >>>> official
> > > > >>>>>>> image
> > > > >>>>>>>>> we
> > > > >>>>>>>>>>> can inherit > > that java containing image via a FROM
> > > > >>>>> statement.
> > > > >>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> -
> > > > >>>>>>>>>>> Josh > > > > > > On Sun, May 17, 2020 at 4:12 PM H W
> > > > >>> wrote: >
> > > > >>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>> We
> > > > >>>>>>>>>>> use Centos 7 as well. > > > > > > On Sun, May 17, 2020 at
> > > > >>>> 10:41
> > > > >>>>>> AM
> > > > >>>>>>>>>> Nicholas
> > > > >>>>>>>>>>> Nezis < > > nicholas.nezis@gmail.com> > > > wrote: > > >
> > > > >>>>>
> > > > >>>>>>>
> > > > >>>>>>>> Some
> > > > >>>>>>>>>>> thoughts I'd like to add to the discussion: > > > > > >
> > > > >>>>
> > > > >>>>>> Debian
> > > > >>>>>>>> and
> > > > >>>>>>>>>>> Ubuntu are somewhat similar. Would it make sense to pick
> > > > >>>
> > > > >>>> one
> > > > >>>>>>
> > > > >>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>> of
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>>> those and pair with Centos? > > > > > > > > I've run
> > > > >>> into
> > > > >>>>>>> issues
> > > > >>>>>>>>> with
> > > > >>>>>>>>>>> Ubuntu images having DNS issues in > > Kubernetes. > > >
> > > > >>>
> > > > >>>>>>>>>>> https://github.com/kubernetes/kubeadm/issues/787 > > > >
> > > > >>>>>
> > > > >>>>>>>
> > > > >>>>>>>>> Ubuntu
> > > > >>>>>>>>>>> 14.04 should be removed. It was end of life last year.
> > > > >> For
> > > > >>>>
> > > > >>>>>> those
> > > > >>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> of
> > > > >>>>>>>>>>>>>>> you using Heron in production, is anyone using
> > > > >>> Ubuntu
> > > > >>>>> 16?
> > > > >>>>>>> My
> > > > >>>>>>>>>> vote >
> > > > >>>>>>>>>>> would > > > be > > > > to remove it also. For what it's
> > > > >>>> worth,
> > > > >>>>> my
> > > > >>>>>>> use
> > > > >>>>>>>>>> case
> > > > >>>>>>>>>>> is in K8s and we > > use > > > > the Centos7 image on a
> > > > >>>> Centos
> > > > >>>>> 7
> > > > >>>>>>> OS.
> > > > >>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>> We should do a test of the official image and
> > > > >>> helm
> > > > >>>>>> chart
> > > > >>>>>>>> in >
> > > > >>>>>>>>>>> Kubernetes > > > > (maybe with Kind
> > > > >>>> https://kind.sigs.k8s.io/
> > > > >>>>> )
> > > > >>>>>> as
> > > > >>>>>>>>> part
> > > > >>>>>>>>>>> of our release > > > > process. This might be nice to add
> > > > >>> as
> > > > >>>> an
> > > > >>>>>>>>>> integration
> > > > >>>>>>>>>>> test. > > > > > > > > > > > > > > > > > > > > > > > > > >
> > > > >>>>>
> > > > >>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>> On
> > > > >>>>>>>>>>> Sun, May 17, 2020, 12:37 PM Ning Wang > wrote: > > > > >
> > > > >>>
> > > > >>>>
> > > > >>>>>
> > > > >>>>>>
> > > > >>>>>>> That
> > > > >>>>>>>>> is
> > > > >>>>>>>>>> a
> > > > >>>>>>>>>>> good point. We need to adjust the OS version accordingly
> > > > >>>
> > > > >>>>
> > > > >>>>>> too. >
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>> On Sun, May 17, 2020 at 2:28 AM Windham Wong <
> > > > >>>
> > > > >>>>
> > > > >>>>>>>>>>> windham.wong@stormeye.io > > > > > > > > > wrote: > > >
> > > > >>>
> > > > >>>>
> > > > >>>>>
> > > > >>>>>>
> > > > >>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>> debian 10 has no python 2.7 support. Ubuntu
> > > > >> 16.04
> > > > >>>> has
> > > > >>>>>> no
> > > > >>>>>>>>>> python >
> > > > >>>>>>>>>>>> 3.7+ > > > > dev > > > > > > library support. hope this
> > > > >>>> helps
> > > > >>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>> -- > > > > > > Sent from myMail for Android Sunday, 17
> > > > >> May
> > > > >>>>> 2020,
> > > > >>>>>>>>> 11:34AM
> > > > >>>>>>>>>>> +08:00 > > from > > > > > Ning > > > > > > Wang
> > > > >>>>>>> wangninggm@gmail.com
> > > > >>>>>>>>> : >
> > > > >>>>>>>>>>>>>>>>>>>>>>> Hi, > > > > > > > > > > > > > >We
> > > > >>> have
> > > > >>>>>>> talked
> > > > >>>>>>>>>> about
> > > > >>>>>>>>>>> it a few times in different places. Let's > > make > > >
> > > > >> a
> > > > >>>>
> > > > >>>>>
> > > > >>>>>>
> > > > >>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>>>> decision here. We don't have enough resources to support
> > > > >>>> many
> > > > >>>>>>
> > > > >>>>>>>
> > > > >>>>>>>>>> docker >
> > > > >>>>>>>>>>>>>>> OSes > > > > > > >(technically we only support
> > > > >>>> Debian,
> > > > >>>>>>>> Ubuntu,
> > > > >>>>>>>>>> and
> > > > >>>>>>>>>>> CentOS but each > > of > > > > them > > > > > > >has
> > > > >>> multiple
> > > > >>>>>>>>> versions).
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>>>>>>>>> Things we have agreed on (my
> > > > >>>>>>> understanding)
> > > > >>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>>>>> - for binary release, we are going to choose one
> > > > >> OS
> > > > >>>> one
> > > > >>>>>>
> > > > >>>>>>>>>> version. I
> > > > >>>>>>>>>>>>>>>>>> remember that Debian 9 was the decision. An
> > > > >>> open
> > > > >>>>>>>> question
> > > > >>>>>>>>>> is:
> > > > >>>>>>>>>>> do > > we > > > > want > > > > > > to > > > > > > >switch
> > > > >>> to
> > > > >>>>>> Debian
> > > > >>>>>>>> 10
> > > > >>>>>>>>>> now?
> > > > >>>>>>>>>>> We haven't had a binary release yet, so > > > there > > >
> > > > >>>>>
> > > > >>>>> is
> > > > >>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>>> still time. As the main target, I think 1. the image
> > > > >>>> should
> > > > >>>>>>
> > > > >>>>>>>
> > > > >>>>>>>>>> compile
> > > > >>>>>>>>>>>>>>>>>> successfully. and 2 the image should be
> > > > >>> tested:
> > > > >>>>> the
> > > > >>>>>>>> tools
> > > > >>>>>>>>>> and
> > > > >>>>>>>>>>> an > > > > example > > > > > > >topology. > > > > > > >
> > > > >>>>
> > > > >>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>> -
> > > > >>>>>>>>>> for
> > > > >>>>>>>>>>> source release, we haven't talked about it very much. > >
> > > > >>>>>
> > > > >>>>>>>>> Personally
> > > > >>>>>>>>>> I
> > > > >>>>>>>>>>>>>>>>>> feel it is a reasonable expectation that, if
> > > > >>> we
> > > > >>>>> have
> > > > >>>>>>> the
> > > > >>>>>>>>>>> Docker > > file > > > > in > > > > > > the > > > > > >
> > > > >>>> core
> > > > >>>>>>> folders,
> > > > >>>>>>>>> it
> > > > >>>>>>>>>>> should at least compile. Otherwise, it's a > failed > >
> > > > >>>>
> > > > >>>>>> build >
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>>> hence a failed release. We are not going to test run
> > > > >> the
> > > > >>>>>> tools >
> > > > >>>>>>>> and
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>> examples > > > > > > >for each release though. >
> > > > >>>
> > > > >>>>
> > > > >>>>>
> > > > >>>>>>
> > > > >>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>>>>> Finally, a question about OS versions. I believe
> > > > >> at
> > > > >>>>> least
> > > > >>>>>>> two
> > > > >>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> releases > > > > > > >should be supported because most
> > > > >>> people
> > > > >>>>>> don't
> > > > >>>>>>>>>> upgrade
> > > > >>>>>>>>>>> to the > > latest > > > > > > version > > > > > > >when
> > > > >> it
> > > > >>> is
> > > > >>>>> out
> > > > >>>>>>> in
> > > > >>>>>>>>> more
> > > > >>>>>>>>>>> serious scenarios, hence IMO the second > > > oldest > >
> > > > >>>
> > > > >>>>
> > > > >>>>>
> > > > >>>>>>
> > > > >>>>>>>> could
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>> often be more useful than the latest one from a
> > > > >>>>>>> convenience
> > > > >>>>>>>>>
> > > > >>>>>>>>>>> point > > of > > > > > view. > > > > > > >On the other
> > > > >>> hand,
> > > > >>>>> this
> > > > >>>>>>> is
> > > > >>>>>>>>> too
> > > > >>>>>>>>>>> flexible, and we will spend time > > > > thinking > > > >
> > > > >>>>>
> > > > >>>>>>>> about
> > > > >>>>>>>> it
> > > > >>>>>>>>>>> again. I think a clear guideline could be helpful to >
> > > > >> us.
> > > > >>>>
> > > > >>>>>
> > > > >>>>>>
> > > > >>>>>>> For
> > > > >>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>>>>> example: > > > > > > >- We only include only
> > > > >> Debian
> > > > >>>>>> (popular
> > > > >>>>>>>> in
> > > > >>>>>>>>>> the
> > > > >>>>>>>>>>> server world) and > > > Ubuntu > > > > > > >(popular in
> > > > >>>> servers
> > > > >>>>>> and
> > > > >>>>>>>>>>> workstations). > > > > > > >- for the main OS (Debian) we
> > > > >>>>> choose
> > > > >>>>>>> the
> > > > >>>>>>>>>> latest
> > > > >>>>>>>>>>> to compile and > > test > > > > for > > > > > > our > >
> > > > >>>>
> > > > >>>>>>
> > > > >>>>>>>>> binary
> > > > >>>>>>>>>>> release. > > > > > > >- for a given OS, only two versions
> > > > >>>> (most
> > > > >>>>>>>> likely
> > > > >>>>>>>>>> the
> > > > >>>>>>>>>>> most and > the > > > > second > > > > > > >recent
> > > > >> releases)
> > > > >>>> are
> > > > >>>>>>>>> included
> > > > >>>>>>>>>> in
> > > > >>>>>>>>>>> the core source code and they > > need > > > to > > > > >
> > > > >>> be
> > > > >>>>>
> > > > >>>>>>
> > > > >>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>>> fixed ASAP if they don't compile. > > > > > > >- OSs and
> > > > >>>> other
> > > > >>>>>>>>> versions
> > > > >>>>>>>>>>> could be included in a special folder > and > > > but > >
> > > > >>>>>
> > > > >>>>>>>
> > > > >>>>>>>>>> there
> > > > >>>>>>>>>> is
> > > > >>>>>>>>>>> no guarantee that the code will compile. Fixes from the >
> > > > >>>>>
> > > > >>>>>>>
> > > > >>>>>>>>>> community
> > > > >>>>>>>>>>>>>>>>>> are welcome. > > > > > > > > > > > > > > > >
> > > > >>>>>
> > > > >>>>>>>
> > > > >>>>>>>>> How
> > > > >>>>>>>>> do
> > > > >>>>>>>>>>> you guys think? > > > > > > >--ning > > > > > > > > > >
> > > > >>>>
> > > > >>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>
> > > > >>>>>>
> > > > >>>>>
> > > > >>>>
> > > > >>>
> > > > >>
> > > >
> > > >
> > >
> >
>

Re: [DISCUSSION] Docker OS support

[Ning Wang <wa...@gmail.com>]

That is the one I am blocked on. Will need to do another full tests before
starting the vote.

On Fri, Jun 5, 2020 at 2:22 PM Josh Fischer <jo...@joshfischer.io> wrote:

> I just opened a PR that fixes javadoc generation for the website build.  Do
> we have anything else outstanding before we can start a vote on the next
> release candidate?
>
> On Wed, May 27, 2020 at 11:31 AM Ning Wang <wa...@gmail.com> wrote:
>
> > Yeah. We have chosen Debian 10 for the binary release and clean up the
> > license requirements for the first binary release.
> >
> > In the source level, we would also like to include a few
> platforms/versions
> > in the main source code to be "supported" and the others will be moved to
> > like a "extra" folder and not actively supported/tested.
> >
> >
> > On Wed, May 27, 2020 at 9:22 AM Dave Fisher <wa...@apache.org> wrote:
> >
> > > I thought that the purpose of this discussion was to find the one or
> two
> > > platforms for Heron to support in order to keep down the complexity of
> > > making the first binary release for the community.
> > >
> > > Other platforms could be on a roadmap for future releases.
> > >
> > > Regards,
> > > Dave
> > >
> > > > On May 26, 2020, at 12:26 PM, Ning Wang <wa...@gmail.com>
> wrote:
> > > >
> > > > My concern about "latest" is that I feel most people hold it for some
> > > time
> > > > before migrating to the latest version (if it ever happens). If there
> > are
> > > > numbers, it could make sense to use the most popular version (plus
> the
> > > > other criterias).
> > > >
> > > > On Tue, May 26, 2020 at 12:14 PM Oliver Bristow <
> > > oliver@oliverbristow.co.uk>
> > > > wrote:
> > > >
> > > >> Sorry if I missed the rationale for Ubuntu 18.04 instead of 20.04,
> but
> > > >> would the latest LTS be a better choice? 18.04 was released before
> > > Debian
> > > >> 10, so that may be an argument for 20.04 re. compatibility if Debian
> > 10
> > > is
> > > >> the standard.
> > > >>
> > > >> There's already a 20.04 Dockerfile + I It looks like the vagrant
> setup
> > > will
> > > >> get an upgrade from 14.04 to 20.04 as a part of the python3 upgrade,
> > and
> > > >> seems to run fine so far.
> > > >>
> > > >> On Tue, 26 May 2020 at 19:30, Josh Fischer <jo...@joshfischer.io>
> > wrote:
> > > >>
> > > >>> Thanks for further clarifying. Yes, that is correct.
> > > >>>
> > > >>>
> > > >>>
> > > >>> On Tue, May 26, 2020 at 1:16 PM Ning Wang <wa...@gmail.com>
> > > wrote:
> > > >>>
> > > >>>> +1.
> > > >>>>
> > > >>>> So my understanding is:
> > > >>>> binary (in future release): Debian 10.
> > > >>>> source: Debian 9/10 + ubuntu 18.04 + centos 7. version numbers
> will
> > be
> > > >>>> updated in the future after more discussions/votes.
> > > >>>>
> > > >>>> others will be moved to a "extra" folder only for reference.
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>> On Tue, May 26, 2020 at 6:13 AM Josh Fischer <josh@joshfischer.io
> >
> > > >>> wrote:
> > > >>>>
> > > >>>>> Hi All,
> > > >>>>>
> > > >>>>> Based on what I've read on this thread a combination of
> > CentOS/Debian
> > > >>>> would
> > > >>>>> be the better options for us to support multiple versions for the
> > > >>>>> community.  We will also support one version of Ubuntu.
> > > >>>>>
> > > >>>>> So..... I will start a vote in 72 hours that calls on the vote on
> > > >> this.
> > > >>>>> I'm going to write down my understanding of what this thread
> covers
> > > >> to
> > > >>>> get
> > > >>>>> some actionable topics to vote on.  If I've written anything down
> > > >>>>> incorrectly please speak up before we start a vote.
> > > >>>>>
> > > >>>>> Debian:
> > > >>>>>    - Debian10 is the official docker image to support and be
> > > >> published
> > > >>>> to
> > > >>>>> Docker Hub, we will also support Debian9.
> > > >>>>> CentOS:
> > > >>>>>    - We will support CentOS7.  Note, I do not think it has native
> > > >>>>> Python3.7 support, but we are also not there yet.
> > > >>>>> Ubuntu:
> > > >>>>>    - We will still support one version. At this time the version
> is
> > > >>>> 18.04.
> > > >>>>>
> > > >>>>> We will upgrade supported versions of each OS as
> > updates/deprecation
> > > >> of
> > > >>>>> support are needed.
> > > >>>>>
> > > >>>>> Again let's wait 72 hours before starting a vote.  Also please
> > check
> > > >>> that
> > > >>>>> the statements I wrote down are what you understood!
> > > >>>>>
> > > >>>>> See below for notes on lazy consensus:
> > > >>>>> https://community.apache.org/committers/lazyConsensus.html
> > > >>>>>
> > > >>>>> - Josh
> > > >>>>>
> > > >>>>>
> > > >>>>> On Wed, May 20, 2020 at 9:41 PM Ning Wang <wa...@gmail.com>
> > > >>> wrote:
> > > >>>>>
> > > >>>>>> +1
> > > >>>>>>
> > > >>>>>> On Wed, May 20, 2020 at 7:26 PM Nicholas Nezis <
> > > >>>> nicholas.nezis@gmail.com
> > > >>>>>>
> > > >>>>>> wrote:
> > > >>>>>>
> > > >>>>>>> I like those options.
> > > >>>>>>>
> > > >>>>>>> On Wed, May 20, 2020 at 10:15 PM Josh Fischer <
> > > >> josh@joshfischer.io
> > > >>>>
> > > >>>>>> wrote:
> > > >>>>>>>
> > > >>>>>>>> Let's wait 3 more days for anyone to give input.  Then we can
> > > >>>> start a
> > > >>>>>>> vote
> > > >>>>>>>> to make a final decision on which containers for us to
> support.
> > > >>>> Keep
> > > >>>>>> in
> > > >>>>>>>> mind that even though we do need to plan for Python3 support
> > > >>> (There
> > > >>>>> is
> > > >>>>>>> work
> > > >>>>>>>> in progress now) we do not need it as of yet.
> > > >>>>>>>>
> > > >>>>>>>> - Josh
> > > >>>>>>>>
> > > >>>>>>>> On Wed, May 20, 2020 at 8:34 PM thinker0 <th...@gmail.com>
> > > >>>> wrote:
> > > >>>>>>>>
> > > >>>>>>>>> My experience is mainly for CentOS 7, so I think it would be
> > > >>> nice
> > > >>>>> if
> > > >>>>>> it
> > > >>>>>>>> was
> > > >>>>>>>>> included.
> > > >>>>>>>>>
> > > >>>>>>>>> 2020년 5월 21일 (목) 오전 9:56, Josh Fischer <josh@joshfischer.io
> > > >>> 님이
> > > >>>> 작성:
> > > >>>>>>>>>
> > > >>>>>>>>>> Well thought out Windham.  I like how you narrowed the
> > > >>> criteria
> > > >>>>>> down
> > > >>>>>>> to
> > > >>>>>>>>>> Debian10/CentOS7(depending on PY3 support) thinking about
> > > >>>> overall
> > > >>>>>>>> support
> > > >>>>>>>>>> and what others would need/want.
> > > >>>>>>>>>>
> > > >>>>>>>>>> +1
> > > >>>>>>>>>>
> > > >>>>>>>>>> - Josh
> > > >>>>>>>>>>
> > > >>>>>>>>>> On Mon, May 18, 2020 at 8:39 PM Windham Wong <
> > > >>>>>>> windham.wong@stormeye.io
> > > >>>>>>>>>
> > > >>>>>>>>>> wrote:
> > > >>>>>>>>>>
> > > >>>>>>>>>>> My experience is that, people tends to pick their
> > > >> container
> > > >>>>> image
> > > >>>>>>> OS
> > > >>>>>>>>>> based
> > > >>>>>>>>>>> on few criterias only:
> > > >>>>>>>>>>> 1) image size, alpine always win, then debian/centos and
> > > >>> 3rd
> > > >>>> is
> > > >>>>>>>>>>> ubuntu/centos. alpine is always the smallest but not much
> > > >>>>> library
> > > >>>>>>>>>> provided
> > > >>>>>>>>>>> and require to compile manually. debian is always good
> > > >>>> because
> > > >>>>> it
> > > >>>>>>> has
> > > >>>>>>>>>>> apt-get to provide loads of library. centos is cool as
> > > >>> debian
> > > >>>>>>> because
> > > >>>>>>>>> it
> > > >>>>>>>>>>> also has good yum repo. ubuntu is always the last because
> > > >>> it
> > > >>>> is
> > > >>>>>>> fat,
> > > >>>>>>>>> but
> > > >>>>>>>>>> it
> > > >>>>>>>>>>> provides very large range of libraries to use.
> > > >>>>>>>>>>> 2) kernel feature, as mentioned last email, ubuntu has
> > > >>> issue
> > > >>>>> with
> > > >>>>>>> DNS
> > > >>>>>>>>> and
> > > >>>>>>>>>>> some sort of network discover issues. I remember alpine
> > > >> has
> > > >>>>> some
> > > >>>>>> as
> > > >>>>>>>>> well,
> > > >>>>>>>>>>> because it is very plain, requires many fine tunes be
> > > >> done
> > > >>>>>>> manually.
> > > >>>>>>>> in
> > > >>>>>>>>>>> this case, centos and debian should win.
> > > >>>>>>>>>>> 3) linux stream, obviously, alpine, debian/ubuntu,
> > > >> centos,
> > > >>>> are
> > > >>>>> 3
> > > >>>>>>>>>> different
> > > >>>>>>>>>>> streams that use different native tools. Poeple just pick
> > > >>>> their
> > > >>>>>>> base
> > > >>>>>>>>>> image
> > > >>>>>>>>>>> based on what they often use in their life. However, I
> > > >> see
> > > >>> a
> > > >>>>> lot
> > > >>>>>>>> debian
> > > >>>>>>>>>> and
> > > >>>>>>>>>>> alpine on docker hub because they are just smaller in
> > > >> size.
> > > >>>>>>>>>>> 4. library support. Python library is a very good example
> > > >>> in
> > > >>>>> this
> > > >>>>>>>> case.
> > > >>>>>>>>>> As
> > > >>>>>>>>>>> Heron is turning to Python 3, which specifically Python
> > > >>> 3.7+,
> > > >>>>>> some
> > > >>>>>>>>>>> discontinued based OS version does not provide the
> > > >> support
> > > >>>>>> anymore.
> > > >>>>>>>> We
> > > >>>>>>>>>> have
> > > >>>>>>>>>>> tested out that, debian9 supports only up to Python3.5,
> > > >>>> ubuntu
> > > >>>>>>> 14.04
> > > >>>>>>>>> does
> > > >>>>>>>>>>> not support python3 and ubuntu 16.04 supports up to
> > > >> Python
> > > >>>> 3.5
> > > >>>>>>> only.
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> I am not a big fan of using centos for docker. Concluding
> > > >>>> all 4
> > > >>>>>>>> points
> > > >>>>>>>>>>> above, I believe keeping centos7 (if it supports
> > > >> python37),
> > > >>>>>>> debian10
> > > >>>>>>>>> (not
> > > >>>>>>>>>>> debian9), ubuntu 18.04 (supports python3.7+) would be the
> > > >>>> best
> > > >>>>>>>> solution
> > > >>>>>>>>>> at
> > > >>>>>>>>>>> the moment. We have to cut off the old images due to our
> > > >>>>> library
> > > >>>>>>>>> upgrade
> > > >>>>>>>>>>> and this is unavoidable. Otherwise, we would have tons of
> > > >>>>> issues
> > > >>>>>>>>> sending
> > > >>>>>>>>>> in
> > > >>>>>>>>>>> about library support in the future and the workload
> > > >> could
> > > >>>> be a
> > > >>>>>> lot
> > > >>>>>>>>> more
> > > >>>>>>>>>>> for just answering questions.
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Windham Wong
> > > >>>>>>>>>>> Co-Founder, Technical Director of
> > > >>>>>>>>>>> Stormeye.io, Hong Kong Managed Security Operation Center
> > > >>>>> Limited
> > > >>>>>>>>>>> Email // windham.wong@stormeye.io (mailto:
> > > >>>>>> windham.wong@stormeye.io
> > > >>>>>>> )
> > > >>>>>>>>>>> Phone // +852_3590_2212_|_+852_9832_0707
> > > >> (tel:+85235902212)
> > > >>>>>>>>>>> Fax // +852_3590_2202 (tel:+852_3590_2202)
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> On 5月 19 2020, at 6:09 早上, Josh Fischer <
> > > >>> josh@joshfischer.io
> > > >>>>>
> > > >>>>>>> wrote:
> > > >>>>>>>>>>>> Hi All, I think using CentOS and either Debian or
> > > >> Ubuntu
> > > >>>> is a
> > > >>>>>>> good
> > > >>>>>>>>>> idea.
> > > >>>>>>>>>>> I would pick Debian over Ubuntu for the reason that Nick
> > > >>>>> pointed
> > > >>>>>>> out
> > > >>>>>>>>> in a
> > > >>>>>>>>>>> previous email of DNS issues in Kubernetes. I"ve copy and
> > > >>>>> pasted
> > > >>>>>> a
> > > >>>>>>>>>> section
> > > >>>>>>>>>>> of Nick's previous email. ### Start I've run into issues
> > > >>> with
> > > >>>>>>> Ubuntu
> > > >>>>>>>>>> images
> > > >>>>>>>>>>> having DNS issues in Kubernetes.
> > > >>>>>>>>>>> https://github.com/kubernetes/kubeadm/issues/787 ### End
> > > >>> As
> > > >>>>> for
> > > >>>>>>> what
> > > >>>>>>>>>> ends
> > > >>>>>>>>>>> up being the official Docker container I'm fine with
> > > >> either
> > > >>>>>> choice
> > > >>>>>>>>> Debian
> > > >>>>>>>>>>> or CentOS. On Mon, May 18, 2020 at 2:10 AM Ning Wang
> > > >>> wrote: >
> > > >>>>>> Yeah.
> > > >>>>>>>> For
> > > >>>>>>>>>>> binary release, the license is a huge factor. > > On Sun,
> > > >>> May
> > > >>>>> 17,
> > > >>>>>>>> 2020
> > > >>>>>>>>> at
> > > >>>>>>>>>>> 2:32 PM Josh Fischer wrote: > > > Hi All, > > > > When it
> > > >>>> comes
> > > >>>>>> to
> > > >>>>>>>>>> choosing
> > > >>>>>>>>>>> the official Apache Docker image for Heron we > > have to
> > > >>>> think
> > > >>>>>> of
> > > >>>>>>>> the
> > > >>>>>>>>>>> licensing issues first. Everything that is > installed >
> > > >>>
> > > >>>> into
> > > >>>>>> the
> > > >>>>>>>>>>> container at build time with the exception of the "FROM"
> > > >>>
> > > >>>>
> > > >>>>>>>> statement
> > > >>>>>>>>> in
> > > >>>>>>>>>>> the Dockerfile must be ALv2 compatible. OpenJDK is under
> > > >>>
> > > >>>>> GPLv2
> > > >>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>>> which
> > > >>>>>>>>>>> is not compatible with Apache. However we can get around
> > > >>> this
> > > >>>>> if
> > > >>>>>>>
> > > >>>>>>> we
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> bring the OpenJDK into the container via a FROM
> > > >> statement.
> > > >>>> I've
> > > >>>>>>>> copied
> > > >>>>>>>>> a
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>> reply from Dave in our dev list in a previous email.
> > > >> See
> > > >>>>> below.
> > > >>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> ######## Start ######## > > Regarding OpenJDK and GPL2 -
> > > >>> here
> > > >>>>> is
> > > >>>>>>> what
> > > >>>>>>>>>> Roman
> > > >>>>>>>>>>> the VP, Legal wrote when > > answering Beam’s questions.
> > > >>>
> > > >>>>
> > > >>>>>
> > > >>>>>>
> > > >>>>>>>> Roman
> > > >>>>>>>>>>> Shaposhnik commented on LEGAL-503: > >
> > > >>>>>>>>>>> ---------------------------------------- > > > > Hey
> > > >>> [~altay]
> > > >>>>> if
> > > >>>>>>> you
> > > >>>>>>>>>> would
> > > >>>>>>>>>>> like to continue linking to the Docker release > >
> > > >> artifact
> > > >>>>> from
> > > >>>>>>> the
> > > >>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>> https://beam.apache.org > > you will have: > > 1.
> > > >>> Transition
> > > >>>>> to
> > > >>>>>>> the
> > > >>>>>>>>>>> official ASF dockerhub org: > >
> > > >>>>> https://hub.docker.com/u/apache
> > > >>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> 2.
> > > >>>>>>>>>>> Start including that binary convenience artifact into
> > > >> your
> > > >>>> VOTE
> > > >>>>>>>>
> > > >>>>>>>>>> threads
> > > >>>>>>>>>>> on Beam releases > > 3. Make sure that all Cat-X licenses
> > > >>> are
> > > >>>>>> ONLY
> > > >>>>>>>>>> brought
> > > >>>>>>>>>>> into your > > container via FROM statements > > ########
> > > >>> End
> > > >>>>>>>> ######## >
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> So at the end of the day, we need to use a container
> > > >> that
> > > >>>> has
> > > >>>>>>> Java
> > > >>>>>>>>>
> > > >>>>>>>>>>> already > > installed in it so that when we create our
> > > >>>> official
> > > >>>>>>> image
> > > >>>>>>>>> we
> > > >>>>>>>>>>> can inherit > > that java containing image via a FROM
> > > >>>>> statement.
> > > >>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>> -
> > > >>>>>>>>>>> Josh > > > > > > On Sun, May 17, 2020 at 4:12 PM H W
> > > >>> wrote: >
> > > >>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>> We
> > > >>>>>>>>>>> use Centos 7 as well. > > > > > > On Sun, May 17, 2020 at
> > > >>>> 10:41
> > > >>>>>> AM
> > > >>>>>>>>>> Nicholas
> > > >>>>>>>>>>> Nezis < > > nicholas.nezis@gmail.com> > > > wrote: > > >
> > > >>>>>
> > > >>>>>>>
> > > >>>>>>>> Some
> > > >>>>>>>>>>> thoughts I'd like to add to the discussion: > > > > > >
> > > >>>>
> > > >>>>>> Debian
> > > >>>>>>>> and
> > > >>>>>>>>>>> Ubuntu are somewhat similar. Would it make sense to pick
> > > >>>
> > > >>>> one
> > > >>>>>>
> > > >>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> of
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>> those and pair with Centos? > > > > > > > > I've run
> > > >>> into
> > > >>>>>>> issues
> > > >>>>>>>>> with
> > > >>>>>>>>>>> Ubuntu images having DNS issues in > > Kubernetes. > > >
> > > >>>
> > > >>>>>>>>>>> https://github.com/kubernetes/kubeadm/issues/787 > > > >
> > > >>>>>
> > > >>>>>>>
> > > >>>>>>>>> Ubuntu
> > > >>>>>>>>>>> 14.04 should be removed. It was end of life last year.
> > > >> For
> > > >>>>
> > > >>>>>> those
> > > >>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>> of
> > > >>>>>>>>>>>>>>> you using Heron in production, is anyone using
> > > >>> Ubuntu
> > > >>>>> 16?
> > > >>>>>>> My
> > > >>>>>>>>>> vote >
> > > >>>>>>>>>>> would > > > be > > > > to remove it also. For what it's
> > > >>>> worth,
> > > >>>>> my
> > > >>>>>>> use
> > > >>>>>>>>>> case
> > > >>>>>>>>>>> is in K8s and we > > use > > > > the Centos7 image on a
> > > >>>> Centos
> > > >>>>> 7
> > > >>>>>>> OS.
> > > >>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> We should do a test of the official image and
> > > >>> helm
> > > >>>>>> chart
> > > >>>>>>>> in >
> > > >>>>>>>>>>> Kubernetes > > > > (maybe with Kind
> > > >>>> https://kind.sigs.k8s.io/
> > > >>>>> )
> > > >>>>>> as
> > > >>>>>>>>> part
> > > >>>>>>>>>>> of our release > > > > process. This might be nice to add
> > > >>> as
> > > >>>> an
> > > >>>>>>>>>> integration
> > > >>>>>>>>>>> test. > > > > > > > > > > > > > > > > > > > > > > > > > >
> > > >>>>>
> > > >>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>> On
> > > >>>>>>>>>>> Sun, May 17, 2020, 12:37 PM Ning Wang > wrote: > > > > >
> > > >>>
> > > >>>>
> > > >>>>>
> > > >>>>>>
> > > >>>>>>> That
> > > >>>>>>>>> is
> > > >>>>>>>>>> a
> > > >>>>>>>>>>> good point. We need to adjust the OS version accordingly
> > > >>>
> > > >>>>
> > > >>>>>> too. >
> > > >>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> On Sun, May 17, 2020 at 2:28 AM Windham Wong <
> > > >>>
> > > >>>>
> > > >>>>>>>>>>> windham.wong@stormeye.io > > > > > > > > > wrote: > > >
> > > >>>
> > > >>>>
> > > >>>>>
> > > >>>>>>
> > > >>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> debian 10 has no python 2.7 support. Ubuntu
> > > >> 16.04
> > > >>>> has
> > > >>>>>> no
> > > >>>>>>>>>> python >
> > > >>>>>>>>>>>> 3.7+ > > > > dev > > > > > > library support. hope this
> > > >>>> helps
> > > >>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>> -- > > > > > > Sent from myMail for Android Sunday, 17
> > > >> May
> > > >>>>> 2020,
> > > >>>>>>>>> 11:34AM
> > > >>>>>>>>>>> +08:00 > > from > > > > > Ning > > > > > > Wang
> > > >>>>>>> wangninggm@gmail.com
> > > >>>>>>>>> : >
> > > >>>>>>>>>>>>>>>>>>>>>>> Hi, > > > > > > > > > > > > > >We
> > > >>> have
> > > >>>>>>> talked
> > > >>>>>>>>>> about
> > > >>>>>>>>>>> it a few times in different places. Let's > > make > > >
> > > >> a
> > > >>>>
> > > >>>>>
> > > >>>>>>
> > > >>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>>>> decision here. We don't have enough resources to support
> > > >>>> many
> > > >>>>>>
> > > >>>>>>>
> > > >>>>>>>>>> docker >
> > > >>>>>>>>>>>>>>> OSes > > > > > > >(technically we only support
> > > >>>> Debian,
> > > >>>>>>>> Ubuntu,
> > > >>>>>>>>>> and
> > > >>>>>>>>>>> CentOS but each > > of > > > > them > > > > > > >has
> > > >>> multiple
> > > >>>>>>>>> versions).
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>>>>>>>>> Things we have agreed on (my
> > > >>>>>>> understanding)
> > > >>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>>>>> - for binary release, we are going to choose one
> > > >> OS
> > > >>>> one
> > > >>>>>>
> > > >>>>>>>>>> version. I
> > > >>>>>>>>>>>>>>>>>> remember that Debian 9 was the decision. An
> > > >>> open
> > > >>>>>>>> question
> > > >>>>>>>>>> is:
> > > >>>>>>>>>>> do > > we > > > > want > > > > > > to > > > > > > >switch
> > > >>> to
> > > >>>>>> Debian
> > > >>>>>>>> 10
> > > >>>>>>>>>> now?
> > > >>>>>>>>>>> We haven't had a binary release yet, so > > > there > > >
> > > >>>>>
> > > >>>>> is
> > > >>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>>> still time. As the main target, I think 1. the image
> > > >>>> should
> > > >>>>>>
> > > >>>>>>>
> > > >>>>>>>>>> compile
> > > >>>>>>>>>>>>>>>>>> successfully. and 2 the image should be
> > > >>> tested:
> > > >>>>> the
> > > >>>>>>>> tools
> > > >>>>>>>>>> and
> > > >>>>>>>>>>> an > > > > example > > > > > > >topology. > > > > > > >
> > > >>>>
> > > >>>>>>
> > > >>>>>>>>
> > > >>>>>>>>> -
> > > >>>>>>>>>> for
> > > >>>>>>>>>>> source release, we haven't talked about it very much. > >
> > > >>>>>
> > > >>>>>>>>> Personally
> > > >>>>>>>>>> I
> > > >>>>>>>>>>>>>>>>>> feel it is a reasonable expectation that, if
> > > >>> we
> > > >>>>> have
> > > >>>>>>> the
> > > >>>>>>>>>>> Docker > > file > > > > in > > > > > > the > > > > > >
> > > >>>> core
> > > >>>>>>> folders,
> > > >>>>>>>>> it
> > > >>>>>>>>>>> should at least compile. Otherwise, it's a > failed > >
> > > >>>>
> > > >>>>>> build >
> > > >>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>> hence a failed release. We are not going to test run
> > > >> the
> > > >>>>>> tools >
> > > >>>>>>>> and
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> examples > > > > > > >for each release though. >
> > > >>>
> > > >>>>
> > > >>>>>
> > > >>>>>>
> > > >>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Finally, a question about OS versions. I believe
> > > >> at
> > > >>>>> least
> > > >>>>>>> two
> > > >>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> releases > > > > > > >should be supported because most
> > > >>> people
> > > >>>>>> don't
> > > >>>>>>>>>> upgrade
> > > >>>>>>>>>>> to the > > latest > > > > > > version > > > > > > >when
> > > >> it
> > > >>> is
> > > >>>>> out
> > > >>>>>>> in
> > > >>>>>>>>> more
> > > >>>>>>>>>>> serious scenarios, hence IMO the second > > > oldest > >
> > > >>>
> > > >>>>
> > > >>>>>
> > > >>>>>>
> > > >>>>>>>> could
> > > >>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> often be more useful than the latest one from a
> > > >>>>>>> convenience
> > > >>>>>>>>>
> > > >>>>>>>>>>> point > > of > > > > > view. > > > > > > >On the other
> > > >>> hand,
> > > >>>>> this
> > > >>>>>>> is
> > > >>>>>>>>> too
> > > >>>>>>>>>>> flexible, and we will spend time > > > > thinking > > > >
> > > >>>>>
> > > >>>>>>>> about
> > > >>>>>>>> it
> > > >>>>>>>>>>> again. I think a clear guideline could be helpful to >
> > > >> us.
> > > >>>>
> > > >>>>>
> > > >>>>>>
> > > >>>>>>> For
> > > >>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>>>>> example: > > > > > > >- We only include only
> > > >> Debian
> > > >>>>>> (popular
> > > >>>>>>>> in
> > > >>>>>>>>>> the
> > > >>>>>>>>>>> server world) and > > > Ubuntu > > > > > > >(popular in
> > > >>>> servers
> > > >>>>>> and
> > > >>>>>>>>>>> workstations). > > > > > > >- for the main OS (Debian) we
> > > >>>>> choose
> > > >>>>>>> the
> > > >>>>>>>>>> latest
> > > >>>>>>>>>>> to compile and > > test > > > > for > > > > > > our > >
> > > >>>>
> > > >>>>>>
> > > >>>>>>>>> binary
> > > >>>>>>>>>>> release. > > > > > > >- for a given OS, only two versions
> > > >>>> (most
> > > >>>>>>>> likely
> > > >>>>>>>>>> the
> > > >>>>>>>>>>> most and > the > > > > second > > > > > > >recent
> > > >> releases)
> > > >>>> are
> > > >>>>>>>>> included
> > > >>>>>>>>>> in
> > > >>>>>>>>>>> the core source code and they > > need > > > to > > > > >
> > > >>> be
> > > >>>>>
> > > >>>>>>
> > > >>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>>> fixed ASAP if they don't compile. > > > > > > >- OSs and
> > > >>>> other
> > > >>>>>>>>> versions
> > > >>>>>>>>>>> could be included in a special folder > and > > > but > >
> > > >>>>>
> > > >>>>>>>
> > > >>>>>>>>>> there
> > > >>>>>>>>>> is
> > > >>>>>>>>>>> no guarantee that the code will compile. Fixes from the >
> > > >>>>>
> > > >>>>>>>
> > > >>>>>>>>>> community
> > > >>>>>>>>>>>>>>>>>> are welcome. > > > > > > > > > > > > > > > >
> > > >>>>>
> > > >>>>>>>
> > > >>>>>>>>> How
> > > >>>>>>>>> do
> > > >>>>>>>>>>> you guys think? > > > > > > >--ning > > > > > > > > > >
> > > >>>>
> > > >>>>>>
> > > >>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>>
> > > >>>
> > > >>
> > >
> > >
> >
>

Re: [DISCUSSION] Docker OS support

[Josh Fischer <jo...@joshfischer.io>]

I just opened a PR that fixes javadoc generation for the website build.  Do
we have anything else outstanding before we can start a vote on the next
release candidate?

On Wed, May 27, 2020 at 11:31 AM Ning Wang <wa...@gmail.com> wrote:

> Yeah. We have chosen Debian 10 for the binary release and clean up the
> license requirements for the first binary release.
>
> In the source level, we would also like to include a few platforms/versions
> in the main source code to be "supported" and the others will be moved to
> like a "extra" folder and not actively supported/tested.
>
>
> On Wed, May 27, 2020 at 9:22 AM Dave Fisher <wa...@apache.org> wrote:
>
> > I thought that the purpose of this discussion was to find the one or two
> > platforms for Heron to support in order to keep down the complexity of
> > making the first binary release for the community.
> >
> > Other platforms could be on a roadmap for future releases.
> >
> > Regards,
> > Dave
> >
> > > On May 26, 2020, at 12:26 PM, Ning Wang <wa...@gmail.com> wrote:
> > >
> > > My concern about "latest" is that I feel most people hold it for some
> > time
> > > before migrating to the latest version (if it ever happens). If there
> are
> > > numbers, it could make sense to use the most popular version (plus the
> > > other criterias).
> > >
> > > On Tue, May 26, 2020 at 12:14 PM Oliver Bristow <
> > oliver@oliverbristow.co.uk>
> > > wrote:
> > >
> > >> Sorry if I missed the rationale for Ubuntu 18.04 instead of 20.04, but
> > >> would the latest LTS be a better choice? 18.04 was released before
> > Debian
> > >> 10, so that may be an argument for 20.04 re. compatibility if Debian
> 10
> > is
> > >> the standard.
> > >>
> > >> There's already a 20.04 Dockerfile + I It looks like the vagrant setup
> > will
> > >> get an upgrade from 14.04 to 20.04 as a part of the python3 upgrade,
> and
> > >> seems to run fine so far.
> > >>
> > >> On Tue, 26 May 2020 at 19:30, Josh Fischer <jo...@joshfischer.io>
> wrote:
> > >>
> > >>> Thanks for further clarifying. Yes, that is correct.
> > >>>
> > >>>
> > >>>
> > >>> On Tue, May 26, 2020 at 1:16 PM Ning Wang <wa...@gmail.com>
> > wrote:
> > >>>
> > >>>> +1.
> > >>>>
> > >>>> So my understanding is:
> > >>>> binary (in future release): Debian 10.
> > >>>> source: Debian 9/10 + ubuntu 18.04 + centos 7. version numbers will
> be
> > >>>> updated in the future after more discussions/votes.
> > >>>>
> > >>>> others will be moved to a "extra" folder only for reference.
> > >>>>
> > >>>>
> > >>>>
> > >>>> On Tue, May 26, 2020 at 6:13 AM Josh Fischer <jo...@joshfischer.io>
> > >>> wrote:
> > >>>>
> > >>>>> Hi All,
> > >>>>>
> > >>>>> Based on what I've read on this thread a combination of
> CentOS/Debian
> > >>>> would
> > >>>>> be the better options for us to support multiple versions for the
> > >>>>> community.  We will also support one version of Ubuntu.
> > >>>>>
> > >>>>> So..... I will start a vote in 72 hours that calls on the vote on
> > >> this.
> > >>>>> I'm going to write down my understanding of what this thread covers
> > >> to
> > >>>> get
> > >>>>> some actionable topics to vote on.  If I've written anything down
> > >>>>> incorrectly please speak up before we start a vote.
> > >>>>>
> > >>>>> Debian:
> > >>>>>    - Debian10 is the official docker image to support and be
> > >> published
> > >>>> to
> > >>>>> Docker Hub, we will also support Debian9.
> > >>>>> CentOS:
> > >>>>>    - We will support CentOS7.  Note, I do not think it has native
> > >>>>> Python3.7 support, but we are also not there yet.
> > >>>>> Ubuntu:
> > >>>>>    - We will still support one version. At this time the version is
> > >>>> 18.04.
> > >>>>>
> > >>>>> We will upgrade supported versions of each OS as
> updates/deprecation
> > >> of
> > >>>>> support are needed.
> > >>>>>
> > >>>>> Again let's wait 72 hours before starting a vote.  Also please
> check
> > >>> that
> > >>>>> the statements I wrote down are what you understood!
> > >>>>>
> > >>>>> See below for notes on lazy consensus:
> > >>>>> https://community.apache.org/committers/lazyConsensus.html
> > >>>>>
> > >>>>> - Josh
> > >>>>>
> > >>>>>
> > >>>>> On Wed, May 20, 2020 at 9:41 PM Ning Wang <wa...@gmail.com>
> > >>> wrote:
> > >>>>>
> > >>>>>> +1
> > >>>>>>
> > >>>>>> On Wed, May 20, 2020 at 7:26 PM Nicholas Nezis <
> > >>>> nicholas.nezis@gmail.com
> > >>>>>>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>>> I like those options.
> > >>>>>>>
> > >>>>>>> On Wed, May 20, 2020 at 10:15 PM Josh Fischer <
> > >> josh@joshfischer.io
> > >>>>
> > >>>>>> wrote:
> > >>>>>>>
> > >>>>>>>> Let's wait 3 more days for anyone to give input.  Then we can
> > >>>> start a
> > >>>>>>> vote
> > >>>>>>>> to make a final decision on which containers for us to support.
> > >>>> Keep
> > >>>>>> in
> > >>>>>>>> mind that even though we do need to plan for Python3 support
> > >>> (There
> > >>>>> is
> > >>>>>>> work
> > >>>>>>>> in progress now) we do not need it as of yet.
> > >>>>>>>>
> > >>>>>>>> - Josh
> > >>>>>>>>
> > >>>>>>>> On Wed, May 20, 2020 at 8:34 PM thinker0 <th...@gmail.com>
> > >>>> wrote:
> > >>>>>>>>
> > >>>>>>>>> My experience is mainly for CentOS 7, so I think it would be
> > >>> nice
> > >>>>> if
> > >>>>>> it
> > >>>>>>>> was
> > >>>>>>>>> included.
> > >>>>>>>>>
> > >>>>>>>>> 2020년 5월 21일 (목) 오전 9:56, Josh Fischer <josh@joshfischer.io
> > >>> 님이
> > >>>> 작성:
> > >>>>>>>>>
> > >>>>>>>>>> Well thought out Windham.  I like how you narrowed the
> > >>> criteria
> > >>>>>> down
> > >>>>>>> to
> > >>>>>>>>>> Debian10/CentOS7(depending on PY3 support) thinking about
> > >>>> overall
> > >>>>>>>> support
> > >>>>>>>>>> and what others would need/want.
> > >>>>>>>>>>
> > >>>>>>>>>> +1
> > >>>>>>>>>>
> > >>>>>>>>>> - Josh
> > >>>>>>>>>>
> > >>>>>>>>>> On Mon, May 18, 2020 at 8:39 PM Windham Wong <
> > >>>>>>> windham.wong@stormeye.io
> > >>>>>>>>>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>> My experience is that, people tends to pick their
> > >> container
> > >>>>> image
> > >>>>>>> OS
> > >>>>>>>>>> based
> > >>>>>>>>>>> on few criterias only:
> > >>>>>>>>>>> 1) image size, alpine always win, then debian/centos and
> > >>> 3rd
> > >>>> is
> > >>>>>>>>>>> ubuntu/centos. alpine is always the smallest but not much
> > >>>>> library
> > >>>>>>>>>> provided
> > >>>>>>>>>>> and require to compile manually. debian is always good
> > >>>> because
> > >>>>> it
> > >>>>>>> has
> > >>>>>>>>>>> apt-get to provide loads of library. centos is cool as
> > >>> debian
> > >>>>>>> because
> > >>>>>>>>> it
> > >>>>>>>>>>> also has good yum repo. ubuntu is always the last because
> > >>> it
> > >>>> is
> > >>>>>>> fat,
> > >>>>>>>>> but
> > >>>>>>>>>> it
> > >>>>>>>>>>> provides very large range of libraries to use.
> > >>>>>>>>>>> 2) kernel feature, as mentioned last email, ubuntu has
> > >>> issue
> > >>>>> with
> > >>>>>>> DNS
> > >>>>>>>>> and
> > >>>>>>>>>>> some sort of network discover issues. I remember alpine
> > >> has
> > >>>>> some
> > >>>>>> as
> > >>>>>>>>> well,
> > >>>>>>>>>>> because it is very plain, requires many fine tunes be
> > >> done
> > >>>>>>> manually.
> > >>>>>>>> in
> > >>>>>>>>>>> this case, centos and debian should win.
> > >>>>>>>>>>> 3) linux stream, obviously, alpine, debian/ubuntu,
> > >> centos,
> > >>>> are
> > >>>>> 3
> > >>>>>>>>>> different
> > >>>>>>>>>>> streams that use different native tools. Poeple just pick
> > >>>> their
> > >>>>>>> base
> > >>>>>>>>>> image
> > >>>>>>>>>>> based on what they often use in their life. However, I
> > >> see
> > >>> a
> > >>>>> lot
> > >>>>>>>> debian
> > >>>>>>>>>> and
> > >>>>>>>>>>> alpine on docker hub because they are just smaller in
> > >> size.
> > >>>>>>>>>>> 4. library support. Python library is a very good example
> > >>> in
> > >>>>> this
> > >>>>>>>> case.
> > >>>>>>>>>> As
> > >>>>>>>>>>> Heron is turning to Python 3, which specifically Python
> > >>> 3.7+,
> > >>>>>> some
> > >>>>>>>>>>> discontinued based OS version does not provide the
> > >> support
> > >>>>>> anymore.
> > >>>>>>>> We
> > >>>>>>>>>> have
> > >>>>>>>>>>> tested out that, debian9 supports only up to Python3.5,
> > >>>> ubuntu
> > >>>>>>> 14.04
> > >>>>>>>>> does
> > >>>>>>>>>>> not support python3 and ubuntu 16.04 supports up to
> > >> Python
> > >>>> 3.5
> > >>>>>>> only.
> > >>>>>>>>>>>
> > >>>>>>>>>>> I am not a big fan of using centos for docker. Concluding
> > >>>> all 4
> > >>>>>>>> points
> > >>>>>>>>>>> above, I believe keeping centos7 (if it supports
> > >> python37),
> > >>>>>>> debian10
> > >>>>>>>>> (not
> > >>>>>>>>>>> debian9), ubuntu 18.04 (supports python3.7+) would be the
> > >>>> best
> > >>>>>>>> solution
> > >>>>>>>>>> at
> > >>>>>>>>>>> the moment. We have to cut off the old images due to our
> > >>>>> library
> > >>>>>>>>> upgrade
> > >>>>>>>>>>> and this is unavoidable. Otherwise, we would have tons of
> > >>>>> issues
> > >>>>>>>>> sending
> > >>>>>>>>>> in
> > >>>>>>>>>>> about library support in the future and the workload
> > >> could
> > >>>> be a
> > >>>>>> lot
> > >>>>>>>>> more
> > >>>>>>>>>>> for just answering questions.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Windham Wong
> > >>>>>>>>>>> Co-Founder, Technical Director of
> > >>>>>>>>>>> Stormeye.io, Hong Kong Managed Security Operation Center
> > >>>>> Limited
> > >>>>>>>>>>> Email // windham.wong@stormeye.io (mailto:
> > >>>>>> windham.wong@stormeye.io
> > >>>>>>> )
> > >>>>>>>>>>> Phone // +852_3590_2212_|_+852_9832_0707
> > >> (tel:+85235902212)
> > >>>>>>>>>>> Fax // +852_3590_2202 (tel:+852_3590_2202)
> > >>>>>>>>>>>
> > >>>>>>>>>>> On 5月 19 2020, at 6:09 早上, Josh Fischer <
> > >>> josh@joshfischer.io
> > >>>>>
> > >>>>>>> wrote:
> > >>>>>>>>>>>> Hi All, I think using CentOS and either Debian or
> > >> Ubuntu
> > >>>> is a
> > >>>>>>> good
> > >>>>>>>>>> idea.
> > >>>>>>>>>>> I would pick Debian over Ubuntu for the reason that Nick
> > >>>>> pointed
> > >>>>>>> out
> > >>>>>>>>> in a
> > >>>>>>>>>>> previous email of DNS issues in Kubernetes. I"ve copy and
> > >>>>> pasted
> > >>>>>> a
> > >>>>>>>>>> section
> > >>>>>>>>>>> of Nick's previous email. ### Start I've run into issues
> > >>> with
> > >>>>>>> Ubuntu
> > >>>>>>>>>> images
> > >>>>>>>>>>> having DNS issues in Kubernetes.
> > >>>>>>>>>>> https://github.com/kubernetes/kubeadm/issues/787 ### End
> > >>> As
> > >>>>> for
> > >>>>>>> what
> > >>>>>>>>>> ends
> > >>>>>>>>>>> up being the official Docker container I'm fine with
> > >> either
> > >>>>>> choice
> > >>>>>>>>> Debian
> > >>>>>>>>>>> or CentOS. On Mon, May 18, 2020 at 2:10 AM Ning Wang
> > >>> wrote: >
> > >>>>>> Yeah.
> > >>>>>>>> For
> > >>>>>>>>>>> binary release, the license is a huge factor. > > On Sun,
> > >>> May
> > >>>>> 17,
> > >>>>>>>> 2020
> > >>>>>>>>> at
> > >>>>>>>>>>> 2:32 PM Josh Fischer wrote: > > > Hi All, > > > > When it
> > >>>> comes
> > >>>>>> to
> > >>>>>>>>>> choosing
> > >>>>>>>>>>> the official Apache Docker image for Heron we > > have to
> > >>>> think
> > >>>>>> of
> > >>>>>>>> the
> > >>>>>>>>>>> licensing issues first. Everything that is > installed >
> > >>>
> > >>>> into
> > >>>>>> the
> > >>>>>>>>>>> container at build time with the exception of the "FROM"
> > >>>
> > >>>>
> > >>>>>>>> statement
> > >>>>>>>>> in
> > >>>>>>>>>>> the Dockerfile must be ALv2 compatible. OpenJDK is under
> > >>>
> > >>>>> GPLv2
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>>>> which
> > >>>>>>>>>>> is not compatible with Apache. However we can get around
> > >>> this
> > >>>>> if
> > >>>>>>>
> > >>>>>>> we
> > >>>>>>>>>>>
> > >>>>>>>>>>> bring the OpenJDK into the container via a FROM
> > >> statement.
> > >>>> I've
> > >>>>>>>> copied
> > >>>>>>>>> a
> > >>>>>>>>>>>
> > >>>>>>>>>>>> reply from Dave in our dev list in a previous email.
> > >> See
> > >>>>> below.
> > >>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> ######## Start ######## > > Regarding OpenJDK and GPL2 -
> > >>> here
> > >>>>> is
> > >>>>>>> what
> > >>>>>>>>>> Roman
> > >>>>>>>>>>> the VP, Legal wrote when > > answering Beam’s questions.
> > >>>
> > >>>>
> > >>>>>
> > >>>>>>
> > >>>>>>>> Roman
> > >>>>>>>>>>> Shaposhnik commented on LEGAL-503: > >
> > >>>>>>>>>>> ---------------------------------------- > > > > Hey
> > >>> [~altay]
> > >>>>> if
> > >>>>>>> you
> > >>>>>>>>>> would
> > >>>>>>>>>>> like to continue linking to the Docker release > >
> > >> artifact
> > >>>>> from
> > >>>>>>> the
> > >>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>> https://beam.apache.org > > you will have: > > 1.
> > >>> Transition
> > >>>>> to
> > >>>>>>> the
> > >>>>>>>>>>> official ASF dockerhub org: > >
> > >>>>> https://hub.docker.com/u/apache
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>> 2.
> > >>>>>>>>>>> Start including that binary convenience artifact into
> > >> your
> > >>>> VOTE
> > >>>>>>>>
> > >>>>>>>>>> threads
> > >>>>>>>>>>> on Beam releases > > 3. Make sure that all Cat-X licenses
> > >>> are
> > >>>>>> ONLY
> > >>>>>>>>>> brought
> > >>>>>>>>>>> into your > > container via FROM statements > > ########
> > >>> End
> > >>>>>>>> ######## >
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> So at the end of the day, we need to use a container
> > >> that
> > >>>> has
> > >>>>>>> Java
> > >>>>>>>>>
> > >>>>>>>>>>> already > > installed in it so that when we create our
> > >>>> official
> > >>>>>>> image
> > >>>>>>>>> we
> > >>>>>>>>>>> can inherit > > that java containing image via a FROM
> > >>>>> statement.
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> -
> > >>>>>>>>>>> Josh > > > > > > On Sun, May 17, 2020 at 4:12 PM H W
> > >>> wrote: >
> > >>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> We
> > >>>>>>>>>>> use Centos 7 as well. > > > > > > On Sun, May 17, 2020 at
> > >>>> 10:41
> > >>>>>> AM
> > >>>>>>>>>> Nicholas
> > >>>>>>>>>>> Nezis < > > nicholas.nezis@gmail.com> > > > wrote: > > >
> > >>>>>
> > >>>>>>>
> > >>>>>>>> Some
> > >>>>>>>>>>> thoughts I'd like to add to the discussion: > > > > > >
> > >>>>
> > >>>>>> Debian
> > >>>>>>>> and
> > >>>>>>>>>>> Ubuntu are somewhat similar. Would it make sense to pick
> > >>>
> > >>>> one
> > >>>>>>
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>> of
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> those and pair with Centos? > > > > > > > > I've run
> > >>> into
> > >>>>>>> issues
> > >>>>>>>>> with
> > >>>>>>>>>>> Ubuntu images having DNS issues in > > Kubernetes. > > >
> > >>>
> > >>>>>>>>>>> https://github.com/kubernetes/kubeadm/issues/787 > > > >
> > >>>>>
> > >>>>>>>
> > >>>>>>>>> Ubuntu
> > >>>>>>>>>>> 14.04 should be removed. It was end of life last year.
> > >> For
> > >>>>
> > >>>>>> those
> > >>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> of
> > >>>>>>>>>>>>>>> you using Heron in production, is anyone using
> > >>> Ubuntu
> > >>>>> 16?
> > >>>>>>> My
> > >>>>>>>>>> vote >
> > >>>>>>>>>>> would > > > be > > > > to remove it also. For what it's
> > >>>> worth,
> > >>>>> my
> > >>>>>>> use
> > >>>>>>>>>> case
> > >>>>>>>>>>> is in K8s and we > > use > > > > the Centos7 image on a
> > >>>> Centos
> > >>>>> 7
> > >>>>>>> OS.
> > >>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>>>>>> We should do a test of the official image and
> > >>> helm
> > >>>>>> chart
> > >>>>>>>> in >
> > >>>>>>>>>>> Kubernetes > > > > (maybe with Kind
> > >>>> https://kind.sigs.k8s.io/
> > >>>>> )
> > >>>>>> as
> > >>>>>>>>> part
> > >>>>>>>>>>> of our release > > > > process. This might be nice to add
> > >>> as
> > >>>> an
> > >>>>>>>>>> integration
> > >>>>>>>>>>> test. > > > > > > > > > > > > > > > > > > > > > > > > > >
> > >>>>>
> > >>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> On
> > >>>>>>>>>>> Sun, May 17, 2020, 12:37 PM Ning Wang > wrote: > > > > >
> > >>>
> > >>>>
> > >>>>>
> > >>>>>>
> > >>>>>>> That
> > >>>>>>>>> is
> > >>>>>>>>>> a
> > >>>>>>>>>>> good point. We need to adjust the OS version accordingly
> > >>>
> > >>>>
> > >>>>>> too. >
> > >>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> On Sun, May 17, 2020 at 2:28 AM Windham Wong <
> > >>>
> > >>>>
> > >>>>>>>>>>> windham.wong@stormeye.io > > > > > > > > > wrote: > > >
> > >>>
> > >>>>
> > >>>>>
> > >>>>>>
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>>>>>> debian 10 has no python 2.7 support. Ubuntu
> > >> 16.04
> > >>>> has
> > >>>>>> no
> > >>>>>>>>>> python >
> > >>>>>>>>>>>> 3.7+ > > > > dev > > > > > > library support. hope this
> > >>>> helps
> > >>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>> -- > > > > > > Sent from myMail for Android Sunday, 17
> > >> May
> > >>>>> 2020,
> > >>>>>>>>> 11:34AM
> > >>>>>>>>>>> +08:00 > > from > > > > > Ning > > > > > > Wang
> > >>>>>>> wangninggm@gmail.com
> > >>>>>>>>> : >
> > >>>>>>>>>>>>>>>>>>>>>>> Hi, > > > > > > > > > > > > > >We
> > >>> have
> > >>>>>>> talked
> > >>>>>>>>>> about
> > >>>>>>>>>>> it a few times in different places. Let's > > make > > >
> > >> a
> > >>>>
> > >>>>>
> > >>>>>>
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>>>> decision here. We don't have enough resources to support
> > >>>> many
> > >>>>>>
> > >>>>>>>
> > >>>>>>>>>> docker >
> > >>>>>>>>>>>>>>> OSes > > > > > > >(technically we only support
> > >>>> Debian,
> > >>>>>>>> Ubuntu,
> > >>>>>>>>>> and
> > >>>>>>>>>>> CentOS but each > > of > > > > them > > > > > > >has
> > >>> multiple
> > >>>>>>>>> versions).
> > >>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> Things we have agreed on (my
> > >>>>>>> understanding)
> > >>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>>>>> - for binary release, we are going to choose one
> > >> OS
> > >>>> one
> > >>>>>>
> > >>>>>>>>>> version. I
> > >>>>>>>>>>>>>>>>>> remember that Debian 9 was the decision. An
> > >>> open
> > >>>>>>>> question
> > >>>>>>>>>> is:
> > >>>>>>>>>>> do > > we > > > > want > > > > > > to > > > > > > >switch
> > >>> to
> > >>>>>> Debian
> > >>>>>>>> 10
> > >>>>>>>>>> now?
> > >>>>>>>>>>> We haven't had a binary release yet, so > > > there > > >
> > >>>>>
> > >>>>> is
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>>> still time. As the main target, I think 1. the image
> > >>>> should
> > >>>>>>
> > >>>>>>>
> > >>>>>>>>>> compile
> > >>>>>>>>>>>>>>>>>> successfully. and 2 the image should be
> > >>> tested:
> > >>>>> the
> > >>>>>>>> tools
> > >>>>>>>>>> and
> > >>>>>>>>>>> an > > > > example > > > > > > >topology. > > > > > > >
> > >>>>
> > >>>>>>
> > >>>>>>>>
> > >>>>>>>>> -
> > >>>>>>>>>> for
> > >>>>>>>>>>> source release, we haven't talked about it very much. > >
> > >>>>>
> > >>>>>>>>> Personally
> > >>>>>>>>>> I
> > >>>>>>>>>>>>>>>>>> feel it is a reasonable expectation that, if
> > >>> we
> > >>>>> have
> > >>>>>>> the
> > >>>>>>>>>>> Docker > > file > > > > in > > > > > > the > > > > > >
> > >>>> core
> > >>>>>>> folders,
> > >>>>>>>>> it
> > >>>>>>>>>>> should at least compile. Otherwise, it's a > failed > >
> > >>>>
> > >>>>>> build >
> > >>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> hence a failed release. We are not going to test run
> > >> the
> > >>>>>> tools >
> > >>>>>>>> and
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>>>> examples > > > > > > >for each release though. >
> > >>>
> > >>>>
> > >>>>>
> > >>>>>>
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>>>>> Finally, a question about OS versions. I believe
> > >> at
> > >>>>> least
> > >>>>>>> two
> > >>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> releases > > > > > > >should be supported because most
> > >>> people
> > >>>>>> don't
> > >>>>>>>>>> upgrade
> > >>>>>>>>>>> to the > > latest > > > > > > version > > > > > > >when
> > >> it
> > >>> is
> > >>>>> out
> > >>>>>>> in
> > >>>>>>>>> more
> > >>>>>>>>>>> serious scenarios, hence IMO the second > > > oldest > >
> > >>>
> > >>>>
> > >>>>>
> > >>>>>>
> > >>>>>>>> could
> > >>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>>>>>> often be more useful than the latest one from a
> > >>>>>>> convenience
> > >>>>>>>>>
> > >>>>>>>>>>> point > > of > > > > > view. > > > > > > >On the other
> > >>> hand,
> > >>>>> this
> > >>>>>>> is
> > >>>>>>>>> too
> > >>>>>>>>>>> flexible, and we will spend time > > > > thinking > > > >
> > >>>>>
> > >>>>>>>> about
> > >>>>>>>> it
> > >>>>>>>>>>> again. I think a clear guideline could be helpful to >
> > >> us.
> > >>>>
> > >>>>>
> > >>>>>>
> > >>>>>>> For
> > >>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>>>>> example: > > > > > > >- We only include only
> > >> Debian
> > >>>>>> (popular
> > >>>>>>>> in
> > >>>>>>>>>> the
> > >>>>>>>>>>> server world) and > > > Ubuntu > > > > > > >(popular in
> > >>>> servers
> > >>>>>> and
> > >>>>>>>>>>> workstations). > > > > > > >- for the main OS (Debian) we
> > >>>>> choose
> > >>>>>>> the
> > >>>>>>>>>> latest
> > >>>>>>>>>>> to compile and > > test > > > > for > > > > > > our > >
> > >>>>
> > >>>>>>
> > >>>>>>>>> binary
> > >>>>>>>>>>> release. > > > > > > >- for a given OS, only two versions
> > >>>> (most
> > >>>>>>>> likely
> > >>>>>>>>>> the
> > >>>>>>>>>>> most and > the > > > > second > > > > > > >recent
> > >> releases)
> > >>>> are
> > >>>>>>>>> included
> > >>>>>>>>>> in
> > >>>>>>>>>>> the core source code and they > > need > > > to > > > > >
> > >>> be
> > >>>>>
> > >>>>>>
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>>> fixed ASAP if they don't compile. > > > > > > >- OSs and
> > >>>> other
> > >>>>>>>>> versions
> > >>>>>>>>>>> could be included in a special folder > and > > > but > >
> > >>>>>
> > >>>>>>>
> > >>>>>>>>>> there
> > >>>>>>>>>> is
> > >>>>>>>>>>> no guarantee that the code will compile. Fixes from the >
> > >>>>>
> > >>>>>>>
> > >>>>>>>>>> community
> > >>>>>>>>>>>>>>>>>> are welcome. > > > > > > > > > > > > > > > >
> > >>>>>
> > >>>>>>>
> > >>>>>>>>> How
> > >>>>>>>>> do
> > >>>>>>>>>>> you guys think? > > > > > > >--ning > > > > > > > > > >
> > >>>>
> > >>>>>>
> > >>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>
> >
> >
>

Re: [DISCUSSION] Docker OS support

[Ning Wang <wa...@gmail.com>]

Yeah. We have chosen Debian 10 for the binary release and clean up the
license requirements for the first binary release.

In the source level, we would also like to include a few platforms/versions
in the main source code to be "supported" and the others will be moved to
like a "extra" folder and not actively supported/tested.


On Wed, May 27, 2020 at 9:22 AM Dave Fisher <wa...@apache.org> wrote:

> I thought that the purpose of this discussion was to find the one or two
> platforms for Heron to support in order to keep down the complexity of
> making the first binary release for the community.
>
> Other platforms could be on a roadmap for future releases.
>
> Regards,
> Dave
>
> > On May 26, 2020, at 12:26 PM, Ning Wang <wa...@gmail.com> wrote:
> >
> > My concern about "latest" is that I feel most people hold it for some
> time
> > before migrating to the latest version (if it ever happens). If there are
> > numbers, it could make sense to use the most popular version (plus the
> > other criterias).
> >
> > On Tue, May 26, 2020 at 12:14 PM Oliver Bristow <
> oliver@oliverbristow.co.uk>
> > wrote:
> >
> >> Sorry if I missed the rationale for Ubuntu 18.04 instead of 20.04, but
> >> would the latest LTS be a better choice? 18.04 was released before
> Debian
> >> 10, so that may be an argument for 20.04 re. compatibility if Debian 10
> is
> >> the standard.
> >>
> >> There's already a 20.04 Dockerfile + I It looks like the vagrant setup
> will
> >> get an upgrade from 14.04 to 20.04 as a part of the python3 upgrade, and
> >> seems to run fine so far.
> >>
> >> On Tue, 26 May 2020 at 19:30, Josh Fischer <jo...@joshfischer.io> wrote:
> >>
> >>> Thanks for further clarifying. Yes, that is correct.
> >>>
> >>>
> >>>
> >>> On Tue, May 26, 2020 at 1:16 PM Ning Wang <wa...@gmail.com>
> wrote:
> >>>
> >>>> +1.
> >>>>
> >>>> So my understanding is:
> >>>> binary (in future release): Debian 10.
> >>>> source: Debian 9/10 + ubuntu 18.04 + centos 7. version numbers will be
> >>>> updated in the future after more discussions/votes.
> >>>>
> >>>> others will be moved to a "extra" folder only for reference.
> >>>>
> >>>>
> >>>>
> >>>> On Tue, May 26, 2020 at 6:13 AM Josh Fischer <jo...@joshfischer.io>
> >>> wrote:
> >>>>
> >>>>> Hi All,
> >>>>>
> >>>>> Based on what I've read on this thread a combination of CentOS/Debian
> >>>> would
> >>>>> be the better options for us to support multiple versions for the
> >>>>> community.  We will also support one version of Ubuntu.
> >>>>>
> >>>>> So..... I will start a vote in 72 hours that calls on the vote on
> >> this.
> >>>>> I'm going to write down my understanding of what this thread covers
> >> to
> >>>> get
> >>>>> some actionable topics to vote on.  If I've written anything down
> >>>>> incorrectly please speak up before we start a vote.
> >>>>>
> >>>>> Debian:
> >>>>>    - Debian10 is the official docker image to support and be
> >> published
> >>>> to
> >>>>> Docker Hub, we will also support Debian9.
> >>>>> CentOS:
> >>>>>    - We will support CentOS7.  Note, I do not think it has native
> >>>>> Python3.7 support, but we are also not there yet.
> >>>>> Ubuntu:
> >>>>>    - We will still support one version. At this time the version is
> >>>> 18.04.
> >>>>>
> >>>>> We will upgrade supported versions of each OS as updates/deprecation
> >> of
> >>>>> support are needed.
> >>>>>
> >>>>> Again let's wait 72 hours before starting a vote.  Also please check
> >>> that
> >>>>> the statements I wrote down are what you understood!
> >>>>>
> >>>>> See below for notes on lazy consensus:
> >>>>> https://community.apache.org/committers/lazyConsensus.html
> >>>>>
> >>>>> - Josh
> >>>>>
> >>>>>
> >>>>> On Wed, May 20, 2020 at 9:41 PM Ning Wang <wa...@gmail.com>
> >>> wrote:
> >>>>>
> >>>>>> +1
> >>>>>>
> >>>>>> On Wed, May 20, 2020 at 7:26 PM Nicholas Nezis <
> >>>> nicholas.nezis@gmail.com
> >>>>>>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> I like those options.
> >>>>>>>
> >>>>>>> On Wed, May 20, 2020 at 10:15 PM Josh Fischer <
> >> josh@joshfischer.io
> >>>>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>>> Let's wait 3 more days for anyone to give input.  Then we can
> >>>> start a
> >>>>>>> vote
> >>>>>>>> to make a final decision on which containers for us to support.
> >>>> Keep
> >>>>>> in
> >>>>>>>> mind that even though we do need to plan for Python3 support
> >>> (There
> >>>>> is
> >>>>>>> work
> >>>>>>>> in progress now) we do not need it as of yet.
> >>>>>>>>
> >>>>>>>> - Josh
> >>>>>>>>
> >>>>>>>> On Wed, May 20, 2020 at 8:34 PM thinker0 <th...@gmail.com>
> >>>> wrote:
> >>>>>>>>
> >>>>>>>>> My experience is mainly for CentOS 7, so I think it would be
> >>> nice
> >>>>> if
> >>>>>> it
> >>>>>>>> was
> >>>>>>>>> included.
> >>>>>>>>>
> >>>>>>>>> 2020년 5월 21일 (목) 오전 9:56, Josh Fischer <josh@joshfischer.io
> >>> 님이
> >>>> 작성:
> >>>>>>>>>
> >>>>>>>>>> Well thought out Windham.  I like how you narrowed the
> >>> criteria
> >>>>>> down
> >>>>>>> to
> >>>>>>>>>> Debian10/CentOS7(depending on PY3 support) thinking about
> >>>> overall
> >>>>>>>> support
> >>>>>>>>>> and what others would need/want.
> >>>>>>>>>>
> >>>>>>>>>> +1
> >>>>>>>>>>
> >>>>>>>>>> - Josh
> >>>>>>>>>>
> >>>>>>>>>> On Mon, May 18, 2020 at 8:39 PM Windham Wong <
> >>>>>>> windham.wong@stormeye.io
> >>>>>>>>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> My experience is that, people tends to pick their
> >> container
> >>>>> image
> >>>>>>> OS
> >>>>>>>>>> based
> >>>>>>>>>>> on few criterias only:
> >>>>>>>>>>> 1) image size, alpine always win, then debian/centos and
> >>> 3rd
> >>>> is
> >>>>>>>>>>> ubuntu/centos. alpine is always the smallest but not much
> >>>>> library
> >>>>>>>>>> provided
> >>>>>>>>>>> and require to compile manually. debian is always good
> >>>> because
> >>>>> it
> >>>>>>> has
> >>>>>>>>>>> apt-get to provide loads of library. centos is cool as
> >>> debian
> >>>>>>> because
> >>>>>>>>> it
> >>>>>>>>>>> also has good yum repo. ubuntu is always the last because
> >>> it
> >>>> is
> >>>>>>> fat,
> >>>>>>>>> but
> >>>>>>>>>> it
> >>>>>>>>>>> provides very large range of libraries to use.
> >>>>>>>>>>> 2) kernel feature, as mentioned last email, ubuntu has
> >>> issue
> >>>>> with
> >>>>>>> DNS
> >>>>>>>>> and
> >>>>>>>>>>> some sort of network discover issues. I remember alpine
> >> has
> >>>>> some
> >>>>>> as
> >>>>>>>>> well,
> >>>>>>>>>>> because it is very plain, requires many fine tunes be
> >> done
> >>>>>>> manually.
> >>>>>>>> in
> >>>>>>>>>>> this case, centos and debian should win.
> >>>>>>>>>>> 3) linux stream, obviously, alpine, debian/ubuntu,
> >> centos,
> >>>> are
> >>>>> 3
> >>>>>>>>>> different
> >>>>>>>>>>> streams that use different native tools. Poeple just pick
> >>>> their
> >>>>>>> base
> >>>>>>>>>> image
> >>>>>>>>>>> based on what they often use in their life. However, I
> >> see
> >>> a
> >>>>> lot
> >>>>>>>> debian
> >>>>>>>>>> and
> >>>>>>>>>>> alpine on docker hub because they are just smaller in
> >> size.
> >>>>>>>>>>> 4. library support. Python library is a very good example
> >>> in
> >>>>> this
> >>>>>>>> case.
> >>>>>>>>>> As
> >>>>>>>>>>> Heron is turning to Python 3, which specifically Python
> >>> 3.7+,
> >>>>>> some
> >>>>>>>>>>> discontinued based OS version does not provide the
> >> support
> >>>>>> anymore.
> >>>>>>>> We
> >>>>>>>>>> have
> >>>>>>>>>>> tested out that, debian9 supports only up to Python3.5,
> >>>> ubuntu
> >>>>>>> 14.04
> >>>>>>>>> does
> >>>>>>>>>>> not support python3 and ubuntu 16.04 supports up to
> >> Python
> >>>> 3.5
> >>>>>>> only.
> >>>>>>>>>>>
> >>>>>>>>>>> I am not a big fan of using centos for docker. Concluding
> >>>> all 4
> >>>>>>>> points
> >>>>>>>>>>> above, I believe keeping centos7 (if it supports
> >> python37),
> >>>>>>> debian10
> >>>>>>>>> (not
> >>>>>>>>>>> debian9), ubuntu 18.04 (supports python3.7+) would be the
> >>>> best
> >>>>>>>> solution
> >>>>>>>>>> at
> >>>>>>>>>>> the moment. We have to cut off the old images due to our
> >>>>> library
> >>>>>>>>> upgrade
> >>>>>>>>>>> and this is unavoidable. Otherwise, we would have tons of
> >>>>> issues
> >>>>>>>>> sending
> >>>>>>>>>> in
> >>>>>>>>>>> about library support in the future and the workload
> >> could
> >>>> be a
> >>>>>> lot
> >>>>>>>>> more
> >>>>>>>>>>> for just answering questions.
> >>>>>>>>>>>
> >>>>>>>>>>> Windham Wong
> >>>>>>>>>>> Co-Founder, Technical Director of
> >>>>>>>>>>> Stormeye.io, Hong Kong Managed Security Operation Center
> >>>>> Limited
> >>>>>>>>>>> Email // windham.wong@stormeye.io (mailto:
> >>>>>> windham.wong@stormeye.io
> >>>>>>> )
> >>>>>>>>>>> Phone // +852_3590_2212_|_+852_9832_0707
> >> (tel:+85235902212)
> >>>>>>>>>>> Fax // +852_3590_2202 (tel:+852_3590_2202)
> >>>>>>>>>>>
> >>>>>>>>>>> On 5月 19 2020, at 6:09 早上, Josh Fischer <
> >>> josh@joshfischer.io
> >>>>>
> >>>>>>> wrote:
> >>>>>>>>>>>> Hi All, I think using CentOS and either Debian or
> >> Ubuntu
> >>>> is a
> >>>>>>> good
> >>>>>>>>>> idea.
> >>>>>>>>>>> I would pick Debian over Ubuntu for the reason that Nick
> >>>>> pointed
> >>>>>>> out
> >>>>>>>>> in a
> >>>>>>>>>>> previous email of DNS issues in Kubernetes. I"ve copy and
> >>>>> pasted
> >>>>>> a
> >>>>>>>>>> section
> >>>>>>>>>>> of Nick's previous email. ### Start I've run into issues
> >>> with
> >>>>>>> Ubuntu
> >>>>>>>>>> images
> >>>>>>>>>>> having DNS issues in Kubernetes.
> >>>>>>>>>>> https://github.com/kubernetes/kubeadm/issues/787 ### End
> >>> As
> >>>>> for
> >>>>>>> what
> >>>>>>>>>> ends
> >>>>>>>>>>> up being the official Docker container I'm fine with
> >> either
> >>>>>> choice
> >>>>>>>>> Debian
> >>>>>>>>>>> or CentOS. On Mon, May 18, 2020 at 2:10 AM Ning Wang
> >>> wrote: >
> >>>>>> Yeah.
> >>>>>>>> For
> >>>>>>>>>>> binary release, the license is a huge factor. > > On Sun,
> >>> May
> >>>>> 17,
> >>>>>>>> 2020
> >>>>>>>>> at
> >>>>>>>>>>> 2:32 PM Josh Fischer wrote: > > > Hi All, > > > > When it
> >>>> comes
> >>>>>> to
> >>>>>>>>>> choosing
> >>>>>>>>>>> the official Apache Docker image for Heron we > > have to
> >>>> think
> >>>>>> of
> >>>>>>>> the
> >>>>>>>>>>> licensing issues first. Everything that is > installed >
> >>>
> >>>> into
> >>>>>> the
> >>>>>>>>>>> container at build time with the exception of the "FROM"
> >>>
> >>>>
> >>>>>>>> statement
> >>>>>>>>> in
> >>>>>>>>>>> the Dockerfile must be ALv2 compatible. OpenJDK is under
> >>>
> >>>>> GPLv2
> >>>>>>>
> >>>>>>>>
> >>>>>>>>>> which
> >>>>>>>>>>> is not compatible with Apache. However we can get around
> >>> this
> >>>>> if
> >>>>>>>
> >>>>>>> we
> >>>>>>>>>>>
> >>>>>>>>>>> bring the OpenJDK into the container via a FROM
> >> statement.
> >>>> I've
> >>>>>>>> copied
> >>>>>>>>> a
> >>>>>>>>>>>
> >>>>>>>>>>>> reply from Dave in our dev list in a previous email.
> >> See
> >>>>> below.
> >>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> ######## Start ######## > > Regarding OpenJDK and GPL2 -
> >>> here
> >>>>> is
> >>>>>>> what
> >>>>>>>>>> Roman
> >>>>>>>>>>> the VP, Legal wrote when > > answering Beam’s questions.
> >>>
> >>>>
> >>>>>
> >>>>>>
> >>>>>>>> Roman
> >>>>>>>>>>> Shaposhnik commented on LEGAL-503: > >
> >>>>>>>>>>> ---------------------------------------- > > > > Hey
> >>> [~altay]
> >>>>> if
> >>>>>>> you
> >>>>>>>>>> would
> >>>>>>>>>>> like to continue linking to the Docker release > >
> >> artifact
> >>>>> from
> >>>>>>> the
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> https://beam.apache.org > > you will have: > > 1.
> >>> Transition
> >>>>> to
> >>>>>>> the
> >>>>>>>>>>> official ASF dockerhub org: > >
> >>>>> https://hub.docker.com/u/apache
> >>>>>>>
> >>>>>>>>
> >>>>>>>> 2.
> >>>>>>>>>>> Start including that binary convenience artifact into
> >> your
> >>>> VOTE
> >>>>>>>>
> >>>>>>>>>> threads
> >>>>>>>>>>> on Beam releases > > 3. Make sure that all Cat-X licenses
> >>> are
> >>>>>> ONLY
> >>>>>>>>>> brought
> >>>>>>>>>>> into your > > container via FROM statements > > ########
> >>> End
> >>>>>>>> ######## >
> >>>>>>>>>>>>
> >>>>>>>>>>>> So at the end of the day, we need to use a container
> >> that
> >>>> has
> >>>>>>> Java
> >>>>>>>>>
> >>>>>>>>>>> already > > installed in it so that when we create our
> >>>> official
> >>>>>>> image
> >>>>>>>>> we
> >>>>>>>>>>> can inherit > > that java containing image via a FROM
> >>>>> statement.
> >>>>>>>
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> -
> >>>>>>>>>>> Josh > > > > > > On Sun, May 17, 2020 at 4:12 PM H W
> >>> wrote: >
> >>>>>>>
> >>>>>>>>>
> >>>>>>>>> We
> >>>>>>>>>>> use Centos 7 as well. > > > > > > On Sun, May 17, 2020 at
> >>>> 10:41
> >>>>>> AM
> >>>>>>>>>> Nicholas
> >>>>>>>>>>> Nezis < > > nicholas.nezis@gmail.com> > > > wrote: > > >
> >>>>>
> >>>>>>>
> >>>>>>>> Some
> >>>>>>>>>>> thoughts I'd like to add to the discussion: > > > > > >
> >>>>
> >>>>>> Debian
> >>>>>>>> and
> >>>>>>>>>>> Ubuntu are somewhat similar. Would it make sense to pick
> >>>
> >>>> one
> >>>>>>
> >>>>>>>
> >>>>>>>>
> >>>>>>>> of
> >>>>>>>>>>>>
> >>>>>>>>>>>>> those and pair with Centos? > > > > > > > > I've run
> >>> into
> >>>>>>> issues
> >>>>>>>>> with
> >>>>>>>>>>> Ubuntu images having DNS issues in > > Kubernetes. > > >
> >>>
> >>>>>>>>>>> https://github.com/kubernetes/kubeadm/issues/787 > > > >
> >>>>>
> >>>>>>>
> >>>>>>>>> Ubuntu
> >>>>>>>>>>> 14.04 should be removed. It was end of life last year.
> >> For
> >>>>
> >>>>>> those
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> of
> >>>>>>>>>>>>>>> you using Heron in production, is anyone using
> >>> Ubuntu
> >>>>> 16?
> >>>>>>> My
> >>>>>>>>>> vote >
> >>>>>>>>>>> would > > > be > > > > to remove it also. For what it's
> >>>> worth,
> >>>>> my
> >>>>>>> use
> >>>>>>>>>> case
> >>>>>>>>>>> is in K8s and we > > use > > > > the Centos7 image on a
> >>>> Centos
> >>>>> 7
> >>>>>>> OS.
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>>>>>> We should do a test of the official image and
> >>> helm
> >>>>>> chart
> >>>>>>>> in >
> >>>>>>>>>>> Kubernetes > > > > (maybe with Kind
> >>>> https://kind.sigs.k8s.io/
> >>>>> )
> >>>>>> as
> >>>>>>>>> part
> >>>>>>>>>>> of our release > > > > process. This might be nice to add
> >>> as
> >>>> an
> >>>>>>>>>> integration
> >>>>>>>>>>> test. > > > > > > > > > > > > > > > > > > > > > > > > > >
> >>>>>
> >>>>>>>
> >>>>>>>>>
> >>>>>>>>> On
> >>>>>>>>>>> Sun, May 17, 2020, 12:37 PM Ning Wang > wrote: > > > > >
> >>>
> >>>>
> >>>>>
> >>>>>>
> >>>>>>> That
> >>>>>>>>> is
> >>>>>>>>>> a
> >>>>>>>>>>> good point. We need to adjust the OS version accordingly
> >>>
> >>>>
> >>>>>> too. >
> >>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Sun, May 17, 2020 at 2:28 AM Windham Wong <
> >>>
> >>>>
> >>>>>>>>>>> windham.wong@stormeye.io > > > > > > > > > wrote: > > >
> >>>
> >>>>
> >>>>>
> >>>>>>
> >>>>>>>
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>>>>>> debian 10 has no python 2.7 support. Ubuntu
> >> 16.04
> >>>> has
> >>>>>> no
> >>>>>>>>>> python >
> >>>>>>>>>>>> 3.7+ > > > > dev > > > > > > library support. hope this
> >>>> helps
> >>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>> -- > > > > > > Sent from myMail for Android Sunday, 17
> >> May
> >>>>> 2020,
> >>>>>>>>> 11:34AM
> >>>>>>>>>>> +08:00 > > from > > > > > Ning > > > > > > Wang
> >>>>>>> wangninggm@gmail.com
> >>>>>>>>> : >
> >>>>>>>>>>>>>>>>>>>>>>> Hi, > > > > > > > > > > > > > >We
> >>> have
> >>>>>>> talked
> >>>>>>>>>> about
> >>>>>>>>>>> it a few times in different places. Let's > > make > > >
> >> a
> >>>>
> >>>>>
> >>>>>>
> >>>>>>>
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>>>>> decision here. We don't have enough resources to support
> >>>> many
> >>>>>>
> >>>>>>>
> >>>>>>>>>> docker >
> >>>>>>>>>>>>>>> OSes > > > > > > >(technically we only support
> >>>> Debian,
> >>>>>>>> Ubuntu,
> >>>>>>>>>> and
> >>>>>>>>>>> CentOS but each > > of > > > > them > > > > > > >has
> >>> multiple
> >>>>>>>>> versions).
> >>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Things we have agreed on (my
> >>>>>>> understanding)
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>>>>> - for binary release, we are going to choose one
> >> OS
> >>>> one
> >>>>>>
> >>>>>>>>>> version. I
> >>>>>>>>>>>>>>>>>> remember that Debian 9 was the decision. An
> >>> open
> >>>>>>>> question
> >>>>>>>>>> is:
> >>>>>>>>>>> do > > we > > > > want > > > > > > to > > > > > > >switch
> >>> to
> >>>>>> Debian
> >>>>>>>> 10
> >>>>>>>>>> now?
> >>>>>>>>>>> We haven't had a binary release yet, so > > > there > > >
> >>>>>
> >>>>> is
> >>>>>>>
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>>> still time. As the main target, I think 1. the image
> >>>> should
> >>>>>>
> >>>>>>>
> >>>>>>>>>> compile
> >>>>>>>>>>>>>>>>>> successfully. and 2 the image should be
> >>> tested:
> >>>>> the
> >>>>>>>> tools
> >>>>>>>>>> and
> >>>>>>>>>>> an > > > > example > > > > > > >topology. > > > > > > >
> >>>>
> >>>>>>
> >>>>>>>>
> >>>>>>>>> -
> >>>>>>>>>> for
> >>>>>>>>>>> source release, we haven't talked about it very much. > >
> >>>>>
> >>>>>>>>> Personally
> >>>>>>>>>> I
> >>>>>>>>>>>>>>>>>> feel it is a reasonable expectation that, if
> >>> we
> >>>>> have
> >>>>>>> the
> >>>>>>>>>>> Docker > > file > > > > in > > > > > > the > > > > > >
> >>>> core
> >>>>>>> folders,
> >>>>>>>>> it
> >>>>>>>>>>> should at least compile. Otherwise, it's a > failed > >
> >>>>
> >>>>>> build >
> >>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>> hence a failed release. We are not going to test run
> >> the
> >>>>>> tools >
> >>>>>>>> and
> >>>>>>>>>>>>
> >>>>>>>>>>>>>>> examples > > > > > > >for each release though. >
> >>>
> >>>>
> >>>>>
> >>>>>>
> >>>>>>>
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>>>>> Finally, a question about OS versions. I believe
> >> at
> >>>>> least
> >>>>>>> two
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> releases > > > > > > >should be supported because most
> >>> people
> >>>>>> don't
> >>>>>>>>>> upgrade
> >>>>>>>>>>> to the > > latest > > > > > > version > > > > > > >when
> >> it
> >>> is
> >>>>> out
> >>>>>>> in
> >>>>>>>>> more
> >>>>>>>>>>> serious scenarios, hence IMO the second > > > oldest > >
> >>>
> >>>>
> >>>>>
> >>>>>>
> >>>>>>>> could
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>>>>>> often be more useful than the latest one from a
> >>>>>>> convenience
> >>>>>>>>>
> >>>>>>>>>>> point > > of > > > > > view. > > > > > > >On the other
> >>> hand,
> >>>>> this
> >>>>>>> is
> >>>>>>>>> too
> >>>>>>>>>>> flexible, and we will spend time > > > > thinking > > > >
> >>>>>
> >>>>>>>> about
> >>>>>>>> it
> >>>>>>>>>>> again. I think a clear guideline could be helpful to >
> >> us.
> >>>>
> >>>>>
> >>>>>>
> >>>>>>> For
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>>>>> example: > > > > > > >- We only include only
> >> Debian
> >>>>>> (popular
> >>>>>>>> in
> >>>>>>>>>> the
> >>>>>>>>>>> server world) and > > > Ubuntu > > > > > > >(popular in
> >>>> servers
> >>>>>> and
> >>>>>>>>>>> workstations). > > > > > > >- for the main OS (Debian) we
> >>>>> choose
> >>>>>>> the
> >>>>>>>>>> latest
> >>>>>>>>>>> to compile and > > test > > > > for > > > > > > our > >
> >>>>
> >>>>>>
> >>>>>>>>> binary
> >>>>>>>>>>> release. > > > > > > >- for a given OS, only two versions
> >>>> (most
> >>>>>>>> likely
> >>>>>>>>>> the
> >>>>>>>>>>> most and > the > > > > second > > > > > > >recent
> >> releases)
> >>>> are
> >>>>>>>>> included
> >>>>>>>>>> in
> >>>>>>>>>>> the core source code and they > > need > > > to > > > > >
> >>> be
> >>>>>
> >>>>>>
> >>>>>>>
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> fixed ASAP if they don't compile. > > > > > > >- OSs and
> >>>> other
> >>>>>>>>> versions
> >>>>>>>>>>> could be included in a special folder > and > > > but > >
> >>>>>
> >>>>>>>
> >>>>>>>>>> there
> >>>>>>>>>> is
> >>>>>>>>>>> no guarantee that the code will compile. Fixes from the >
> >>>>>
> >>>>>>>
> >>>>>>>>>> community
> >>>>>>>>>>>>>>>>>> are welcome. > > > > > > > > > > > > > > > >
> >>>>>
> >>>>>>>
> >>>>>>>>> How
> >>>>>>>>> do
> >>>>>>>>>>> you guys think? > > > > > > >--ning > > > > > > > > > >
> >>>>
> >>>>>>
> >>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
>
>

Re: [DISCUSSION] Docker OS support

[Dave Fisher <wa...@apache.org>]

I thought that the purpose of this discussion was to find the one or two platforms for Heron to support in order to keep down the complexity of making the first binary release for the community.

Other platforms could be on a roadmap for future releases.

Regards,
Dave

> On May 26, 2020, at 12:26 PM, Ning Wang <wa...@gmail.com> wrote:
> 
> My concern about "latest" is that I feel most people hold it for some time
> before migrating to the latest version (if it ever happens). If there are
> numbers, it could make sense to use the most popular version (plus the
> other criterias).
> 
> On Tue, May 26, 2020 at 12:14 PM Oliver Bristow <ol...@oliverbristow.co.uk>
> wrote:
> 
>> Sorry if I missed the rationale for Ubuntu 18.04 instead of 20.04, but
>> would the latest LTS be a better choice? 18.04 was released before Debian
>> 10, so that may be an argument for 20.04 re. compatibility if Debian 10 is
>> the standard.
>> 
>> There's already a 20.04 Dockerfile + I It looks like the vagrant setup will
>> get an upgrade from 14.04 to 20.04 as a part of the python3 upgrade, and
>> seems to run fine so far.
>> 
>> On Tue, 26 May 2020 at 19:30, Josh Fischer <jo...@joshfischer.io> wrote:
>> 
>>> Thanks for further clarifying. Yes, that is correct.
>>> 
>>> 
>>> 
>>> On Tue, May 26, 2020 at 1:16 PM Ning Wang <wa...@gmail.com> wrote:
>>> 
>>>> +1.
>>>> 
>>>> So my understanding is:
>>>> binary (in future release): Debian 10.
>>>> source: Debian 9/10 + ubuntu 18.04 + centos 7. version numbers will be
>>>> updated in the future after more discussions/votes.
>>>> 
>>>> others will be moved to a "extra" folder only for reference.
>>>> 
>>>> 
>>>> 
>>>> On Tue, May 26, 2020 at 6:13 AM Josh Fischer <jo...@joshfischer.io>
>>> wrote:
>>>> 
>>>>> Hi All,
>>>>> 
>>>>> Based on what I've read on this thread a combination of CentOS/Debian
>>>> would
>>>>> be the better options for us to support multiple versions for the
>>>>> community.  We will also support one version of Ubuntu.
>>>>> 
>>>>> So..... I will start a vote in 72 hours that calls on the vote on
>> this.
>>>>> I'm going to write down my understanding of what this thread covers
>> to
>>>> get
>>>>> some actionable topics to vote on.  If I've written anything down
>>>>> incorrectly please speak up before we start a vote.
>>>>> 
>>>>> Debian:
>>>>>    - Debian10 is the official docker image to support and be
>> published
>>>> to
>>>>> Docker Hub, we will also support Debian9.
>>>>> CentOS:
>>>>>    - We will support CentOS7.  Note, I do not think it has native
>>>>> Python3.7 support, but we are also not there yet.
>>>>> Ubuntu:
>>>>>    - We will still support one version. At this time the version is
>>>> 18.04.
>>>>> 
>>>>> We will upgrade supported versions of each OS as updates/deprecation
>> of
>>>>> support are needed.
>>>>> 
>>>>> Again let's wait 72 hours before starting a vote.  Also please check
>>> that
>>>>> the statements I wrote down are what you understood!
>>>>> 
>>>>> See below for notes on lazy consensus:
>>>>> https://community.apache.org/committers/lazyConsensus.html
>>>>> 
>>>>> - Josh
>>>>> 
>>>>> 
>>>>> On Wed, May 20, 2020 at 9:41 PM Ning Wang <wa...@gmail.com>
>>> wrote:
>>>>> 
>>>>>> +1
>>>>>> 
>>>>>> On Wed, May 20, 2020 at 7:26 PM Nicholas Nezis <
>>>> nicholas.nezis@gmail.com
>>>>>> 
>>>>>> wrote:
>>>>>> 
>>>>>>> I like those options.
>>>>>>> 
>>>>>>> On Wed, May 20, 2020 at 10:15 PM Josh Fischer <
>> josh@joshfischer.io
>>>> 
>>>>>> wrote:
>>>>>>> 
>>>>>>>> Let's wait 3 more days for anyone to give input.  Then we can
>>>> start a
>>>>>>> vote
>>>>>>>> to make a final decision on which containers for us to support.
>>>> Keep
>>>>>> in
>>>>>>>> mind that even though we do need to plan for Python3 support
>>> (There
>>>>> is
>>>>>>> work
>>>>>>>> in progress now) we do not need it as of yet.
>>>>>>>> 
>>>>>>>> - Josh
>>>>>>>> 
>>>>>>>> On Wed, May 20, 2020 at 8:34 PM thinker0 <th...@gmail.com>
>>>> wrote:
>>>>>>>> 
>>>>>>>>> My experience is mainly for CentOS 7, so I think it would be
>>> nice
>>>>> if
>>>>>> it
>>>>>>>> was
>>>>>>>>> included.
>>>>>>>>> 
>>>>>>>>> 2020년 5월 21일 (목) 오전 9:56, Josh Fischer <josh@joshfischer.io
>>> 님이
>>>> 작성:
>>>>>>>>> 
>>>>>>>>>> Well thought out Windham.  I like how you narrowed the
>>> criteria
>>>>>> down
>>>>>>> to
>>>>>>>>>> Debian10/CentOS7(depending on PY3 support) thinking about
>>>> overall
>>>>>>>> support
>>>>>>>>>> and what others would need/want.
>>>>>>>>>> 
>>>>>>>>>> +1
>>>>>>>>>> 
>>>>>>>>>> - Josh
>>>>>>>>>> 
>>>>>>>>>> On Mon, May 18, 2020 at 8:39 PM Windham Wong <
>>>>>>> windham.wong@stormeye.io
>>>>>>>>> 
>>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> My experience is that, people tends to pick their
>> container
>>>>> image
>>>>>>> OS
>>>>>>>>>> based
>>>>>>>>>>> on few criterias only:
>>>>>>>>>>> 1) image size, alpine always win, then debian/centos and
>>> 3rd
>>>> is
>>>>>>>>>>> ubuntu/centos. alpine is always the smallest but not much
>>>>> library
>>>>>>>>>> provided
>>>>>>>>>>> and require to compile manually. debian is always good
>>>> because
>>>>> it
>>>>>>> has
>>>>>>>>>>> apt-get to provide loads of library. centos is cool as
>>> debian
>>>>>>> because
>>>>>>>>> it
>>>>>>>>>>> also has good yum repo. ubuntu is always the last because
>>> it
>>>> is
>>>>>>> fat,
>>>>>>>>> but
>>>>>>>>>> it
>>>>>>>>>>> provides very large range of libraries to use.
>>>>>>>>>>> 2) kernel feature, as mentioned last email, ubuntu has
>>> issue
>>>>> with
>>>>>>> DNS
>>>>>>>>> and
>>>>>>>>>>> some sort of network discover issues. I remember alpine
>> has
>>>>> some
>>>>>> as
>>>>>>>>> well,
>>>>>>>>>>> because it is very plain, requires many fine tunes be
>> done
>>>>>>> manually.
>>>>>>>> in
>>>>>>>>>>> this case, centos and debian should win.
>>>>>>>>>>> 3) linux stream, obviously, alpine, debian/ubuntu,
>> centos,
>>>> are
>>>>> 3
>>>>>>>>>> different
>>>>>>>>>>> streams that use different native tools. Poeple just pick
>>>> their
>>>>>>> base
>>>>>>>>>> image
>>>>>>>>>>> based on what they often use in their life. However, I
>> see
>>> a
>>>>> lot
>>>>>>>> debian
>>>>>>>>>> and
>>>>>>>>>>> alpine on docker hub because they are just smaller in
>> size.
>>>>>>>>>>> 4. library support. Python library is a very good example
>>> in
>>>>> this
>>>>>>>> case.
>>>>>>>>>> As
>>>>>>>>>>> Heron is turning to Python 3, which specifically Python
>>> 3.7+,
>>>>>> some
>>>>>>>>>>> discontinued based OS version does not provide the
>> support
>>>>>> anymore.
>>>>>>>> We
>>>>>>>>>> have
>>>>>>>>>>> tested out that, debian9 supports only up to Python3.5,
>>>> ubuntu
>>>>>>> 14.04
>>>>>>>>> does
>>>>>>>>>>> not support python3 and ubuntu 16.04 supports up to
>> Python
>>>> 3.5
>>>>>>> only.
>>>>>>>>>>> 
>>>>>>>>>>> I am not a big fan of using centos for docker. Concluding
>>>> all 4
>>>>>>>> points
>>>>>>>>>>> above, I believe keeping centos7 (if it supports
>> python37),
>>>>>>> debian10
>>>>>>>>> (not
>>>>>>>>>>> debian9), ubuntu 18.04 (supports python3.7+) would be the
>>>> best
>>>>>>>> solution
>>>>>>>>>> at
>>>>>>>>>>> the moment. We have to cut off the old images due to our
>>>>> library
>>>>>>>>> upgrade
>>>>>>>>>>> and this is unavoidable. Otherwise, we would have tons of
>>>>> issues
>>>>>>>>> sending
>>>>>>>>>> in
>>>>>>>>>>> about library support in the future and the workload
>> could
>>>> be a
>>>>>> lot
>>>>>>>>> more
>>>>>>>>>>> for just answering questions.
>>>>>>>>>>> 
>>>>>>>>>>> Windham Wong
>>>>>>>>>>> Co-Founder, Technical Director of
>>>>>>>>>>> Stormeye.io, Hong Kong Managed Security Operation Center
>>>>> Limited
>>>>>>>>>>> Email // windham.wong@stormeye.io (mailto:
>>>>>> windham.wong@stormeye.io
>>>>>>> )
>>>>>>>>>>> Phone // +852_3590_2212_|_+852_9832_0707
>> (tel:+85235902212)
>>>>>>>>>>> Fax // +852_3590_2202 (tel:+852_3590_2202)
>>>>>>>>>>> 
>>>>>>>>>>> On 5月 19 2020, at 6:09 早上, Josh Fischer <
>>> josh@joshfischer.io
>>>>> 
>>>>>>> wrote:
>>>>>>>>>>>> Hi All, I think using CentOS and either Debian or
>> Ubuntu
>>>> is a
>>>>>>> good
>>>>>>>>>> idea.
>>>>>>>>>>> I would pick Debian over Ubuntu for the reason that Nick
>>>>> pointed
>>>>>>> out
>>>>>>>>> in a
>>>>>>>>>>> previous email of DNS issues in Kubernetes. I"ve copy and
>>>>> pasted
>>>>>> a
>>>>>>>>>> section
>>>>>>>>>>> of Nick's previous email. ### Start I've run into issues
>>> with
>>>>>>> Ubuntu
>>>>>>>>>> images
>>>>>>>>>>> having DNS issues in Kubernetes.
>>>>>>>>>>> https://github.com/kubernetes/kubeadm/issues/787 ### End
>>> As
>>>>> for
>>>>>>> what
>>>>>>>>>> ends
>>>>>>>>>>> up being the official Docker container I'm fine with
>> either
>>>>>> choice
>>>>>>>>> Debian
>>>>>>>>>>> or CentOS. On Mon, May 18, 2020 at 2:10 AM Ning Wang
>>> wrote: >
>>>>>> Yeah.
>>>>>>>> For
>>>>>>>>>>> binary release, the license is a huge factor. > > On Sun,
>>> May
>>>>> 17,
>>>>>>>> 2020
>>>>>>>>> at
>>>>>>>>>>> 2:32 PM Josh Fischer wrote: > > > Hi All, > > > > When it
>>>> comes
>>>>>> to
>>>>>>>>>> choosing
>>>>>>>>>>> the official Apache Docker image for Heron we > > have to
>>>> think
>>>>>> of
>>>>>>>> the
>>>>>>>>>>> licensing issues first. Everything that is > installed >
>>> 
>>>> into
>>>>>> the
>>>>>>>>>>> container at build time with the exception of the "FROM"
>>> 
>>>> 
>>>>>>>> statement
>>>>>>>>> in
>>>>>>>>>>> the Dockerfile must be ALv2 compatible. OpenJDK is under
>>> 
>>>>> GPLv2
>>>>>>> 
>>>>>>>> 
>>>>>>>>>> which
>>>>>>>>>>> is not compatible with Apache. However we can get around
>>> this
>>>>> if
>>>>>>> 
>>>>>>> we
>>>>>>>>>>> 
>>>>>>>>>>> bring the OpenJDK into the container via a FROM
>> statement.
>>>> I've
>>>>>>>> copied
>>>>>>>>> a
>>>>>>>>>>> 
>>>>>>>>>>>> reply from Dave in our dev list in a previous email.
>> See
>>>>> below.
>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> ######## Start ######## > > Regarding OpenJDK and GPL2 -
>>> here
>>>>> is
>>>>>>> what
>>>>>>>>>> Roman
>>>>>>>>>>> the VP, Legal wrote when > > answering Beam’s questions.
>>> 
>>>> 
>>>>> 
>>>>>> 
>>>>>>>> Roman
>>>>>>>>>>> Shaposhnik commented on LEGAL-503: > >
>>>>>>>>>>> ---------------------------------------- > > > > Hey
>>> [~altay]
>>>>> if
>>>>>>> you
>>>>>>>>>> would
>>>>>>>>>>> like to continue linking to the Docker release > >
>> artifact
>>>>> from
>>>>>>> the
>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> https://beam.apache.org > > you will have: > > 1.
>>> Transition
>>>>> to
>>>>>>> the
>>>>>>>>>>> official ASF dockerhub org: > >
>>>>> https://hub.docker.com/u/apache
>>>>>>> 
>>>>>>>> 
>>>>>>>> 2.
>>>>>>>>>>> Start including that binary convenience artifact into
>> your
>>>> VOTE
>>>>>>>> 
>>>>>>>>>> threads
>>>>>>>>>>> on Beam releases > > 3. Make sure that all Cat-X licenses
>>> are
>>>>>> ONLY
>>>>>>>>>> brought
>>>>>>>>>>> into your > > container via FROM statements > > ########
>>> End
>>>>>>>> ######## >
>>>>>>>>>>>> 
>>>>>>>>>>>> So at the end of the day, we need to use a container
>> that
>>>> has
>>>>>>> Java
>>>>>>>>> 
>>>>>>>>>>> already > > installed in it so that when we create our
>>>> official
>>>>>>> image
>>>>>>>>> we
>>>>>>>>>>> can inherit > > that java containing image via a FROM
>>>>> statement.
>>>>>>> 
>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> -
>>>>>>>>>>> Josh > > > > > > On Sun, May 17, 2020 at 4:12 PM H W
>>> wrote: >
>>>>>>> 
>>>>>>>>> 
>>>>>>>>> We
>>>>>>>>>>> use Centos 7 as well. > > > > > > On Sun, May 17, 2020 at
>>>> 10:41
>>>>>> AM
>>>>>>>>>> Nicholas
>>>>>>>>>>> Nezis < > > nicholas.nezis@gmail.com> > > > wrote: > > >
>>>>> 
>>>>>>> 
>>>>>>>> Some
>>>>>>>>>>> thoughts I'd like to add to the discussion: > > > > > >
>>>> 
>>>>>> Debian
>>>>>>>> and
>>>>>>>>>>> Ubuntu are somewhat similar. Would it make sense to pick
>>> 
>>>> one
>>>>>> 
>>>>>>> 
>>>>>>>> 
>>>>>>>> of
>>>>>>>>>>>> 
>>>>>>>>>>>>> those and pair with Centos? > > > > > > > > I've run
>>> into
>>>>>>> issues
>>>>>>>>> with
>>>>>>>>>>> Ubuntu images having DNS issues in > > Kubernetes. > > >
>>> 
>>>>>>>>>>> https://github.com/kubernetes/kubeadm/issues/787 > > > >
>>>>> 
>>>>>>> 
>>>>>>>>> Ubuntu
>>>>>>>>>>> 14.04 should be removed. It was end of life last year.
>> For
>>>> 
>>>>>> those
>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> of
>>>>>>>>>>>>>>> you using Heron in production, is anyone using
>>> Ubuntu
>>>>> 16?
>>>>>>> My
>>>>>>>>>> vote >
>>>>>>>>>>> would > > > be > > > > to remove it also. For what it's
>>>> worth,
>>>>> my
>>>>>>> use
>>>>>>>>>> case
>>>>>>>>>>> is in K8s and we > > use > > > > the Centos7 image on a
>>>> Centos
>>>>> 7
>>>>>>> OS.
>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>>>>>> We should do a test of the official image and
>>> helm
>>>>>> chart
>>>>>>>> in >
>>>>>>>>>>> Kubernetes > > > > (maybe with Kind
>>>> https://kind.sigs.k8s.io/
>>>>> )
>>>>>> as
>>>>>>>>> part
>>>>>>>>>>> of our release > > > > process. This might be nice to add
>>> as
>>>> an
>>>>>>>>>> integration
>>>>>>>>>>> test. > > > > > > > > > > > > > > > > > > > > > > > > > >
>>>>> 
>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On
>>>>>>>>>>> Sun, May 17, 2020, 12:37 PM Ning Wang > wrote: > > > > >
>>> 
>>>> 
>>>>> 
>>>>>> 
>>>>>>> That
>>>>>>>>> is
>>>>>>>>>> a
>>>>>>>>>>> good point. We need to adjust the OS version accordingly
>>> 
>>>> 
>>>>>> too. >
>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On Sun, May 17, 2020 at 2:28 AM Windham Wong <
>>> 
>>>> 
>>>>>>>>>>> windham.wong@stormeye.io > > > > > > > > > wrote: > > >
>>> 
>>>> 
>>>>> 
>>>>>> 
>>>>>>> 
>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>>>>>> debian 10 has no python 2.7 support. Ubuntu
>> 16.04
>>>> has
>>>>>> no
>>>>>>>>>> python >
>>>>>>>>>>>> 3.7+ > > > > dev > > > > > > library support. hope this
>>>> helps
>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>> -- > > > > > > Sent from myMail for Android Sunday, 17
>> May
>>>>> 2020,
>>>>>>>>> 11:34AM
>>>>>>>>>>> +08:00 > > from > > > > > Ning > > > > > > Wang
>>>>>>> wangninggm@gmail.com
>>>>>>>>> : >
>>>>>>>>>>>>>>>>>>>>>>> Hi, > > > > > > > > > > > > > >We
>>> have
>>>>>>> talked
>>>>>>>>>> about
>>>>>>>>>>> it a few times in different places. Let's > > make > > >
>> a
>>>> 
>>>>> 
>>>>>> 
>>>>>>> 
>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>>>> decision here. We don't have enough resources to support
>>>> many
>>>>>> 
>>>>>>> 
>>>>>>>>>> docker >
>>>>>>>>>>>>>>> OSes > > > > > > >(technically we only support
>>>> Debian,
>>>>>>>> Ubuntu,
>>>>>>>>>> and
>>>>>>>>>>> CentOS but each > > of > > > > them > > > > > > >has
>>> multiple
>>>>>>>>> versions).
>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> Things we have agreed on (my
>>>>>>> understanding)
>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>>>>> - for binary release, we are going to choose one
>> OS
>>>> one
>>>>>> 
>>>>>>>>>> version. I
>>>>>>>>>>>>>>>>>> remember that Debian 9 was the decision. An
>>> open
>>>>>>>> question
>>>>>>>>>> is:
>>>>>>>>>>> do > > we > > > > want > > > > > > to > > > > > > >switch
>>> to
>>>>>> Debian
>>>>>>>> 10
>>>>>>>>>> now?
>>>>>>>>>>> We haven't had a binary release yet, so > > > there > > >
>>>>> 
>>>>> is
>>>>>>> 
>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>>> still time. As the main target, I think 1. the image
>>>> should
>>>>>> 
>>>>>>> 
>>>>>>>>>> compile
>>>>>>>>>>>>>>>>>> successfully. and 2 the image should be
>>> tested:
>>>>> the
>>>>>>>> tools
>>>>>>>>>> and
>>>>>>>>>>> an > > > > example > > > > > > >topology. > > > > > > >
>>>> 
>>>>>> 
>>>>>>>> 
>>>>>>>>> -
>>>>>>>>>> for
>>>>>>>>>>> source release, we haven't talked about it very much. > >
>>>>> 
>>>>>>>>> Personally
>>>>>>>>>> I
>>>>>>>>>>>>>>>>>> feel it is a reasonable expectation that, if
>>> we
>>>>> have
>>>>>>> the
>>>>>>>>>>> Docker > > file > > > > in > > > > > > the > > > > > >
>>>> core
>>>>>>> folders,
>>>>>>>>> it
>>>>>>>>>>> should at least compile. Otherwise, it's a > failed > >
>>>> 
>>>>>> build >
>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>>> hence a failed release. We are not going to test run
>> the
>>>>>> tools >
>>>>>>>> and
>>>>>>>>>>>> 
>>>>>>>>>>>>>>> examples > > > > > > >for each release though. >
>>> 
>>>> 
>>>>> 
>>>>>> 
>>>>>>> 
>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>>>>> Finally, a question about OS versions. I believe
>> at
>>>>> least
>>>>>>> two
>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> releases > > > > > > >should be supported because most
>>> people
>>>>>> don't
>>>>>>>>>> upgrade
>>>>>>>>>>> to the > > latest > > > > > > version > > > > > > >when
>> it
>>> is
>>>>> out
>>>>>>> in
>>>>>>>>> more
>>>>>>>>>>> serious scenarios, hence IMO the second > > > oldest > >
>>> 
>>>> 
>>>>> 
>>>>>> 
>>>>>>>> could
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>>>>>> often be more useful than the latest one from a
>>>>>>> convenience
>>>>>>>>> 
>>>>>>>>>>> point > > of > > > > > view. > > > > > > >On the other
>>> hand,
>>>>> this
>>>>>>> is
>>>>>>>>> too
>>>>>>>>>>> flexible, and we will spend time > > > > thinking > > > >
>>>>> 
>>>>>>>> about
>>>>>>>> it
>>>>>>>>>>> again. I think a clear guideline could be helpful to >
>> us.
>>>> 
>>>>> 
>>>>>> 
>>>>>>> For
>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>>>>> example: > > > > > > >- We only include only
>> Debian
>>>>>> (popular
>>>>>>>> in
>>>>>>>>>> the
>>>>>>>>>>> server world) and > > > Ubuntu > > > > > > >(popular in
>>>> servers
>>>>>> and
>>>>>>>>>>> workstations). > > > > > > >- for the main OS (Debian) we
>>>>> choose
>>>>>>> the
>>>>>>>>>> latest
>>>>>>>>>>> to compile and > > test > > > > for > > > > > > our > >
>>>> 
>>>>>> 
>>>>>>>>> binary
>>>>>>>>>>> release. > > > > > > >- for a given OS, only two versions
>>>> (most
>>>>>>>> likely
>>>>>>>>>> the
>>>>>>>>>>> most and > the > > > > second > > > > > > >recent
>> releases)
>>>> are
>>>>>>>>> included
>>>>>>>>>> in
>>>>>>>>>>> the core source code and they > > need > > > to > > > > >
>>> be
>>>>> 
>>>>>> 
>>>>>>> 
>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>>> fixed ASAP if they don't compile. > > > > > > >- OSs and
>>>> other
>>>>>>>>> versions
>>>>>>>>>>> could be included in a special folder > and > > > but > >
>>>>> 
>>>>>>> 
>>>>>>>>>> there
>>>>>>>>>> is
>>>>>>>>>>> no guarantee that the code will compile. Fixes from the >
>>>>> 
>>>>>>> 
>>>>>>>>>> community
>>>>>>>>>>>>>>>>>> are welcome. > > > > > > > > > > > > > > > >
>>>>> 
>>>>>>> 
>>>>>>>>> How
>>>>>>>>> do
>>>>>>>>>>> you guys think? > > > > > > >--ning > > > > > > > > > >
>>>> 
>>>>>> 
>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>> 
>> 

Re: [DISCUSSION] Docker OS support

[Ning Wang <wa...@gmail.com>]

My concern about "latest" is that I feel most people hold it for some time
before migrating to the latest version (if it ever happens). If there are
numbers, it could make sense to use the most popular version (plus the
other criterias).

On Tue, May 26, 2020 at 12:14 PM Oliver Bristow <ol...@oliverbristow.co.uk>
wrote:

> Sorry if I missed the rationale for Ubuntu 18.04 instead of 20.04, but
> would the latest LTS be a better choice? 18.04 was released before Debian
> 10, so that may be an argument for 20.04 re. compatibility if Debian 10 is
> the standard.
>
> There's already a 20.04 Dockerfile + I It looks like the vagrant setup will
> get an upgrade from 14.04 to 20.04 as a part of the python3 upgrade, and
> seems to run fine so far.
>
> On Tue, 26 May 2020 at 19:30, Josh Fischer <jo...@joshfischer.io> wrote:
>
> > Thanks for further clarifying. Yes, that is correct.
> >
> >
> >
> > On Tue, May 26, 2020 at 1:16 PM Ning Wang <wa...@gmail.com> wrote:
> >
> > > +1.
> > >
> > > So my understanding is:
> > > binary (in future release): Debian 10.
> > > source: Debian 9/10 + ubuntu 18.04 + centos 7. version numbers will be
> > > updated in the future after more discussions/votes.
> > >
> > > others will be moved to a "extra" folder only for reference.
> > >
> > >
> > >
> > > On Tue, May 26, 2020 at 6:13 AM Josh Fischer <jo...@joshfischer.io>
> > wrote:
> > >
> > > > Hi All,
> > > >
> > > > Based on what I've read on this thread a combination of CentOS/Debian
> > > would
> > > > be the better options for us to support multiple versions for the
> > > > community.  We will also support one version of Ubuntu.
> > > >
> > > > So..... I will start a vote in 72 hours that calls on the vote on
> this.
> > > > I'm going to write down my understanding of what this thread covers
> to
> > > get
> > > > some actionable topics to vote on.  If I've written anything down
> > > > incorrectly please speak up before we start a vote.
> > > >
> > > > Debian:
> > > >     - Debian10 is the official docker image to support and be
> published
> > > to
> > > > Docker Hub, we will also support Debian9.
> > > > CentOS:
> > > >     - We will support CentOS7.  Note, I do not think it has native
> > > > Python3.7 support, but we are also not there yet.
> > > > Ubuntu:
> > > >     - We will still support one version. At this time the version is
> > > 18.04.
> > > >
> > > > We will upgrade supported versions of each OS as updates/deprecation
> of
> > > > support are needed.
> > > >
> > > > Again let's wait 72 hours before starting a vote.  Also please check
> > that
> > > > the statements I wrote down are what you understood!
> > > >
> > > > See below for notes on lazy consensus:
> > > > https://community.apache.org/committers/lazyConsensus.html
> > > >
> > > > - Josh
> > > >
> > > >
> > > > On Wed, May 20, 2020 at 9:41 PM Ning Wang <wa...@gmail.com>
> > wrote:
> > > >
> > > > > +1
> > > > >
> > > > > On Wed, May 20, 2020 at 7:26 PM Nicholas Nezis <
> > > nicholas.nezis@gmail.com
> > > > >
> > > > > wrote:
> > > > >
> > > > > > I like those options.
> > > > > >
> > > > > > On Wed, May 20, 2020 at 10:15 PM Josh Fischer <
> josh@joshfischer.io
> > >
> > > > > wrote:
> > > > > >
> > > > > > > Let's wait 3 more days for anyone to give input.  Then we can
> > > start a
> > > > > > vote
> > > > > > > to make a final decision on which containers for us to support.
> > > Keep
> > > > > in
> > > > > > > mind that even though we do need to plan for Python3 support
> > (There
> > > > is
> > > > > > work
> > > > > > > in progress now) we do not need it as of yet.
> > > > > > >
> > > > > > > - Josh
> > > > > > >
> > > > > > > On Wed, May 20, 2020 at 8:34 PM thinker0 <th...@gmail.com>
> > > wrote:
> > > > > > >
> > > > > > > > My experience is mainly for CentOS 7, so I think it would be
> > nice
> > > > if
> > > > > it
> > > > > > > was
> > > > > > > > included.
> > > > > > > >
> > > > > > > > 2020년 5월 21일 (목) 오전 9:56, Josh Fischer <josh@joshfischer.io
> >님이
> > > 작성:
> > > > > > > >
> > > > > > > > > Well thought out Windham.  I like how you narrowed the
> > criteria
> > > > > down
> > > > > > to
> > > > > > > > > Debian10/CentOS7(depending on PY3 support) thinking about
> > > overall
> > > > > > > support
> > > > > > > > > and what others would need/want.
> > > > > > > > >
> > > > > > > > > +1
> > > > > > > > >
> > > > > > > > > - Josh
> > > > > > > > >
> > > > > > > > > On Mon, May 18, 2020 at 8:39 PM Windham Wong <
> > > > > > windham.wong@stormeye.io
> > > > > > > >
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > My experience is that, people tends to pick their
> container
> > > > image
> > > > > > OS
> > > > > > > > > based
> > > > > > > > > > on few criterias only:
> > > > > > > > > > 1) image size, alpine always win, then debian/centos and
> > 3rd
> > > is
> > > > > > > > > > ubuntu/centos. alpine is always the smallest but not much
> > > > library
> > > > > > > > > provided
> > > > > > > > > > and require to compile manually. debian is always good
> > > because
> > > > it
> > > > > > has
> > > > > > > > > > apt-get to provide loads of library. centos is cool as
> > debian
> > > > > > because
> > > > > > > > it
> > > > > > > > > > also has good yum repo. ubuntu is always the last because
> > it
> > > is
> > > > > > fat,
> > > > > > > > but
> > > > > > > > > it
> > > > > > > > > > provides very large range of libraries to use.
> > > > > > > > > > 2) kernel feature, as mentioned last email, ubuntu has
> > issue
> > > > with
> > > > > > DNS
> > > > > > > > and
> > > > > > > > > > some sort of network discover issues. I remember alpine
> has
> > > > some
> > > > > as
> > > > > > > > well,
> > > > > > > > > > because it is very plain, requires many fine tunes be
> done
> > > > > > manually.
> > > > > > > in
> > > > > > > > > > this case, centos and debian should win.
> > > > > > > > > > 3) linux stream, obviously, alpine, debian/ubuntu,
> centos,
> > > are
> > > > 3
> > > > > > > > > different
> > > > > > > > > > streams that use different native tools. Poeple just pick
> > > their
> > > > > > base
> > > > > > > > > image
> > > > > > > > > > based on what they often use in their life. However, I
> see
> > a
> > > > lot
> > > > > > > debian
> > > > > > > > > and
> > > > > > > > > > alpine on docker hub because they are just smaller in
> size.
> > > > > > > > > > 4. library support. Python library is a very good example
> > in
> > > > this
> > > > > > > case.
> > > > > > > > > As
> > > > > > > > > > Heron is turning to Python 3, which specifically Python
> > 3.7+,
> > > > > some
> > > > > > > > > > discontinued based OS version does not provide the
> support
> > > > > anymore.
> > > > > > > We
> > > > > > > > > have
> > > > > > > > > > tested out that, debian9 supports only up to Python3.5,
> > > ubuntu
> > > > > > 14.04
> > > > > > > > does
> > > > > > > > > > not support python3 and ubuntu 16.04 supports up to
> Python
> > > 3.5
> > > > > > only.
> > > > > > > > > >
> > > > > > > > > > I am not a big fan of using centos for docker. Concluding
> > > all 4
> > > > > > > points
> > > > > > > > > > above, I believe keeping centos7 (if it supports
> python37),
> > > > > > debian10
> > > > > > > > (not
> > > > > > > > > > debian9), ubuntu 18.04 (supports python3.7+) would be the
> > > best
> > > > > > > solution
> > > > > > > > > at
> > > > > > > > > > the moment. We have to cut off the old images due to our
> > > > library
> > > > > > > > upgrade
> > > > > > > > > > and this is unavoidable. Otherwise, we would have tons of
> > > > issues
> > > > > > > > sending
> > > > > > > > > in
> > > > > > > > > > about library support in the future and the workload
> could
> > > be a
> > > > > lot
> > > > > > > > more
> > > > > > > > > > for just answering questions.
> > > > > > > > > >
> > > > > > > > > > Windham Wong
> > > > > > > > > > Co-Founder, Technical Director of
> > > > > > > > > > Stormeye.io, Hong Kong Managed Security Operation Center
> > > > Limited
> > > > > > > > > > Email // windham.wong@stormeye.io (mailto:
> > > > > windham.wong@stormeye.io
> > > > > > )
> > > > > > > > > > Phone // +852_3590_2212_|_+852_9832_0707
> (tel:+85235902212)
> > > > > > > > > > Fax // +852_3590_2202 (tel:+852_3590_2202)
> > > > > > > > > >
> > > > > > > > > > On 5月 19 2020, at 6:09 早上, Josh Fischer <
> > josh@joshfischer.io
> > > >
> > > > > > wrote:
> > > > > > > > > > > Hi All, I think using CentOS and either Debian or
> Ubuntu
> > > is a
> > > > > > good
> > > > > > > > > idea.
> > > > > > > > > > I would pick Debian over Ubuntu for the reason that Nick
> > > > pointed
> > > > > > out
> > > > > > > > in a
> > > > > > > > > > previous email of DNS issues in Kubernetes. I"ve copy and
> > > > pasted
> > > > > a
> > > > > > > > > section
> > > > > > > > > > of Nick's previous email. ### Start I've run into issues
> > with
> > > > > > Ubuntu
> > > > > > > > > images
> > > > > > > > > > having DNS issues in Kubernetes.
> > > > > > > > > > https://github.com/kubernetes/kubeadm/issues/787 ### End
> > As
> > > > for
> > > > > > what
> > > > > > > > > ends
> > > > > > > > > > up being the official Docker container I'm fine with
> either
> > > > > choice
> > > > > > > > Debian
> > > > > > > > > > or CentOS. On Mon, May 18, 2020 at 2:10 AM Ning Wang
> > wrote: >
> > > > > Yeah.
> > > > > > > For
> > > > > > > > > > binary release, the license is a huge factor. > > On Sun,
> > May
> > > > 17,
> > > > > > > 2020
> > > > > > > > at
> > > > > > > > > > 2:32 PM Josh Fischer wrote: > > > Hi All, > > > > When it
> > > comes
> > > > > to
> > > > > > > > > choosing
> > > > > > > > > > the official Apache Docker image for Heron we > > have to
> > > think
> > > > > of
> > > > > > > the
> > > > > > > > > > licensing issues first. Everything that is > installed >
> >
> > > into
> > > > > the
> > > > > > > > > > container at build time with the exception of the "FROM"
> >
> > >
> > > > > > > statement
> > > > > > > > in
> > > > > > > > > > the Dockerfile must be ALv2 compatible. OpenJDK is under
> >
> > > > GPLv2
> > > > > >
> > > > > > >
> > > > > > > > > which
> > > > > > > > > > is not compatible with Apache. However we can get around
> > this
> > > > if
> > > > > >
> > > > > > we
> > > > > > > > > >
> > > > > > > > > > bring the OpenJDK into the container via a FROM
> statement.
> > > I've
> > > > > > > copied
> > > > > > > > a
> > > > > > > > > >
> > > > > > > > > > > reply from Dave in our dev list in a previous email.
> See
> > > > below.
> > > > > > > >
> > > > > > > > > >
> > > > > > > > > > ######## Start ######## > > Regarding OpenJDK and GPL2 -
> > here
> > > > is
> > > > > > what
> > > > > > > > > Roman
> > > > > > > > > > the VP, Legal wrote when > > answering Beam’s questions.
> >
> > >
> > > >
> > > > >
> > > > > > > Roman
> > > > > > > > > > Shaposhnik commented on LEGAL-503: > >
> > > > > > > > > > ---------------------------------------- > > > > Hey
> > [~altay]
> > > > if
> > > > > > you
> > > > > > > > > would
> > > > > > > > > > like to continue linking to the Docker release > >
> artifact
> > > > from
> > > > > > the
> > > > > > > >
> > > > > > > > >
> > > > > > > > > > https://beam.apache.org > > you will have: > > 1.
> > Transition
> > > > to
> > > > > > the
> > > > > > > > > > official ASF dockerhub org: > >
> > > > https://hub.docker.com/u/apache
> > > > > >
> > > > > > >
> > > > > > > 2.
> > > > > > > > > > Start including that binary convenience artifact into
> your
> > > VOTE
> > > > > > >
> > > > > > > > > threads
> > > > > > > > > > on Beam releases > > 3. Make sure that all Cat-X licenses
> > are
> > > > > ONLY
> > > > > > > > > brought
> > > > > > > > > > into your > > container via FROM statements > > ########
> > End
> > > > > > > ######## >
> > > > > > > > > > >
> > > > > > > > > > > So at the end of the day, we need to use a container
> that
> > > has
> > > > > > Java
> > > > > > > >
> > > > > > > > > > already > > installed in it so that when we create our
> > > official
> > > > > > image
> > > > > > > > we
> > > > > > > > > > can inherit > > that java containing image via a FROM
> > > > statement.
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > -
> > > > > > > > > > Josh > > > > > > On Sun, May 17, 2020 at 4:12 PM H W
> > wrote: >
> > > > > >
> > > > > > > >
> > > > > > > > We
> > > > > > > > > > use Centos 7 as well. > > > > > > On Sun, May 17, 2020 at
> > > 10:41
> > > > > AM
> > > > > > > > > Nicholas
> > > > > > > > > > Nezis < > > nicholas.nezis@gmail.com> > > > wrote: > > >
> > > >
> > > > > >
> > > > > > > Some
> > > > > > > > > > thoughts I'd like to add to the discussion: > > > > > >
> > >
> > > > > Debian
> > > > > > > and
> > > > > > > > > > Ubuntu are somewhat similar. Would it make sense to pick
> >
> > > one
> > > > >
> > > > > >
> > > > > > >
> > > > > > > of
> > > > > > > > > > >
> > > > > > > > > > > > those and pair with Centos? > > > > > > > > I've run
> > into
> > > > > > issues
> > > > > > > > with
> > > > > > > > > > Ubuntu images having DNS issues in > > Kubernetes. > > >
> >
> > > > > > > > > > https://github.com/kubernetes/kubeadm/issues/787 > > > >
> > > >
> > > > > >
> > > > > > > > Ubuntu
> > > > > > > > > > 14.04 should be removed. It was end of life last year.
> For
> > >
> > > > > those
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > of
> > > > > > > > > > > > > > you using Heron in production, is anyone using
> > Ubuntu
> > > > 16?
> > > > > > My
> > > > > > > > > vote >
> > > > > > > > > > would > > > be > > > > to remove it also. For what it's
> > > worth,
> > > > my
> > > > > > use
> > > > > > > > > case
> > > > > > > > > > is in K8s and we > > use > > > > the Centos7 image on a
> > > Centos
> > > > 7
> > > > > > OS.
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > > > > We should do a test of the official image and
> > helm
> > > > > chart
> > > > > > > in >
> > > > > > > > > > Kubernetes > > > > (maybe with Kind
> > > https://kind.sigs.k8s.io/
> > > > )
> > > > > as
> > > > > > > > part
> > > > > > > > > > of our release > > > > process. This might be nice to add
> > as
> > > an
> > > > > > > > > integration
> > > > > > > > > > test. > > > > > > > > > > > > > > > > > > > > > > > > > >
> > > >
> > > > > >
> > > > > > > >
> > > > > > > > On
> > > > > > > > > > Sun, May 17, 2020, 12:37 PM Ning Wang > wrote: > > > > >
> >
> > >
> > > >
> > > > >
> > > > > > That
> > > > > > > > is
> > > > > > > > > a
> > > > > > > > > > good point. We need to adjust the OS version accordingly
> >
> > >
> > > > > too. >
> > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > > > > > On Sun, May 17, 2020 at 2:28 AM Windham Wong <
> >
> > >
> > > > > > > > > > windham.wong@stormeye.io > > > > > > > > > wrote: > > >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > > > > debian 10 has no python 2.7 support. Ubuntu
> 16.04
> > > has
> > > > > no
> > > > > > > > > python >
> > > > > > > > > > > 3.7+ > > > > dev > > > > > > library support. hope this
> > > helps
> > > > > > >
> > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > -- > > > > > > Sent from myMail for Android Sunday, 17
> May
> > > > 2020,
> > > > > > > > 11:34AM
> > > > > > > > > > +08:00 > > from > > > > > Ning > > > > > > Wang
> > > > > > wangninggm@gmail.com
> > > > > > > > : >
> > > > > > > > > > > > > > > > > > > > > >Hi, > > > > > > > > > > > > > >We
> > have
> > > > > > talked
> > > > > > > > > about
> > > > > > > > > > it a few times in different places. Let's > > make > > >
> a
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > >decision here. We don't have enough resources to support
> > > many
> > > > >
> > > > > >
> > > > > > > > > docker >
> > > > > > > > > > > > > > OSes > > > > > > >(technically we only support
> > > Debian,
> > > > > > > Ubuntu,
> > > > > > > > > and
> > > > > > > > > > CentOS but each > > of > > > > them > > > > > > >has
> > multiple
> > > > > > > > versions).
> > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > >Things we have agreed on (my
> > > > > > understanding)
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > > >- for binary release, we are going to choose one
> OS
> > > one
> > > > >
> > > > > > > > > version. I
> > > > > > > > > > > > > > > > >remember that Debian 9 was the decision. An
> > open
> > > > > > > question
> > > > > > > > > is:
> > > > > > > > > > do > > we > > > > want > > > > > > to > > > > > > >switch
> > to
> > > > > Debian
> > > > > > > 10
> > > > > > > > > now?
> > > > > > > > > > We haven't had a binary release yet, so > > > there > > >
> > > >
> > > > is
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > >still time. As the main target, I think 1. the image
> > > should
> > > > >
> > > > > >
> > > > > > > > > compile
> > > > > > > > > > > > > > > > >successfully. and 2 the image should be
> > tested:
> > > > the
> > > > > > > tools
> > > > > > > > > and
> > > > > > > > > > an > > > > example > > > > > > >topology. > > > > > > >
> > >
> > > > >
> > > > > > >
> > > > > > > >-
> > > > > > > > > for
> > > > > > > > > > source release, we haven't talked about it very much. > >
> > > >
> > > > > > > > Personally
> > > > > > > > > I
> > > > > > > > > > > > > > > > >feel it is a reasonable expectation that, if
> > we
> > > > have
> > > > > > the
> > > > > > > > > > Docker > > file > > > > in > > > > > > the > > > > > >
> > >core
> > > > > > folders,
> > > > > > > > it
> > > > > > > > > > should at least compile. Otherwise, it's a > failed > >
> > >
> > > > > build >
> > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >hence a failed release. We are not going to test run
> the
> > > > > tools >
> > > > > > > and
> > > > > > > > > > >
> > > > > > > > > > > > > > examples > > > > > > >for each release though. >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > > >Finally, a question about OS versions. I believe
> at
> > > > least
> > > > > > two
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > releases > > > > > > >should be supported because most
> > people
> > > > > don't
> > > > > > > > > upgrade
> > > > > > > > > > to the > > latest > > > > > > version > > > > > > >when
> it
> > is
> > > > out
> > > > > > in
> > > > > > > > more
> > > > > > > > > > serious scenarios, hence IMO the second > > > oldest > >
> >
> > >
> > > >
> > > > >
> > > > > > > could
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > > > >often be more useful than the latest one from a
> > > > > > convenience
> > > > > > > >
> > > > > > > > > > point > > of > > > > > view. > > > > > > >On the other
> > hand,
> > > > this
> > > > > > is
> > > > > > > > too
> > > > > > > > > > flexible, and we will spend time > > > > thinking > > > >
> > > >
> > > > > > >about
> > > > > > > it
> > > > > > > > > > again. I think a clear guideline could be helpful to >
> us.
> > >
> > > >
> > > > >
> > > > > > For
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > > >example: > > > > > > >- We only include only
> Debian
> > > > > (popular
> > > > > > > in
> > > > > > > > > the
> > > > > > > > > > server world) and > > > Ubuntu > > > > > > >(popular in
> > > servers
> > > > > and
> > > > > > > > > > workstations). > > > > > > >- for the main OS (Debian) we
> > > > choose
> > > > > > the
> > > > > > > > > latest
> > > > > > > > > > to compile and > > test > > > > for > > > > > > our > >
> > >
> > > > >
> > > > > > > >binary
> > > > > > > > > > release. > > > > > > >- for a given OS, only two versions
> > > (most
> > > > > > > likely
> > > > > > > > > the
> > > > > > > > > > most and > the > > > > second > > > > > > >recent
> releases)
> > > are
> > > > > > > > included
> > > > > > > > > in
> > > > > > > > > > the core source code and they > > need > > > to > > > > >
> > be
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > > >fixed ASAP if they don't compile. > > > > > > >- OSs and
> > > other
> > > > > > > > versions
> > > > > > > > > > could be included in a special folder > and > > > but > >
> > > >
> > > > > >
> > > > > > > > >there
> > > > > > > > > is
> > > > > > > > > > no guarantee that the code will compile. Fixes from the >
> > > >
> > > > > >
> > > > > > > > > community
> > > > > > > > > > > > > > > > >are welcome. > > > > > > > > > > > > > > > >
> > > >
> > > > > >
> > > > > > > >How
> > > > > > > > do
> > > > > > > > > > you guys think? > > > > > > >--ning > > > > > > > > > >
> > >
> > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSSION] Docker OS support

[Oliver Bristow <ol...@oliverbristow.co.uk>]

Sorry if I missed the rationale for Ubuntu 18.04 instead of 20.04, but
would the latest LTS be a better choice? 18.04 was released before Debian
10, so that may be an argument for 20.04 re. compatibility if Debian 10 is
the standard.

There's already a 20.04 Dockerfile + I It looks like the vagrant setup will
get an upgrade from 14.04 to 20.04 as a part of the python3 upgrade, and
seems to run fine so far.

On Tue, 26 May 2020 at 19:30, Josh Fischer <jo...@joshfischer.io> wrote:

> Thanks for further clarifying. Yes, that is correct.
>
>
>
> On Tue, May 26, 2020 at 1:16 PM Ning Wang <wa...@gmail.com> wrote:
>
> > +1.
> >
> > So my understanding is:
> > binary (in future release): Debian 10.
> > source: Debian 9/10 + ubuntu 18.04 + centos 7. version numbers will be
> > updated in the future after more discussions/votes.
> >
> > others will be moved to a "extra" folder only for reference.
> >
> >
> >
> > On Tue, May 26, 2020 at 6:13 AM Josh Fischer <jo...@joshfischer.io>
> wrote:
> >
> > > Hi All,
> > >
> > > Based on what I've read on this thread a combination of CentOS/Debian
> > would
> > > be the better options for us to support multiple versions for the
> > > community.  We will also support one version of Ubuntu.
> > >
> > > So..... I will start a vote in 72 hours that calls on the vote on this.
> > > I'm going to write down my understanding of what this thread covers to
> > get
> > > some actionable topics to vote on.  If I've written anything down
> > > incorrectly please speak up before we start a vote.
> > >
> > > Debian:
> > >     - Debian10 is the official docker image to support and be published
> > to
> > > Docker Hub, we will also support Debian9.
> > > CentOS:
> > >     - We will support CentOS7.  Note, I do not think it has native
> > > Python3.7 support, but we are also not there yet.
> > > Ubuntu:
> > >     - We will still support one version. At this time the version is
> > 18.04.
> > >
> > > We will upgrade supported versions of each OS as updates/deprecation of
> > > support are needed.
> > >
> > > Again let's wait 72 hours before starting a vote.  Also please check
> that
> > > the statements I wrote down are what you understood!
> > >
> > > See below for notes on lazy consensus:
> > > https://community.apache.org/committers/lazyConsensus.html
> > >
> > > - Josh
> > >
> > >
> > > On Wed, May 20, 2020 at 9:41 PM Ning Wang <wa...@gmail.com>
> wrote:
> > >
> > > > +1
> > > >
> > > > On Wed, May 20, 2020 at 7:26 PM Nicholas Nezis <
> > nicholas.nezis@gmail.com
> > > >
> > > > wrote:
> > > >
> > > > > I like those options.
> > > > >
> > > > > On Wed, May 20, 2020 at 10:15 PM Josh Fischer <josh@joshfischer.io
> >
> > > > wrote:
> > > > >
> > > > > > Let's wait 3 more days for anyone to give input.  Then we can
> > start a
> > > > > vote
> > > > > > to make a final decision on which containers for us to support.
> > Keep
> > > > in
> > > > > > mind that even though we do need to plan for Python3 support
> (There
> > > is
> > > > > work
> > > > > > in progress now) we do not need it as of yet.
> > > > > >
> > > > > > - Josh
> > > > > >
> > > > > > On Wed, May 20, 2020 at 8:34 PM thinker0 <th...@gmail.com>
> > wrote:
> > > > > >
> > > > > > > My experience is mainly for CentOS 7, so I think it would be
> nice
> > > if
> > > > it
> > > > > > was
> > > > > > > included.
> > > > > > >
> > > > > > > 2020년 5월 21일 (목) 오전 9:56, Josh Fischer <jo...@joshfischer.io>님이
> > 작성:
> > > > > > >
> > > > > > > > Well thought out Windham.  I like how you narrowed the
> criteria
> > > > down
> > > > > to
> > > > > > > > Debian10/CentOS7(depending on PY3 support) thinking about
> > overall
> > > > > > support
> > > > > > > > and what others would need/want.
> > > > > > > >
> > > > > > > > +1
> > > > > > > >
> > > > > > > > - Josh
> > > > > > > >
> > > > > > > > On Mon, May 18, 2020 at 8:39 PM Windham Wong <
> > > > > windham.wong@stormeye.io
> > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > My experience is that, people tends to pick their container
> > > image
> > > > > OS
> > > > > > > > based
> > > > > > > > > on few criterias only:
> > > > > > > > > 1) image size, alpine always win, then debian/centos and
> 3rd
> > is
> > > > > > > > > ubuntu/centos. alpine is always the smallest but not much
> > > library
> > > > > > > > provided
> > > > > > > > > and require to compile manually. debian is always good
> > because
> > > it
> > > > > has
> > > > > > > > > apt-get to provide loads of library. centos is cool as
> debian
> > > > > because
> > > > > > > it
> > > > > > > > > also has good yum repo. ubuntu is always the last because
> it
> > is
> > > > > fat,
> > > > > > > but
> > > > > > > > it
> > > > > > > > > provides very large range of libraries to use.
> > > > > > > > > 2) kernel feature, as mentioned last email, ubuntu has
> issue
> > > with
> > > > > DNS
> > > > > > > and
> > > > > > > > > some sort of network discover issues. I remember alpine has
> > > some
> > > > as
> > > > > > > well,
> > > > > > > > > because it is very plain, requires many fine tunes be done
> > > > > manually.
> > > > > > in
> > > > > > > > > this case, centos and debian should win.
> > > > > > > > > 3) linux stream, obviously, alpine, debian/ubuntu, centos,
> > are
> > > 3
> > > > > > > > different
> > > > > > > > > streams that use different native tools. Poeple just pick
> > their
> > > > > base
> > > > > > > > image
> > > > > > > > > based on what they often use in their life. However, I see
> a
> > > lot
> > > > > > debian
> > > > > > > > and
> > > > > > > > > alpine on docker hub because they are just smaller in size.
> > > > > > > > > 4. library support. Python library is a very good example
> in
> > > this
> > > > > > case.
> > > > > > > > As
> > > > > > > > > Heron is turning to Python 3, which specifically Python
> 3.7+,
> > > > some
> > > > > > > > > discontinued based OS version does not provide the support
> > > > anymore.
> > > > > > We
> > > > > > > > have
> > > > > > > > > tested out that, debian9 supports only up to Python3.5,
> > ubuntu
> > > > > 14.04
> > > > > > > does
> > > > > > > > > not support python3 and ubuntu 16.04 supports up to Python
> > 3.5
> > > > > only.
> > > > > > > > >
> > > > > > > > > I am not a big fan of using centos for docker. Concluding
> > all 4
> > > > > > points
> > > > > > > > > above, I believe keeping centos7 (if it supports python37),
> > > > > debian10
> > > > > > > (not
> > > > > > > > > debian9), ubuntu 18.04 (supports python3.7+) would be the
> > best
> > > > > > solution
> > > > > > > > at
> > > > > > > > > the moment. We have to cut off the old images due to our
> > > library
> > > > > > > upgrade
> > > > > > > > > and this is unavoidable. Otherwise, we would have tons of
> > > issues
> > > > > > > sending
> > > > > > > > in
> > > > > > > > > about library support in the future and the workload could
> > be a
> > > > lot
> > > > > > > more
> > > > > > > > > for just answering questions.
> > > > > > > > >
> > > > > > > > > Windham Wong
> > > > > > > > > Co-Founder, Technical Director of
> > > > > > > > > Stormeye.io, Hong Kong Managed Security Operation Center
> > > Limited
> > > > > > > > > Email // windham.wong@stormeye.io (mailto:
> > > > windham.wong@stormeye.io
> > > > > )
> > > > > > > > > Phone // +852_3590_2212_|_+852_9832_0707 (tel:+85235902212)
> > > > > > > > > Fax // +852_3590_2202 (tel:+852_3590_2202)
> > > > > > > > >
> > > > > > > > > On 5月 19 2020, at 6:09 早上, Josh Fischer <
> josh@joshfischer.io
> > >
> > > > > wrote:
> > > > > > > > > > Hi All, I think using CentOS and either Debian or Ubuntu
> > is a
> > > > > good
> > > > > > > > idea.
> > > > > > > > > I would pick Debian over Ubuntu for the reason that Nick
> > > pointed
> > > > > out
> > > > > > > in a
> > > > > > > > > previous email of DNS issues in Kubernetes. I"ve copy and
> > > pasted
> > > > a
> > > > > > > > section
> > > > > > > > > of Nick's previous email. ### Start I've run into issues
> with
> > > > > Ubuntu
> > > > > > > > images
> > > > > > > > > having DNS issues in Kubernetes.
> > > > > > > > > https://github.com/kubernetes/kubeadm/issues/787 ### End
> As
> > > for
> > > > > what
> > > > > > > > ends
> > > > > > > > > up being the official Docker container I'm fine with either
> > > > choice
> > > > > > > Debian
> > > > > > > > > or CentOS. On Mon, May 18, 2020 at 2:10 AM Ning Wang
> wrote: >
> > > > Yeah.
> > > > > > For
> > > > > > > > > binary release, the license is a huge factor. > > On Sun,
> May
> > > 17,
> > > > > > 2020
> > > > > > > at
> > > > > > > > > 2:32 PM Josh Fischer wrote: > > > Hi All, > > > > When it
> > comes
> > > > to
> > > > > > > > choosing
> > > > > > > > > the official Apache Docker image for Heron we > > have to
> > think
> > > > of
> > > > > > the
> > > > > > > > > licensing issues first. Everything that is > installed > >
> > into
> > > > the
> > > > > > > > > container at build time with the exception of the "FROM" >
> >
> > > > > > statement
> > > > > > > in
> > > > > > > > > the Dockerfile must be ALv2 compatible. OpenJDK is under >
> > > GPLv2
> > > > >
> > > > > >
> > > > > > > > which
> > > > > > > > > is not compatible with Apache. However we can get around
> this
> > > if
> > > > >
> > > > > we
> > > > > > > > >
> > > > > > > > > bring the OpenJDK into the container via a FROM statement.
> > I've
> > > > > > copied
> > > > > > > a
> > > > > > > > >
> > > > > > > > > > reply from Dave in our dev list in a previous email. See
> > > below.
> > > > > > >
> > > > > > > > >
> > > > > > > > > ######## Start ######## > > Regarding OpenJDK and GPL2 -
> here
> > > is
> > > > > what
> > > > > > > > Roman
> > > > > > > > > the VP, Legal wrote when > > answering Beam’s questions. >
> >
> > >
> > > >
> > > > > > Roman
> > > > > > > > > Shaposhnik commented on LEGAL-503: > >
> > > > > > > > > ---------------------------------------- > > > > Hey
> [~altay]
> > > if
> > > > > you
> > > > > > > > would
> > > > > > > > > like to continue linking to the Docker release > > artifact
> > > from
> > > > > the
> > > > > > >
> > > > > > > >
> > > > > > > > > https://beam.apache.org > > you will have: > > 1.
> Transition
> > > to
> > > > > the
> > > > > > > > > official ASF dockerhub org: > >
> > > https://hub.docker.com/u/apache
> > > > >
> > > > > >
> > > > > > 2.
> > > > > > > > > Start including that binary convenience artifact into your
> > VOTE
> > > > > >
> > > > > > > > threads
> > > > > > > > > on Beam releases > > 3. Make sure that all Cat-X licenses
> are
> > > > ONLY
> > > > > > > > brought
> > > > > > > > > into your > > container via FROM statements > > ########
> End
> > > > > > ######## >
> > > > > > > > > >
> > > > > > > > > > So at the end of the day, we need to use a container that
> > has
> > > > > Java
> > > > > > >
> > > > > > > > > already > > installed in it so that when we create our
> > official
> > > > > image
> > > > > > > we
> > > > > > > > > can inherit > > that java containing image via a FROM
> > > statement.
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > -
> > > > > > > > > Josh > > > > > > On Sun, May 17, 2020 at 4:12 PM H W
> wrote: >
> > > > >
> > > > > > >
> > > > > > > We
> > > > > > > > > use Centos 7 as well. > > > > > > On Sun, May 17, 2020 at
> > 10:41
> > > > AM
> > > > > > > > Nicholas
> > > > > > > > > Nezis < > > nicholas.nezis@gmail.com> > > > wrote: > > >
> > >
> > > > >
> > > > > > Some
> > > > > > > > > thoughts I'd like to add to the discussion: > > > > > > > >
> > > > Debian
> > > > > > and
> > > > > > > > > Ubuntu are somewhat similar. Would it make sense to pick >
> > one
> > > >
> > > > >
> > > > > >
> > > > > > of
> > > > > > > > > >
> > > > > > > > > > > those and pair with Centos? > > > > > > > > I've run
> into
> > > > > issues
> > > > > > > with
> > > > > > > > > Ubuntu images having DNS issues in > > Kubernetes. > > > >
> > > > > > > > > https://github.com/kubernetes/kubeadm/issues/787 > > > >
> > >
> > > > >
> > > > > > > Ubuntu
> > > > > > > > > 14.04 should be removed. It was end of life last year. For
> >
> > > > those
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > of
> > > > > > > > > > > > > you using Heron in production, is anyone using
> Ubuntu
> > > 16?
> > > > > My
> > > > > > > > vote >
> > > > > > > > > would > > > be > > > > to remove it also. For what it's
> > worth,
> > > my
> > > > > use
> > > > > > > > case
> > > > > > > > > is in K8s and we > > use > > > > the Centos7 image on a
> > Centos
> > > 7
> > > > > OS.
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > > > > > > We should do a test of the official image and
> helm
> > > > chart
> > > > > > in >
> > > > > > > > > Kubernetes > > > > (maybe with Kind
> > https://kind.sigs.k8s.io/
> > > )
> > > > as
> > > > > > > part
> > > > > > > > > of our release > > > > process. This might be nice to add
> as
> > an
> > > > > > > > integration
> > > > > > > > > test. > > > > > > > > > > > > > > > > > > > > > > > > > >
> > >
> > > > >
> > > > > > >
> > > > > > > On
> > > > > > > > > Sun, May 17, 2020, 12:37 PM Ning Wang > wrote: > > > > > >
> >
> > >
> > > >
> > > > > That
> > > > > > > is
> > > > > > > > a
> > > > > > > > > good point. We need to adjust the OS version accordingly >
> >
> > > > too. >
> > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > > > On Sun, May 17, 2020 at 2:28 AM Windham Wong < >
> >
> > > > > > > > > windham.wong@stormeye.io > > > > > > > > > wrote: > > > >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > > > > > > debian 10 has no python 2.7 support. Ubuntu 16.04
> > has
> > > > no
> > > > > > > > python >
> > > > > > > > > > 3.7+ > > > > dev > > > > > > library support. hope this
> > helps
> > > > > >
> > > > > > > >
> > > > > > > > > >
> > > > > > > > > -- > > > > > > Sent from myMail for Android Sunday, 17 May
> > > 2020,
> > > > > > > 11:34AM
> > > > > > > > > +08:00 > > from > > > > > Ning > > > > > > Wang
> > > > > wangninggm@gmail.com
> > > > > > > : >
> > > > > > > > > > > > > > > > > > > > >Hi, > > > > > > > > > > > > > >We
> have
> > > > > talked
> > > > > > > > about
> > > > > > > > > it a few times in different places. Let's > > make > > > a
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > >decision here. We don't have enough resources to support
> > many
> > > >
> > > > >
> > > > > > > > docker >
> > > > > > > > > > > > > OSes > > > > > > >(technically we only support
> > Debian,
> > > > > > Ubuntu,
> > > > > > > > and
> > > > > > > > > CentOS but each > > of > > > > them > > > > > > >has
> multiple
> > > > > > > versions).
> > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > >Things we have agreed on (my
> > > > > understanding)
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > > > > >- for binary release, we are going to choose one OS
> > one
> > > >
> > > > > > > > version. I
> > > > > > > > > > > > > > > >remember that Debian 9 was the decision. An
> open
> > > > > > question
> > > > > > > > is:
> > > > > > > > > do > > we > > > > want > > > > > > to > > > > > > >switch
> to
> > > > Debian
> > > > > > 10
> > > > > > > > now?
> > > > > > > > > We haven't had a binary release yet, so > > > there > > >
> > >
> > > is
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > > >still time. As the main target, I think 1. the image
> > should
> > > >
> > > > >
> > > > > > > > compile
> > > > > > > > > > > > > > > >successfully. and 2 the image should be
> tested:
> > > the
> > > > > > tools
> > > > > > > > and
> > > > > > > > > an > > > > example > > > > > > >topology. > > > > > > > > >
> > > >
> > > > > >
> > > > > > >-
> > > > > > > > for
> > > > > > > > > source release, we haven't talked about it very much. > >
> > >
> > > > > > > Personally
> > > > > > > > I
> > > > > > > > > > > > > > > >feel it is a reasonable expectation that, if
> we
> > > have
> > > > > the
> > > > > > > > > Docker > > file > > > > in > > > > > > the > > > > > >
> >core
> > > > > folders,
> > > > > > > it
> > > > > > > > > should at least compile. Otherwise, it's a > failed > > > >
> > > > build >
> > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >hence a failed release. We are not going to test run the
> > > > tools >
> > > > > > and
> > > > > > > > > >
> > > > > > > > > > > > > examples > > > > > > >for each release though. > >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > > > > >Finally, a question about OS versions. I believe at
> > > least
> > > > > two
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > releases > > > > > > >should be supported because most
> people
> > > > don't
> > > > > > > > upgrade
> > > > > > > > > to the > > latest > > > > > > version > > > > > > >when it
> is
> > > out
> > > > > in
> > > > > > > more
> > > > > > > > > serious scenarios, hence IMO the second > > > oldest > > >
> >
> > >
> > > >
> > > > > > could
> > > > > > > >
> > > > > > > > >
> > > > > > > > > > > > > >often be more useful than the latest one from a
> > > > > convenience
> > > > > > >
> > > > > > > > > point > > of > > > > > view. > > > > > > >On the other
> hand,
> > > this
> > > > > is
> > > > > > > too
> > > > > > > > > flexible, and we will spend time > > > > thinking > > > >
> > >
> > > > > >about
> > > > > > it
> > > > > > > > > again. I think a clear guideline could be helpful to > us.
> >
> > >
> > > >
> > > > > For
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > > > > >example: > > > > > > >- We only include only Debian
> > > > (popular
> > > > > > in
> > > > > > > > the
> > > > > > > > > server world) and > > > Ubuntu > > > > > > >(popular in
> > servers
> > > > and
> > > > > > > > > workstations). > > > > > > >- for the main OS (Debian) we
> > > choose
> > > > > the
> > > > > > > > latest
> > > > > > > > > to compile and > > test > > > > for > > > > > > our > > > >
> > > >
> > > > > > >binary
> > > > > > > > > release. > > > > > > >- for a given OS, only two versions
> > (most
> > > > > > likely
> > > > > > > > the
> > > > > > > > > most and > the > > > > second > > > > > > >recent releases)
> > are
> > > > > > > included
> > > > > > > > in
> > > > > > > > > the core source code and they > > need > > > to > > > > >
> be
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > >fixed ASAP if they don't compile. > > > > > > >- OSs and
> > other
> > > > > > > versions
> > > > > > > > > could be included in a special folder > and > > > but > >
> > >
> > > > >
> > > > > > > >there
> > > > > > > > is
> > > > > > > > > no guarantee that the code will compile. Fixes from the >
> > >
> > > > >
> > > > > > > > community
> > > > > > > > > > > > > > > >are welcome. > > > > > > > > > > > > > > > >
> > >
> > > > >
> > > > > > >How
> > > > > > > do
> > > > > > > > > you guys think? > > > > > > >--ning > > > > > > > > > > > >
> > > >
> > > > > >
> > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSSION] Docker OS support

[Josh Fischer <jo...@joshfischer.io>]

Thanks for further clarifying. Yes, that is correct.



On Tue, May 26, 2020 at 1:16 PM Ning Wang <wa...@gmail.com> wrote:

> +1.
>
> So my understanding is:
> binary (in future release): Debian 10.
> source: Debian 9/10 + ubuntu 18.04 + centos 7. version numbers will be
> updated in the future after more discussions/votes.
>
> others will be moved to a "extra" folder only for reference.
>
>
>
> On Tue, May 26, 2020 at 6:13 AM Josh Fischer <jo...@joshfischer.io> wrote:
>
> > Hi All,
> >
> > Based on what I've read on this thread a combination of CentOS/Debian
> would
> > be the better options for us to support multiple versions for the
> > community.  We will also support one version of Ubuntu.
> >
> > So..... I will start a vote in 72 hours that calls on the vote on this.
> > I'm going to write down my understanding of what this thread covers to
> get
> > some actionable topics to vote on.  If I've written anything down
> > incorrectly please speak up before we start a vote.
> >
> > Debian:
> >     - Debian10 is the official docker image to support and be published
> to
> > Docker Hub, we will also support Debian9.
> > CentOS:
> >     - We will support CentOS7.  Note, I do not think it has native
> > Python3.7 support, but we are also not there yet.
> > Ubuntu:
> >     - We will still support one version. At this time the version is
> 18.04.
> >
> > We will upgrade supported versions of each OS as updates/deprecation of
> > support are needed.
> >
> > Again let's wait 72 hours before starting a vote.  Also please check that
> > the statements I wrote down are what you understood!
> >
> > See below for notes on lazy consensus:
> > https://community.apache.org/committers/lazyConsensus.html
> >
> > - Josh
> >
> >
> > On Wed, May 20, 2020 at 9:41 PM Ning Wang <wa...@gmail.com> wrote:
> >
> > > +1
> > >
> > > On Wed, May 20, 2020 at 7:26 PM Nicholas Nezis <
> nicholas.nezis@gmail.com
> > >
> > > wrote:
> > >
> > > > I like those options.
> > > >
> > > > On Wed, May 20, 2020 at 10:15 PM Josh Fischer <jo...@joshfischer.io>
> > > wrote:
> > > >
> > > > > Let's wait 3 more days for anyone to give input.  Then we can
> start a
> > > > vote
> > > > > to make a final decision on which containers for us to support.
> Keep
> > > in
> > > > > mind that even though we do need to plan for Python3 support (There
> > is
> > > > work
> > > > > in progress now) we do not need it as of yet.
> > > > >
> > > > > - Josh
> > > > >
> > > > > On Wed, May 20, 2020 at 8:34 PM thinker0 <th...@gmail.com>
> wrote:
> > > > >
> > > > > > My experience is mainly for CentOS 7, so I think it would be nice
> > if
> > > it
> > > > > was
> > > > > > included.
> > > > > >
> > > > > > 2020년 5월 21일 (목) 오전 9:56, Josh Fischer <jo...@joshfischer.io>님이
> 작성:
> > > > > >
> > > > > > > Well thought out Windham.  I like how you narrowed the criteria
> > > down
> > > > to
> > > > > > > Debian10/CentOS7(depending on PY3 support) thinking about
> overall
> > > > > support
> > > > > > > and what others would need/want.
> > > > > > >
> > > > > > > +1
> > > > > > >
> > > > > > > - Josh
> > > > > > >
> > > > > > > On Mon, May 18, 2020 at 8:39 PM Windham Wong <
> > > > windham.wong@stormeye.io
> > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > My experience is that, people tends to pick their container
> > image
> > > > OS
> > > > > > > based
> > > > > > > > on few criterias only:
> > > > > > > > 1) image size, alpine always win, then debian/centos and 3rd
> is
> > > > > > > > ubuntu/centos. alpine is always the smallest but not much
> > library
> > > > > > > provided
> > > > > > > > and require to compile manually. debian is always good
> because
> > it
> > > > has
> > > > > > > > apt-get to provide loads of library. centos is cool as debian
> > > > because
> > > > > > it
> > > > > > > > also has good yum repo. ubuntu is always the last because it
> is
> > > > fat,
> > > > > > but
> > > > > > > it
> > > > > > > > provides very large range of libraries to use.
> > > > > > > > 2) kernel feature, as mentioned last email, ubuntu has issue
> > with
> > > > DNS
> > > > > > and
> > > > > > > > some sort of network discover issues. I remember alpine has
> > some
> > > as
> > > > > > well,
> > > > > > > > because it is very plain, requires many fine tunes be done
> > > > manually.
> > > > > in
> > > > > > > > this case, centos and debian should win.
> > > > > > > > 3) linux stream, obviously, alpine, debian/ubuntu, centos,
> are
> > 3
> > > > > > > different
> > > > > > > > streams that use different native tools. Poeple just pick
> their
> > > > base
> > > > > > > image
> > > > > > > > based on what they often use in their life. However, I see a
> > lot
> > > > > debian
> > > > > > > and
> > > > > > > > alpine on docker hub because they are just smaller in size.
> > > > > > > > 4. library support. Python library is a very good example in
> > this
> > > > > case.
> > > > > > > As
> > > > > > > > Heron is turning to Python 3, which specifically Python 3.7+,
> > > some
> > > > > > > > discontinued based OS version does not provide the support
> > > anymore.
> > > > > We
> > > > > > > have
> > > > > > > > tested out that, debian9 supports only up to Python3.5,
> ubuntu
> > > > 14.04
> > > > > > does
> > > > > > > > not support python3 and ubuntu 16.04 supports up to Python
> 3.5
> > > > only.
> > > > > > > >
> > > > > > > > I am not a big fan of using centos for docker. Concluding
> all 4
> > > > > points
> > > > > > > > above, I believe keeping centos7 (if it supports python37),
> > > > debian10
> > > > > > (not
> > > > > > > > debian9), ubuntu 18.04 (supports python3.7+) would be the
> best
> > > > > solution
> > > > > > > at
> > > > > > > > the moment. We have to cut off the old images due to our
> > library
> > > > > > upgrade
> > > > > > > > and this is unavoidable. Otherwise, we would have tons of
> > issues
> > > > > > sending
> > > > > > > in
> > > > > > > > about library support in the future and the workload could
> be a
> > > lot
> > > > > > more
> > > > > > > > for just answering questions.
> > > > > > > >
> > > > > > > > Windham Wong
> > > > > > > > Co-Founder, Technical Director of
> > > > > > > > Stormeye.io, Hong Kong Managed Security Operation Center
> > Limited
> > > > > > > > Email // windham.wong@stormeye.io (mailto:
> > > windham.wong@stormeye.io
> > > > )
> > > > > > > > Phone // +852_3590_2212_|_+852_9832_0707 (tel:+85235902212)
> > > > > > > > Fax // +852_3590_2202 (tel:+852_3590_2202)
> > > > > > > >
> > > > > > > > On 5月 19 2020, at 6:09 早上, Josh Fischer <josh@joshfischer.io
> >
> > > > wrote:
> > > > > > > > > Hi All, I think using CentOS and either Debian or Ubuntu
> is a
> > > > good
> > > > > > > idea.
> > > > > > > > I would pick Debian over Ubuntu for the reason that Nick
> > pointed
> > > > out
> > > > > > in a
> > > > > > > > previous email of DNS issues in Kubernetes. I"ve copy and
> > pasted
> > > a
> > > > > > > section
> > > > > > > > of Nick's previous email. ### Start I've run into issues with
> > > > Ubuntu
> > > > > > > images
> > > > > > > > having DNS issues in Kubernetes.
> > > > > > > > https://github.com/kubernetes/kubeadm/issues/787 ### End As
> > for
> > > > what
> > > > > > > ends
> > > > > > > > up being the official Docker container I'm fine with either
> > > choice
> > > > > > Debian
> > > > > > > > or CentOS. On Mon, May 18, 2020 at 2:10 AM Ning Wang wrote: >
> > > Yeah.
> > > > > For
> > > > > > > > binary release, the license is a huge factor. > > On Sun, May
> > 17,
> > > > > 2020
> > > > > > at
> > > > > > > > 2:32 PM Josh Fischer wrote: > > > Hi All, > > > > When it
> comes
> > > to
> > > > > > > choosing
> > > > > > > > the official Apache Docker image for Heron we > > have to
> think
> > > of
> > > > > the
> > > > > > > > licensing issues first. Everything that is > installed > >
> into
> > > the
> > > > > > > > container at build time with the exception of the "FROM" > >
> > > > > statement
> > > > > > in
> > > > > > > > the Dockerfile must be ALv2 compatible. OpenJDK is under >
> > GPLv2
> > > >
> > > > >
> > > > > > > which
> > > > > > > > is not compatible with Apache. However we can get around this
> > if
> > > >
> > > > we
> > > > > > > >
> > > > > > > > bring the OpenJDK into the container via a FROM statement.
> I've
> > > > > copied
> > > > > > a
> > > > > > > >
> > > > > > > > > reply from Dave in our dev list in a previous email. See
> > below.
> > > > > >
> > > > > > > >
> > > > > > > > ######## Start ######## > > Regarding OpenJDK and GPL2 - here
> > is
> > > > what
> > > > > > > Roman
> > > > > > > > the VP, Legal wrote when > > answering Beam’s questions. > >
> >
> > >
> > > > > Roman
> > > > > > > > Shaposhnik commented on LEGAL-503: > >
> > > > > > > > ---------------------------------------- > > > > Hey [~altay]
> > if
> > > > you
> > > > > > > would
> > > > > > > > like to continue linking to the Docker release > > artifact
> > from
> > > > the
> > > > > >
> > > > > > >
> > > > > > > > https://beam.apache.org > > you will have: > > 1. Transition
> > to
> > > > the
> > > > > > > > official ASF dockerhub org: > >
> > https://hub.docker.com/u/apache
> > > >
> > > > >
> > > > > 2.
> > > > > > > > Start including that binary convenience artifact into your
> VOTE
> > > > >
> > > > > > > threads
> > > > > > > > on Beam releases > > 3. Make sure that all Cat-X licenses are
> > > ONLY
> > > > > > > brought
> > > > > > > > into your > > container via FROM statements > > ######## End
> > > > > ######## >
> > > > > > > > >
> > > > > > > > > So at the end of the day, we need to use a container that
> has
> > > > Java
> > > > > >
> > > > > > > > already > > installed in it so that when we create our
> official
> > > > image
> > > > > > we
> > > > > > > > can inherit > > that java containing image via a FROM
> > statement.
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > -
> > > > > > > > Josh > > > > > > On Sun, May 17, 2020 at 4:12 PM H W wrote: >
> > > >
> > > > > >
> > > > > > We
> > > > > > > > use Centos 7 as well. > > > > > > On Sun, May 17, 2020 at
> 10:41
> > > AM
> > > > > > > Nicholas
> > > > > > > > Nezis < > > nicholas.nezis@gmail.com> > > > wrote: > > > > >
> > > >
> > > > > Some
> > > > > > > > thoughts I'd like to add to the discussion: > > > > > > > >
> > > Debian
> > > > > and
> > > > > > > > Ubuntu are somewhat similar. Would it make sense to pick >
> one
> > >
> > > >
> > > > >
> > > > > of
> > > > > > > > >
> > > > > > > > > > those and pair with Centos? > > > > > > > > I've run into
> > > > issues
> > > > > > with
> > > > > > > > Ubuntu images having DNS issues in > > Kubernetes. > > > >
> > > > > > > > https://github.com/kubernetes/kubeadm/issues/787 > > > > > >
> > > >
> > > > > > Ubuntu
> > > > > > > > 14.04 should be removed. It was end of life last year. For >
> > > those
> > > > >
> > > > > >
> > > > > > >
> > > > > > > of
> > > > > > > > > > > > you using Heron in production, is anyone using Ubuntu
> > 16?
> > > > My
> > > > > > > vote >
> > > > > > > > would > > > be > > > > to remove it also. For what it's
> worth,
> > my
> > > > use
> > > > > > > case
> > > > > > > > is in K8s and we > > use > > > > the Centos7 image on a
> Centos
> > 7
> > > > OS.
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > > > > We should do a test of the official image and helm
> > > chart
> > > > > in >
> > > > > > > > Kubernetes > > > > (maybe with Kind
> https://kind.sigs.k8s.io/
> > )
> > > as
> > > > > > part
> > > > > > > > of our release > > > > process. This might be nice to add as
> an
> > > > > > > integration
> > > > > > > > test. > > > > > > > > > > > > > > > > > > > > > > > > > > > >
> > > >
> > > > > >
> > > > > > On
> > > > > > > > Sun, May 17, 2020, 12:37 PM Ning Wang > wrote: > > > > > > >
> >
> > >
> > > > That
> > > > > > is
> > > > > > > a
> > > > > > > > good point. We need to adjust the OS version accordingly > >
> > > too. >
> > > > > > >
> > > > > > > > >
> > > > > > > > > > > > > On Sun, May 17, 2020 at 2:28 AM Windham Wong < > >
> > > > > > > > windham.wong@stormeye.io > > > > > > > > > wrote: > > > > >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > > > > debian 10 has no python 2.7 support. Ubuntu 16.04
> has
> > > no
> > > > > > > python >
> > > > > > > > > 3.7+ > > > > dev > > > > > > library support. hope this
> helps
> > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > -- > > > > > > Sent from myMail for Android Sunday, 17 May
> > 2020,
> > > > > > 11:34AM
> > > > > > > > +08:00 > > from > > > > > Ning > > > > > > Wang
> > > > wangninggm@gmail.com
> > > > > > : >
> > > > > > > > > > > > > > > > > > > >Hi, > > > > > > > > > > > > > >We have
> > > > talked
> > > > > > > about
> > > > > > > > it a few times in different places. Let's > > make > > > a >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > >decision here. We don't have enough resources to support
> many
> > >
> > > >
> > > > > > > docker >
> > > > > > > > > > > > OSes > > > > > > >(technically we only support
> Debian,
> > > > > Ubuntu,
> > > > > > > and
> > > > > > > > CentOS but each > > of > > > > them > > > > > > >has multiple
> > > > > > versions).
> > > > > > > >
> > > > > > > > > > > > > > > > > > > > >Things we have agreed on (my
> > > > understanding)
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > > >- for binary release, we are going to choose one OS
> one
> > >
> > > > > > > version. I
> > > > > > > > > > > > > > >remember that Debian 9 was the decision. An open
> > > > > question
> > > > > > > is:
> > > > > > > > do > > we > > > > want > > > > > > to > > > > > > >switch to
> > > Debian
> > > > > 10
> > > > > > > now?
> > > > > > > > We haven't had a binary release yet, so > > > there > > > > >
> > is
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > >still time. As the main target, I think 1. the image
> should
> > >
> > > >
> > > > > > > compile
> > > > > > > > > > > > > > >successfully. and 2 the image should be tested:
> > the
> > > > > tools
> > > > > > > and
> > > > > > > > an > > > > example > > > > > > >topology. > > > > > > > > >
> > >
> > > > >
> > > > > >-
> > > > > > > for
> > > > > > > > source release, we haven't talked about it very much. > > > >
> > > > > > Personally
> > > > > > > I
> > > > > > > > > > > > > > >feel it is a reasonable expectation that, if we
> > have
> > > > the
> > > > > > > > Docker > > file > > > > in > > > > > > the > > > > > > >core
> > > > folders,
> > > > > > it
> > > > > > > > should at least compile. Otherwise, it's a > failed > > > >
> > > build >
> > > > > > >
> > > > > > > > >
> > > > > > > > > >hence a failed release. We are not going to test run the
> > > tools >
> > > > > and
> > > > > > > > >
> > > > > > > > > > > > examples > > > > > > >for each release though. > > >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > > >Finally, a question about OS versions. I believe at
> > least
> > > > two
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > releases > > > > > > >should be supported because most people
> > > don't
> > > > > > > upgrade
> > > > > > > > to the > > latest > > > > > > version > > > > > > >when it is
> > out
> > > > in
> > > > > > more
> > > > > > > > serious scenarios, hence IMO the second > > > oldest > > > >
> >
> > >
> > > > > could
> > > > > > >
> > > > > > > >
> > > > > > > > > > > > >often be more useful than the latest one from a
> > > > convenience
> > > > > >
> > > > > > > > point > > of > > > > > view. > > > > > > >On the other hand,
> > this
> > > > is
> > > > > > too
> > > > > > > > flexible, and we will spend time > > > > thinking > > > > > >
> > > > >about
> > > > > it
> > > > > > > > again. I think a clear guideline could be helpful to > us. >
> >
> > >
> > > > For
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > > > >example: > > > > > > >- We only include only Debian
> > > (popular
> > > > > in
> > > > > > > the
> > > > > > > > server world) and > > > Ubuntu > > > > > > >(popular in
> servers
> > > and
> > > > > > > > workstations). > > > > > > >- for the main OS (Debian) we
> > choose
> > > > the
> > > > > > > latest
> > > > > > > > to compile and > > test > > > > for > > > > > > our > > > >
> > >
> > > > > >binary
> > > > > > > > release. > > > > > > >- for a given OS, only two versions
> (most
> > > > > likely
> > > > > > > the
> > > > > > > > most and > the > > > > second > > > > > > >recent releases)
> are
> > > > > > included
> > > > > > > in
> > > > > > > > the core source code and they > > need > > > to > > > > > be
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >fixed ASAP if they don't compile. > > > > > > >- OSs and
> other
> > > > > > versions
> > > > > > > > could be included in a special folder > and > > > but > > > >
> > > >
> > > > > > >there
> > > > > > > is
> > > > > > > > no guarantee that the code will compile. Fixes from the > > >
> > > >
> > > > > > > community
> > > > > > > > > > > > > > >are welcome. > > > > > > > > > > > > > > > > > >
> > > >
> > > > > >How
> > > > > > do
> > > > > > > > you guys think? > > > > > > >--ning > > > > > > > > > > > >
> > >
> > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSSION] Docker OS support

[Ning Wang <wa...@gmail.com>]

+1.

So my understanding is:
binary (in future release): Debian 10.
source: Debian 9/10 + ubuntu 18.04 + centos 7. version numbers will be
updated in the future after more discussions/votes.

others will be moved to a "extra" folder only for reference.



On Tue, May 26, 2020 at 6:13 AM Josh Fischer <jo...@joshfischer.io> wrote:

> Hi All,
>
> Based on what I've read on this thread a combination of CentOS/Debian would
> be the better options for us to support multiple versions for the
> community.  We will also support one version of Ubuntu.
>
> So..... I will start a vote in 72 hours that calls on the vote on this.
> I'm going to write down my understanding of what this thread covers to get
> some actionable topics to vote on.  If I've written anything down
> incorrectly please speak up before we start a vote.
>
> Debian:
>     - Debian10 is the official docker image to support and be published to
> Docker Hub, we will also support Debian9.
> CentOS:
>     - We will support CentOS7.  Note, I do not think it has native
> Python3.7 support, but we are also not there yet.
> Ubuntu:
>     - We will still support one version. At this time the version is 18.04.
>
> We will upgrade supported versions of each OS as updates/deprecation of
> support are needed.
>
> Again let's wait 72 hours before starting a vote.  Also please check that
> the statements I wrote down are what you understood!
>
> See below for notes on lazy consensus:
> https://community.apache.org/committers/lazyConsensus.html
>
> - Josh
>
>
> On Wed, May 20, 2020 at 9:41 PM Ning Wang <wa...@gmail.com> wrote:
>
> > +1
> >
> > On Wed, May 20, 2020 at 7:26 PM Nicholas Nezis <nicholas.nezis@gmail.com
> >
> > wrote:
> >
> > > I like those options.
> > >
> > > On Wed, May 20, 2020 at 10:15 PM Josh Fischer <jo...@joshfischer.io>
> > wrote:
> > >
> > > > Let's wait 3 more days for anyone to give input.  Then we can start a
> > > vote
> > > > to make a final decision on which containers for us to support.  Keep
> > in
> > > > mind that even though we do need to plan for Python3 support (There
> is
> > > work
> > > > in progress now) we do not need it as of yet.
> > > >
> > > > - Josh
> > > >
> > > > On Wed, May 20, 2020 at 8:34 PM thinker0 <th...@gmail.com> wrote:
> > > >
> > > > > My experience is mainly for CentOS 7, so I think it would be nice
> if
> > it
> > > > was
> > > > > included.
> > > > >
> > > > > 2020년 5월 21일 (목) 오전 9:56, Josh Fischer <jo...@joshfischer.io>님이 작성:
> > > > >
> > > > > > Well thought out Windham.  I like how you narrowed the criteria
> > down
> > > to
> > > > > > Debian10/CentOS7(depending on PY3 support) thinking about overall
> > > > support
> > > > > > and what others would need/want.
> > > > > >
> > > > > > +1
> > > > > >
> > > > > > - Josh
> > > > > >
> > > > > > On Mon, May 18, 2020 at 8:39 PM Windham Wong <
> > > windham.wong@stormeye.io
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > My experience is that, people tends to pick their container
> image
> > > OS
> > > > > > based
> > > > > > > on few criterias only:
> > > > > > > 1) image size, alpine always win, then debian/centos and 3rd is
> > > > > > > ubuntu/centos. alpine is always the smallest but not much
> library
> > > > > > provided
> > > > > > > and require to compile manually. debian is always good because
> it
> > > has
> > > > > > > apt-get to provide loads of library. centos is cool as debian
> > > because
> > > > > it
> > > > > > > also has good yum repo. ubuntu is always the last because it is
> > > fat,
> > > > > but
> > > > > > it
> > > > > > > provides very large range of libraries to use.
> > > > > > > 2) kernel feature, as mentioned last email, ubuntu has issue
> with
> > > DNS
> > > > > and
> > > > > > > some sort of network discover issues. I remember alpine has
> some
> > as
> > > > > well,
> > > > > > > because it is very plain, requires many fine tunes be done
> > > manually.
> > > > in
> > > > > > > this case, centos and debian should win.
> > > > > > > 3) linux stream, obviously, alpine, debian/ubuntu, centos, are
> 3
> > > > > > different
> > > > > > > streams that use different native tools. Poeple just pick their
> > > base
> > > > > > image
> > > > > > > based on what they often use in their life. However, I see a
> lot
> > > > debian
> > > > > > and
> > > > > > > alpine on docker hub because they are just smaller in size.
> > > > > > > 4. library support. Python library is a very good example in
> this
> > > > case.
> > > > > > As
> > > > > > > Heron is turning to Python 3, which specifically Python 3.7+,
> > some
> > > > > > > discontinued based OS version does not provide the support
> > anymore.
> > > > We
> > > > > > have
> > > > > > > tested out that, debian9 supports only up to Python3.5, ubuntu
> > > 14.04
> > > > > does
> > > > > > > not support python3 and ubuntu 16.04 supports up to Python 3.5
> > > only.
> > > > > > >
> > > > > > > I am not a big fan of using centos for docker. Concluding all 4
> > > > points
> > > > > > > above, I believe keeping centos7 (if it supports python37),
> > > debian10
> > > > > (not
> > > > > > > debian9), ubuntu 18.04 (supports python3.7+) would be the best
> > > > solution
> > > > > > at
> > > > > > > the moment. We have to cut off the old images due to our
> library
> > > > > upgrade
> > > > > > > and this is unavoidable. Otherwise, we would have tons of
> issues
> > > > > sending
> > > > > > in
> > > > > > > about library support in the future and the workload could be a
> > lot
> > > > > more
> > > > > > > for just answering questions.
> > > > > > >
> > > > > > > Windham Wong
> > > > > > > Co-Founder, Technical Director of
> > > > > > > Stormeye.io, Hong Kong Managed Security Operation Center
> Limited
> > > > > > > Email // windham.wong@stormeye.io (mailto:
> > windham.wong@stormeye.io
> > > )
> > > > > > > Phone // +852_3590_2212_|_+852_9832_0707 (tel:+85235902212)
> > > > > > > Fax // +852_3590_2202 (tel:+852_3590_2202)
> > > > > > >
> > > > > > > On 5月 19 2020, at 6:09 早上, Josh Fischer <jo...@joshfischer.io>
> > > wrote:
> > > > > > > > Hi All, I think using CentOS and either Debian or Ubuntu is a
> > > good
> > > > > > idea.
> > > > > > > I would pick Debian over Ubuntu for the reason that Nick
> pointed
> > > out
> > > > > in a
> > > > > > > previous email of DNS issues in Kubernetes. I"ve copy and
> pasted
> > a
> > > > > > section
> > > > > > > of Nick's previous email. ### Start I've run into issues with
> > > Ubuntu
> > > > > > images
> > > > > > > having DNS issues in Kubernetes.
> > > > > > > https://github.com/kubernetes/kubeadm/issues/787 ### End As
> for
> > > what
> > > > > > ends
> > > > > > > up being the official Docker container I'm fine with either
> > choice
> > > > > Debian
> > > > > > > or CentOS. On Mon, May 18, 2020 at 2:10 AM Ning Wang wrote: >
> > Yeah.
> > > > For
> > > > > > > binary release, the license is a huge factor. > > On Sun, May
> 17,
> > > > 2020
> > > > > at
> > > > > > > 2:32 PM Josh Fischer wrote: > > > Hi All, > > > > When it comes
> > to
> > > > > > choosing
> > > > > > > the official Apache Docker image for Heron we > > have to think
> > of
> > > > the
> > > > > > > licensing issues first. Everything that is > installed > > into
> > the
> > > > > > > container at build time with the exception of the "FROM" > >
> > > > statement
> > > > > in
> > > > > > > the Dockerfile must be ALv2 compatible. OpenJDK is under >
> GPLv2
> > >
> > > >
> > > > > > which
> > > > > > > is not compatible with Apache. However we can get around this
> if
> > >
> > > we
> > > > > > >
> > > > > > > bring the OpenJDK into the container via a FROM statement. I've
> > > > copied
> > > > > a
> > > > > > >
> > > > > > > > reply from Dave in our dev list in a previous email. See
> below.
> > > > >
> > > > > > >
> > > > > > > ######## Start ######## > > Regarding OpenJDK and GPL2 - here
> is
> > > what
> > > > > > Roman
> > > > > > > the VP, Legal wrote when > > answering Beam’s questions. > > >
> >
> > > > Roman
> > > > > > > Shaposhnik commented on LEGAL-503: > >
> > > > > > > ---------------------------------------- > > > > Hey [~altay]
> if
> > > you
> > > > > > would
> > > > > > > like to continue linking to the Docker release > > artifact
> from
> > > the
> > > > >
> > > > > >
> > > > > > > https://beam.apache.org > > you will have: > > 1. Transition
> to
> > > the
> > > > > > > official ASF dockerhub org: > >
> https://hub.docker.com/u/apache
> > >
> > > >
> > > > 2.
> > > > > > > Start including that binary convenience artifact into your VOTE
> > > >
> > > > > > threads
> > > > > > > on Beam releases > > 3. Make sure that all Cat-X licenses are
> > ONLY
> > > > > > brought
> > > > > > > into your > > container via FROM statements > > ######## End
> > > > ######## >
> > > > > > > >
> > > > > > > > So at the end of the day, we need to use a container that has
> > > Java
> > > > >
> > > > > > > already > > installed in it so that when we create our official
> > > image
> > > > > we
> > > > > > > can inherit > > that java containing image via a FROM
> statement.
> > >
> > > >
> > > > >
> > > > > >
> > > > > > -
> > > > > > > Josh > > > > > > On Sun, May 17, 2020 at 4:12 PM H W wrote: >
> > >
> > > > >
> > > > > We
> > > > > > > use Centos 7 as well. > > > > > > On Sun, May 17, 2020 at 10:41
> > AM
> > > > > > Nicholas
> > > > > > > Nezis < > > nicholas.nezis@gmail.com> > > > wrote: > > > > >
> > >
> > > > Some
> > > > > > > thoughts I'd like to add to the discussion: > > > > > > > >
> > Debian
> > > > and
> > > > > > > Ubuntu are somewhat similar. Would it make sense to pick > one
> >
> > >
> > > >
> > > > of
> > > > > > > >
> > > > > > > > > those and pair with Centos? > > > > > > > > I've run into
> > > issues
> > > > > with
> > > > > > > Ubuntu images having DNS issues in > > Kubernetes. > > > >
> > > > > > > https://github.com/kubernetes/kubeadm/issues/787 > > > > > >
> > >
> > > > > Ubuntu
> > > > > > > 14.04 should be removed. It was end of life last year. For >
> > those
> > > >
> > > > >
> > > > > >
> > > > > > of
> > > > > > > > > > > you using Heron in production, is anyone using Ubuntu
> 16?
> > > My
> > > > > > vote >
> > > > > > > would > > > be > > > > to remove it also. For what it's worth,
> my
> > > use
> > > > > > case
> > > > > > > is in K8s and we > > use > > > > the Centos7 image on a Centos
> 7
> > > OS.
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > > > We should do a test of the official image and helm
> > chart
> > > > in >
> > > > > > > Kubernetes > > > > (maybe with Kind https://kind.sigs.k8s.io/
> )
> > as
> > > > > part
> > > > > > > of our release > > > > process. This might be nice to add as an
> > > > > > integration
> > > > > > > test. > > > > > > > > > > > > > > > > > > > > > > > > > > > >
> > >
> > > > >
> > > > > On
> > > > > > > Sun, May 17, 2020, 12:37 PM Ning Wang > wrote: > > > > > > > >
> >
> > > That
> > > > > is
> > > > > > a
> > > > > > > good point. We need to adjust the OS version accordingly > >
> > too. >
> > > > > >
> > > > > > > >
> > > > > > > > > > > > On Sun, May 17, 2020 at 2:28 AM Windham Wong < > >
> > > > > > > windham.wong@stormeye.io > > > > > > > > > wrote: > > > > > >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > > > debian 10 has no python 2.7 support. Ubuntu 16.04 has
> > no
> > > > > > python >
> > > > > > > > 3.7+ > > > > dev > > > > > > library support. hope this helps
> > > >
> > > > > >
> > > > > > > >
> > > > > > > -- > > > > > > Sent from myMail for Android Sunday, 17 May
> 2020,
> > > > > 11:34AM
> > > > > > > +08:00 > > from > > > > > Ning > > > > > > Wang
> > > wangninggm@gmail.com
> > > > > : >
> > > > > > > > > > > > > > > > > > >Hi, > > > > > > > > > > > > > >We have
> > > talked
> > > > > > about
> > > > > > > it a few times in different places. Let's > > make > > > a > >
> >
> > >
> > > >
> > > > >
> > > > > > > >decision here. We don't have enough resources to support many
> >
> > >
> > > > > > docker >
> > > > > > > > > > > OSes > > > > > > >(technically we only support Debian,
> > > > Ubuntu,
> > > > > > and
> > > > > > > CentOS but each > > of > > > > them > > > > > > >has multiple
> > > > > versions).
> > > > > > >
> > > > > > > > > > > > > > > > > > > >Things we have agreed on (my
> > > understanding)
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > >- for binary release, we are going to choose one OS one
> >
> > > > > > version. I
> > > > > > > > > > > > > >remember that Debian 9 was the decision. An open
> > > > question
> > > > > > is:
> > > > > > > do > > we > > > > want > > > > > > to > > > > > > >switch to
> > Debian
> > > > 10
> > > > > > now?
> > > > > > > We haven't had a binary release yet, so > > > there > > > > >
> is
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > >still time. As the main target, I think 1. the image should
> >
> > >
> > > > > > compile
> > > > > > > > > > > > > >successfully. and 2 the image should be tested:
> the
> > > > tools
> > > > > > and
> > > > > > > an > > > > example > > > > > > >topology. > > > > > > > > > > >
> > > >
> > > > >-
> > > > > > for
> > > > > > > source release, we haven't talked about it very much. > > > >
> > > > > Personally
> > > > > > I
> > > > > > > > > > > > > >feel it is a reasonable expectation that, if we
> have
> > > the
> > > > > > > Docker > > file > > > > in > > > > > > the > > > > > > >core
> > > folders,
> > > > > it
> > > > > > > should at least compile. Otherwise, it's a > failed > > > >
> > build >
> > > > > >
> > > > > > > >
> > > > > > > > >hence a failed release. We are not going to test run the
> > tools >
> > > > and
> > > > > > > >
> > > > > > > > > > > examples > > > > > > >for each release though. > > > >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > >Finally, a question about OS versions. I believe at
> least
> > > two
> > > > >
> > > > > >
> > > > > > >
> > > > > > > releases > > > > > > >should be supported because most people
> > don't
> > > > > > upgrade
> > > > > > > to the > > latest > > > > > > version > > > > > > >when it is
> out
> > > in
> > > > > more
> > > > > > > serious scenarios, hence IMO the second > > > oldest > > > > >
> >
> > > > could
> > > > > >
> > > > > > >
> > > > > > > > > > > >often be more useful than the latest one from a
> > > convenience
> > > > >
> > > > > > > point > > of > > > > > view. > > > > > > >On the other hand,
> this
> > > is
> > > > > too
> > > > > > > flexible, and we will spend time > > > > thinking > > > > > >
> > > >about
> > > > it
> > > > > > > again. I think a clear guideline could be helpful to > us. > >
> >
> > > For
> > > > >
> > > > > >
> > > > > > >
> > > > > > > > > > >example: > > > > > > >- We only include only Debian
> > (popular
> > > > in
> > > > > > the
> > > > > > > server world) and > > > Ubuntu > > > > > > >(popular in servers
> > and
> > > > > > > workstations). > > > > > > >- for the main OS (Debian) we
> choose
> > > the
> > > > > > latest
> > > > > > > to compile and > > test > > > > for > > > > > > our > > > > > >
> > > > >binary
> > > > > > > release. > > > > > > >- for a given OS, only two versions (most
> > > > likely
> > > > > > the
> > > > > > > most and > the > > > > second > > > > > > >recent releases) are
> > > > > included
> > > > > > in
> > > > > > > the core source code and they > > need > > > to > > > > > be >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >fixed ASAP if they don't compile. > > > > > > >- OSs and other
> > > > > versions
> > > > > > > could be included in a special folder > and > > > but > > > >
> > >
> > > > > >there
> > > > > > is
> > > > > > > no guarantee that the code will compile. Fixes from the > > >
> > >
> > > > > > community
> > > > > > > > > > > > > >are welcome. > > > > > > > > > > > > > > > > > >
> > >
> > > > >How
> > > > > do
> > > > > > > you guys think? > > > > > > >--ning > > > > > > > > > > > > > >
> > > >
> > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSSION] Docker OS support

[Josh Fischer <jo...@joshfischer.io>]

Hi All,

Based on what I've read on this thread a combination of CentOS/Debian would
be the better options for us to support multiple versions for the
community.  We will also support one version of Ubuntu.

So..... I will start a vote in 72 hours that calls on the vote on this.
I'm going to write down my understanding of what this thread covers to get
some actionable topics to vote on.  If I've written anything down
incorrectly please speak up before we start a vote.

Debian:
    - Debian10 is the official docker image to support and be published to
Docker Hub, we will also support Debian9.
CentOS:
    - We will support CentOS7.  Note, I do not think it has native
Python3.7 support, but we are also not there yet.
Ubuntu:
    - We will still support one version. At this time the version is 18.04.

We will upgrade supported versions of each OS as updates/deprecation of
support are needed.

Again let's wait 72 hours before starting a vote.  Also please check that
the statements I wrote down are what you understood!

See below for notes on lazy consensus:
https://community.apache.org/committers/lazyConsensus.html

- Josh


On Wed, May 20, 2020 at 9:41 PM Ning Wang <wa...@gmail.com> wrote:

> +1
>
> On Wed, May 20, 2020 at 7:26 PM Nicholas Nezis <ni...@gmail.com>
> wrote:
>
> > I like those options.
> >
> > On Wed, May 20, 2020 at 10:15 PM Josh Fischer <jo...@joshfischer.io>
> wrote:
> >
> > > Let's wait 3 more days for anyone to give input.  Then we can start a
> > vote
> > > to make a final decision on which containers for us to support.  Keep
> in
> > > mind that even though we do need to plan for Python3 support (There is
> > work
> > > in progress now) we do not need it as of yet.
> > >
> > > - Josh
> > >
> > > On Wed, May 20, 2020 at 8:34 PM thinker0 <th...@gmail.com> wrote:
> > >
> > > > My experience is mainly for CentOS 7, so I think it would be nice if
> it
> > > was
> > > > included.
> > > >
> > > > 2020년 5월 21일 (목) 오전 9:56, Josh Fischer <jo...@joshfischer.io>님이 작성:
> > > >
> > > > > Well thought out Windham.  I like how you narrowed the criteria
> down
> > to
> > > > > Debian10/CentOS7(depending on PY3 support) thinking about overall
> > > support
> > > > > and what others would need/want.
> > > > >
> > > > > +1
> > > > >
> > > > > - Josh
> > > > >
> > > > > On Mon, May 18, 2020 at 8:39 PM Windham Wong <
> > windham.wong@stormeye.io
> > > >
> > > > > wrote:
> > > > >
> > > > > > My experience is that, people tends to pick their container image
> > OS
> > > > > based
> > > > > > on few criterias only:
> > > > > > 1) image size, alpine always win, then debian/centos and 3rd is
> > > > > > ubuntu/centos. alpine is always the smallest but not much library
> > > > > provided
> > > > > > and require to compile manually. debian is always good because it
> > has
> > > > > > apt-get to provide loads of library. centos is cool as debian
> > because
> > > > it
> > > > > > also has good yum repo. ubuntu is always the last because it is
> > fat,
> > > > but
> > > > > it
> > > > > > provides very large range of libraries to use.
> > > > > > 2) kernel feature, as mentioned last email, ubuntu has issue with
> > DNS
> > > > and
> > > > > > some sort of network discover issues. I remember alpine has some
> as
> > > > well,
> > > > > > because it is very plain, requires many fine tunes be done
> > manually.
> > > in
> > > > > > this case, centos and debian should win.
> > > > > > 3) linux stream, obviously, alpine, debian/ubuntu, centos, are 3
> > > > > different
> > > > > > streams that use different native tools. Poeple just pick their
> > base
> > > > > image
> > > > > > based on what they often use in their life. However, I see a lot
> > > debian
> > > > > and
> > > > > > alpine on docker hub because they are just smaller in size.
> > > > > > 4. library support. Python library is a very good example in this
> > > case.
> > > > > As
> > > > > > Heron is turning to Python 3, which specifically Python 3.7+,
> some
> > > > > > discontinued based OS version does not provide the support
> anymore.
> > > We
> > > > > have
> > > > > > tested out that, debian9 supports only up to Python3.5, ubuntu
> > 14.04
> > > > does
> > > > > > not support python3 and ubuntu 16.04 supports up to Python 3.5
> > only.
> > > > > >
> > > > > > I am not a big fan of using centos for docker. Concluding all 4
> > > points
> > > > > > above, I believe keeping centos7 (if it supports python37),
> > debian10
> > > > (not
> > > > > > debian9), ubuntu 18.04 (supports python3.7+) would be the best
> > > solution
> > > > > at
> > > > > > the moment. We have to cut off the old images due to our library
> > > > upgrade
> > > > > > and this is unavoidable. Otherwise, we would have tons of issues
> > > > sending
> > > > > in
> > > > > > about library support in the future and the workload could be a
> lot
> > > > more
> > > > > > for just answering questions.
> > > > > >
> > > > > > Windham Wong
> > > > > > Co-Founder, Technical Director of
> > > > > > Stormeye.io, Hong Kong Managed Security Operation Center Limited
> > > > > > Email // windham.wong@stormeye.io (mailto:
> windham.wong@stormeye.io
> > )
> > > > > > Phone // +852_3590_2212_|_+852_9832_0707 (tel:+85235902212)
> > > > > > Fax // +852_3590_2202 (tel:+852_3590_2202)
> > > > > >
> > > > > > On 5月 19 2020, at 6:09 早上, Josh Fischer <jo...@joshfischer.io>
> > wrote:
> > > > > > > Hi All, I think using CentOS and either Debian or Ubuntu is a
> > good
> > > > > idea.
> > > > > > I would pick Debian over Ubuntu for the reason that Nick pointed
> > out
> > > > in a
> > > > > > previous email of DNS issues in Kubernetes. I"ve copy and pasted
> a
> > > > > section
> > > > > > of Nick's previous email. ### Start I've run into issues with
> > Ubuntu
> > > > > images
> > > > > > having DNS issues in Kubernetes.
> > > > > > https://github.com/kubernetes/kubeadm/issues/787 ### End As for
> > what
> > > > > ends
> > > > > > up being the official Docker container I'm fine with either
> choice
> > > > Debian
> > > > > > or CentOS. On Mon, May 18, 2020 at 2:10 AM Ning Wang wrote: >
> Yeah.
> > > For
> > > > > > binary release, the license is a huge factor. > > On Sun, May 17,
> > > 2020
> > > > at
> > > > > > 2:32 PM Josh Fischer wrote: > > > Hi All, > > > > When it comes
> to
> > > > > choosing
> > > > > > the official Apache Docker image for Heron we > > have to think
> of
> > > the
> > > > > > licensing issues first. Everything that is > installed > > into
> the
> > > > > > container at build time with the exception of the "FROM" > >
> > > statement
> > > > in
> > > > > > the Dockerfile must be ALv2 compatible. OpenJDK is under > GPLv2
> >
> > >
> > > > > which
> > > > > > is not compatible with Apache. However we can get around this if
> >
> > we
> > > > > >
> > > > > > bring the OpenJDK into the container via a FROM statement. I've
> > > copied
> > > > a
> > > > > >
> > > > > > > reply from Dave in our dev list in a previous email. See below.
> > > >
> > > > > >
> > > > > > ######## Start ######## > > Regarding OpenJDK and GPL2 - here is
> > what
> > > > > Roman
> > > > > > the VP, Legal wrote when > > answering Beam’s questions. > > > >
> > > Roman
> > > > > > Shaposhnik commented on LEGAL-503: > >
> > > > > > ---------------------------------------- > > > > Hey [~altay] if
> > you
> > > > > would
> > > > > > like to continue linking to the Docker release > > artifact from
> > the
> > > >
> > > > >
> > > > > > https://beam.apache.org > > you will have: > > 1. Transition to
> > the
> > > > > > official ASF dockerhub org: > > https://hub.docker.com/u/apache
> >
> > >
> > > 2.
> > > > > > Start including that binary convenience artifact into your VOTE
> > >
> > > > > threads
> > > > > > on Beam releases > > 3. Make sure that all Cat-X licenses are
> ONLY
> > > > > brought
> > > > > > into your > > container via FROM statements > > ######## End
> > > ######## >
> > > > > > >
> > > > > > > So at the end of the day, we need to use a container that has
> > Java
> > > >
> > > > > > already > > installed in it so that when we create our official
> > image
> > > > we
> > > > > > can inherit > > that java containing image via a FROM statement.
> >
> > >
> > > >
> > > > >
> > > > > -
> > > > > > Josh > > > > > > On Sun, May 17, 2020 at 4:12 PM H W wrote: > > >
> > > >
> > > > We
> > > > > > use Centos 7 as well. > > > > > > On Sun, May 17, 2020 at 10:41
> AM
> > > > > Nicholas
> > > > > > Nezis < > > nicholas.nezis@gmail.com> > > > wrote: > > > > > > >
> > > Some
> > > > > > thoughts I'd like to add to the discussion: > > > > > > > >
> Debian
> > > and
> > > > > > Ubuntu are somewhat similar. Would it make sense to pick > one >
> >
> > >
> > > of
> > > > > > >
> > > > > > > > those and pair with Centos? > > > > > > > > I've run into
> > issues
> > > > with
> > > > > > Ubuntu images having DNS issues in > > Kubernetes. > > > >
> > > > > > https://github.com/kubernetes/kubeadm/issues/787 > > > > > > > >
> > > > Ubuntu
> > > > > > 14.04 should be removed. It was end of life last year. For >
> those
> > >
> > > >
> > > > >
> > > > > of
> > > > > > > > > > you using Heron in production, is anyone using Ubuntu 16?
> > My
> > > > > vote >
> > > > > > would > > > be > > > > to remove it also. For what it's worth, my
> > use
> > > > > case
> > > > > > is in K8s and we > > use > > > > the Centos7 image on a Centos 7
> > OS.
> > > >
> > > > >
> > > > > >
> > > > > > > > > > > We should do a test of the official image and helm
> chart
> > > in >
> > > > > > Kubernetes > > > > (maybe with Kind https://kind.sigs.k8s.io/ )
> as
> > > > part
> > > > > > of our release > > > > process. This might be nice to add as an
> > > > > integration
> > > > > > test. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
> > > >
> > > > On
> > > > > > Sun, May 17, 2020, 12:37 PM Ning Wang > wrote: > > > > > > > > >
> > That
> > > > is
> > > > > a
> > > > > > good point. We need to adjust the OS version accordingly > >
> too. >
> > > > >
> > > > > > >
> > > > > > > > > > > On Sun, May 17, 2020 at 2:28 AM Windham Wong < > >
> > > > > > windham.wong@stormeye.io > > > > > > > > > wrote: > > > > > > >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > > > debian 10 has no python 2.7 support. Ubuntu 16.04 has
> no
> > > > > python >
> > > > > > > 3.7+ > > > > dev > > > > > > library support. hope this helps
> > >
> > > > >
> > > > > > >
> > > > > > -- > > > > > > Sent from myMail for Android Sunday, 17 May 2020,
> > > > 11:34AM
> > > > > > +08:00 > > from > > > > > Ning > > > > > > Wang
> > wangninggm@gmail.com
> > > > : >
> > > > > > > > > > > > > > > > > >Hi, > > > > > > > > > > > > > >We have
> > talked
> > > > > about
> > > > > > it a few times in different places. Let's > > make > > > a > > >
> >
> > >
> > > >
> > > > > > >decision here. We don't have enough resources to support many >
> >
> > > > > docker >
> > > > > > > > > > OSes > > > > > > >(technically we only support Debian,
> > > Ubuntu,
> > > > > and
> > > > > > CentOS but each > > of > > > > them > > > > > > >has multiple
> > > > versions).
> > > > > >
> > > > > > > > > > > > > > > > > > >Things we have agreed on (my
> > understanding)
> > > >
> > > > >
> > > > > >
> > > > > > > > > >- for binary release, we are going to choose one OS one >
> > > > > version. I
> > > > > > > > > > > > >remember that Debian 9 was the decision. An open
> > > question
> > > > > is:
> > > > > > do > > we > > > > want > > > > > > to > > > > > > >switch to
> Debian
> > > 10
> > > > > now?
> > > > > > We haven't had a binary release yet, so > > > there > > > > > is
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > >still time. As the main target, I think 1. the image should >
> >
> > > > > compile
> > > > > > > > > > > > >successfully. and 2 the image should be tested: the
> > > tools
> > > > > and
> > > > > > an > > > > example > > > > > > >topology. > > > > > > > > > > >
> > >
> > > >-
> > > > > for
> > > > > > source release, we haven't talked about it very much. > > > >
> > > > Personally
> > > > > I
> > > > > > > > > > > > >feel it is a reasonable expectation that, if we have
> > the
> > > > > > Docker > > file > > > > in > > > > > > the > > > > > > >core
> > folders,
> > > > it
> > > > > > should at least compile. Otherwise, it's a > failed > > > >
> build >
> > > > >
> > > > > > >
> > > > > > > >hence a failed release. We are not going to test run the
> tools >
> > > and
> > > > > > >
> > > > > > > > > > examples > > > > > > >for each release though. > > > > >
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > > > >Finally, a question about OS versions. I believe at least
> > two
> > > >
> > > > >
> > > > > >
> > > > > > releases > > > > > > >should be supported because most people
> don't
> > > > > upgrade
> > > > > > to the > > latest > > > > > > version > > > > > > >when it is out
> > in
> > > > more
> > > > > > serious scenarios, hence IMO the second > > > oldest > > > > > >
> > > could
> > > > >
> > > > > >
> > > > > > > > > > >often be more useful than the latest one from a
> > convenience
> > > >
> > > > > > point > > of > > > > > view. > > > > > > >On the other hand, this
> > is
> > > > too
> > > > > > flexible, and we will spend time > > > > thinking > > > > > >
> > >about
> > > it
> > > > > > again. I think a clear guideline could be helpful to > us. > > >
> > For
> > > >
> > > > >
> > > > > >
> > > > > > > > > >example: > > > > > > >- We only include only Debian
> (popular
> > > in
> > > > > the
> > > > > > server world) and > > > Ubuntu > > > > > > >(popular in servers
> and
> > > > > > workstations). > > > > > > >- for the main OS (Debian) we choose
> > the
> > > > > latest
> > > > > > to compile and > > test > > > > for > > > > > > our > > > > > >
> > > >binary
> > > > > > release. > > > > > > >- for a given OS, only two versions (most
> > > likely
> > > > > the
> > > > > > most and > the > > > > second > > > > > > >recent releases) are
> > > > included
> > > > > in
> > > > > > the core source code and they > > need > > > to > > > > > be > >
> >
> > >
> > > >
> > > > >
> > > > > > >fixed ASAP if they don't compile. > > > > > > >- OSs and other
> > > > versions
> > > > > > could be included in a special folder > and > > > but > > > > > >
> > > > >there
> > > > > is
> > > > > > no guarantee that the code will compile. Fixes from the > > > > >
> > > > > community
> > > > > > > > > > > > >are welcome. > > > > > > > > > > > > > > > > > > > >
> > > >How
> > > > do
> > > > > > you guys think? > > > > > > >--ning > > > > > > > > > > > > > >
> > >
> > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSSION] Docker OS support

[Ning Wang <wa...@gmail.com>]

+1

On Wed, May 20, 2020 at 7:26 PM Nicholas Nezis <ni...@gmail.com>
wrote:

> I like those options.
>
> On Wed, May 20, 2020 at 10:15 PM Josh Fischer <jo...@joshfischer.io> wrote:
>
> > Let's wait 3 more days for anyone to give input.  Then we can start a
> vote
> > to make a final decision on which containers for us to support.  Keep in
> > mind that even though we do need to plan for Python3 support (There is
> work
> > in progress now) we do not need it as of yet.
> >
> > - Josh
> >
> > On Wed, May 20, 2020 at 8:34 PM thinker0 <th...@gmail.com> wrote:
> >
> > > My experience is mainly for CentOS 7, so I think it would be nice if it
> > was
> > > included.
> > >
> > > 2020년 5월 21일 (목) 오전 9:56, Josh Fischer <jo...@joshfischer.io>님이 작성:
> > >
> > > > Well thought out Windham.  I like how you narrowed the criteria down
> to
> > > > Debian10/CentOS7(depending on PY3 support) thinking about overall
> > support
> > > > and what others would need/want.
> > > >
> > > > +1
> > > >
> > > > - Josh
> > > >
> > > > On Mon, May 18, 2020 at 8:39 PM Windham Wong <
> windham.wong@stormeye.io
> > >
> > > > wrote:
> > > >
> > > > > My experience is that, people tends to pick their container image
> OS
> > > > based
> > > > > on few criterias only:
> > > > > 1) image size, alpine always win, then debian/centos and 3rd is
> > > > > ubuntu/centos. alpine is always the smallest but not much library
> > > > provided
> > > > > and require to compile manually. debian is always good because it
> has
> > > > > apt-get to provide loads of library. centos is cool as debian
> because
> > > it
> > > > > also has good yum repo. ubuntu is always the last because it is
> fat,
> > > but
> > > > it
> > > > > provides very large range of libraries to use.
> > > > > 2) kernel feature, as mentioned last email, ubuntu has issue with
> DNS
> > > and
> > > > > some sort of network discover issues. I remember alpine has some as
> > > well,
> > > > > because it is very plain, requires many fine tunes be done
> manually.
> > in
> > > > > this case, centos and debian should win.
> > > > > 3) linux stream, obviously, alpine, debian/ubuntu, centos, are 3
> > > > different
> > > > > streams that use different native tools. Poeple just pick their
> base
> > > > image
> > > > > based on what they often use in their life. However, I see a lot
> > debian
> > > > and
> > > > > alpine on docker hub because they are just smaller in size.
> > > > > 4. library support. Python library is a very good example in this
> > case.
> > > > As
> > > > > Heron is turning to Python 3, which specifically Python 3.7+, some
> > > > > discontinued based OS version does not provide the support anymore.
> > We
> > > > have
> > > > > tested out that, debian9 supports only up to Python3.5, ubuntu
> 14.04
> > > does
> > > > > not support python3 and ubuntu 16.04 supports up to Python 3.5
> only.
> > > > >
> > > > > I am not a big fan of using centos for docker. Concluding all 4
> > points
> > > > > above, I believe keeping centos7 (if it supports python37),
> debian10
> > > (not
> > > > > debian9), ubuntu 18.04 (supports python3.7+) would be the best
> > solution
> > > > at
> > > > > the moment. We have to cut off the old images due to our library
> > > upgrade
> > > > > and this is unavoidable. Otherwise, we would have tons of issues
> > > sending
> > > > in
> > > > > about library support in the future and the workload could be a lot
> > > more
> > > > > for just answering questions.
> > > > >
> > > > > Windham Wong
> > > > > Co-Founder, Technical Director of
> > > > > Stormeye.io, Hong Kong Managed Security Operation Center Limited
> > > > > Email // windham.wong@stormeye.io (mailto:windham.wong@stormeye.io
> )
> > > > > Phone // +852_3590_2212_|_+852_9832_0707 (tel:+85235902212)
> > > > > Fax // +852_3590_2202 (tel:+852_3590_2202)
> > > > >
> > > > > On 5月 19 2020, at 6:09 早上, Josh Fischer <jo...@joshfischer.io>
> wrote:
> > > > > > Hi All, I think using CentOS and either Debian or Ubuntu is a
> good
> > > > idea.
> > > > > I would pick Debian over Ubuntu for the reason that Nick pointed
> out
> > > in a
> > > > > previous email of DNS issues in Kubernetes. I"ve copy and pasted a
> > > > section
> > > > > of Nick's previous email. ### Start I've run into issues with
> Ubuntu
> > > > images
> > > > > having DNS issues in Kubernetes.
> > > > > https://github.com/kubernetes/kubeadm/issues/787 ### End As for
> what
> > > > ends
> > > > > up being the official Docker container I'm fine with either choice
> > > Debian
> > > > > or CentOS. On Mon, May 18, 2020 at 2:10 AM Ning Wang wrote: > Yeah.
> > For
> > > > > binary release, the license is a huge factor. > > On Sun, May 17,
> > 2020
> > > at
> > > > > 2:32 PM Josh Fischer wrote: > > > Hi All, > > > > When it comes to
> > > > choosing
> > > > > the official Apache Docker image for Heron we > > have to think of
> > the
> > > > > licensing issues first. Everything that is > installed > > into the
> > > > > container at build time with the exception of the "FROM" > >
> > statement
> > > in
> > > > > the Dockerfile must be ALv2 compatible. OpenJDK is under > GPLv2 >
> >
> > > > which
> > > > > is not compatible with Apache. However we can get around this if >
> we
> > > > >
> > > > > bring the OpenJDK into the container via a FROM statement. I've
> > copied
> > > a
> > > > >
> > > > > > reply from Dave in our dev list in a previous email. See below.
> > >
> > > > >
> > > > > ######## Start ######## > > Regarding OpenJDK and GPL2 - here is
> what
> > > > Roman
> > > > > the VP, Legal wrote when > > answering Beam’s questions. > > > >
> > Roman
> > > > > Shaposhnik commented on LEGAL-503: > >
> > > > > ---------------------------------------- > > > > Hey [~altay] if
> you
> > > > would
> > > > > like to continue linking to the Docker release > > artifact from
> the
> > >
> > > >
> > > > > https://beam.apache.org > > you will have: > > 1. Transition to
> the
> > > > > official ASF dockerhub org: > > https://hub.docker.com/u/apache >
> >
> > 2.
> > > > > Start including that binary convenience artifact into your VOTE > >
> > > > threads
> > > > > on Beam releases > > 3. Make sure that all Cat-X licenses are ONLY
> > > > brought
> > > > > into your > > container via FROM statements > > ######## End
> > ######## >
> > > > > >
> > > > > > So at the end of the day, we need to use a container that has
> Java
> > >
> > > > > already > > installed in it so that when we create our official
> image
> > > we
> > > > > can inherit > > that java containing image via a FROM statement. >
> >
> > >
> > > >
> > > > -
> > > > > Josh > > > > > > On Sun, May 17, 2020 at 4:12 PM H W wrote: > > >
> > >
> > > We
> > > > > use Centos 7 as well. > > > > > > On Sun, May 17, 2020 at 10:41 AM
> > > > Nicholas
> > > > > Nezis < > > nicholas.nezis@gmail.com> > > > wrote: > > > > > > >
> > Some
> > > > > thoughts I'd like to add to the discussion: > > > > > > > > Debian
> > and
> > > > > Ubuntu are somewhat similar. Would it make sense to pick > one > >
> >
> > of
> > > > > >
> > > > > > > those and pair with Centos? > > > > > > > > I've run into
> issues
> > > with
> > > > > Ubuntu images having DNS issues in > > Kubernetes. > > > >
> > > > > https://github.com/kubernetes/kubeadm/issues/787 > > > > > > > >
> > > Ubuntu
> > > > > 14.04 should be removed. It was end of life last year. For > those
> >
> > >
> > > >
> > > > of
> > > > > > > > > you using Heron in production, is anyone using Ubuntu 16?
> My
> > > > vote >
> > > > > would > > > be > > > > to remove it also. For what it's worth, my
> use
> > > > case
> > > > > is in K8s and we > > use > > > > the Centos7 image on a Centos 7
> OS.
> > >
> > > >
> > > > >
> > > > > > > > > > We should do a test of the official image and helm chart
> > in >
> > > > > Kubernetes > > > > (maybe with Kind https://kind.sigs.k8s.io/ ) as
> > > part
> > > > > of our release > > > > process. This might be nice to add as an
> > > > integration
> > > > > test. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
> > >
> > > On
> > > > > Sun, May 17, 2020, 12:37 PM Ning Wang > wrote: > > > > > > > > >
> That
> > > is
> > > > a
> > > > > good point. We need to adjust the OS version accordingly > > too. >
> > > >
> > > > > >
> > > > > > > > > > On Sun, May 17, 2020 at 2:28 AM Windham Wong < > >
> > > > > windham.wong@stormeye.io > > > > > > > > > wrote: > > > > > > > >
> >
> > >
> > > >
> > > > >
> > > > > > > > > > debian 10 has no python 2.7 support. Ubuntu 16.04 has no
> > > > python >
> > > > > > 3.7+ > > > > dev > > > > > > library support. hope this helps > >
> > > >
> > > > > >
> > > > > -- > > > > > > Sent from myMail for Android Sunday, 17 May 2020,
> > > 11:34AM
> > > > > +08:00 > > from > > > > > Ning > > > > > > Wang
> wangninggm@gmail.com
> > > : >
> > > > > > > > > > > > > > > > >Hi, > > > > > > > > > > > > > >We have
> talked
> > > > about
> > > > > it a few times in different places. Let's > > make > > > a > > > >
> >
> > >
> > > > > >decision here. We don't have enough resources to support many > >
> > > > docker >
> > > > > > > > > OSes > > > > > > >(technically we only support Debian,
> > Ubuntu,
> > > > and
> > > > > CentOS but each > > of > > > > them > > > > > > >has multiple
> > > versions).
> > > > >
> > > > > > > > > > > > > > > > > >Things we have agreed on (my
> understanding)
> > >
> > > >
> > > > >
> > > > > > > > >- for binary release, we are going to choose one OS one >
> > > > version. I
> > > > > > > > > > > >remember that Debian 9 was the decision. An open
> > question
> > > > is:
> > > > > do > > we > > > > want > > > > > > to > > > > > > >switch to Debian
> > 10
> > > > now?
> > > > > We haven't had a binary release yet, so > > > there > > > > > is >
> >
> > >
> > > >
> > > > >
> > > > > > >still time. As the main target, I think 1. the image should > >
> > > > compile
> > > > > > > > > > > >successfully. and 2 the image should be tested: the
> > tools
> > > > and
> > > > > an > > > > example > > > > > > >topology. > > > > > > > > > > > > >
> > >-
> > > > for
> > > > > source release, we haven't talked about it very much. > > > >
> > > Personally
> > > > I
> > > > > > > > > > > >feel it is a reasonable expectation that, if we have
> the
> > > > > Docker > > file > > > > in > > > > > > the > > > > > > >core
> folders,
> > > it
> > > > > should at least compile. Otherwise, it's a > failed > > > > build >
> > > >
> > > > > >
> > > > > > >hence a failed release. We are not going to test run the tools >
> > and
> > > > > >
> > > > > > > > > examples > > > > > > >for each release though. > > > > > >
> >
> > >
> > > >
> > > > >
> > > > > > > > >Finally, a question about OS versions. I believe at least
> two
> > >
> > > >
> > > > >
> > > > > releases > > > > > > >should be supported because most people don't
> > > > upgrade
> > > > > to the > > latest > > > > > > version > > > > > > >when it is out
> in
> > > more
> > > > > serious scenarios, hence IMO the second > > > oldest > > > > > >
> > could
> > > >
> > > > >
> > > > > > > > > >often be more useful than the latest one from a
> convenience
> > >
> > > > > point > > of > > > > > view. > > > > > > >On the other hand, this
> is
> > > too
> > > > > flexible, and we will spend time > > > > thinking > > > > > >
> >about
> > it
> > > > > again. I think a clear guideline could be helpful to > us. > > >
> For
> > >
> > > >
> > > > >
> > > > > > > > >example: > > > > > > >- We only include only Debian (popular
> > in
> > > > the
> > > > > server world) and > > > Ubuntu > > > > > > >(popular in servers and
> > > > > workstations). > > > > > > >- for the main OS (Debian) we choose
> the
> > > > latest
> > > > > to compile and > > test > > > > for > > > > > > our > > > > > >
> > >binary
> > > > > release. > > > > > > >- for a given OS, only two versions (most
> > likely
> > > > the
> > > > > most and > the > > > > second > > > > > > >recent releases) are
> > > included
> > > > in
> > > > > the core source code and they > > need > > > to > > > > > be > > >
> >
> > >
> > > >
> > > > > >fixed ASAP if they don't compile. > > > > > > >- OSs and other
> > > versions
> > > > > could be included in a special folder > and > > > but > > > > > >
> > > >there
> > > > is
> > > > > no guarantee that the code will compile. Fixes from the > > > > >
> > > > community
> > > > > > > > > > > >are welcome. > > > > > > > > > > > > > > > > > > > >
> > >How
> > > do
> > > > > you guys think? > > > > > > >--ning > > > > > > > > > > > > > > > >
> > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSSION] Docker OS support

[Nicholas Nezis <ni...@gmail.com>]

I like those options.

On Wed, May 20, 2020 at 10:15 PM Josh Fischer <jo...@joshfischer.io> wrote:

> Let's wait 3 more days for anyone to give input.  Then we can start a vote
> to make a final decision on which containers for us to support.  Keep in
> mind that even though we do need to plan for Python3 support (There is work
> in progress now) we do not need it as of yet.
>
> - Josh
>
> On Wed, May 20, 2020 at 8:34 PM thinker0 <th...@gmail.com> wrote:
>
> > My experience is mainly for CentOS 7, so I think it would be nice if it
> was
> > included.
> >
> > 2020년 5월 21일 (목) 오전 9:56, Josh Fischer <jo...@joshfischer.io>님이 작성:
> >
> > > Well thought out Windham.  I like how you narrowed the criteria down to
> > > Debian10/CentOS7(depending on PY3 support) thinking about overall
> support
> > > and what others would need/want.
> > >
> > > +1
> > >
> > > - Josh
> > >
> > > On Mon, May 18, 2020 at 8:39 PM Windham Wong <windham.wong@stormeye.io
> >
> > > wrote:
> > >
> > > > My experience is that, people tends to pick their container image OS
> > > based
> > > > on few criterias only:
> > > > 1) image size, alpine always win, then debian/centos and 3rd is
> > > > ubuntu/centos. alpine is always the smallest but not much library
> > > provided
> > > > and require to compile manually. debian is always good because it has
> > > > apt-get to provide loads of library. centos is cool as debian because
> > it
> > > > also has good yum repo. ubuntu is always the last because it is fat,
> > but
> > > it
> > > > provides very large range of libraries to use.
> > > > 2) kernel feature, as mentioned last email, ubuntu has issue with DNS
> > and
> > > > some sort of network discover issues. I remember alpine has some as
> > well,
> > > > because it is very plain, requires many fine tunes be done manually.
> in
> > > > this case, centos and debian should win.
> > > > 3) linux stream, obviously, alpine, debian/ubuntu, centos, are 3
> > > different
> > > > streams that use different native tools. Poeple just pick their base
> > > image
> > > > based on what they often use in their life. However, I see a lot
> debian
> > > and
> > > > alpine on docker hub because they are just smaller in size.
> > > > 4. library support. Python library is a very good example in this
> case.
> > > As
> > > > Heron is turning to Python 3, which specifically Python 3.7+, some
> > > > discontinued based OS version does not provide the support anymore.
> We
> > > have
> > > > tested out that, debian9 supports only up to Python3.5, ubuntu 14.04
> > does
> > > > not support python3 and ubuntu 16.04 supports up to Python 3.5 only.
> > > >
> > > > I am not a big fan of using centos for docker. Concluding all 4
> points
> > > > above, I believe keeping centos7 (if it supports python37), debian10
> > (not
> > > > debian9), ubuntu 18.04 (supports python3.7+) would be the best
> solution
> > > at
> > > > the moment. We have to cut off the old images due to our library
> > upgrade
> > > > and this is unavoidable. Otherwise, we would have tons of issues
> > sending
> > > in
> > > > about library support in the future and the workload could be a lot
> > more
> > > > for just answering questions.
> > > >
> > > > Windham Wong
> > > > Co-Founder, Technical Director of
> > > > Stormeye.io, Hong Kong Managed Security Operation Center Limited
> > > > Email // windham.wong@stormeye.io (mailto:windham.wong@stormeye.io)
> > > > Phone // +852_3590_2212_|_+852_9832_0707 (tel:+85235902212)
> > > > Fax // +852_3590_2202 (tel:+852_3590_2202)
> > > >
> > > > On 5月 19 2020, at 6:09 早上, Josh Fischer <jo...@joshfischer.io> wrote:
> > > > > Hi All, I think using CentOS and either Debian or Ubuntu is a good
> > > idea.
> > > > I would pick Debian over Ubuntu for the reason that Nick pointed out
> > in a
> > > > previous email of DNS issues in Kubernetes. I"ve copy and pasted a
> > > section
> > > > of Nick's previous email. ### Start I've run into issues with Ubuntu
> > > images
> > > > having DNS issues in Kubernetes.
> > > > https://github.com/kubernetes/kubeadm/issues/787 ### End As for what
> > > ends
> > > > up being the official Docker container I'm fine with either choice
> > Debian
> > > > or CentOS. On Mon, May 18, 2020 at 2:10 AM Ning Wang wrote: > Yeah.
> For
> > > > binary release, the license is a huge factor. > > On Sun, May 17,
> 2020
> > at
> > > > 2:32 PM Josh Fischer wrote: > > > Hi All, > > > > When it comes to
> > > choosing
> > > > the official Apache Docker image for Heron we > > have to think of
> the
> > > > licensing issues first. Everything that is > installed > > into the
> > > > container at build time with the exception of the "FROM" > >
> statement
> > in
> > > > the Dockerfile must be ALv2 compatible. OpenJDK is under > GPLv2 > >
> > > which
> > > > is not compatible with Apache. However we can get around this if > we
> > > >
> > > > bring the OpenJDK into the container via a FROM statement. I've
> copied
> > a
> > > >
> > > > > reply from Dave in our dev list in a previous email. See below. > >
> > > >
> > > > ######## Start ######## > > Regarding OpenJDK and GPL2 - here is what
> > > Roman
> > > > the VP, Legal wrote when > > answering Beam’s questions. > > > >
> Roman
> > > > Shaposhnik commented on LEGAL-503: > >
> > > > ---------------------------------------- > > > > Hey [~altay] if you
> > > would
> > > > like to continue linking to the Docker release > > artifact from the
> >
> > >
> > > > https://beam.apache.org > > you will have: > > 1. Transition to the
> > > > official ASF dockerhub org: > > https://hub.docker.com/u/apache > >
> 2.
> > > > Start including that binary convenience artifact into your VOTE > >
> > > threads
> > > > on Beam releases > > 3. Make sure that all Cat-X licenses are ONLY
> > > brought
> > > > into your > > container via FROM statements > > ######## End
> ######## >
> > > > >
> > > > > So at the end of the day, we need to use a container that has Java
> >
> > > > already > > installed in it so that when we create our official image
> > we
> > > > can inherit > > that java containing image via a FROM statement. > >
> >
> > >
> > > -
> > > > Josh > > > > > > On Sun, May 17, 2020 at 4:12 PM H W wrote: > > > > >
> > We
> > > > use Centos 7 as well. > > > > > > On Sun, May 17, 2020 at 10:41 AM
> > > Nicholas
> > > > Nezis < > > nicholas.nezis@gmail.com> > > > wrote: > > > > > > >
> Some
> > > > thoughts I'd like to add to the discussion: > > > > > > > > Debian
> and
> > > > Ubuntu are somewhat similar. Would it make sense to pick > one > > >
> of
> > > > >
> > > > > > those and pair with Centos? > > > > > > > > I've run into issues
> > with
> > > > Ubuntu images having DNS issues in > > Kubernetes. > > > >
> > > > https://github.com/kubernetes/kubeadm/issues/787 > > > > > > > >
> > Ubuntu
> > > > 14.04 should be removed. It was end of life last year. For > those >
> >
> > >
> > > of
> > > > > > > > you using Heron in production, is anyone using Ubuntu 16? My
> > > vote >
> > > > would > > > be > > > > to remove it also. For what it's worth, my use
> > > case
> > > > is in K8s and we > > use > > > > the Centos7 image on a Centos 7 OS.
> >
> > >
> > > >
> > > > > > > > > We should do a test of the official image and helm chart
> in >
> > > > Kubernetes > > > > (maybe with Kind https://kind.sigs.k8s.io/ ) as
> > part
> > > > of our release > > > > process. This might be nice to add as an
> > > integration
> > > > test. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
> > On
> > > > Sun, May 17, 2020, 12:37 PM Ning Wang > wrote: > > > > > > > > > That
> > is
> > > a
> > > > good point. We need to adjust the OS version accordingly > > too. >
> > >
> > > > >
> > > > > > > > > On Sun, May 17, 2020 at 2:28 AM Windham Wong < > >
> > > > windham.wong@stormeye.io > > > > > > > > > wrote: > > > > > > > > >
> >
> > >
> > > >
> > > > > > > > > debian 10 has no python 2.7 support. Ubuntu 16.04 has no
> > > python >
> > > > > 3.7+ > > > > dev > > > > > > library support. hope this helps > >
> > >
> > > > >
> > > > -- > > > > > > Sent from myMail for Android Sunday, 17 May 2020,
> > 11:34AM
> > > > +08:00 > > from > > > > > Ning > > > > > > Wang wangninggm@gmail.com
> > : >
> > > > > > > > > > > > > > > >Hi, > > > > > > > > > > > > > >We have talked
> > > about
> > > > it a few times in different places. Let's > > make > > > a > > > > >
> >
> > > > >decision here. We don't have enough resources to support many > >
> > > docker >
> > > > > > > > OSes > > > > > > >(technically we only support Debian,
> Ubuntu,
> > > and
> > > > CentOS but each > > of > > > > them > > > > > > >has multiple
> > versions).
> > > >
> > > > > > > > > > > > > > > > >Things we have agreed on (my understanding)
> >
> > >
> > > >
> > > > > > > >- for binary release, we are going to choose one OS one >
> > > version. I
> > > > > > > > > > >remember that Debian 9 was the decision. An open
> question
> > > is:
> > > > do > > we > > > > want > > > > > > to > > > > > > >switch to Debian
> 10
> > > now?
> > > > We haven't had a binary release yet, so > > > there > > > > > is > >
> >
> > >
> > > >
> > > > > >still time. As the main target, I think 1. the image should > >
> > > compile
> > > > > > > > > > >successfully. and 2 the image should be tested: the
> tools
> > > and
> > > > an > > > > example > > > > > > >topology. > > > > > > > > > > > > >
> >-
> > > for
> > > > source release, we haven't talked about it very much. > > > >
> > Personally
> > > I
> > > > > > > > > > >feel it is a reasonable expectation that, if we have the
> > > > Docker > > file > > > > in > > > > > > the > > > > > > >core folders,
> > it
> > > > should at least compile. Otherwise, it's a > failed > > > > build >
> > >
> > > > >
> > > > > >hence a failed release. We are not going to test run the tools >
> and
> > > > >
> > > > > > > > examples > > > > > > >for each release though. > > > > > > >
> >
> > >
> > > >
> > > > > > > >Finally, a question about OS versions. I believe at least two
> >
> > >
> > > >
> > > > releases > > > > > > >should be supported because most people don't
> > > upgrade
> > > > to the > > latest > > > > > > version > > > > > > >when it is out in
> > more
> > > > serious scenarios, hence IMO the second > > > oldest > > > > > >
> could
> > >
> > > >
> > > > > > > > >often be more useful than the latest one from a convenience
> >
> > > > point > > of > > > > > view. > > > > > > >On the other hand, this is
> > too
> > > > flexible, and we will spend time > > > > thinking > > > > > > >about
> it
> > > > again. I think a clear guideline could be helpful to > us. > > > For
> >
> > >
> > > >
> > > > > > > >example: > > > > > > >- We only include only Debian (popular
> in
> > > the
> > > > server world) and > > > Ubuntu > > > > > > >(popular in servers and
> > > > workstations). > > > > > > >- for the main OS (Debian) we choose the
> > > latest
> > > > to compile and > > test > > > > for > > > > > > our > > > > > >
> >binary
> > > > release. > > > > > > >- for a given OS, only two versions (most
> likely
> > > the
> > > > most and > the > > > > second > > > > > > >recent releases) are
> > included
> > > in
> > > > the core source code and they > > need > > > to > > > > > be > > > >
> >
> > >
> > > > >fixed ASAP if they don't compile. > > > > > > >- OSs and other
> > versions
> > > > could be included in a special folder > and > > > but > > > > > >
> > >there
> > > is
> > > > no guarantee that the code will compile. Fixes from the > > > > >
> > > community
> > > > > > > > > > >are welcome. > > > > > > > > > > > > > > > > > > > >
> >How
> > do
> > > > you guys think? > > > > > > >--ning > > > > > > > > > > > > > > > >
> > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> >
>

Re: [DISCUSSION] Docker OS support

[Josh Fischer <jo...@joshfischer.io>]

Let's wait 3 more days for anyone to give input.  Then we can start a vote
to make a final decision on which containers for us to support.  Keep in
mind that even though we do need to plan for Python3 support (There is work
in progress now) we do not need it as of yet.

- Josh

On Wed, May 20, 2020 at 8:34 PM thinker0 <th...@gmail.com> wrote:

> My experience is mainly for CentOS 7, so I think it would be nice if it was
> included.
>
> 2020년 5월 21일 (목) 오전 9:56, Josh Fischer <jo...@joshfischer.io>님이 작성:
>
> > Well thought out Windham.  I like how you narrowed the criteria down to
> > Debian10/CentOS7(depending on PY3 support) thinking about overall support
> > and what others would need/want.
> >
> > +1
> >
> > - Josh
> >
> > On Mon, May 18, 2020 at 8:39 PM Windham Wong <wi...@stormeye.io>
> > wrote:
> >
> > > My experience is that, people tends to pick their container image OS
> > based
> > > on few criterias only:
> > > 1) image size, alpine always win, then debian/centos and 3rd is
> > > ubuntu/centos. alpine is always the smallest but not much library
> > provided
> > > and require to compile manually. debian is always good because it has
> > > apt-get to provide loads of library. centos is cool as debian because
> it
> > > also has good yum repo. ubuntu is always the last because it is fat,
> but
> > it
> > > provides very large range of libraries to use.
> > > 2) kernel feature, as mentioned last email, ubuntu has issue with DNS
> and
> > > some sort of network discover issues. I remember alpine has some as
> well,
> > > because it is very plain, requires many fine tunes be done manually. in
> > > this case, centos and debian should win.
> > > 3) linux stream, obviously, alpine, debian/ubuntu, centos, are 3
> > different
> > > streams that use different native tools. Poeple just pick their base
> > image
> > > based on what they often use in their life. However, I see a lot debian
> > and
> > > alpine on docker hub because they are just smaller in size.
> > > 4. library support. Python library is a very good example in this case.
> > As
> > > Heron is turning to Python 3, which specifically Python 3.7+, some
> > > discontinued based OS version does not provide the support anymore. We
> > have
> > > tested out that, debian9 supports only up to Python3.5, ubuntu 14.04
> does
> > > not support python3 and ubuntu 16.04 supports up to Python 3.5 only.
> > >
> > > I am not a big fan of using centos for docker. Concluding all 4 points
> > > above, I believe keeping centos7 (if it supports python37), debian10
> (not
> > > debian9), ubuntu 18.04 (supports python3.7+) would be the best solution
> > at
> > > the moment. We have to cut off the old images due to our library
> upgrade
> > > and this is unavoidable. Otherwise, we would have tons of issues
> sending
> > in
> > > about library support in the future and the workload could be a lot
> more
> > > for just answering questions.
> > >
> > > Windham Wong
> > > Co-Founder, Technical Director of
> > > Stormeye.io, Hong Kong Managed Security Operation Center Limited
> > > Email // windham.wong@stormeye.io (mailto:windham.wong@stormeye.io)
> > > Phone // +852_3590_2212_|_+852_9832_0707 (tel:+85235902212)
> > > Fax // +852_3590_2202 (tel:+852_3590_2202)
> > >
> > > On 5月 19 2020, at 6:09 早上, Josh Fischer <jo...@joshfischer.io> wrote:
> > > > Hi All, I think using CentOS and either Debian or Ubuntu is a good
> > idea.
> > > I would pick Debian over Ubuntu for the reason that Nick pointed out
> in a
> > > previous email of DNS issues in Kubernetes. I"ve copy and pasted a
> > section
> > > of Nick's previous email. ### Start I've run into issues with Ubuntu
> > images
> > > having DNS issues in Kubernetes.
> > > https://github.com/kubernetes/kubeadm/issues/787 ### End As for what
> > ends
> > > up being the official Docker container I'm fine with either choice
> Debian
> > > or CentOS. On Mon, May 18, 2020 at 2:10 AM Ning Wang wrote: > Yeah. For
> > > binary release, the license is a huge factor. > > On Sun, May 17, 2020
> at
> > > 2:32 PM Josh Fischer wrote: > > > Hi All, > > > > When it comes to
> > choosing
> > > the official Apache Docker image for Heron we > > have to think of the
> > > licensing issues first. Everything that is > installed > > into the
> > > container at build time with the exception of the "FROM" > > statement
> in
> > > the Dockerfile must be ALv2 compatible. OpenJDK is under > GPLv2 > >
> > which
> > > is not compatible with Apache. However we can get around this if > we
> > >
> > > bring the OpenJDK into the container via a FROM statement. I've copied
> a
> > >
> > > > reply from Dave in our dev list in a previous email. See below. > >
> > >
> > > ######## Start ######## > > Regarding OpenJDK and GPL2 - here is what
> > Roman
> > > the VP, Legal wrote when > > answering Beam’s questions. > > > > Roman
> > > Shaposhnik commented on LEGAL-503: > >
> > > ---------------------------------------- > > > > Hey [~altay] if you
> > would
> > > like to continue linking to the Docker release > > artifact from the >
> >
> > > https://beam.apache.org > > you will have: > > 1. Transition to the
> > > official ASF dockerhub org: > > https://hub.docker.com/u/apache > > 2.
> > > Start including that binary convenience artifact into your VOTE > >
> > threads
> > > on Beam releases > > 3. Make sure that all Cat-X licenses are ONLY
> > brought
> > > into your > > container via FROM statements > > ######## End ######## >
> > > >
> > > > So at the end of the day, we need to use a container that has Java >
> > > already > > installed in it so that when we create our official image
> we
> > > can inherit > > that java containing image via a FROM statement. > > >
> >
> > -
> > > Josh > > > > > > On Sun, May 17, 2020 at 4:12 PM H W wrote: > > > > >
> We
> > > use Centos 7 as well. > > > > > > On Sun, May 17, 2020 at 10:41 AM
> > Nicholas
> > > Nezis < > > nicholas.nezis@gmail.com> > > > wrote: > > > > > > > Some
> > > thoughts I'd like to add to the discussion: > > > > > > > > Debian and
> > > Ubuntu are somewhat similar. Would it make sense to pick > one > > > of
> > > >
> > > > > those and pair with Centos? > > > > > > > > I've run into issues
> with
> > > Ubuntu images having DNS issues in > > Kubernetes. > > > >
> > > https://github.com/kubernetes/kubeadm/issues/787 > > > > > > > >
> Ubuntu
> > > 14.04 should be removed. It was end of life last year. For > those > >
> >
> > of
> > > > > > > you using Heron in production, is anyone using Ubuntu 16? My
> > vote >
> > > would > > > be > > > > to remove it also. For what it's worth, my use
> > case
> > > is in K8s and we > > use > > > > the Centos7 image on a Centos 7 OS. >
> >
> > >
> > > > > > > > We should do a test of the official image and helm chart in >
> > > Kubernetes > > > > (maybe with Kind https://kind.sigs.k8s.io/ ) as
> part
> > > of our release > > > > process. This might be nice to add as an
> > integration
> > > test. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
> On
> > > Sun, May 17, 2020, 12:37 PM Ning Wang > wrote: > > > > > > > > > That
> is
> > a
> > > good point. We need to adjust the OS version accordingly > > too. > > >
> > > >
> > > > > > > > On Sun, May 17, 2020 at 2:28 AM Windham Wong < > >
> > > windham.wong@stormeye.io > > > > > > > > > wrote: > > > > > > > > > >
> >
> > >
> > > > > > > > debian 10 has no python 2.7 support. Ubuntu 16.04 has no
> > python >
> > > > 3.7+ > > > > dev > > > > > > library support. hope this helps > > > >
> > > >
> > > -- > > > > > > Sent from myMail for Android Sunday, 17 May 2020,
> 11:34AM
> > > +08:00 > > from > > > > > Ning > > > > > > Wang wangninggm@gmail.com
> : >
> > > > > > > > > > > > > > >Hi, > > > > > > > > > > > > > >We have talked
> > about
> > > it a few times in different places. Let's > > make > > > a > > > > > >
> > > >decision here. We don't have enough resources to support many > >
> > docker >
> > > > > > > OSes > > > > > > >(technically we only support Debian, Ubuntu,
> > and
> > > CentOS but each > > of > > > > them > > > > > > >has multiple
> versions).
> > >
> > > > > > > > > > > > > > > >Things we have agreed on (my understanding) >
> >
> > >
> > > > > > >- for binary release, we are going to choose one OS one >
> > version. I
> > > > > > > > > >remember that Debian 9 was the decision. An open question
> > is:
> > > do > > we > > > > want > > > > > > to > > > > > > >switch to Debian 10
> > now?
> > > We haven't had a binary release yet, so > > > there > > > > > is > > >
> >
> > >
> > > > >still time. As the main target, I think 1. the image should > >
> > compile
> > > > > > > > > >successfully. and 2 the image should be tested: the tools
> > and
> > > an > > > > example > > > > > > >topology. > > > > > > > > > > > > > >-
> > for
> > > source release, we haven't talked about it very much. > > > >
> Personally
> > I
> > > > > > > > > >feel it is a reasonable expectation that, if we have the
> > > Docker > > file > > > > in > > > > > > the > > > > > > >core folders,
> it
> > > should at least compile. Otherwise, it's a > failed > > > > build > > >
> > > >
> > > > >hence a failed release. We are not going to test run the tools > and
> > > >
> > > > > > > examples > > > > > > >for each release though. > > > > > > > >
> >
> > >
> > > > > > >Finally, a question about OS versions. I believe at least two >
> >
> > >
> > > releases > > > > > > >should be supported because most people don't
> > upgrade
> > > to the > > latest > > > > > > version > > > > > > >when it is out in
> more
> > > serious scenarios, hence IMO the second > > > oldest > > > > > > could
> >
> > >
> > > > > > > >often be more useful than the latest one from a convenience >
> > > point > > of > > > > > view. > > > > > > >On the other hand, this is
> too
> > > flexible, and we will spend time > > > > thinking > > > > > > >about it
> > > again. I think a clear guideline could be helpful to > us. > > > For >
> >
> > >
> > > > > > >example: > > > > > > >- We only include only Debian (popular in
> > the
> > > server world) and > > > Ubuntu > > > > > > >(popular in servers and
> > > workstations). > > > > > > >- for the main OS (Debian) we choose the
> > latest
> > > to compile and > > test > > > > for > > > > > > our > > > > > > >binary
> > > release. > > > > > > >- for a given OS, only two versions (most likely
> > the
> > > most and > the > > > > second > > > > > > >recent releases) are
> included
> > in
> > > the core source code and they > > need > > > to > > > > > be > > > > >
> >
> > > >fixed ASAP if they don't compile. > > > > > > >- OSs and other
> versions
> > > could be included in a special folder > and > > > but > > > > > >
> >there
> > is
> > > no guarantee that the code will compile. Fixes from the > > > > >
> > community
> > > > > > > > > >are welcome. > > > > > > > > > > > > > > > > > > > > >How
> do
> > > you guys think? > > > > > > >--ning > > > > > > > > > > > > > > > > > >
> > > >
> > > >
> > >
> > >
> >
>

Re: [DISCUSSION] Docker OS support

[thinker0 <th...@gmail.com>]

My experience is mainly for CentOS 7, so I think it would be nice if it was
included.

2020년 5월 21일 (목) 오전 9:56, Josh Fischer <jo...@joshfischer.io>님이 작성:

> Well thought out Windham.  I like how you narrowed the criteria down to
> Debian10/CentOS7(depending on PY3 support) thinking about overall support
> and what others would need/want.
>
> +1
>
> - Josh
>
> On Mon, May 18, 2020 at 8:39 PM Windham Wong <wi...@stormeye.io>
> wrote:
>
> > My experience is that, people tends to pick their container image OS
> based
> > on few criterias only:
> > 1) image size, alpine always win, then debian/centos and 3rd is
> > ubuntu/centos. alpine is always the smallest but not much library
> provided
> > and require to compile manually. debian is always good because it has
> > apt-get to provide loads of library. centos is cool as debian because it
> > also has good yum repo. ubuntu is always the last because it is fat, but
> it
> > provides very large range of libraries to use.
> > 2) kernel feature, as mentioned last email, ubuntu has issue with DNS and
> > some sort of network discover issues. I remember alpine has some as well,
> > because it is very plain, requires many fine tunes be done manually. in
> > this case, centos and debian should win.
> > 3) linux stream, obviously, alpine, debian/ubuntu, centos, are 3
> different
> > streams that use different native tools. Poeple just pick their base
> image
> > based on what they often use in their life. However, I see a lot debian
> and
> > alpine on docker hub because they are just smaller in size.
> > 4. library support. Python library is a very good example in this case.
> As
> > Heron is turning to Python 3, which specifically Python 3.7+, some
> > discontinued based OS version does not provide the support anymore. We
> have
> > tested out that, debian9 supports only up to Python3.5, ubuntu 14.04 does
> > not support python3 and ubuntu 16.04 supports up to Python 3.5 only.
> >
> > I am not a big fan of using centos for docker. Concluding all 4 points
> > above, I believe keeping centos7 (if it supports python37), debian10 (not
> > debian9), ubuntu 18.04 (supports python3.7+) would be the best solution
> at
> > the moment. We have to cut off the old images due to our library upgrade
> > and this is unavoidable. Otherwise, we would have tons of issues sending
> in
> > about library support in the future and the workload could be a lot more
> > for just answering questions.
> >
> > Windham Wong
> > Co-Founder, Technical Director of
> > Stormeye.io, Hong Kong Managed Security Operation Center Limited
> > Email // windham.wong@stormeye.io (mailto:windham.wong@stormeye.io)
> > Phone // +852_3590_2212_|_+852_9832_0707 (tel:+85235902212)
> > Fax // +852_3590_2202 (tel:+852_3590_2202)
> >
> > On 5月 19 2020, at 6:09 早上, Josh Fischer <jo...@joshfischer.io> wrote:
> > > Hi All, I think using CentOS and either Debian or Ubuntu is a good
> idea.
> > I would pick Debian over Ubuntu for the reason that Nick pointed out in a
> > previous email of DNS issues in Kubernetes. I"ve copy and pasted a
> section
> > of Nick's previous email. ### Start I've run into issues with Ubuntu
> images
> > having DNS issues in Kubernetes.
> > https://github.com/kubernetes/kubeadm/issues/787 ### End As for what
> ends
> > up being the official Docker container I'm fine with either choice Debian
> > or CentOS. On Mon, May 18, 2020 at 2:10 AM Ning Wang wrote: > Yeah. For
> > binary release, the license is a huge factor. > > On Sun, May 17, 2020 at
> > 2:32 PM Josh Fischer wrote: > > > Hi All, > > > > When it comes to
> choosing
> > the official Apache Docker image for Heron we > > have to think of the
> > licensing issues first. Everything that is > installed > > into the
> > container at build time with the exception of the "FROM" > > statement in
> > the Dockerfile must be ALv2 compatible. OpenJDK is under > GPLv2 > >
> which
> > is not compatible with Apache. However we can get around this if > we > >
> > bring the OpenJDK into the container via a FROM statement. I've copied a
> >
> > > reply from Dave in our dev list in a previous email. See below. > > > >
> > ######## Start ######## > > Regarding OpenJDK and GPL2 - here is what
> Roman
> > the VP, Legal wrote when > > answering Beam’s questions. > > > > Roman
> > Shaposhnik commented on LEGAL-503: > >
> > ---------------------------------------- > > > > Hey [~altay] if you
> would
> > like to continue linking to the Docker release > > artifact from the > >
> > https://beam.apache.org > > you will have: > > 1. Transition to the
> > official ASF dockerhub org: > > https://hub.docker.com/u/apache > > 2.
> > Start including that binary convenience artifact into your VOTE > >
> threads
> > on Beam releases > > 3. Make sure that all Cat-X licenses are ONLY
> brought
> > into your > > container via FROM statements > > ######## End ######## >
> > >
> > > So at the end of the day, we need to use a container that has Java >
> > already > > installed in it so that when we create our official image we
> > can inherit > > that java containing image via a FROM statement. > > > >
> -
> > Josh > > > > > > On Sun, May 17, 2020 at 4:12 PM H W wrote: > > > > > We
> > use Centos 7 as well. > > > > > > On Sun, May 17, 2020 at 10:41 AM
> Nicholas
> > Nezis < > > nicholas.nezis@gmail.com> > > > wrote: > > > > > > > Some
> > thoughts I'd like to add to the discussion: > > > > > > > > Debian and
> > Ubuntu are somewhat similar. Would it make sense to pick > one > > > of
> > >
> > > > those and pair with Centos? > > > > > > > > I've run into issues with
> > Ubuntu images having DNS issues in > > Kubernetes. > > > >
> > https://github.com/kubernetes/kubeadm/issues/787 > > > > > > > > Ubuntu
> > 14.04 should be removed. It was end of life last year. For > those > > >
> of
> > > > > > you using Heron in production, is anyone using Ubuntu 16? My
> vote >
> > would > > > be > > > > to remove it also. For what it's worth, my use
> case
> > is in K8s and we > > use > > > > the Centos7 image on a Centos 7 OS. > >
> >
> > > > > > > We should do a test of the official image and helm chart in >
> > Kubernetes > > > > (maybe with Kind https://kind.sigs.k8s.io/ ) as part
> > of our release > > > > process. This might be nice to add as an
> integration
> > test. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On
> > Sun, May 17, 2020, 12:37 PM Ning Wang > wrote: > > > > > > > > > That is
> a
> > good point. We need to adjust the OS version accordingly > > too. > > >
> > >
> > > > > > > On Sun, May 17, 2020 at 2:28 AM Windham Wong < > >
> > windham.wong@stormeye.io > > > > > > > > > wrote: > > > > > > > > > > >
> >
> > > > > > > debian 10 has no python 2.7 support. Ubuntu 16.04 has no
> python >
> > > 3.7+ > > > > dev > > > > > > library support. hope this helps > > > >
> > >
> > -- > > > > > > Sent from myMail for Android Sunday, 17 May 2020, 11:34AM
> > +08:00 > > from > > > > > Ning > > > > > > Wang wangninggm@gmail.com : >
> > > > > > > > > > > > > >Hi, > > > > > > > > > > > > > >We have talked
> about
> > it a few times in different places. Let's > > make > > > a > > > > > >
> > >decision here. We don't have enough resources to support many > >
> docker >
> > > > > > OSes > > > > > > >(technically we only support Debian, Ubuntu,
> and
> > CentOS but each > > of > > > > them > > > > > > >has multiple versions).
> >
> > > > > > > > > > > > > > >Things we have agreed on (my understanding) > >
> >
> > > > > >- for binary release, we are going to choose one OS one >
> version. I
> > > > > > > > >remember that Debian 9 was the decision. An open question
> is:
> > do > > we > > > > want > > > > > > to > > > > > > >switch to Debian 10
> now?
> > We haven't had a binary release yet, so > > > there > > > > > is > > > >
> >
> > > >still time. As the main target, I think 1. the image should > >
> compile
> > > > > > > > >successfully. and 2 the image should be tested: the tools
> and
> > an > > > > example > > > > > > >topology. > > > > > > > > > > > > > >-
> for
> > source release, we haven't talked about it very much. > > > > Personally
> I
> > > > > > > > >feel it is a reasonable expectation that, if we have the
> > Docker > > file > > > > in > > > > > > the > > > > > > >core folders, it
> > should at least compile. Otherwise, it's a > failed > > > > build > > >
> > >
> > > >hence a failed release. We are not going to test run the tools > and
> > >
> > > > > > examples > > > > > > >for each release though. > > > > > > > > >
> >
> > > > > >Finally, a question about OS versions. I believe at least two > >
> >
> > releases > > > > > > >should be supported because most people don't
> upgrade
> > to the > > latest > > > > > > version > > > > > > >when it is out in more
> > serious scenarios, hence IMO the second > > > oldest > > > > > > could >
> >
> > > > > > >often be more useful than the latest one from a convenience >
> > point > > of > > > > > view. > > > > > > >On the other hand, this is too
> > flexible, and we will spend time > > > > thinking > > > > > > >about it
> > again. I think a clear guideline could be helpful to > us. > > > For > >
> >
> > > > > >example: > > > > > > >- We only include only Debian (popular in
> the
> > server world) and > > > Ubuntu > > > > > > >(popular in servers and
> > workstations). > > > > > > >- for the main OS (Debian) we choose the
> latest
> > to compile and > > test > > > > for > > > > > > our > > > > > > >binary
> > release. > > > > > > >- for a given OS, only two versions (most likely
> the
> > most and > the > > > > second > > > > > > >recent releases) are included
> in
> > the core source code and they > > need > > > to > > > > > be > > > > > >
> > >fixed ASAP if they don't compile. > > > > > > >- OSs and other versions
> > could be included in a special folder > and > > > but > > > > > > >there
> is
> > no guarantee that the code will compile. Fixes from the > > > > >
> community
> > > > > > > > >are welcome. > > > > > > > > > > > > > > > > > > > > >How do
> > you guys think? > > > > > > >--ning > > > > > > > > > > > > > > > > > >
> > >
> > >
> >
> >
>

Re: [DISCUSSION] Docker OS support

[Josh Fischer <jo...@joshfischer.io>]

Well thought out Windham.  I like how you narrowed the criteria down to
Debian10/CentOS7(depending on PY3 support) thinking about overall support
and what others would need/want.

+1

- Josh

On Mon, May 18, 2020 at 8:39 PM Windham Wong <wi...@stormeye.io>
wrote:

> My experience is that, people tends to pick their container image OS based
> on few criterias only:
> 1) image size, alpine always win, then debian/centos and 3rd is
> ubuntu/centos. alpine is always the smallest but not much library provided
> and require to compile manually. debian is always good because it has
> apt-get to provide loads of library. centos is cool as debian because it
> also has good yum repo. ubuntu is always the last because it is fat, but it
> provides very large range of libraries to use.
> 2) kernel feature, as mentioned last email, ubuntu has issue with DNS and
> some sort of network discover issues. I remember alpine has some as well,
> because it is very plain, requires many fine tunes be done manually. in
> this case, centos and debian should win.
> 3) linux stream, obviously, alpine, debian/ubuntu, centos, are 3 different
> streams that use different native tools. Poeple just pick their base image
> based on what they often use in their life. However, I see a lot debian and
> alpine on docker hub because they are just smaller in size.
> 4. library support. Python library is a very good example in this case. As
> Heron is turning to Python 3, which specifically Python 3.7+, some
> discontinued based OS version does not provide the support anymore. We have
> tested out that, debian9 supports only up to Python3.5, ubuntu 14.04 does
> not support python3 and ubuntu 16.04 supports up to Python 3.5 only.
>
> I am not a big fan of using centos for docker. Concluding all 4 points
> above, I believe keeping centos7 (if it supports python37), debian10 (not
> debian9), ubuntu 18.04 (supports python3.7+) would be the best solution at
> the moment. We have to cut off the old images due to our library upgrade
> and this is unavoidable. Otherwise, we would have tons of issues sending in
> about library support in the future and the workload could be a lot more
> for just answering questions.
>
> Windham Wong
> Co-Founder, Technical Director of
> Stormeye.io, Hong Kong Managed Security Operation Center Limited
> Email // windham.wong@stormeye.io (mailto:windham.wong@stormeye.io)
> Phone // +852_3590_2212_|_+852_9832_0707 (tel:+85235902212)
> Fax // +852_3590_2202 (tel:+852_3590_2202)
>
> On 5月 19 2020, at 6:09 早上, Josh Fischer <jo...@joshfischer.io> wrote:
> > Hi All, I think using CentOS and either Debian or Ubuntu is a good idea.
> I would pick Debian over Ubuntu for the reason that Nick pointed out in a
> previous email of DNS issues in Kubernetes. I"ve copy and pasted a section
> of Nick's previous email. ### Start I've run into issues with Ubuntu images
> having DNS issues in Kubernetes.
> https://github.com/kubernetes/kubeadm/issues/787 ### End As for what ends
> up being the official Docker container I'm fine with either choice Debian
> or CentOS. On Mon, May 18, 2020 at 2:10 AM Ning Wang wrote: > Yeah. For
> binary release, the license is a huge factor. > > On Sun, May 17, 2020 at
> 2:32 PM Josh Fischer wrote: > > > Hi All, > > > > When it comes to choosing
> the official Apache Docker image for Heron we > > have to think of the
> licensing issues first. Everything that is > installed > > into the
> container at build time with the exception of the "FROM" > > statement in
> the Dockerfile must be ALv2 compatible. OpenJDK is under > GPLv2 > > which
> is not compatible with Apache. However we can get around this if > we > >
> bring the OpenJDK into the container via a FROM statement. I've copied a >
> > reply from Dave in our dev list in a previous email. See below. > > > >
> ######## Start ######## > > Regarding OpenJDK and GPL2 - here is what Roman
> the VP, Legal wrote when > > answering Beam’s questions. > > > > Roman
> Shaposhnik commented on LEGAL-503: > >
> ---------------------------------------- > > > > Hey [~altay] if you would
> like to continue linking to the Docker release > > artifact from the > >
> https://beam.apache.org > > you will have: > > 1. Transition to the
> official ASF dockerhub org: > > https://hub.docker.com/u/apache > > 2.
> Start including that binary convenience artifact into your VOTE > > threads
> on Beam releases > > 3. Make sure that all Cat-X licenses are ONLY brought
> into your > > container via FROM statements > > ######## End ######## > > >
> > So at the end of the day, we need to use a container that has Java >
> already > > installed in it so that when we create our official image we
> can inherit > > that java containing image via a FROM statement. > > > > -
> Josh > > > > > > On Sun, May 17, 2020 at 4:12 PM H W wrote: > > > > > We
> use Centos 7 as well. > > > > > > On Sun, May 17, 2020 at 10:41 AM Nicholas
> Nezis < > > nicholas.nezis@gmail.com> > > > wrote: > > > > > > > Some
> thoughts I'd like to add to the discussion: > > > > > > > > Debian and
> Ubuntu are somewhat similar. Would it make sense to pick > one > > > of > >
> > > those and pair with Centos? > > > > > > > > I've run into issues with
> Ubuntu images having DNS issues in > > Kubernetes. > > > >
> https://github.com/kubernetes/kubeadm/issues/787 > > > > > > > > Ubuntu
> 14.04 should be removed. It was end of life last year. For > those > > > of
> > > > > you using Heron in production, is anyone using Ubuntu 16? My vote >
> would > > > be > > > > to remove it also. For what it's worth, my use case
> is in K8s and we > > use > > > > the Centos7 image on a Centos 7 OS. > > >
> > > > > > We should do a test of the official image and helm chart in >
> Kubernetes > > > > (maybe with Kind https://kind.sigs.k8s.io/ ) as part
> of our release > > > > process. This might be nice to add as an integration
> test. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On
> Sun, May 17, 2020, 12:37 PM Ning Wang > wrote: > > > > > > > > > That is a
> good point. We need to adjust the OS version accordingly > > too. > > > > >
> > > > > > On Sun, May 17, 2020 at 2:28 AM Windham Wong < > >
> windham.wong@stormeye.io > > > > > > > > > wrote: > > > > > > > > > > > >
> > > > > > debian 10 has no python 2.7 support. Ubuntu 16.04 has no python >
> > 3.7+ > > > > dev > > > > > > library support. hope this helps > > > > > >
> -- > > > > > > Sent from myMail for Android Sunday, 17 May 2020, 11:34AM
> +08:00 > > from > > > > > Ning > > > > > > Wang wangninggm@gmail.com : >
> > > > > > > > > > > > >Hi, > > > > > > > > > > > > > >We have talked about
> it a few times in different places. Let's > > make > > > a > > > > > >
> >decision here. We don't have enough resources to support many > > docker >
> > > > > OSes > > > > > > >(technically we only support Debian, Ubuntu, and
> CentOS but each > > of > > > > them > > > > > > >has multiple versions). >
> > > > > > > > > > > > > >Things we have agreed on (my understanding) > > >
> > > > >- for binary release, we are going to choose one OS one > version. I
> > > > > > > >remember that Debian 9 was the decision. An open question is:
> do > > we > > > > want > > > > > > to > > > > > > >switch to Debian 10 now?
> We haven't had a binary release yet, so > > > there > > > > > is > > > > >
> > >still time. As the main target, I think 1. the image should > > compile
> > > > > > > >successfully. and 2 the image should be tested: the tools and
> an > > > > example > > > > > > >topology. > > > > > > > > > > > > > >- for
> source release, we haven't talked about it very much. > > > > Personally I
> > > > > > > >feel it is a reasonable expectation that, if we have the
> Docker > > file > > > > in > > > > > > the > > > > > > >core folders, it
> should at least compile. Otherwise, it's a > failed > > > > build > > > > >
> > >hence a failed release. We are not going to test run the tools > and > >
> > > > > examples > > > > > > >for each release though. > > > > > > > > > >
> > > > >Finally, a question about OS versions. I believe at least two > > >
> releases > > > > > > >should be supported because most people don't upgrade
> to the > > latest > > > > > > version > > > > > > >when it is out in more
> serious scenarios, hence IMO the second > > > oldest > > > > > > could > >
> > > > > >often be more useful than the latest one from a convenience >
> point > > of > > > > > view. > > > > > > >On the other hand, this is too
> flexible, and we will spend time > > > > thinking > > > > > > >about it
> again. I think a clear guideline could be helpful to > us. > > > For > > >
> > > > >example: > > > > > > >- We only include only Debian (popular in the
> server world) and > > > Ubuntu > > > > > > >(popular in servers and
> workstations). > > > > > > >- for the main OS (Debian) we choose the latest
> to compile and > > test > > > > for > > > > > > our > > > > > > >binary
> release. > > > > > > >- for a given OS, only two versions (most likely the
> most and > the > > > > second > > > > > > >recent releases) are included in
> the core source code and they > > need > > > to > > > > > be > > > > > >
> >fixed ASAP if they don't compile. > > > > > > >- OSs and other versions
> could be included in a special folder > and > > > but > > > > > > >there is
> no guarantee that the code will compile. Fixes from the > > > > > community
> > > > > > > >are welcome. > > > > > > > > > > > > > > > > > > > > >How do
> you guys think? > > > > > > >--ning > > > > > > > > > > > > > > > > > > > >
> >
>
>

Re: [DISCUSSION] Docker OS support

[Ning Wang <wa...@gmail.com>]

Good reasoning! Thanks!

On Mon, May 18, 2020 at 6:39 PM Windham Wong <wi...@stormeye.io>
wrote:

> My experience is that, people tends to pick their container image OS based
> on few criterias only:
> 1) image size, alpine always win, then debian/centos and 3rd is
> ubuntu/centos. alpine is always the smallest but not much library provided
> and require to compile manually. debian is always good because it has
> apt-get to provide loads of library. centos is cool as debian because it
> also has good yum repo. ubuntu is always the last because it is fat, but it
> provides very large range of libraries to use.
> 2) kernel feature, as mentioned last email, ubuntu has issue with DNS and
> some sort of network discover issues. I remember alpine has some as well,
> because it is very plain, requires many fine tunes be done manually. in
> this case, centos and debian should win.
> 3) linux stream, obviously, alpine, debian/ubuntu, centos, are 3 different
> streams that use different native tools. Poeple just pick their base image
> based on what they often use in their life. However, I see a lot debian and
> alpine on docker hub because they are just smaller in size.
> 4. library support. Python library is a very good example in this case. As
> Heron is turning to Python 3, which specifically Python 3.7+, some
> discontinued based OS version does not provide the support anymore. We have
> tested out that, debian9 supports only up to Python3.5, ubuntu 14.04 does
> not support python3 and ubuntu 16.04 supports up to Python 3.5 only.
>
> I am not a big fan of using centos for docker. Concluding all 4 points
> above, I believe keeping centos7 (if it supports python37), debian10 (not
> debian9), ubuntu 18.04 (supports python3.7+) would be the best solution at
> the moment. We have to cut off the old images due to our library upgrade
> and this is unavoidable. Otherwise, we would have tons of issues sending in
> about library support in the future and the workload could be a lot more
> for just answering questions.
>
> Windham Wong
> Co-Founder, Technical Director of
> Stormeye.io, Hong Kong Managed Security Operation Center Limited
> Email // windham.wong@stormeye.io (mailto:windham.wong@stormeye.io)
> Phone // +852_3590_2212_|_+852_9832_0707 (tel:+85235902212)
> Fax // +852_3590_2202 (tel:+852_3590_2202)
>
> On 5月 19 2020, at 6:09 早上, Josh Fischer <jo...@joshfischer.io> wrote:
> > Hi All, I think using CentOS and either Debian or Ubuntu is a good idea.
> I would pick Debian over Ubuntu for the reason that Nick pointed out in a
> previous email of DNS issues in Kubernetes. I"ve copy and pasted a section
> of Nick's previous email. ### Start I've run into issues with Ubuntu images
> having DNS issues in Kubernetes.
> https://github.com/kubernetes/kubeadm/issues/787 ### End As for what ends
> up being the official Docker container I'm fine with either choice Debian
> or CentOS. On Mon, May 18, 2020 at 2:10 AM Ning Wang wrote: > Yeah. For
> binary release, the license is a huge factor. > > On Sun, May 17, 2020 at
> 2:32 PM Josh Fischer wrote: > > > Hi All, > > > > When it comes to choosing
> the official Apache Docker image for Heron we > > have to think of the
> licensing issues first. Everything that is > installed > > into the
> container at build time with the exception of the "FROM" > > statement in
> the Dockerfile must be ALv2 compatible. OpenJDK is under > GPLv2 > > which
> is not compatible with Apache. However we can get around this if > we > >
> bring the OpenJDK into the container via a FROM statement. I've copied a >
> > reply from Dave in our dev list in a previous email. See below. > > > >
> ######## Start ######## > > Regarding OpenJDK and GPL2 - here is what Roman
> the VP, Legal wrote when > > answering Beam’s questions. > > > > Roman
> Shaposhnik commented on LEGAL-503: > >
> ---------------------------------------- > > > > Hey [~altay] if you would
> like to continue linking to the Docker release > > artifact from the > >
> https://beam.apache.org > > you will have: > > 1. Transition to the
> official ASF dockerhub org: > > https://hub.docker.com/u/apache > > 2.
> Start including that binary convenience artifact into your VOTE > > threads
> on Beam releases > > 3. Make sure that all Cat-X licenses are ONLY brought
> into your > > container via FROM statements > > ######## End ######## > > >
> > So at the end of the day, we need to use a container that has Java >
> already > > installed in it so that when we create our official image we
> can inherit > > that java containing image via a FROM statement. > > > > -
> Josh > > > > > > On Sun, May 17, 2020 at 4:12 PM H W wrote: > > > > > We
> use Centos 7 as well. > > > > > > On Sun, May 17, 2020 at 10:41 AM Nicholas
> Nezis < > > nicholas.nezis@gmail.com> > > > wrote: > > > > > > > Some
> thoughts I'd like to add to the discussion: > > > > > > > > Debian and
> Ubuntu are somewhat similar. Would it make sense to pick > one > > > of > >
> > > those and pair with Centos? > > > > > > > > I've run into issues with
> Ubuntu images having DNS issues in > > Kubernetes. > > > >
> https://github.com/kubernetes/kubeadm/issues/787 > > > > > > > > Ubuntu
> 14.04 should be removed. It was end of life last year. For > those > > > of
> > > > > you using Heron in production, is anyone using Ubuntu 16? My vote >
> would > > > be > > > > to remove it also. For what it's worth, my use case
> is in K8s and we > > use > > > > the Centos7 image on a Centos 7 OS. > > >
> > > > > > We should do a test of the official image and helm chart in >
> Kubernetes > > > > (maybe with Kind https://kind.sigs.k8s.io/ ) as part
> of our release > > > > process. This might be nice to add as an integration
> test. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On
> Sun, May 17, 2020, 12:37 PM Ning Wang > wrote: > > > > > > > > > That is a
> good point. We need to adjust the OS version accordingly > > too. > > > > >
> > > > > > On Sun, May 17, 2020 at 2:28 AM Windham Wong < > >
> windham.wong@stormeye.io > > > > > > > > > wrote: > > > > > > > > > > > >
> > > > > > debian 10 has no python 2.7 support. Ubuntu 16.04 has no python >
> > 3.7+ > > > > dev > > > > > > library support. hope this helps > > > > > >
> -- > > > > > > Sent from myMail for Android Sunday, 17 May 2020, 11:34AM
> +08:00 > > from > > > > > Ning > > > > > > Wang wangninggm@gmail.com : >
> > > > > > > > > > > > >Hi, > > > > > > > > > > > > > >We have talked about
> it a few times in different places. Let's > > make > > > a > > > > > >
> >decision here. We don't have enough resources to support many > > docker >
> > > > > OSes > > > > > > >(technically we only support Debian, Ubuntu, and
> CentOS but each > > of > > > > them > > > > > > >has multiple versions). >
> > > > > > > > > > > > > >Things we have agreed on (my understanding) > > >
> > > > >- for binary release, we are going to choose one OS one > version. I
> > > > > > > >remember that Debian 9 was the decision. An open question is:
> do > > we > > > > want > > > > > > to > > > > > > >switch to Debian 10 now?
> We haven't had a binary release yet, so > > > there > > > > > is > > > > >
> > >still time. As the main target, I think 1. the image should > > compile
> > > > > > > >successfully. and 2 the image should be tested: the tools and
> an > > > > example > > > > > > >topology. > > > > > > > > > > > > > >- for
> source release, we haven't talked about it very much. > > > > Personally I
> > > > > > > >feel it is a reasonable expectation that, if we have the
> Docker > > file > > > > in > > > > > > the > > > > > > >core folders, it
> should at least compile. Otherwise, it's a > failed > > > > build > > > > >
> > >hence a failed release. We are not going to test run the tools > and > >
> > > > > examples > > > > > > >for each release though. > > > > > > > > > >
> > > > >Finally, a question about OS versions. I believe at least two > > >
> releases > > > > > > >should be supported because most people don't upgrade
> to the > > latest > > > > > > version > > > > > > >when it is out in more
> serious scenarios, hence IMO the second > > > oldest > > > > > > could > >
> > > > > >often be more useful than the latest one from a convenience >
> point > > of > > > > > view. > > > > > > >On the other hand, this is too
> flexible, and we will spend time > > > > thinking > > > > > > >about it
> again. I think a clear guideline could be helpful to > us. > > > For > > >
> > > > >example: > > > > > > >- We only include only Debian (popular in the
> server world) and > > > Ubuntu > > > > > > >(popular in servers and
> workstations). > > > > > > >- for the main OS (Debian) we choose the latest
> to compile and > > test > > > > for > > > > > > our > > > > > > >binary
> release. > > > > > > >- for a given OS, only two versions (most likely the
> most and > the > > > > second > > > > > > >recent releases) are included in
> the core source code and they > > need > > > to > > > > > be > > > > > >
> >fixed ASAP if they don't compile. > > > > > > >- OSs and other versions
> could be included in a special folder > and > > > but > > > > > > >there is
> no guarantee that the code will compile. Fixes from the > > > > > community
> > > > > > > >are welcome. > > > > > > > > > > > > > > > > > > > > >How do
> you guys think? > > > > > > >--ning > > > > > > > > > > > > > > > > > > > >
> >
>
>

Re: [DISCUSSION] Docker OS support

[Windham Wong <wi...@stormeye.io>]

My experience is that, people tends to pick their container image OS based on few criterias only:
1) image size, alpine always win, then debian/centos and 3rd is ubuntu/centos. alpine is always the smallest but not much library provided and require to compile manually. debian is always good because it has apt-get to provide loads of library. centos is cool as debian because it also has good yum repo. ubuntu is always the last because it is fat, but it provides very large range of libraries to use.
2) kernel feature, as mentioned last email, ubuntu has issue with DNS and some sort of network discover issues. I remember alpine has some as well, because it is very plain, requires many fine tunes be done manually. in this case, centos and debian should win.
3) linux stream, obviously, alpine, debian/ubuntu, centos, are 3 different streams that use different native tools. Poeple just pick their base image based on what they often use in their life. However, I see a lot debian and alpine on docker hub because they are just smaller in size.
4. library support. Python library is a very good example in this case. As Heron is turning to Python 3, which specifically Python 3.7+, some discontinued based OS version does not provide the support anymore. We have tested out that, debian9 supports only up to Python3.5, ubuntu 14.04 does not support python3 and ubuntu 16.04 supports up to Python 3.5 only.

I am not a big fan of using centos for docker. Concluding all 4 points above, I believe keeping centos7 (if it supports python37), debian10 (not debian9), ubuntu 18.04 (supports python3.7+) would be the best solution at the moment. We have to cut off the old images due to our library upgrade and this is unavoidable. Otherwise, we would have tons of issues sending in about library support in the future and the workload could be a lot more for just answering questions.

Windham Wong
Co-Founder, Technical Director of
Stormeye.io, Hong Kong Managed Security Operation Center Limited
Email // windham.wong@stormeye.io (mailto:windham.wong@stormeye.io)
Phone // +852_3590_2212_|_+852_9832_0707 (tel:+85235902212)
Fax // +852_3590_2202 (tel:+852_3590_2202)

On 5月 19 2020, at 6:09 早上, Josh Fischer <jo...@joshfischer.io> wrote:
> Hi All, I think using CentOS and either Debian or Ubuntu is a good idea. I would pick Debian over Ubuntu for the reason that Nick pointed out in a previous email of DNS issues in Kubernetes. I"ve copy and pasted a section of Nick's previous email. ### Start I've run into issues with Ubuntu images having DNS issues in Kubernetes. https://github.com/kubernetes/kubeadm/issues/787 ### End As for what ends up being the official Docker container I'm fine with either choice Debian or CentOS. On Mon, May 18, 2020 at 2:10 AM Ning Wang wrote: > Yeah. For binary release, the license is a huge factor. > > On Sun, May 17, 2020 at 2:32 PM Josh Fischer wrote: > > > Hi All, > > > > When it comes to choosing the official Apache Docker image for Heron we > > have to think of the licensing issues first. Everything that is > installed > > into the container at build time with the exception of the "FROM" > > statement in the Dockerfile must be ALv2 compatible. OpenJDK is under > GPLv2 > > which is not compatible with Apache. However we can get around this if > we > > bring the OpenJDK into the container via a FROM statement. I've copied a > > reply from Dave in our dev list in a previous email. See below. > > > > ######## Start ######## > > Regarding OpenJDK and GPL2 - here is what Roman the VP, Legal wrote when > > answering Beam’s questions. > > > > Roman Shaposhnik commented on LEGAL-503: > > ---------------------------------------- > > > > Hey [~altay] if you would like to continue linking to the Docker release > > artifact from the > > https://beam.apache.org > > you will have: > > 1. Transition to the official ASF dockerhub org: > > https://hub.docker.com/u/apache > > 2. Start including that binary convenience artifact into your VOTE > > threads on Beam releases > > 3. Make sure that all Cat-X licenses are ONLY brought into your > > container via FROM statements > > ######## End ######## > > > > So at the end of the day, we need to use a container that has Java > already > > installed in it so that when we create our official image we can inherit > > that java containing image via a FROM statement. > > > > - Josh > > > > > > On Sun, May 17, 2020 at 4:12 PM H W wrote: > > > > > We use Centos 7 as well. > > > > > > On Sun, May 17, 2020 at 10:41 AM Nicholas Nezis < > > nicholas.nezis@gmail.com> > > > wrote: > > > > > > > Some thoughts I'd like to add to the discussion: > > > > > > > > Debian and Ubuntu are somewhat similar. Would it make sense to pick > one > > > of > > > > those and pair with Centos? > > > > > > > > I've run into issues with Ubuntu images having DNS issues in > > Kubernetes. > > > > https://github.com/kubernetes/kubeadm/issues/787 > > > > > > > > Ubuntu 14.04 should be removed. It was end of life last year. For > those > > > of > > > > you using Heron in production, is anyone using Ubuntu 16? My vote > would > > > be > > > > to remove it also. For what it's worth, my use case is in K8s and we > > use > > > > the Centos7 image on a Centos 7 OS. > > > > > > > > We should do a test of the official image and helm chart in > Kubernetes > > > > (maybe with Kind https://kind.sigs.k8s.io/ ) as part of our release > > > > process. This might be nice to add as an integration test. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sun, May 17, 2020, 12:37 PM Ning Wang > wrote: > > > > > > > > > That is a good point. We need to adjust the OS version accordingly > > too. > > > > > > > > > > On Sun, May 17, 2020 at 2:28 AM Windham Wong < > > windham.wong@stormeye.io > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > debian 10 has no python 2.7 support. Ubuntu 16.04 has no python > > 3.7+ > > > > dev > > > > > > library support. hope this helps > > > > > > -- > > > > > > Sent from myMail for Android Sunday, 17 May 2020, 11:34AM +08:00 > > from > > > > > Ning > > > > > > Wang wangninggm@gmail.com : > > > > > > > > > > > > >Hi, > > > > > > > > > > > > > >We have talked about it a few times in different places. Let's > > make > > > a > > > > > > >decision here. We don't have enough resources to support many > > docker > > > > > OSes > > > > > > >(technically we only support Debian, Ubuntu, and CentOS but each > > of > > > > them > > > > > > >has multiple versions). > > > > > > > > > > > > > >Things we have agreed on (my understanding) > > > > > > >- for binary release, we are going to choose one OS one > version. I > > > > > > >remember that Debian 9 was the decision. An open question is: do > > we > > > > want > > > > > > to > > > > > > >switch to Debian 10 now? We haven't had a binary release yet, so > > > there > > > > > is > > > > > > >still time. As the main target, I think 1. the image should > > compile > > > > > > >successfully. and 2 the image should be tested: the tools and an > > > > example > > > > > > >topology. > > > > > > > > > > > > > >- for source release, we haven't talked about it very much. > > > > Personally I > > > > > > >feel it is a reasonable expectation that, if we have the Docker > > file > > > > in > > > > > > the > > > > > > >core folders, it should at least compile. Otherwise, it's a > failed > > > > build > > > > > > >hence a failed release. We are not going to test run the tools > and > > > > > > examples > > > > > > >for each release though. > > > > > > > > > > > > > >Finally, a question about OS versions. I believe at least two > > > releases > > > > > > >should be supported because most people don't upgrade to the > > latest > > > > > > version > > > > > > >when it is out in more serious scenarios, hence IMO the second > > > oldest > > > > > > could > > > > > > >often be more useful than the latest one from a convenience > point > > of > > > > > view. > > > > > > >On the other hand, this is too flexible, and we will spend time > > > > thinking > > > > > > >about it again. I think a clear guideline could be helpful to > us. > > > For > > > > > > >example: > > > > > > >- We only include only Debian (popular in the server world) and > > > Ubuntu > > > > > > >(popular in servers and workstations). > > > > > > >- for the main OS (Debian) we choose the latest to compile and > > test > > > > for > > > > > > our > > > > > > >binary release. > > > > > > >- for a given OS, only two versions (most likely the most and > the > > > > second > > > > > > >recent releases) are included in the core source code and they > > need > > > to > > > > > be > > > > > > >fixed ASAP if they don't compile. > > > > > > >- OSs and other versions could be included in a special folder > and > > > but > > > > > > >there is no guarantee that the code will compile. Fixes from the > > > > > community > > > > > > >are welcome. > > > > > > > > > > > > > > > > > > > > >How do you guys think? > > > > > > >--ning > > > > > > > > > > > > > > > > > > > > >

Re: [DISCUSSION] Docker OS support

[Josh Fischer <jo...@joshfischer.io>]

Hi All,

I think using CentOS and either Debian or Ubuntu is a good idea.  I would
pick Debian over Ubuntu for the reason that Nick pointed out in a previous
email of DNS issues in Kubernetes.  I"ve copy and pasted a section of
Nick's previous email.

###  Start
I've run into issues with Ubuntu images having DNS issues in Kubernetes.
https://github.com/kubernetes/kubeadm/issues/787
### End

 As for what ends up being the official Docker container I'm fine with
either choice Debian or CentOS.


On Mon, May 18, 2020 at 2:10 AM Ning Wang <wa...@gmail.com> wrote:

> Yeah. For binary release, the license is a huge factor.
>
> On Sun, May 17, 2020 at 2:32 PM Josh Fischer <jo...@joshfischer.io> wrote:
>
> > Hi All,
> >
> > When it comes to choosing the official Apache Docker image for Heron we
> > have to think of the licensing issues first.  Everything that is
> installed
> > into the container at build time with the exception of the "FROM"
> > statement in the Dockerfile must be ALv2 compatible. OpenJDK is under
> GPLv2
> > which is not compatible with Apache.  However we can get around this if
> we
> > bring the OpenJDK into the container via a FROM statement.  I've copied a
> > reply from Dave in our dev list in a previous email.  See below.
> >
> > ######## Start ########
> > Regarding OpenJDK and GPL2 - here is what Roman the VP, Legal wrote when
> > answering Beam’s questions.
> >
> > Roman Shaposhnik commented on LEGAL-503:
> > ----------------------------------------
> >
> > Hey [~altay] if you would like to continue linking to the Docker release
> > artifact from the
> > https://beam.apache.org
> >  you will have:
> >    1. Transition to the official ASF dockerhub org:
> > https://hub.docker.com/u/apache
> >    2. Start including that binary convenience artifact into your VOTE
> > threads on Beam releases
> >    3. Make sure that all Cat-X licenses are ONLY brought into your
> > container via FROM statements
> > ######## End ########
> >
> > So at the end of the day, we need to use a container that has Java
> already
> > installed in it so that when we create our official image we can inherit
> > that java containing image via a FROM statement.
> >
> > - Josh
> >
> >
> > On Sun, May 17, 2020 at 4:12 PM H W <hu...@gmail.com> wrote:
> >
> > > We use Centos 7 as well.
> > >
> > > On Sun, May 17, 2020 at 10:41 AM Nicholas Nezis <
> > nicholas.nezis@gmail.com>
> > > wrote:
> > >
> > > > Some thoughts I'd like to add to the discussion:
> > > >
> > > > Debian and Ubuntu are somewhat similar. Would it make sense to pick
> one
> > > of
> > > > those and pair with Centos?
> > > >
> > > > I've run into issues with Ubuntu images having DNS issues in
> > Kubernetes.
> > > > https://github.com/kubernetes/kubeadm/issues/787
> > > >
> > > > Ubuntu 14.04 should be removed. It was end of life last year. For
> those
> > > of
> > > > you using Heron in production, is anyone using Ubuntu 16? My vote
> would
> > > be
> > > > to remove it also. For what it's worth, my use case is in K8s and we
> > use
> > > > the Centos7 image on a Centos 7 OS.
> > > >
> > > > We should do a test of the official image and helm chart in
> Kubernetes
> > > > (maybe with Kind https://kind.sigs.k8s.io/ ) as part of our release
> > > > process. This might be nice to add as an integration test.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Sun, May 17, 2020, 12:37 PM Ning Wang <wa...@gmail.com>
> wrote:
> > > >
> > > > > That is a good point. We need to adjust the OS version accordingly
> > too.
> > > > >
> > > > > On Sun, May 17, 2020 at 2:28 AM Windham Wong <
> > windham.wong@stormeye.io
> > > >
> > > > > wrote:
> > > > >
> > > > > >
> > > > > > debian 10 has no python 2.7 support. Ubuntu 16.04 has no python
> > 3.7+
> > > > dev
> > > > > > library support. hope this helps
> > > > > > --
> > > > > > Sent from myMail for Android Sunday, 17 May 2020, 11:34AM +08:00
> > from
> > > > > Ning
> > > > > > Wang  wangninggm@gmail.com :
> > > > > >
> > > > > > >Hi,
> > > > > > >
> > > > > > >We have talked about it a few times in different places. Let's
> > make
> > > a
> > > > > > >decision here. We don't have enough resources to support many
> > docker
> > > > > OSes
> > > > > > >(technically we only support Debian, Ubuntu, and CentOS but each
> > of
> > > > them
> > > > > > >has multiple versions).
> > > > > > >
> > > > > > >Things we have agreed on (my understanding)
> > > > > > >- for binary release, we are going to choose one OS one
> version. I
> > > > > > >remember that Debian 9 was the decision. An open question is: do
> > we
> > > > want
> > > > > > to
> > > > > > >switch to Debian 10 now? We haven't had a binary release yet, so
> > > there
> > > > > is
> > > > > > >still time. As the main target, I think 1. the image should
> > compile
> > > > > > >successfully. and 2 the image should be tested: the tools and an
> > > > example
> > > > > > >topology.
> > > > > > >
> > > > > > >- for source release, we haven't talked about it very much.
> > > > Personally I
> > > > > > >feel it is a reasonable expectation that, if we have the Docker
> > file
> > > > in
> > > > > > the
> > > > > > >core folders, it should at least compile. Otherwise, it's a
> failed
> > > > build
> > > > > > >hence a failed release. We are not going to test run the tools
> and
> > > > > > examples
> > > > > > >for each release though.
> > > > > > >
> > > > > > >Finally, a question about OS versions. I believe at least two
> > > releases
> > > > > > >should be supported because most people don't upgrade to the
> > latest
> > > > > > version
> > > > > > >when it is out in more serious scenarios, hence IMO the second
> > > oldest
> > > > > > could
> > > > > > >often be more useful than the latest one from a convenience
> point
> > of
> > > > > view.
> > > > > > >On the other hand, this is too flexible, and we will spend time
> > > > thinking
> > > > > > >about it again. I think a clear guideline could be helpful to
> us.
> > > For
> > > > > > >example:
> > > > > > >- We only include only Debian (popular in the server world) and
> > > Ubuntu
> > > > > > >(popular in servers and workstations).
> > > > > > >- for the main OS (Debian) we choose the latest to compile and
> > test
> > > > for
> > > > > > our
> > > > > > >binary release.
> > > > > > >- for a given OS, only two versions (most likely the most and
> the
> > > > second
> > > > > > >recent releases) are included in the core source code and they
> > need
> > > to
> > > > > be
> > > > > > >fixed ASAP if they don't compile.
> > > > > > >- OSs and other versions could be included in a special folder
> and
> > > but
> > > > > > >there is no guarantee that the code will compile. Fixes from the
> > > > > community
> > > > > > >are welcome.
> > > > > > >
> > > > > > >
> > > > > > >How do you guys think?
> > > > > > >--ning
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSSION] Docker OS support

[Ning Wang <wa...@gmail.com>]

Yeah. For binary release, the license is a huge factor.

On Sun, May 17, 2020 at 2:32 PM Josh Fischer <jo...@joshfischer.io> wrote:

> Hi All,
>
> When it comes to choosing the official Apache Docker image for Heron we
> have to think of the licensing issues first.  Everything that is installed
> into the container at build time with the exception of the "FROM"
> statement in the Dockerfile must be ALv2 compatible. OpenJDK is under GPLv2
> which is not compatible with Apache.  However we can get around this if we
> bring the OpenJDK into the container via a FROM statement.  I've copied a
> reply from Dave in our dev list in a previous email.  See below.
>
> ######## Start ########
> Regarding OpenJDK and GPL2 - here is what Roman the VP, Legal wrote when
> answering Beam’s questions.
>
> Roman Shaposhnik commented on LEGAL-503:
> ----------------------------------------
>
> Hey [~altay] if you would like to continue linking to the Docker release
> artifact from the
> https://beam.apache.org
>  you will have:
>    1. Transition to the official ASF dockerhub org:
> https://hub.docker.com/u/apache
>    2. Start including that binary convenience artifact into your VOTE
> threads on Beam releases
>    3. Make sure that all Cat-X licenses are ONLY brought into your
> container via FROM statements
> ######## End ########
>
> So at the end of the day, we need to use a container that has Java already
> installed in it so that when we create our official image we can inherit
> that java containing image via a FROM statement.
>
> - Josh
>
>
> On Sun, May 17, 2020 at 4:12 PM H W <hu...@gmail.com> wrote:
>
> > We use Centos 7 as well.
> >
> > On Sun, May 17, 2020 at 10:41 AM Nicholas Nezis <
> nicholas.nezis@gmail.com>
> > wrote:
> >
> > > Some thoughts I'd like to add to the discussion:
> > >
> > > Debian and Ubuntu are somewhat similar. Would it make sense to pick one
> > of
> > > those and pair with Centos?
> > >
> > > I've run into issues with Ubuntu images having DNS issues in
> Kubernetes.
> > > https://github.com/kubernetes/kubeadm/issues/787
> > >
> > > Ubuntu 14.04 should be removed. It was end of life last year. For those
> > of
> > > you using Heron in production, is anyone using Ubuntu 16? My vote would
> > be
> > > to remove it also. For what it's worth, my use case is in K8s and we
> use
> > > the Centos7 image on a Centos 7 OS.
> > >
> > > We should do a test of the official image and helm chart in Kubernetes
> > > (maybe with Kind https://kind.sigs.k8s.io/ ) as part of our release
> > > process. This might be nice to add as an integration test.
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Sun, May 17, 2020, 12:37 PM Ning Wang <wa...@gmail.com> wrote:
> > >
> > > > That is a good point. We need to adjust the OS version accordingly
> too.
> > > >
> > > > On Sun, May 17, 2020 at 2:28 AM Windham Wong <
> windham.wong@stormeye.io
> > >
> > > > wrote:
> > > >
> > > > >
> > > > > debian 10 has no python 2.7 support. Ubuntu 16.04 has no python
> 3.7+
> > > dev
> > > > > library support. hope this helps
> > > > > --
> > > > > Sent from myMail for Android Sunday, 17 May 2020, 11:34AM +08:00
> from
> > > > Ning
> > > > > Wang  wangninggm@gmail.com :
> > > > >
> > > > > >Hi,
> > > > > >
> > > > > >We have talked about it a few times in different places. Let's
> make
> > a
> > > > > >decision here. We don't have enough resources to support many
> docker
> > > > OSes
> > > > > >(technically we only support Debian, Ubuntu, and CentOS but each
> of
> > > them
> > > > > >has multiple versions).
> > > > > >
> > > > > >Things we have agreed on (my understanding)
> > > > > >- for binary release, we are going to choose one OS one version. I
> > > > > >remember that Debian 9 was the decision. An open question is: do
> we
> > > want
> > > > > to
> > > > > >switch to Debian 10 now? We haven't had a binary release yet, so
> > there
> > > > is
> > > > > >still time. As the main target, I think 1. the image should
> compile
> > > > > >successfully. and 2 the image should be tested: the tools and an
> > > example
> > > > > >topology.
> > > > > >
> > > > > >- for source release, we haven't talked about it very much.
> > > Personally I
> > > > > >feel it is a reasonable expectation that, if we have the Docker
> file
> > > in
> > > > > the
> > > > > >core folders, it should at least compile. Otherwise, it's a failed
> > > build
> > > > > >hence a failed release. We are not going to test run the tools and
> > > > > examples
> > > > > >for each release though.
> > > > > >
> > > > > >Finally, a question about OS versions. I believe at least two
> > releases
> > > > > >should be supported because most people don't upgrade to the
> latest
> > > > > version
> > > > > >when it is out in more serious scenarios, hence IMO the second
> > oldest
> > > > > could
> > > > > >often be more useful than the latest one from a convenience point
> of
> > > > view.
> > > > > >On the other hand, this is too flexible, and we will spend time
> > > thinking
> > > > > >about it again. I think a clear guideline could be helpful to us.
> > For
> > > > > >example:
> > > > > >- We only include only Debian (popular in the server world) and
> > Ubuntu
> > > > > >(popular in servers and workstations).
> > > > > >- for the main OS (Debian) we choose the latest to compile and
> test
> > > for
> > > > > our
> > > > > >binary release.
> > > > > >- for a given OS, only two versions (most likely the most and the
> > > second
> > > > > >recent releases) are included in the core source code and they
> need
> > to
> > > > be
> > > > > >fixed ASAP if they don't compile.
> > > > > >- OSs and other versions could be included in a special folder and
> > but
> > > > > >there is no guarantee that the code will compile. Fixes from the
> > > > community
> > > > > >are welcome.
> > > > > >
> > > > > >
> > > > > >How do you guys think?
> > > > > >--ning
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSSION] Docker OS support

[Josh Fischer <jo...@joshfischer.io>]

Hi All,

When it comes to choosing the official Apache Docker image for Heron we
have to think of the licensing issues first.  Everything that is installed
into the container at build time with the exception of the "FROM"
statement in the Dockerfile must be ALv2 compatible. OpenJDK is under GPLv2
which is not compatible with Apache.  However we can get around this if we
bring the OpenJDK into the container via a FROM statement.  I've copied a
reply from Dave in our dev list in a previous email.  See below.

######## Start ########
Regarding OpenJDK and GPL2 - here is what Roman the VP, Legal wrote when
answering Beam’s questions.

Roman Shaposhnik commented on LEGAL-503:
----------------------------------------

Hey [~altay] if you would like to continue linking to the Docker release
artifact from the
https://beam.apache.org
 you will have:
   1. Transition to the official ASF dockerhub org:
https://hub.docker.com/u/apache
   2. Start including that binary convenience artifact into your VOTE
threads on Beam releases
   3. Make sure that all Cat-X licenses are ONLY brought into your
container via FROM statements
######## End ########

So at the end of the day, we need to use a container that has Java already
installed in it so that when we create our official image we can inherit
that java containing image via a FROM statement.

- Josh


On Sun, May 17, 2020 at 4:12 PM H W <hu...@gmail.com> wrote:

> We use Centos 7 as well.
>
> On Sun, May 17, 2020 at 10:41 AM Nicholas Nezis <ni...@gmail.com>
> wrote:
>
> > Some thoughts I'd like to add to the discussion:
> >
> > Debian and Ubuntu are somewhat similar. Would it make sense to pick one
> of
> > those and pair with Centos?
> >
> > I've run into issues with Ubuntu images having DNS issues in Kubernetes.
> > https://github.com/kubernetes/kubeadm/issues/787
> >
> > Ubuntu 14.04 should be removed. It was end of life last year. For those
> of
> > you using Heron in production, is anyone using Ubuntu 16? My vote would
> be
> > to remove it also. For what it's worth, my use case is in K8s and we use
> > the Centos7 image on a Centos 7 OS.
> >
> > We should do a test of the official image and helm chart in Kubernetes
> > (maybe with Kind https://kind.sigs.k8s.io/ ) as part of our release
> > process. This might be nice to add as an integration test.
> >
> >
> >
> >
> >
> >
> >
> > On Sun, May 17, 2020, 12:37 PM Ning Wang <wa...@gmail.com> wrote:
> >
> > > That is a good point. We need to adjust the OS version accordingly too.
> > >
> > > On Sun, May 17, 2020 at 2:28 AM Windham Wong <windham.wong@stormeye.io
> >
> > > wrote:
> > >
> > > >
> > > > debian 10 has no python 2.7 support. Ubuntu 16.04 has no python 3.7+
> > dev
> > > > library support. hope this helps
> > > > --
> > > > Sent from myMail for Android Sunday, 17 May 2020, 11:34AM +08:00 from
> > > Ning
> > > > Wang  wangninggm@gmail.com :
> > > >
> > > > >Hi,
> > > > >
> > > > >We have talked about it a few times in different places. Let's make
> a
> > > > >decision here. We don't have enough resources to support many docker
> > > OSes
> > > > >(technically we only support Debian, Ubuntu, and CentOS but each of
> > them
> > > > >has multiple versions).
> > > > >
> > > > >Things we have agreed on (my understanding)
> > > > >- for binary release, we are going to choose one OS one version. I
> > > > >remember that Debian 9 was the decision. An open question is: do we
> > want
> > > > to
> > > > >switch to Debian 10 now? We haven't had a binary release yet, so
> there
> > > is
> > > > >still time. As the main target, I think 1. the image should compile
> > > > >successfully. and 2 the image should be tested: the tools and an
> > example
> > > > >topology.
> > > > >
> > > > >- for source release, we haven't talked about it very much.
> > Personally I
> > > > >feel it is a reasonable expectation that, if we have the Docker file
> > in
> > > > the
> > > > >core folders, it should at least compile. Otherwise, it's a failed
> > build
> > > > >hence a failed release. We are not going to test run the tools and
> > > > examples
> > > > >for each release though.
> > > > >
> > > > >Finally, a question about OS versions. I believe at least two
> releases
> > > > >should be supported because most people don't upgrade to the latest
> > > > version
> > > > >when it is out in more serious scenarios, hence IMO the second
> oldest
> > > > could
> > > > >often be more useful than the latest one from a convenience point of
> > > view.
> > > > >On the other hand, this is too flexible, and we will spend time
> > thinking
> > > > >about it again. I think a clear guideline could be helpful to us.
> For
> > > > >example:
> > > > >- We only include only Debian (popular in the server world) and
> Ubuntu
> > > > >(popular in servers and workstations).
> > > > >- for the main OS (Debian) we choose the latest to compile and test
> > for
> > > > our
> > > > >binary release.
> > > > >- for a given OS, only two versions (most likely the most and the
> > second
> > > > >recent releases) are included in the core source code and they need
> to
> > > be
> > > > >fixed ASAP if they don't compile.
> > > > >- OSs and other versions could be included in a special folder and
> but
> > > > >there is no guarantee that the code will compile. Fixes from the
> > > community
> > > > >are welcome.
> > > > >
> > > > >
> > > > >How do you guys think?
> > > > >--ning
> > > >
> > >
> >
>

Re: [DISCUSSION] Docker OS support

[H W <hu...@gmail.com>]

We use Centos 7 as well.

On Sun, May 17, 2020 at 10:41 AM Nicholas Nezis <ni...@gmail.com>
wrote:

> Some thoughts I'd like to add to the discussion:
>
> Debian and Ubuntu are somewhat similar. Would it make sense to pick one of
> those and pair with Centos?
>
> I've run into issues with Ubuntu images having DNS issues in Kubernetes.
> https://github.com/kubernetes/kubeadm/issues/787
>
> Ubuntu 14.04 should be removed. It was end of life last year. For those of
> you using Heron in production, is anyone using Ubuntu 16? My vote would be
> to remove it also. For what it's worth, my use case is in K8s and we use
> the Centos7 image on a Centos 7 OS.
>
> We should do a test of the official image and helm chart in Kubernetes
> (maybe with Kind https://kind.sigs.k8s.io/ ) as part of our release
> process. This might be nice to add as an integration test.
>
>
>
>
>
>
>
> On Sun, May 17, 2020, 12:37 PM Ning Wang <wa...@gmail.com> wrote:
>
> > That is a good point. We need to adjust the OS version accordingly too.
> >
> > On Sun, May 17, 2020 at 2:28 AM Windham Wong <wi...@stormeye.io>
> > wrote:
> >
> > >
> > > debian 10 has no python 2.7 support. Ubuntu 16.04 has no python 3.7+
> dev
> > > library support. hope this helps
> > > --
> > > Sent from myMail for Android Sunday, 17 May 2020, 11:34AM +08:00 from
> > Ning
> > > Wang  wangninggm@gmail.com :
> > >
> > > >Hi,
> > > >
> > > >We have talked about it a few times in different places. Let's make a
> > > >decision here. We don't have enough resources to support many docker
> > OSes
> > > >(technically we only support Debian, Ubuntu, and CentOS but each of
> them
> > > >has multiple versions).
> > > >
> > > >Things we have agreed on (my understanding)
> > > >- for binary release, we are going to choose one OS one version. I
> > > >remember that Debian 9 was the decision. An open question is: do we
> want
> > > to
> > > >switch to Debian 10 now? We haven't had a binary release yet, so there
> > is
> > > >still time. As the main target, I think 1. the image should compile
> > > >successfully. and 2 the image should be tested: the tools and an
> example
> > > >topology.
> > > >
> > > >- for source release, we haven't talked about it very much.
> Personally I
> > > >feel it is a reasonable expectation that, if we have the Docker file
> in
> > > the
> > > >core folders, it should at least compile. Otherwise, it's a failed
> build
> > > >hence a failed release. We are not going to test run the tools and
> > > examples
> > > >for each release though.
> > > >
> > > >Finally, a question about OS versions. I believe at least two releases
> > > >should be supported because most people don't upgrade to the latest
> > > version
> > > >when it is out in more serious scenarios, hence IMO the second oldest
> > > could
> > > >often be more useful than the latest one from a convenience point of
> > view.
> > > >On the other hand, this is too flexible, and we will spend time
> thinking
> > > >about it again. I think a clear guideline could be helpful to us. For
> > > >example:
> > > >- We only include only Debian (popular in the server world) and Ubuntu
> > > >(popular in servers and workstations).
> > > >- for the main OS (Debian) we choose the latest to compile and test
> for
> > > our
> > > >binary release.
> > > >- for a given OS, only two versions (most likely the most and the
> second
> > > >recent releases) are included in the core source code and they need to
> > be
> > > >fixed ASAP if they don't compile.
> > > >- OSs and other versions could be included in a special folder and but
> > > >there is no guarantee that the code will compile. Fixes from the
> > community
> > > >are welcome.
> > > >
> > > >
> > > >How do you guys think?
> > > >--ning
> > >
> >
>

Re: [DISCUSSION] Docker OS support

[Nicholas Nezis <ni...@gmail.com>]

Some thoughts I'd like to add to the discussion:

Debian and Ubuntu are somewhat similar. Would it make sense to pick one of
those and pair with Centos?

I've run into issues with Ubuntu images having DNS issues in Kubernetes.
https://github.com/kubernetes/kubeadm/issues/787

Ubuntu 14.04 should be removed. It was end of life last year. For those of
you using Heron in production, is anyone using Ubuntu 16? My vote would be
to remove it also. For what it's worth, my use case is in K8s and we use
the Centos7 image on a Centos 7 OS.

We should do a test of the official image and helm chart in Kubernetes
(maybe with Kind https://kind.sigs.k8s.io/ ) as part of our release
process. This might be nice to add as an integration test.







On Sun, May 17, 2020, 12:37 PM Ning Wang <wa...@gmail.com> wrote:

> That is a good point. We need to adjust the OS version accordingly too.
>
> On Sun, May 17, 2020 at 2:28 AM Windham Wong <wi...@stormeye.io>
> wrote:
>
> >
> > debian 10 has no python 2.7 support. Ubuntu 16.04 has no python 3.7+ dev
> > library support. hope this helps
> > --
> > Sent from myMail for Android Sunday, 17 May 2020, 11:34AM +08:00 from
> Ning
> > Wang  wangninggm@gmail.com :
> >
> > >Hi,
> > >
> > >We have talked about it a few times in different places. Let's make a
> > >decision here. We don't have enough resources to support many docker
> OSes
> > >(technically we only support Debian, Ubuntu, and CentOS but each of them
> > >has multiple versions).
> > >
> > >Things we have agreed on (my understanding)
> > >- for binary release, we are going to choose one OS one version. I
> > >remember that Debian 9 was the decision. An open question is: do we want
> > to
> > >switch to Debian 10 now? We haven't had a binary release yet, so there
> is
> > >still time. As the main target, I think 1. the image should compile
> > >successfully. and 2 the image should be tested: the tools and an example
> > >topology.
> > >
> > >- for source release, we haven't talked about it very much. Personally I
> > >feel it is a reasonable expectation that, if we have the Docker file in
> > the
> > >core folders, it should at least compile. Otherwise, it's a failed build
> > >hence a failed release. We are not going to test run the tools and
> > examples
> > >for each release though.
> > >
> > >Finally, a question about OS versions. I believe at least two releases
> > >should be supported because most people don't upgrade to the latest
> > version
> > >when it is out in more serious scenarios, hence IMO the second oldest
> > could
> > >often be more useful than the latest one from a convenience point of
> view.
> > >On the other hand, this is too flexible, and we will spend time thinking
> > >about it again. I think a clear guideline could be helpful to us. For
> > >example:
> > >- We only include only Debian (popular in the server world) and Ubuntu
> > >(popular in servers and workstations).
> > >- for the main OS (Debian) we choose the latest to compile and test for
> > our
> > >binary release.
> > >- for a given OS, only two versions (most likely the most and the second
> > >recent releases) are included in the core source code and they need to
> be
> > >fixed ASAP if they don't compile.
> > >- OSs and other versions could be included in a special folder and but
> > >there is no guarantee that the code will compile. Fixes from the
> community
> > >are welcome.
> > >
> > >
> > >How do you guys think?
> > >--ning
> >
>

Re: [DISCUSSION] Docker OS support

[Ning Wang <wa...@gmail.com>]

That is a good point. We need to adjust the OS version accordingly too.

On Sun, May 17, 2020 at 2:28 AM Windham Wong <wi...@stormeye.io>
wrote:

>
> debian 10 has no python 2.7 support. Ubuntu 16.04 has no python 3.7+ dev
> library support. hope this helps
> --
> Sent from myMail for Android Sunday, 17 May 2020, 11:34AM +08:00 from Ning
> Wang  wangninggm@gmail.com :
>
> >Hi,
> >
> >We have talked about it a few times in different places. Let's make a
> >decision here. We don't have enough resources to support many docker OSes
> >(technically we only support Debian, Ubuntu, and CentOS but each of them
> >has multiple versions).
> >
> >Things we have agreed on (my understanding)
> >- for binary release, we are going to choose one OS one version. I
> >remember that Debian 9 was the decision. An open question is: do we want
> to
> >switch to Debian 10 now? We haven't had a binary release yet, so there is
> >still time. As the main target, I think 1. the image should compile
> >successfully. and 2 the image should be tested: the tools and an example
> >topology.
> >
> >- for source release, we haven't talked about it very much. Personally I
> >feel it is a reasonable expectation that, if we have the Docker file in
> the
> >core folders, it should at least compile. Otherwise, it's a failed build
> >hence a failed release. We are not going to test run the tools and
> examples
> >for each release though.
> >
> >Finally, a question about OS versions. I believe at least two releases
> >should be supported because most people don't upgrade to the latest
> version
> >when it is out in more serious scenarios, hence IMO the second oldest
> could
> >often be more useful than the latest one from a convenience point of view.
> >On the other hand, this is too flexible, and we will spend time thinking
> >about it again. I think a clear guideline could be helpful to us. For
> >example:
> >- We only include only Debian (popular in the server world) and Ubuntu
> >(popular in servers and workstations).
> >- for the main OS (Debian) we choose the latest to compile and test for
> our
> >binary release.
> >- for a given OS, only two versions (most likely the most and the second
> >recent releases) are included in the core source code and they need to be
> >fixed ASAP if they don't compile.
> >- OSs and other versions could be included in a special folder and but
> >there is no guarantee that the code will compile. Fixes from the community
> >are welcome.
> >
> >
> >How do you guys think?
> >--ning
>

Re: [DISCUSSION] Docker OS support

[Windham Wong <wi...@stormeye.io>]

debian 10 has no python 2.7 support. Ubuntu 16.04 has no python 3.7+ dev library support. hope this helps
--
Sent from myMail for Android Sunday, 17 May 2020, 11:34AM +08:00 from Ning Wang  wangninggm@gmail.com :

>Hi,
>
>We have talked about it a few times in different places. Let's make a
>decision here. We don't have enough resources to support many docker OSes
>(technically we only support Debian, Ubuntu, and CentOS but each of them
>has multiple versions).
>
>Things we have agreed on (my understanding)
>- for binary release, we are going to choose one OS one version. I
>remember that Debian 9 was the decision. An open question is: do we want to
>switch to Debian 10 now? We haven't had a binary release yet, so there is
>still time. As the main target, I think 1. the image should compile
>successfully. and 2 the image should be tested: the tools and an example
>topology.
>
>- for source release, we haven't talked about it very much. Personally I
>feel it is a reasonable expectation that, if we have the Docker file in the
>core folders, it should at least compile. Otherwise, it's a failed build
>hence a failed release. We are not going to test run the tools and examples
>for each release though.
>
>Finally, a question about OS versions. I believe at least two releases
>should be supported because most people don't upgrade to the latest version
>when it is out in more serious scenarios, hence IMO the second oldest could
>often be more useful than the latest one from a convenience point of view.
>On the other hand, this is too flexible, and we will spend time thinking
>about it again. I think a clear guideline could be helpful to us. For
>example:
>- We only include only Debian (popular in the server world) and Ubuntu
>(popular in servers and workstations).
>- for the main OS (Debian) we choose the latest to compile and test for our
>binary release.
>- for a given OS, only two versions (most likely the most and the second
>recent releases) are included in the core source code and they need to be
>fixed ASAP if they don't compile.
>- OSs and other versions could be included in a special folder and but
>there is no guarantee that the code will compile. Fixes from the community
>are welcome.
>
>
>How do you guys think?
>--ning