You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Jarek Potiuk <Ja...@polidea.com> on 2020/07/11 18:43:38 UTC
Release cadence of the Docker Images
Hello Everyone,
Before we agree on release cadence of the Charts I have a proposal for Prod
images. They have not been released yesterday with 1.10.11.
The relevant Dockerfile changes are part of the officially voted and
released sources and those sources are enough to build the image and push
it to the registry. In fact - I am already doing so.
I think this is perfectly fine with ASF release policy that we release the
images without extra voting in such case (it's built using the released
sources): http://www.apache.org/legal/release-policy.html#compiled-packages
I think we should continue this way and do not introduce separate cadence
to release the docker images - unless we decide to make an ad-hoc release
at some point.
I believe for the image (unlike for the Helm Chart), such ad-hoc releases
are not really possible. We will test the future Helm chart against
those released versions of images, and we should maintain backward
compatibility.
With my proposal (which I am going to talk about next week at my
"Production Image talk" and ask for feedback from the users) that if
someone wants to customize the image, they should rebuild the image on
their own using either stable (if cautions) or master (if adventurous)
Dockerfile. This way they can get the image much better size-optimized.
With the latest Breeze release, it's super-easy to build your own image
using our Dockerfile - with your own dependencies both on Python. apt dev
and apt runtime dependencies.
J.
--
Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer
M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>
Re: Release cadence of the Docker Images
Posted by Jarek Potiuk <Ja...@polidea.com>.
>
> Curious as to why 3.6 is the default, not 3.7?
No strong reasoning. It's the minimum version required (f-strings) and
I thought that should be the default version then. But I am happy to
change it to 3.7 if we think it's better. My original line of thought
was that Airlfow is mostly used in corporate environments which are
usually pretty conservative (And if you watched the "How large
companies are using Airflow" Airflow Summit Keynote
(https://airflowsummit.org/sessions/how-airbnb-twitter-lyft-use-airflow/)
som of even "startup born" companies are still struggling with
migrating out of 2.7. So I thought 3.6 is rather "safe bet" for
default.
But I think it would be great to have some reasoning/rule which
version we make as default and stick to it now (and update in the
future).
Anyone has some thoughts on why we could decide some of those (3.6,
3.7 or even 3.8) is the default one ?
--
Jarek Potiuk
Polidea | Principal Software Engineer
M: +48 660 796 129
Re: Release cadence of the Docker Images
Posted by Ash Berlin-Taylor <as...@apache.org>.
Curious as to why 3.6 is the default, not 3.7?
-a
On Jul 12 2020, at 9:22 am, Jarek Potiuk <Ja...@polidea.com> wrote:
> FYI -> the images for 1.10.11 are in DockerHub now. They also include
> the "latest" tag - which means that anyone running `docker pull
> apache/airflow` will get the python3.6 variant of 1.10.11 version of
> image. Also apache/airflow:1.10.11 points to 3.6 variant.
>
> J.
>
>
>
>
>> On Sat, Jul 11, 2020 at 8:43 PM Jarek Potiuk
>> <Ja...@polidea.com> wrote:
>>
>>> Hello Everyone,
>>>
>>> Before we agree on release cadence of the Charts I have a proposal
>>> for Prod images. They have not been released yesterday with 1.10.11.
>>>
>>> The relevant Dockerfile changes are part of the officially voted and
>>> released sources and those sources are enough to build the image and
>>> push it to the registry. In fact - I am already doing so.
>>>
>>> I think this is perfectly fine with ASF release policy that we
>>> release the images without extra voting in such case (it's built
>>> using the released sources): http://www.apache.org/legal/release-policy.html#compiled-packages
>>>
>>> I think we should continue this way and do not introduce separate
>>> cadence to release the docker images - unless we decide to make an
>>> ad-hoc release at some point.
>>>
>>> I believe for the image (unlike for the Helm Chart), such ad-hoc
>>> releases are not really possible. We will test the future Helm chart
>>> against those released versions of images, and we should maintain
>>> backward compatibility.
>>>
>>> With my proposal (which I am going to talk about next week at my
>>> "Production Image talk" and ask for feedback from the users) that if
>>> someone wants to customize the image, they should rebuild the image
>>> on their own using either stable (if cautions) or master (if
>>> adventurous) Dockerfile. This way they can get the image much better
>>> size-optimized. With the latest Breeze release, it's super-easy to
>>> build your own image using our Dockerfile - with your own
>>> dependencies both on Python. apt dev and apt runtime dependencies.
>>>
>>> J.
>>>
>>> --
>>> Jarek Potiuk
>>> Polidea | Principal Software Engineer
>>> M: +48 660 796 129
>>>
>
>
> --
>
> Jarek Potiuk
> Polidea | Principal Software Engineer
> M: +48 660 796 129
Re: Release cadence of the Docker Images
Posted by Zikun Zhu <ku...@gmail.com>.
Great. Thanks!
On Tue, 14 Jul 2020 at 15:21, Jarek Potiuk <Ja...@polidea.com> wrote:
> The image for 3.8 is pushed now. I missed the update of 3.8 in 1.10.11 :)
>
> On Mon, Jul 13, 2020 at 3:55 PM Jarek Potiuk <Ja...@polidea.com>
> wrote:
>
> > BTW. For those interested - here is a sneak peek into the presentation I
> > am going to run tomorrow (or today depending on where you are) at the
> > Airflow Summit:
> >
> > *Production Docker Image for Apache Airflow*
> >
> > From the sneak peek you can find:
> > * what questions will be answered
> > * what will not be covered by the presentation
> > * who the talk is for
> >
> > The talk is Tuesday mid-day Tokio, early morning EU and late evening
> today
> > in US. So watch the time-zone (but it is recorded and will be available
> to
> > watch pretty much immediately after the talk).
> >
> > Link to the talk here:
> > https://airflowsummit.org/sessions/production-docker-image/
> >
> > And sneak peek of the presentation here:
> >
> >
> https://docs.google.com/presentation/d/1iyIdxihGl87cL8Llvppwla02fikxVYGsH7pHToEZ9A8/edit#slide=id.g58a0b9556c_0_1144
> >
> > J.
> >
> >
> > On Mon, Jul 13, 2020 at 3:46 PM Jarek Potiuk <Ja...@polidea.com>
> > wrote:
> >
> >>
> >>>
> >>> "Add Support for Python 3.8 (#8836
> >>> <https://github.com/apache/airflow/pull/8836>)(#8823
> >>> <https://github.com/apache/airflow/pull/8823>)" is in the changelog
> but
> >>> seems like *1.10.11-python3.8* tag is missing in DockerHub?
> >>>
> >>
> >> We do not support Python 3.8 in 1.10.* line:
> >>
> >> https://github.com/apache/airflow/blob/master/README.md#requirements
> >>
> >> J.
> >>
> >>
> >>
> >>> On Sun, 12 Jul 2020 at 16:22, Jarek Potiuk <Ja...@polidea.com>
> >>> wrote:
> >>>
> >>> > FYI -> the images for 1.10.11 are in DockerHub now. They also include
> >>> the
> >>> > "latest" tag - which means that anyone running `*docker pull
> >>> > apache/airflow*` will get the python3.6 variant of 1.10.11 version of
> >>> > image. Also *apache/airflow:1.10.11* points to 3.6 variant.
> >>> >
> >>> > J.
> >>> >
> >>> > [image: Screenshot from 2020-07-12 10-19-23.png]
> >>> >
> >>> > On Sat, Jul 11, 2020 at 8:43 PM Jarek Potiuk <
> Jarek.Potiuk@polidea.com
> >>> >
> >>> > wrote:
> >>> >
> >>> >> Hello Everyone,
> >>> >>
> >>> >> Before we agree on release cadence of the Charts I have a proposal
> for
> >>> >> Prod images. They have not been released yesterday with 1.10.11.
> >>> >>
> >>> >> The relevant Dockerfile changes are part of the officially voted and
> >>> >> released sources and those sources are enough to build the image and
> >>> push
> >>> >> it to the registry. In fact - I am already doing so.
> >>> >>
> >>> >> I think this is perfectly fine with ASF release policy that we
> release
> >>> >> the images without extra voting in such case (it's built using the
> >>> released
> >>> >> sources):
> >>> >> http://www.apache.org/legal/release-policy.html#compiled-packages
> >>> >>
> >>> >> I think we should continue this way and do not introduce separate
> >>> cadence
> >>> >> to release the docker images - unless we decide to make an ad-hoc
> >>> release
> >>> >> at some point.
> >>> >>
> >>> >> I believe for the image (unlike for the Helm Chart), such ad-hoc
> >>> releases
> >>> >> are not really possible. We will test the future Helm chart against
> >>> >> those released versions of images, and we should maintain backward
> >>> >> compatibility.
> >>> >>
> >>> >> With my proposal (which I am going to talk about next week at my
> >>> >> "Production Image talk" and ask for feedback from the users) that if
> >>> >> someone wants to customize the image, they should rebuild the image
> on
> >>> >> their own using either stable (if cautions) or master (if
> adventurous)
> >>> >> Dockerfile. This way they can get the image much better
> >>> size-optimized.
> >>> >> With the latest Breeze release, it's super-easy to build your own
> >>> image
> >>> >> using our Dockerfile - with your own dependencies both on Python.
> apt
> >>> dev
> >>> >> and apt runtime dependencies.
> >>> >>
> >>> >> J.
> >>> >>
> >>> >> --
> >>> >>
> >>> >> Jarek Potiuk
> >>> >> Polidea <https://www.polidea.com/> | Principal Software Engineer
> >>> >>
> >>> >> M: +48 660 796 129 <+48660796129>
> >>> >> [image: Polidea] <https://www.polidea.com/>
> >>> >>
> >>> >>
> >>> >
> >>> > --
> >>> >
> >>> > Jarek Potiuk
> >>> > Polidea <https://www.polidea.com/> | Principal Software Engineer
> >>> >
> >>> > M: +48 660 796 129 <+48660796129>
> >>> > [image: Polidea] <https://www.polidea.com/>
> >>> >
> >>> >
> >>>
> >>
> >>
> >> --
> >>
> >> Jarek Potiuk
> >> Polidea <https://www.polidea.com/> | Principal Software Engineer
> >>
> >> M: +48 660 796 129 <+48660796129>
> >> [image: Polidea] <https://www.polidea.com/>
> >>
> >>
> >
> > --
> >
> > Jarek Potiuk
> > Polidea <https://www.polidea.com/> | Principal Software Engineer
> >
> > M: +48 660 796 129 <+48660796129>
> > [image: Polidea] <https://www.polidea.com/>
> >
> >
>
> --
>
> Jarek Potiuk
> Polidea <https://www.polidea.com/> | Principal Software Engineer
>
> M: +48 660 796 129 <+48660796129>
> [image: Polidea] <https://www.polidea.com/>
>
Re: Release cadence of the Docker Images
Posted by Jarek Potiuk <Ja...@polidea.com>.
The image for 3.8 is pushed now. I missed the update of 3.8 in 1.10.11 :)
On Mon, Jul 13, 2020 at 3:55 PM Jarek Potiuk <Ja...@polidea.com>
wrote:
> BTW. For those interested - here is a sneak peek into the presentation I
> am going to run tomorrow (or today depending on where you are) at the
> Airflow Summit:
>
> *Production Docker Image for Apache Airflow*
>
> From the sneak peek you can find:
> * what questions will be answered
> * what will not be covered by the presentation
> * who the talk is for
>
> The talk is Tuesday mid-day Tokio, early morning EU and late evening today
> in US. So watch the time-zone (but it is recorded and will be available to
> watch pretty much immediately after the talk).
>
> Link to the talk here:
> https://airflowsummit.org/sessions/production-docker-image/
>
> And sneak peek of the presentation here:
>
> https://docs.google.com/presentation/d/1iyIdxihGl87cL8Llvppwla02fikxVYGsH7pHToEZ9A8/edit#slide=id.g58a0b9556c_0_1144
>
> J.
>
>
> On Mon, Jul 13, 2020 at 3:46 PM Jarek Potiuk <Ja...@polidea.com>
> wrote:
>
>>
>>>
>>> "Add Support for Python 3.8 (#8836
>>> <https://github.com/apache/airflow/pull/8836>)(#8823
>>> <https://github.com/apache/airflow/pull/8823>)" is in the changelog but
>>> seems like *1.10.11-python3.8* tag is missing in DockerHub?
>>>
>>
>> We do not support Python 3.8 in 1.10.* line:
>>
>> https://github.com/apache/airflow/blob/master/README.md#requirements
>>
>> J.
>>
>>
>>
>>> On Sun, 12 Jul 2020 at 16:22, Jarek Potiuk <Ja...@polidea.com>
>>> wrote:
>>>
>>> > FYI -> the images for 1.10.11 are in DockerHub now. They also include
>>> the
>>> > "latest" tag - which means that anyone running `*docker pull
>>> > apache/airflow*` will get the python3.6 variant of 1.10.11 version of
>>> > image. Also *apache/airflow:1.10.11* points to 3.6 variant.
>>> >
>>> > J.
>>> >
>>> > [image: Screenshot from 2020-07-12 10-19-23.png]
>>> >
>>> > On Sat, Jul 11, 2020 at 8:43 PM Jarek Potiuk <Jarek.Potiuk@polidea.com
>>> >
>>> > wrote:
>>> >
>>> >> Hello Everyone,
>>> >>
>>> >> Before we agree on release cadence of the Charts I have a proposal for
>>> >> Prod images. They have not been released yesterday with 1.10.11.
>>> >>
>>> >> The relevant Dockerfile changes are part of the officially voted and
>>> >> released sources and those sources are enough to build the image and
>>> push
>>> >> it to the registry. In fact - I am already doing so.
>>> >>
>>> >> I think this is perfectly fine with ASF release policy that we release
>>> >> the images without extra voting in such case (it's built using the
>>> released
>>> >> sources):
>>> >> http://www.apache.org/legal/release-policy.html#compiled-packages
>>> >>
>>> >> I think we should continue this way and do not introduce separate
>>> cadence
>>> >> to release the docker images - unless we decide to make an ad-hoc
>>> release
>>> >> at some point.
>>> >>
>>> >> I believe for the image (unlike for the Helm Chart), such ad-hoc
>>> releases
>>> >> are not really possible. We will test the future Helm chart against
>>> >> those released versions of images, and we should maintain backward
>>> >> compatibility.
>>> >>
>>> >> With my proposal (which I am going to talk about next week at my
>>> >> "Production Image talk" and ask for feedback from the users) that if
>>> >> someone wants to customize the image, they should rebuild the image on
>>> >> their own using either stable (if cautions) or master (if adventurous)
>>> >> Dockerfile. This way they can get the image much better
>>> size-optimized.
>>> >> With the latest Breeze release, it's super-easy to build your own
>>> image
>>> >> using our Dockerfile - with your own dependencies both on Python. apt
>>> dev
>>> >> and apt runtime dependencies.
>>> >>
>>> >> J.
>>> >>
>>> >> --
>>> >>
>>> >> Jarek Potiuk
>>> >> Polidea <https://www.polidea.com/> | Principal Software Engineer
>>> >>
>>> >> M: +48 660 796 129 <+48660796129>
>>> >> [image: Polidea] <https://www.polidea.com/>
>>> >>
>>> >>
>>> >
>>> > --
>>> >
>>> > Jarek Potiuk
>>> > Polidea <https://www.polidea.com/> | Principal Software Engineer
>>> >
>>> > M: +48 660 796 129 <+48660796129>
>>> > [image: Polidea] <https://www.polidea.com/>
>>> >
>>> >
>>>
>>
>>
>> --
>>
>> Jarek Potiuk
>> Polidea <https://www.polidea.com/> | Principal Software Engineer
>>
>> M: +48 660 796 129 <+48660796129>
>> [image: Polidea] <https://www.polidea.com/>
>>
>>
>
> --
>
> Jarek Potiuk
> Polidea <https://www.polidea.com/> | Principal Software Engineer
>
> M: +48 660 796 129 <+48660796129>
> [image: Polidea] <https://www.polidea.com/>
>
>
--
Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer
M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>
Re: Release cadence of the Docker Images
Posted by Jarek Potiuk <Ja...@polidea.com>.
BTW. For those interested - here is a sneak peek into the presentation I am
going to run tomorrow (or today depending on where you are) at the Airflow
Summit:
*Production Docker Image for Apache Airflow*
From the sneak peek you can find:
* what questions will be answered
* what will not be covered by the presentation
* who the talk is for
The talk is Tuesday mid-day Tokio, early morning EU and late evening today
in US. So watch the time-zone (but it is recorded and will be available to
watch pretty much immediately after the talk).
Link to the talk here:
https://airflowsummit.org/sessions/production-docker-image/
And sneak peek of the presentation here:
https://docs.google.com/presentation/d/1iyIdxihGl87cL8Llvppwla02fikxVYGsH7pHToEZ9A8/edit#slide=id.g58a0b9556c_0_1144
J.
On Mon, Jul 13, 2020 at 3:46 PM Jarek Potiuk <Ja...@polidea.com>
wrote:
>
>>
>> "Add Support for Python 3.8 (#8836
>> <https://github.com/apache/airflow/pull/8836>)(#8823
>> <https://github.com/apache/airflow/pull/8823>)" is in the changelog but
>> seems like *1.10.11-python3.8* tag is missing in DockerHub?
>>
>
> We do not support Python 3.8 in 1.10.* line:
>
> https://github.com/apache/airflow/blob/master/README.md#requirements
>
> J.
>
>
>
>> On Sun, 12 Jul 2020 at 16:22, Jarek Potiuk <Ja...@polidea.com>
>> wrote:
>>
>> > FYI -> the images for 1.10.11 are in DockerHub now. They also include
>> the
>> > "latest" tag - which means that anyone running `*docker pull
>> > apache/airflow*` will get the python3.6 variant of 1.10.11 version of
>> > image. Also *apache/airflow:1.10.11* points to 3.6 variant.
>> >
>> > J.
>> >
>> > [image: Screenshot from 2020-07-12 10-19-23.png]
>> >
>> > On Sat, Jul 11, 2020 at 8:43 PM Jarek Potiuk <Ja...@polidea.com>
>> > wrote:
>> >
>> >> Hello Everyone,
>> >>
>> >> Before we agree on release cadence of the Charts I have a proposal for
>> >> Prod images. They have not been released yesterday with 1.10.11.
>> >>
>> >> The relevant Dockerfile changes are part of the officially voted and
>> >> released sources and those sources are enough to build the image and
>> push
>> >> it to the registry. In fact - I am already doing so.
>> >>
>> >> I think this is perfectly fine with ASF release policy that we release
>> >> the images without extra voting in such case (it's built using the
>> released
>> >> sources):
>> >> http://www.apache.org/legal/release-policy.html#compiled-packages
>> >>
>> >> I think we should continue this way and do not introduce separate
>> cadence
>> >> to release the docker images - unless we decide to make an ad-hoc
>> release
>> >> at some point.
>> >>
>> >> I believe for the image (unlike for the Helm Chart), such ad-hoc
>> releases
>> >> are not really possible. We will test the future Helm chart against
>> >> those released versions of images, and we should maintain backward
>> >> compatibility.
>> >>
>> >> With my proposal (which I am going to talk about next week at my
>> >> "Production Image talk" and ask for feedback from the users) that if
>> >> someone wants to customize the image, they should rebuild the image on
>> >> their own using either stable (if cautions) or master (if adventurous)
>> >> Dockerfile. This way they can get the image much better size-optimized.
>> >> With the latest Breeze release, it's super-easy to build your own image
>> >> using our Dockerfile - with your own dependencies both on Python. apt
>> dev
>> >> and apt runtime dependencies.
>> >>
>> >> J.
>> >>
>> >> --
>> >>
>> >> Jarek Potiuk
>> >> Polidea <https://www.polidea.com/> | Principal Software Engineer
>> >>
>> >> M: +48 660 796 129 <+48660796129>
>> >> [image: Polidea] <https://www.polidea.com/>
>> >>
>> >>
>> >
>> > --
>> >
>> > Jarek Potiuk
>> > Polidea <https://www.polidea.com/> | Principal Software Engineer
>> >
>> > M: +48 660 796 129 <+48660796129>
>> > [image: Polidea] <https://www.polidea.com/>
>> >
>> >
>>
>
>
> --
>
> Jarek Potiuk
> Polidea <https://www.polidea.com/> | Principal Software Engineer
>
> M: +48 660 796 129 <+48660796129>
> [image: Polidea] <https://www.polidea.com/>
>
>
--
Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer
M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>
Re: Release cadence of the Docker Images
Posted by Kaxil Naik <ka...@gmail.com>.
Whoops I forgot to update that line on Master but we were running tests on
Py 3.8 too.
Example: https://github.com/apache/airflow/runs/858692643
https://github.com/apache/airflow/tree/v1-10-stable#stable-version
On Mon, Jul 13, 2020 at 2:47 PM Jarek Potiuk <Ja...@polidea.com>
wrote:
> >
> >
> > "Add Support for Python 3.8 (#8836
> > <https://github.com/apache/airflow/pull/8836>)(#8823
> > <https://github.com/apache/airflow/pull/8823>)" is in the changelog but
> > seems like *1.10.11-python3.8* tag is missing in DockerHub?
> >
>
> We do not support Python 3.8 in 1.10.* line:
>
> https://github.com/apache/airflow/blob/master/README.md#requirements
>
> J.
>
>
>
> > On Sun, 12 Jul 2020 at 16:22, Jarek Potiuk <Ja...@polidea.com>
> > wrote:
> >
> > > FYI -> the images for 1.10.11 are in DockerHub now. They also include
> the
> > > "latest" tag - which means that anyone running `*docker pull
> > > apache/airflow*` will get the python3.6 variant of 1.10.11 version of
> > > image. Also *apache/airflow:1.10.11* points to 3.6 variant.
> > >
> > > J.
> > >
> > > [image: Screenshot from 2020-07-12 10-19-23.png]
> > >
> > > On Sat, Jul 11, 2020 at 8:43 PM Jarek Potiuk <Jarek.Potiuk@polidea.com
> >
> > > wrote:
> > >
> > >> Hello Everyone,
> > >>
> > >> Before we agree on release cadence of the Charts I have a proposal for
> > >> Prod images. They have not been released yesterday with 1.10.11.
> > >>
> > >> The relevant Dockerfile changes are part of the officially voted and
> > >> released sources and those sources are enough to build the image and
> > push
> > >> it to the registry. In fact - I am already doing so.
> > >>
> > >> I think this is perfectly fine with ASF release policy that we release
> > >> the images without extra voting in such case (it's built using the
> > released
> > >> sources):
> > >> http://www.apache.org/legal/release-policy.html#compiled-packages
> > >>
> > >> I think we should continue this way and do not introduce separate
> > cadence
> > >> to release the docker images - unless we decide to make an ad-hoc
> > release
> > >> at some point.
> > >>
> > >> I believe for the image (unlike for the Helm Chart), such ad-hoc
> > releases
> > >> are not really possible. We will test the future Helm chart against
> > >> those released versions of images, and we should maintain backward
> > >> compatibility.
> > >>
> > >> With my proposal (which I am going to talk about next week at my
> > >> "Production Image talk" and ask for feedback from the users) that if
> > >> someone wants to customize the image, they should rebuild the image on
> > >> their own using either stable (if cautions) or master (if adventurous)
> > >> Dockerfile. This way they can get the image much better
> size-optimized.
> > >> With the latest Breeze release, it's super-easy to build your own
> image
> > >> using our Dockerfile - with your own dependencies both on Python. apt
> > dev
> > >> and apt runtime dependencies.
> > >>
> > >> J.
> > >>
> > >> --
> > >>
> > >> Jarek Potiuk
> > >> Polidea <https://www.polidea.com/> | Principal Software Engineer
> > >>
> > >> M: +48 660 796 129 <+48660796129>
> > >> [image: Polidea] <https://www.polidea.com/>
> > >>
> > >>
> > >
> > > --
> > >
> > > Jarek Potiuk
> > > Polidea <https://www.polidea.com/> | Principal Software Engineer
> > >
> > > M: +48 660 796 129 <+48660796129>
> > > [image: Polidea] <https://www.polidea.com/>
> > >
> > >
> >
>
>
> --
>
> Jarek Potiuk
> Polidea <https://www.polidea.com/> | Principal Software Engineer
>
> M: +48 660 796 129 <+48660796129>
> [image: Polidea] <https://www.polidea.com/>
>
Re: Release cadence of the Docker Images
Posted by Jarek Potiuk <Ja...@polidea.com>.
>
>
> "Add Support for Python 3.8 (#8836
> <https://github.com/apache/airflow/pull/8836>)(#8823
> <https://github.com/apache/airflow/pull/8823>)" is in the changelog but
> seems like *1.10.11-python3.8* tag is missing in DockerHub?
>
We do not support Python 3.8 in 1.10.* line:
https://github.com/apache/airflow/blob/master/README.md#requirements
J.
> On Sun, 12 Jul 2020 at 16:22, Jarek Potiuk <Ja...@polidea.com>
> wrote:
>
> > FYI -> the images for 1.10.11 are in DockerHub now. They also include the
> > "latest" tag - which means that anyone running `*docker pull
> > apache/airflow*` will get the python3.6 variant of 1.10.11 version of
> > image. Also *apache/airflow:1.10.11* points to 3.6 variant.
> >
> > J.
> >
> > [image: Screenshot from 2020-07-12 10-19-23.png]
> >
> > On Sat, Jul 11, 2020 at 8:43 PM Jarek Potiuk <Ja...@polidea.com>
> > wrote:
> >
> >> Hello Everyone,
> >>
> >> Before we agree on release cadence of the Charts I have a proposal for
> >> Prod images. They have not been released yesterday with 1.10.11.
> >>
> >> The relevant Dockerfile changes are part of the officially voted and
> >> released sources and those sources are enough to build the image and
> push
> >> it to the registry. In fact - I am already doing so.
> >>
> >> I think this is perfectly fine with ASF release policy that we release
> >> the images without extra voting in such case (it's built using the
> released
> >> sources):
> >> http://www.apache.org/legal/release-policy.html#compiled-packages
> >>
> >> I think we should continue this way and do not introduce separate
> cadence
> >> to release the docker images - unless we decide to make an ad-hoc
> release
> >> at some point.
> >>
> >> I believe for the image (unlike for the Helm Chart), such ad-hoc
> releases
> >> are not really possible. We will test the future Helm chart against
> >> those released versions of images, and we should maintain backward
> >> compatibility.
> >>
> >> With my proposal (which I am going to talk about next week at my
> >> "Production Image talk" and ask for feedback from the users) that if
> >> someone wants to customize the image, they should rebuild the image on
> >> their own using either stable (if cautions) or master (if adventurous)
> >> Dockerfile. This way they can get the image much better size-optimized.
> >> With the latest Breeze release, it's super-easy to build your own image
> >> using our Dockerfile - with your own dependencies both on Python. apt
> dev
> >> and apt runtime dependencies.
> >>
> >> J.
> >>
> >> --
> >>
> >> Jarek Potiuk
> >> Polidea <https://www.polidea.com/> | Principal Software Engineer
> >>
> >> M: +48 660 796 129 <+48660796129>
> >> [image: Polidea] <https://www.polidea.com/>
> >>
> >>
> >
> > --
> >
> > Jarek Potiuk
> > Polidea <https://www.polidea.com/> | Principal Software Engineer
> >
> > M: +48 660 796 129 <+48660796129>
> > [image: Polidea] <https://www.polidea.com/>
> >
> >
>
--
Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer
M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>
Re: Release cadence of the Docker Images
Posted by Zikun Zhu <ku...@gmail.com>.
Thanks for this.
"Add Support for Python 3.8 (#8836
<https://github.com/apache/airflow/pull/8836>)(#8823
<https://github.com/apache/airflow/pull/8823>)" is in the changelog but
seems like *1.10.11-python3.8* tag is missing in DockerHub?
On Sun, 12 Jul 2020 at 16:22, Jarek Potiuk <Ja...@polidea.com> wrote:
> FYI -> the images for 1.10.11 are in DockerHub now. They also include the
> "latest" tag - which means that anyone running `*docker pull
> apache/airflow*` will get the python3.6 variant of 1.10.11 version of
> image. Also *apache/airflow:1.10.11* points to 3.6 variant.
>
> J.
>
> [image: Screenshot from 2020-07-12 10-19-23.png]
>
> On Sat, Jul 11, 2020 at 8:43 PM Jarek Potiuk <Ja...@polidea.com>
> wrote:
>
>> Hello Everyone,
>>
>> Before we agree on release cadence of the Charts I have a proposal for
>> Prod images. They have not been released yesterday with 1.10.11.
>>
>> The relevant Dockerfile changes are part of the officially voted and
>> released sources and those sources are enough to build the image and push
>> it to the registry. In fact - I am already doing so.
>>
>> I think this is perfectly fine with ASF release policy that we release
>> the images without extra voting in such case (it's built using the released
>> sources):
>> http://www.apache.org/legal/release-policy.html#compiled-packages
>>
>> I think we should continue this way and do not introduce separate cadence
>> to release the docker images - unless we decide to make an ad-hoc release
>> at some point.
>>
>> I believe for the image (unlike for the Helm Chart), such ad-hoc releases
>> are not really possible. We will test the future Helm chart against
>> those released versions of images, and we should maintain backward
>> compatibility.
>>
>> With my proposal (which I am going to talk about next week at my
>> "Production Image talk" and ask for feedback from the users) that if
>> someone wants to customize the image, they should rebuild the image on
>> their own using either stable (if cautions) or master (if adventurous)
>> Dockerfile. This way they can get the image much better size-optimized.
>> With the latest Breeze release, it's super-easy to build your own image
>> using our Dockerfile - with your own dependencies both on Python. apt dev
>> and apt runtime dependencies.
>>
>> J.
>>
>> --
>>
>> Jarek Potiuk
>> Polidea <https://www.polidea.com/> | Principal Software Engineer
>>
>> M: +48 660 796 129 <+48660796129>
>> [image: Polidea] <https://www.polidea.com/>
>>
>>
>
> --
>
> Jarek Potiuk
> Polidea <https://www.polidea.com/> | Principal Software Engineer
>
> M: +48 660 796 129 <+48660796129>
> [image: Polidea] <https://www.polidea.com/>
>
>
Re: Release cadence of the Docker Images
Posted by Jarek Potiuk <Ja...@polidea.com>.
FYI -> the images for 1.10.11 are in DockerHub now. They also include the
"latest" tag - which means that anyone running `*docker pull apache/airflow*`
will get the python3.6 variant of 1.10.11 version of image. Also
*apache/airflow:1.10.11* points to 3.6 variant.
J.
[image: Screenshot from 2020-07-12 10-19-23.png]
On Sat, Jul 11, 2020 at 8:43 PM Jarek Potiuk <Ja...@polidea.com>
wrote:
> Hello Everyone,
>
> Before we agree on release cadence of the Charts I have a proposal for
> Prod images. They have not been released yesterday with 1.10.11.
>
> The relevant Dockerfile changes are part of the officially voted and
> released sources and those sources are enough to build the image and push
> it to the registry. In fact - I am already doing so.
>
> I think this is perfectly fine with ASF release policy that we release the
> images without extra voting in such case (it's built using the released
> sources):
> http://www.apache.org/legal/release-policy.html#compiled-packages
>
> I think we should continue this way and do not introduce separate cadence
> to release the docker images - unless we decide to make an ad-hoc release
> at some point.
>
> I believe for the image (unlike for the Helm Chart), such ad-hoc releases
> are not really possible. We will test the future Helm chart against
> those released versions of images, and we should maintain backward
> compatibility.
>
> With my proposal (which I am going to talk about next week at my
> "Production Image talk" and ask for feedback from the users) that if
> someone wants to customize the image, they should rebuild the image on
> their own using either stable (if cautions) or master (if adventurous)
> Dockerfile. This way they can get the image much better size-optimized.
> With the latest Breeze release, it's super-easy to build your own image
> using our Dockerfile - with your own dependencies both on Python. apt dev
> and apt runtime dependencies.
>
> J.
>
> --
>
> Jarek Potiuk
> Polidea <https://www.polidea.com/> | Principal Software Engineer
>
> M: +48 660 796 129 <+48660796129>
> [image: Polidea] <https://www.polidea.com/>
>
>
--
Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer
M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>