You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Jarek Potiuk <ja...@potiuk.com> on 2025/07/05 08:06:30 UTC
Re: [DISCUSS] Switching base Python container images to python-standalone ?
FYI - we just merged Aritra's change to build Python in CI image from
sources https://github.com/apache/airflow/pull/52265 -> we will run it for
a while in CI, i also want to test some self-upgrade scenarios.
Then we will look at applying it to the PROD image if we find no surprises.
It looks really good and we might have slightly more secure images produced
(i.e. getting read of some libraries CVEs faster than the "official"
images.
Good job Aritra!
J.
On Thu, Jun 26, 2025 at 9:16 PM Jarek Potiuk <ja...@potiuk.com> wrote:
> It looks like building from sources on our CI using base debian image
> (with enabled optimizations) "just works" and takes 1m36s. -> and taking
> into account that we already have pretty sophisticated CI-level remote
> caching that will allow to rebuild it only when either base image is
> released or python is released - building sounds like a very good idea.
>
> J.
>
> On Thu, Jun 26, 2025 at 7:02 PM Damian Shaw <ds...@striketechnologies.com>
> wrote:
>
>> Going from source is certainly going to give the most control, I
>> personally choose python-build-standalone for my own images rather than
>> building from source to outsource all the build choices, such as what
>> optimization flags to enable, what compiler tool chains to use, etc.
>>
>> Damian
>>
>>
>>
>> -----Original Message-----
>> From: Jarek Potiuk <ja...@potiuk.com>
>> Sent: Thursday, June 26, 2025 10:15 AM
>> To: dev@airflow.apache.org
>> Subject: Re: [DISCUSS] Switching base Python container images to
>> python-standalone ?
>>
>> Thanks Damian - very insightful. And yes - I did not really want to
>> "diminish" the value of community images and work of the maintainers, it
>> was really more on the "we base our image security on the assumptions that
>> it's coming from "official" sources and surely there are some guardrails" -
>> which turned out to be not really true.
>>
>> But.... We might not have to even use python standalone.
>>
>> Aritra just tested installation of Python straight from sources ->
>> https://github.com/apache/airflow/pull/52265 and I was actually pretty
>> surprised how non-problematic and fast it was - and it seems it passes all
>> our CI tests. We just need to add a little pre-commit magic to get
>> notified when we should update patchlevel version of Python when a new one
>> is released, and we should be able to try it out in CI image - once it gets
>> battle-tested with CI/breeze etc. we can transfer this to PROD image as
>> well. We already have an idea how to do it - our PROD images are optimized
>> for size and do not contain "build essentials" - but I think we should be
>> able to build Python in the "build" segment and simply copy resulting
>> binaries to the "main" segment - since in both cases we use the same base
>> image, such 1-1 copy should **just work** - we already do the same with
>> installed python packages - we install them (including building when
>> needed) in build segment and we copy-over the installed .venv to the main
>> segment.
>>
>> So ... we might even not need a discussion - installing from Python
>> sources is THE BEST
>>
>> J.
>>
>>
>> On Wed, Jun 25, 2025 at 6:25 PM Damian Shaw <dshaw@striketechnologies.com
>> >
>> wrote:
>>
>> > First, I would like to thank the community members who have been
>> > maintaining the Python docker images, it's one of those thankless
>> > opensource infrastructure volunteer roles that they've been doing for
>> > a long time. Unfortunately Docker assigns the title "Official Image"
>> > for various community run images, which creates a misconception on the
>> > guarantees being provided, and if I were a suspicious person I would
>> > say Docker creates this misconception on purpose to both get free work
>> > from community members and make Docker seem more supported by third
>> > party organizations than it actually is.
>> >
>> > On the topic of python-build-standalone, I've been using it in
>> > production for several months now and I'm fairly happy with it.
>> >
>> > However, one minor reproducibility issue I have when installing
>> > python-build-standalone via uv, is that uv does not have an ability to
>> > pin to a specific build between uv versions, as uv hard codes a
>> > mapping of Python version and platform to a specific build and then
>> > updates that mapping between releases. So, updating the version of uv
>> > between runs, or having two users run different versions of uv to
>> > initialize the environment can change the results.
>> >
>> > While normally such build changes are trivial, if you look at the uv
>> > 0.7.8 and 0.7.9 changelogs you will see that sometimes they can have
>> > significant
>> > impact: https://github.com/astral-sh/uv/releases/tag/0.7.8. Also, this
>> > email finally prompted make an issue on this topic:
>> > https://github.com/astral-sh/uv/issues/14263.
>> >
>> > There are other ways to source python-build-standalone, such as
>> > pbs-installer or writing your own script, but I've not yet spent any
>> > time investigating them, so I can't comment on them.
>> >
>> > Damian
>> >
>> > -----Original Message-----
>> > From: Jarek Potiuk <ja...@potiuk.com>
>> > Sent: Wednesday, June 25, 2025 10:44 AM
>> > To: dev@airflow.apache.org
>> > Subject: [DISCUSS] Switching base Python container images to
>> > python-standalone ?
>> >
>> > Hello here,
>> >
>> > Together with Aritra we are looking into adding a few more things to
>> > our images (golang tool chain for CI image), also Shahar is
>> > experimenting with Rust tool chain and I also recently realized (by
>> > some of the issues we had) that 'Docker Official Python Image' that is
>> > part of 'Official Program' [1] is not as 'Official' as I thought so we
>> > discuss about changing the base of our images (first CI and then when
>> > we see it works fine - PROD)
>> >
>> > Currently we are using the 'Official' image - but after some issues
>> > and discussions with people at PyCon and FOSS Backstage (I had a
>> > chance to talk to Python maintainers and even had a few beers with
>> > them) - it turned out that the official Python Image' is maintained by
>> > 'a community's which really is a few pretty random people - and that
>> > explains for example why we have sometimes unpacked security
>> > vulnerabilities in setuptools etc. - because they made some
>> > compatibility choices and decisions that do not allow them to upgrade
>> > easily, also they had some delays in releasing updated Python
>> > versions. And Docker does not **really** do much vetting there.
>> >
>> > So I think it would be good to switch how we build the base for our
>> images.
>> > And following the experience of `uv python` [2] - it seems that maybe
>> > using "python-standalone" [3] project is a good alternative. It's
>> > managed by Astral now (so yes - another dependency on them), but what
>> > you have with it you have practically 100% complete Python interpreter
>> > installed in seconds.
>> > We could continue using debian-slim as a "base, base image" - and
>> > install python using "python-standalone". There are a few
>> > incompatibilities [4] of the distributions of Python, but there are
>> > very few and mostly related to some obscure systems (compatibilities
>> > with terminal in REPL, and gtk / UI integration that is anyhow not
>> > really working in "standard" Python distributions).
>> >
>> > I would love to hear what you think - happy to get any feedback/
>> > insights, suggestions and answer additional questions, provide some
>> > links to past "troubles" we had with Python "Official" images etc.
>> >
>> > J.
>> >
>> >
>> > [1] Official Python Images - https://hub.docker.com/_/python [2] UV
>> > Python installation - https://docs.astral.sh/uv/guides/install-python/
>> > [3] Python Standalone project -
>> > https://github.com/astral-sh/python-build-standalone
>> > [4] Python Standalone incompatibilities -
>> > https://gregoryszorc.com/docs/python-build-standalone/main/quirks.html
>> > ________________________________
>> > Strike Technologies, LLC (“Strike”) is part of the GTS family of
>> > companies. Strike is a technology solutions provider, and is not a
>> > broker or dealer and does not transact any securities related business
>> > directly whatsoever. This communication is the property of Strike and
>> > its affiliates, and does not constitute an offer to sell or the
>> > solicitation of an offer to buy any security in any jurisdiction. It
>> > is intended only for the person to whom it is addressed and may
>> > contain information that is privileged, confidential, or otherwise
>> protected from disclosure.
>> > Distribution or copying of this communication, or the information
>> > contained herein, by anyone other than the intended recipient is
>> > prohibited. If you have received this communication in error, please
>> > immediately notify Strike at info@striketechnologies.com, and delete
>> and destroy any copies hereof.
>> > ________________________________
>> >
>> > CONFIDENTIALITY / PRIVILEGE NOTICE: This transmission and any
>> > attachments are intended solely for the addressee. This transmission
>> > is covered by the Electronic Communications Privacy Act, 18 U.S.C
>> > ''2510-2521. The information contained in this transmission is
>> > confidential in nature and protected from further use or disclosure
>> > under U.S. Pub. L. 106-102, 113 U.S. Stat. 1338 (1999), and may be
>> > subject to attorney-client or other legal privilege. Your use or
>> > disclosure of this information for any purpose other than that
>> > intended by its transmittal is strictly prohibited, and may subject
>> > you to fines and/or penalties under federal and state law. If you are
>> > not the intended recipient of this transmission, please DESTROY ALL
>> > COPIES RECEIVED and confirm destruction to the sender via return
>> transmittal.
>> >
>> ________________________________
>> Strike Technologies, LLC (“Strike”) is part of the GTS family of
>> companies. Strike is a technology solutions provider, and is not a broker
>> or dealer and does not transact any securities related business directly
>> whatsoever. This communication is the property of Strike and its
>> affiliates, and does not constitute an offer to sell or the solicitation of
>> an offer to buy any security in any jurisdiction. It is intended only for
>> the person to whom it is addressed and may contain information that is
>> privileged, confidential, or otherwise protected from disclosure.
>> Distribution or copying of this communication, or the information contained
>> herein, by anyone other than the intended recipient is prohibited. If you
>> have received this communication in error, please immediately notify Strike
>> at info@striketechnologies.com, and delete and destroy any copies hereof.
>> ________________________________
>>
>> CONFIDENTIALITY / PRIVILEGE NOTICE: This transmission and any attachments
>> are intended solely for the addressee. This transmission is covered by the
>> Electronic Communications Privacy Act, 18 U.S.C ''2510-2521. The
>> information contained in this transmission is confidential in nature and
>> protected from further use or disclosure under U.S. Pub. L. 106-102, 113
>> U.S. Stat. 1338 (1999), and may be subject to attorney-client or other
>> legal privilege. Your use or disclosure of this information for any purpose
>> other than that intended by its transmittal is strictly prohibited, and may
>> subject you to fines and/or penalties under federal and state law. If you
>> are not the intended recipient of this transmission, please DESTROY ALL
>> COPIES RECEIVED and confirm destruction to the sender via return
>> transmittal.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
>> For additional commands, e-mail: dev-help@airflow.apache.org
>>
>
Re: [DISCUSS] Switching base Python container images to python-standalone ?
Posted by Jarek Potiuk <ja...@potiuk.com>.
Yep. And merging the instalki script ci/prod :)
pt., 25 lip 2025, 15:13 użytkownik Aritra Basu <ar...@gmail.com>
napisał:
> Hey Jarek,
>
> I haven't seen any suspicious side effects either. I can go ahead and start
> working on a pr later today, if nothing breaks I should be able to raise it
> by tonight. I think it should be a fairly small change (🤞) just moving the
> python install directory within the docker file hopefully.
>
> --
> Regards,
> Aritra Basu
>
> On Fri, 25 Jul 2025, 1:39 pm Jarek Potiuk, <ja...@potiuk.com> wrote:
>
> > Did anyone experience any suspicious problems with the breeze CI image
> > prepared this way ?
> >
> > I did not - python works as usual, all airflow features are working, I
> have
> > not seen any "negative" side effect of changing python to source-built
> one
> > - except slightly longer build time occasionally.
> >
> > If ther are no other observations or issues - Aritra, I tink it might be
> > good time to look at trasplanting it to PROD image, Happy to either just
> > help and review or do it and have you review it - up to you :)
> >
> > J.
> >
> > On Fri, Jul 11, 2025 at 6:54 PM Jarek Potiuk <ja...@potiuk.com> wrote:
> >
> > > And now I have to rebase my Python 3.13 PR to see if 3.13 also compiles
> > > well :D
> > >
> > > On Fri, Jul 11, 2025 at 6:53 PM Aritra Basu <ar...@gmail.com>
> > > wrote:
> > >
> > >> Yay! 🙌
> > >> --
> > >> Regards,
> > >> Aritra Basu
> > >>
> > >> On Fri, 11 Jul 2025, 9:48 pm Jarek Potiuk, <ja...@potiuk.com> wrote:
> > >>
> > >> > Ok. We got it merged again !
> > >> https://github.com/apache/airflow/pull/53150
> > >> > is merged. Once you build breeze images after rebase, python there
> > will
> > >> be
> > >> > built directly from Python official sources (latest released
> > patchlevel
> > >> for
> > >> > each version - and we have automation to upgrade them soon after new
> > >> > patchlevel are released).
> > >> >
> > >> > We are going to try it for a week or so and see if it all looks
> good,
> > we
> > >> > will proceed to apply it to PROD images.
> > >> >
> > >> > J.
> > >> >
> > >> >
> > >> > On Sat, Jul 5, 2025 at 10:24 AM Jarek Potiuk <ja...@potiuk.com>
> > wrote:
> > >> >
> > >> > > I merget it too early - reverting here
> > >> > > https://github.com/apache/airflow/pull/52900 as I found an issue
> > with
> > >> > > other python versions that I overlooked
> > >> > >
> > >> > > On Sat, Jul 5, 2025 at 10:06 AM Jarek Potiuk <ja...@potiuk.com>
> > >> wrote:
> > >> > >
> > >> > >> FYI - we just merged Aritra's change to build Python in CI image
> > from
> > >> > >> sources https://github.com/apache/airflow/pull/52265 -> we will
> > run
> > >> it
> > >> > >> for a while in CI, i also want to test some self-upgrade
> scenarios.
> > >> > >>
> > >> > >> Then we will look at applying it to the PROD image if we find no
> > >> > >> surprises.
> > >> > >>
> > >> > >> It looks really good and we might have slightly more secure
> images
> > >> > >> produced (i.e. getting read of some libraries CVEs faster than
> the
> > >> > >> "official" images.
> > >> > >>
> > >> > >> Good job Aritra!
> > >> > >> J.
> > >> > >>
> > >> > >>
> > >> > >> On Thu, Jun 26, 2025 at 9:16 PM Jarek Potiuk <ja...@potiuk.com>
> > >> wrote:
> > >> > >>
> > >> > >>> It looks like building from sources on our CI using base debian
> > >> image
> > >> > >>> (with enabled optimizations) "just works" and takes 1m36s. ->
> and
> > >> > taking
> > >> > >>> into account that we already have pretty sophisticated CI-level
> > >> remote
> > >> > >>> caching that will allow to rebuild it only when either base
> image
> > is
> > >> > >>> released or python is released - building sounds like a very
> good
> > >> idea.
> > >> > >>>
> > >> > >>> J.
> > >> > >>>
> > >> > >>> On Thu, Jun 26, 2025 at 7:02 PM Damian Shaw <
> > >> > >>> dshaw@striketechnologies.com> wrote:
> > >> > >>>
> > >> > >>>> Going from source is certainly going to give the most control,
> I
> > >> > >>>> personally choose python-build-standalone for my own images
> > rather
> > >> > than
> > >> > >>>> building from source to outsource all the build choices, such
> as
> > >> what
> > >> > >>>> optimization flags to enable, what compiler tool chains to use,
> > >> etc.
> > >> > >>>>
> > >> > >>>> Damian
> > >> > >>>>
> > >> > >>>>
> > >> > >>>>
> > >> > >>>> -----Original Message-----
> > >> > >>>> From: Jarek Potiuk <ja...@potiuk.com>
> > >> > >>>> Sent: Thursday, June 26, 2025 10:15 AM
> > >> > >>>> To: dev@airflow.apache.org
> > >> > >>>> Subject: Re: [DISCUSS] Switching base Python container images
> to
> > >> > >>>> python-standalone ?
> > >> > >>>>
> > >> > >>>> Thanks Damian - very insightful. And yes - I did not really
> want
> > to
> > >> > >>>> "diminish" the value of community images and work of the
> > >> maintainers,
> > >> > it
> > >> > >>>> was really more on the "we base our image security on the
> > >> assumptions
> > >> > that
> > >> > >>>> it's coming from "official" sources and surely there are some
> > >> > guardrails" -
> > >> > >>>> which turned out to be not really true.
> > >> > >>>>
> > >> > >>>> But.... We might not have to even use python standalone.
> > >> > >>>>
> > >> > >>>> Aritra just tested installation of Python straight from sources
> > ->
> > >> > >>>> https://github.com/apache/airflow/pull/52265 and I was
> actually
> > >> > pretty
> > >> > >>>> surprised how non-problematic and fast it was - and it seems it
> > >> > passes all
> > >> > >>>> our CI tests. We just need to add a little pre-commit magic to
> > get
> > >> > >>>> notified when we should update patchlevel version of Python
> when
> > a
> > >> > new one
> > >> > >>>> is released, and we should be able to try it out in CI image -
> > once
> > >> > it gets
> > >> > >>>> battle-tested with CI/breeze etc. we can transfer this to PROD
> > >> image
> > >> > as
> > >> > >>>> well. We already have an idea how to do it - our PROD images
> are
> > >> > optimized
> > >> > >>>> for size and do not contain "build essentials" - but I think we
> > >> > should be
> > >> > >>>> able to build Python in the "build" segment and simply copy
> > >> resulting
> > >> > >>>> binaries to the "main" segment - since in both cases we use the
> > >> same
> > >> > base
> > >> > >>>> image, such 1-1 copy should **just work** - we already do the
> > same
> > >> > with
> > >> > >>>> installed python packages - we install them (including building
> > >> when
> > >> > >>>> needed) in build segment and we copy-over the installed .venv
> to
> > >> the
> > >> > >>>> main segment.
> > >> > >>>>
> > >> > >>>> So ... we might even not need a discussion - installing from
> > Python
> > >> > >>>> sources is THE BEST
> > >> > >>>>
> > >> > >>>> J.
> > >> > >>>>
> > >> > >>>>
> > >> > >>>> On Wed, Jun 25, 2025 at 6:25 PM Damian Shaw <
> > >> > >>>> dshaw@striketechnologies.com>
> > >> > >>>> wrote:
> > >> > >>>>
> > >> > >>>> > First, I would like to thank the community members who have
> > been
> > >> > >>>> > maintaining the Python docker images, it's one of those
> > thankless
> > >> > >>>> > opensource infrastructure volunteer roles that they've been
> > doing
> > >> > for
> > >> > >>>> > a long time. Unfortunately Docker assigns the title "Official
> > >> Image"
> > >> > >>>> > for various community run images, which creates a
> misconception
> > >> on
> > >> > the
> > >> > >>>> > guarantees being provided, and if I were a suspicious person
> I
> > >> would
> > >> > >>>> > say Docker creates this misconception on purpose to both get
> > free
> > >> > work
> > >> > >>>> > from community members and make Docker seem more supported by
> > >> third
> > >> > >>>> > party organizations than it actually is.
> > >> > >>>> >
> > >> > >>>> > On the topic of python-build-standalone, I've been using it
> in
> > >> > >>>> > production for several months now and I'm fairly happy with
> it.
> > >> > >>>> >
> > >> > >>>> > However, one minor reproducibility issue I have when
> installing
> > >> > >>>> > python-build-standalone via uv, is that uv does not have an
> > >> ability
> > >> > to
> > >> > >>>> > pin to a specific build between uv versions, as uv hard
> codes a
> > >> > >>>> > mapping of Python version and platform to a specific build
> and
> > >> then
> > >> > >>>> > updates that mapping between releases. So, updating the
> version
> > >> of
> > >> > uv
> > >> > >>>> > between runs, or having two users run different versions of
> uv
> > to
> > >> > >>>> > initialize the environment can change the results.
> > >> > >>>> >
> > >> > >>>> > While normally such build changes are trivial, if you look at
> > >> the uv
> > >> > >>>> > 0.7.8 and 0.7.9 changelogs you will see that sometimes they
> can
> > >> have
> > >> > >>>> > significant
> > >> > >>>> > impact: https://github.com/astral-sh/uv/releases/tag/0.7.8.
> > >> Also,
> > >> > >>>> this
> > >> > >>>> > email finally prompted make an issue on this topic:
> > >> > >>>> > https://github.com/astral-sh/uv/issues/14263.
> > >> > >>>> >
> > >> > >>>> > There are other ways to source python-build-standalone, such
> as
> > >> > >>>> > pbs-installer or writing your own script, but I've not yet
> > spent
> > >> any
> > >> > >>>> > time investigating them, so I can't comment on them.
> > >> > >>>> >
> > >> > >>>> > Damian
> > >> > >>>> >
> > >> > >>>> > -----Original Message-----
> > >> > >>>> > From: Jarek Potiuk <ja...@potiuk.com>
> > >> > >>>> > Sent: Wednesday, June 25, 2025 10:44 AM
> > >> > >>>> > To: dev@airflow.apache.org
> > >> > >>>> > Subject: [DISCUSS] Switching base Python container images to
> > >> > >>>> > python-standalone ?
> > >> > >>>> >
> > >> > >>>> > Hello here,
> > >> > >>>> >
> > >> > >>>> > Together with Aritra we are looking into adding a few more
> > >> things to
> > >> > >>>> > our images (golang tool chain for CI image), also Shahar is
> > >> > >>>> > experimenting with Rust tool chain and I also recently
> realized
> > >> (by
> > >> > >>>> > some of the issues we had) that 'Docker Official Python
> Image'
> > >> that
> > >> > is
> > >> > >>>> > part of 'Official Program' [1] is not as 'Official' as I
> > thought
> > >> so
> > >> > we
> > >> > >>>> > discuss about changing the base of our images (first CI and
> > then
> > >> > when
> > >> > >>>> > we see it works fine - PROD)
> > >> > >>>> >
> > >> > >>>> > Currently we are using the 'Official' image - but after some
> > >> issues
> > >> > >>>> > and discussions with people at PyCon and FOSS Backstage (I
> had
> > a
> > >> > >>>> > chance to talk to Python maintainers and even had a few beers
> > >> with
> > >> > >>>> > them) - it turned out that the official Python Image' is
> > >> maintained
> > >> > by
> > >> > >>>> > 'a community's which really is a few pretty random people -
> and
> > >> that
> > >> > >>>> > explains for example why we have sometimes unpacked security
> > >> > >>>> > vulnerabilities in setuptools etc. - because they made some
> > >> > >>>> > compatibility choices and decisions that do not allow them to
> > >> > upgrade
> > >> > >>>> > easily, also they had some delays in releasing updated Python
> > >> > >>>> > versions. And Docker does not **really** do much vetting
> there.
> > >> > >>>> >
> > >> > >>>> > So I think it would be good to switch how we build the base
> for
> > >> our
> > >> > >>>> images.
> > >> > >>>> > And following the experience of `uv python` [2] - it seems
> that
> > >> > maybe
> > >> > >>>> > using "python-standalone" [3] project is a good alternative.
> > It's
> > >> > >>>> > managed by Astral now (so yes - another dependency on them),
> > but
> > >> > what
> > >> > >>>> > you have with it you have practically 100% complete Python
> > >> > interpreter
> > >> > >>>> > installed in seconds.
> > >> > >>>> > We could continue using debian-slim as a "base, base image" -
> > and
> > >> > >>>> > install python using "python-standalone". There are a few
> > >> > >>>> > incompatibilities [4] of the distributions of Python, but
> there
> > >> are
> > >> > >>>> > very few and mostly related to some obscure systems
> > >> (compatibilities
> > >> > >>>> > with terminal in REPL, and gtk / UI integration that is
> anyhow
> > >> not
> > >> > >>>> > really working in "standard" Python distributions).
> > >> > >>>> >
> > >> > >>>> > I would love to hear what you think - happy to get any
> > feedback/
> > >> > >>>> > insights, suggestions and answer additional questions,
> provide
> > >> some
> > >> > >>>> > links to past "troubles" we had with Python "Official" images
> > >> etc.
> > >> > >>>> >
> > >> > >>>> > J.
> > >> > >>>> >
> > >> > >>>> >
> > >> > >>>> > [1] Official Python Images - https://hub.docker.com/_/python
> > >> [2] UV
> > >> > >>>> > Python installation -
> > >> > >>>> https://docs.astral.sh/uv/guides/install-python/
> > >> > >>>> > [3] Python Standalone project -
> > >> > >>>> > https://github.com/astral-sh/python-build-standalone
> > >> > >>>> > [4] Python Standalone incompatibilities -
> > >> > >>>> >
> > >> > >>>>
> > >> >
> > https://gregoryszorc.com/docs/python-build-standalone/main/quirks.html
> > >> > >>>> > ________________________________
> > >> > >>>> > Strike Technologies, LLC (“Strike”) is part of the GTS
> family
> > of
> > >> > >>>> > companies. Strike is a technology solutions provider, and is
> > not
> > >> a
> > >> > >>>> > broker or dealer and does not transact any securities related
> > >> > business
> > >> > >>>> > directly whatsoever. This communication is the property of
> > Strike
> > >> > and
> > >> > >>>> > its affiliates, and does not constitute an offer to sell or
> the
> > >> > >>>> > solicitation of an offer to buy any security in any
> > >> jurisdiction. It
> > >> > >>>> > is intended only for the person to whom it is addressed and
> may
> > >> > >>>> > contain information that is privileged, confidential, or
> > >> otherwise
> > >> > >>>> protected from disclosure.
> > >> > >>>> > Distribution or copying of this communication, or the
> > information
> > >> > >>>> > contained herein, by anyone other than the intended recipient
> > is
> > >> > >>>> > prohibited. If you have received this communication in error,
> > >> please
> > >> > >>>> > immediately notify Strike at info@striketechnologies.com,
> and
> > >> > delete
> > >> > >>>> and destroy any copies hereof.
> > >> > >>>> > ________________________________
> > >> > >>>> >
> > >> > >>>> > CONFIDENTIALITY / PRIVILEGE NOTICE: This transmission and any
> > >> > >>>> > attachments are intended solely for the addressee. This
> > >> transmission
> > >> > >>>> > is covered by the Electronic Communications Privacy Act, 18
> > U.S.C
> > >> > >>>> > ''2510-2521. The information contained in this transmission
> is
> > >> > >>>> > confidential in nature and protected from further use or
> > >> disclosure
> > >> > >>>> > under U.S. Pub. L. 106-102, 113 U.S. Stat. 1338 (1999), and
> may
> > >> be
> > >> > >>>> > subject to attorney-client or other legal privilege. Your use
> > or
> > >> > >>>> > disclosure of this information for any purpose other than
> that
> > >> > >>>> > intended by its transmittal is strictly prohibited, and may
> > >> subject
> > >> > >>>> > you to fines and/or penalties under federal and state law. If
> > you
> > >> > are
> > >> > >>>> > not the intended recipient of this transmission, please
> DESTROY
> > >> ALL
> > >> > >>>> > COPIES RECEIVED and confirm destruction to the sender via
> > return
> > >> > >>>> transmittal.
> > >> > >>>> >
> > >> > >>>> ________________________________
> > >> > >>>> Strike Technologies, LLC (“Strike”) is part of the GTS family
> of
> > >> > >>>> companies. Strike is a technology solutions provider, and is
> not
> > a
> > >> > broker
> > >> > >>>> or dealer and does not transact any securities related business
> > >> > directly
> > >> > >>>> whatsoever. This communication is the property of Strike and
> its
> > >> > >>>> affiliates, and does not constitute an offer to sell or the
> > >> > solicitation of
> > >> > >>>> an offer to buy any security in any jurisdiction. It is
> intended
> > >> only
> > >> > for
> > >> > >>>> the person to whom it is addressed and may contain information
> > >> that is
> > >> > >>>> privileged, confidential, or otherwise protected from
> disclosure.
> > >> > >>>> Distribution or copying of this communication, or the
> information
> > >> > contained
> > >> > >>>> herein, by anyone other than the intended recipient is
> > prohibited.
> > >> If
> > >> > you
> > >> > >>>> have received this communication in error, please immediately
> > >> notify
> > >> > Strike
> > >> > >>>> at info@striketechnologies.com, and delete and destroy any
> > copies
> > >> > >>>> hereof.
> > >> > >>>> ________________________________
> > >> > >>>>
> > >> > >>>> CONFIDENTIALITY / PRIVILEGE NOTICE: This transmission and any
> > >> > >>>> attachments are intended solely for the addressee. This
> > >> transmission
> > >> > is
> > >> > >>>> covered by the Electronic Communications Privacy Act, 18 U.S.C
> > >> > ''2510-2521.
> > >> > >>>> The information contained in this transmission is confidential
> in
> > >> > nature
> > >> > >>>> and protected from further use or disclosure under U.S. Pub. L.
> > >> > 106-102,
> > >> > >>>> 113 U.S. Stat. 1338 (1999), and may be subject to
> attorney-client
> > >> or
> > >> > other
> > >> > >>>> legal privilege. Your use or disclosure of this information for
> > any
> > >> > purpose
> > >> > >>>> other than that intended by its transmittal is strictly
> > prohibited,
> > >> > and may
> > >> > >>>> subject you to fines and/or penalties under federal and state
> > law.
> > >> If
> > >> > you
> > >> > >>>> are not the intended recipient of this transmission, please
> > DESTROY
> > >> > ALL
> > >> > >>>> COPIES RECEIVED and confirm destruction to the sender via
> return
> > >> > >>>> transmittal.
> > >> > >>>>
> > >> > >>>>
> > >> ---------------------------------------------------------------------
> > >> > >>>> To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
> > >> > >>>> For additional commands, e-mail: dev-help@airflow.apache.org
> > >> > >>>>
> > >> > >>>
> > >> >
> > >>
> > >
> >
>
Re: [DISCUSS] Switching base Python container images to python-standalone ?
Posted by Aritra Basu <ar...@gmail.com>.
Hey Jarek,
I haven't seen any suspicious side effects either. I can go ahead and start
working on a pr later today, if nothing breaks I should be able to raise it
by tonight. I think it should be a fairly small change (🤞) just moving the
python install directory within the docker file hopefully.
--
Regards,
Aritra Basu
On Fri, 25 Jul 2025, 1:39 pm Jarek Potiuk, <ja...@potiuk.com> wrote:
> Did anyone experience any suspicious problems with the breeze CI image
> prepared this way ?
>
> I did not - python works as usual, all airflow features are working, I have
> not seen any "negative" side effect of changing python to source-built one
> - except slightly longer build time occasionally.
>
> If ther are no other observations or issues - Aritra, I tink it might be
> good time to look at trasplanting it to PROD image, Happy to either just
> help and review or do it and have you review it - up to you :)
>
> J.
>
> On Fri, Jul 11, 2025 at 6:54 PM Jarek Potiuk <ja...@potiuk.com> wrote:
>
> > And now I have to rebase my Python 3.13 PR to see if 3.13 also compiles
> > well :D
> >
> > On Fri, Jul 11, 2025 at 6:53 PM Aritra Basu <ar...@gmail.com>
> > wrote:
> >
> >> Yay! 🙌
> >> --
> >> Regards,
> >> Aritra Basu
> >>
> >> On Fri, 11 Jul 2025, 9:48 pm Jarek Potiuk, <ja...@potiuk.com> wrote:
> >>
> >> > Ok. We got it merged again !
> >> https://github.com/apache/airflow/pull/53150
> >> > is merged. Once you build breeze images after rebase, python there
> will
> >> be
> >> > built directly from Python official sources (latest released
> patchlevel
> >> for
> >> > each version - and we have automation to upgrade them soon after new
> >> > patchlevel are released).
> >> >
> >> > We are going to try it for a week or so and see if it all looks good,
> we
> >> > will proceed to apply it to PROD images.
> >> >
> >> > J.
> >> >
> >> >
> >> > On Sat, Jul 5, 2025 at 10:24 AM Jarek Potiuk <ja...@potiuk.com>
> wrote:
> >> >
> >> > > I merget it too early - reverting here
> >> > > https://github.com/apache/airflow/pull/52900 as I found an issue
> with
> >> > > other python versions that I overlooked
> >> > >
> >> > > On Sat, Jul 5, 2025 at 10:06 AM Jarek Potiuk <ja...@potiuk.com>
> >> wrote:
> >> > >
> >> > >> FYI - we just merged Aritra's change to build Python in CI image
> from
> >> > >> sources https://github.com/apache/airflow/pull/52265 -> we will
> run
> >> it
> >> > >> for a while in CI, i also want to test some self-upgrade scenarios.
> >> > >>
> >> > >> Then we will look at applying it to the PROD image if we find no
> >> > >> surprises.
> >> > >>
> >> > >> It looks really good and we might have slightly more secure images
> >> > >> produced (i.e. getting read of some libraries CVEs faster than the
> >> > >> "official" images.
> >> > >>
> >> > >> Good job Aritra!
> >> > >> J.
> >> > >>
> >> > >>
> >> > >> On Thu, Jun 26, 2025 at 9:16 PM Jarek Potiuk <ja...@potiuk.com>
> >> wrote:
> >> > >>
> >> > >>> It looks like building from sources on our CI using base debian
> >> image
> >> > >>> (with enabled optimizations) "just works" and takes 1m36s. -> and
> >> > taking
> >> > >>> into account that we already have pretty sophisticated CI-level
> >> remote
> >> > >>> caching that will allow to rebuild it only when either base image
> is
> >> > >>> released or python is released - building sounds like a very good
> >> idea.
> >> > >>>
> >> > >>> J.
> >> > >>>
> >> > >>> On Thu, Jun 26, 2025 at 7:02 PM Damian Shaw <
> >> > >>> dshaw@striketechnologies.com> wrote:
> >> > >>>
> >> > >>>> Going from source is certainly going to give the most control, I
> >> > >>>> personally choose python-build-standalone for my own images
> rather
> >> > than
> >> > >>>> building from source to outsource all the build choices, such as
> >> what
> >> > >>>> optimization flags to enable, what compiler tool chains to use,
> >> etc.
> >> > >>>>
> >> > >>>> Damian
> >> > >>>>
> >> > >>>>
> >> > >>>>
> >> > >>>> -----Original Message-----
> >> > >>>> From: Jarek Potiuk <ja...@potiuk.com>
> >> > >>>> Sent: Thursday, June 26, 2025 10:15 AM
> >> > >>>> To: dev@airflow.apache.org
> >> > >>>> Subject: Re: [DISCUSS] Switching base Python container images to
> >> > >>>> python-standalone ?
> >> > >>>>
> >> > >>>> Thanks Damian - very insightful. And yes - I did not really want
> to
> >> > >>>> "diminish" the value of community images and work of the
> >> maintainers,
> >> > it
> >> > >>>> was really more on the "we base our image security on the
> >> assumptions
> >> > that
> >> > >>>> it's coming from "official" sources and surely there are some
> >> > guardrails" -
> >> > >>>> which turned out to be not really true.
> >> > >>>>
> >> > >>>> But.... We might not have to even use python standalone.
> >> > >>>>
> >> > >>>> Aritra just tested installation of Python straight from sources
> ->
> >> > >>>> https://github.com/apache/airflow/pull/52265 and I was actually
> >> > pretty
> >> > >>>> surprised how non-problematic and fast it was - and it seems it
> >> > passes all
> >> > >>>> our CI tests. We just need to add a little pre-commit magic to
> get
> >> > >>>> notified when we should update patchlevel version of Python when
> a
> >> > new one
> >> > >>>> is released, and we should be able to try it out in CI image -
> once
> >> > it gets
> >> > >>>> battle-tested with CI/breeze etc. we can transfer this to PROD
> >> image
> >> > as
> >> > >>>> well. We already have an idea how to do it - our PROD images are
> >> > optimized
> >> > >>>> for size and do not contain "build essentials" - but I think we
> >> > should be
> >> > >>>> able to build Python in the "build" segment and simply copy
> >> resulting
> >> > >>>> binaries to the "main" segment - since in both cases we use the
> >> same
> >> > base
> >> > >>>> image, such 1-1 copy should **just work** - we already do the
> same
> >> > with
> >> > >>>> installed python packages - we install them (including building
> >> when
> >> > >>>> needed) in build segment and we copy-over the installed .venv to
> >> the
> >> > >>>> main segment.
> >> > >>>>
> >> > >>>> So ... we might even not need a discussion - installing from
> Python
> >> > >>>> sources is THE BEST
> >> > >>>>
> >> > >>>> J.
> >> > >>>>
> >> > >>>>
> >> > >>>> On Wed, Jun 25, 2025 at 6:25 PM Damian Shaw <
> >> > >>>> dshaw@striketechnologies.com>
> >> > >>>> wrote:
> >> > >>>>
> >> > >>>> > First, I would like to thank the community members who have
> been
> >> > >>>> > maintaining the Python docker images, it's one of those
> thankless
> >> > >>>> > opensource infrastructure volunteer roles that they've been
> doing
> >> > for
> >> > >>>> > a long time. Unfortunately Docker assigns the title "Official
> >> Image"
> >> > >>>> > for various community run images, which creates a misconception
> >> on
> >> > the
> >> > >>>> > guarantees being provided, and if I were a suspicious person I
> >> would
> >> > >>>> > say Docker creates this misconception on purpose to both get
> free
> >> > work
> >> > >>>> > from community members and make Docker seem more supported by
> >> third
> >> > >>>> > party organizations than it actually is.
> >> > >>>> >
> >> > >>>> > On the topic of python-build-standalone, I've been using it in
> >> > >>>> > production for several months now and I'm fairly happy with it.
> >> > >>>> >
> >> > >>>> > However, one minor reproducibility issue I have when installing
> >> > >>>> > python-build-standalone via uv, is that uv does not have an
> >> ability
> >> > to
> >> > >>>> > pin to a specific build between uv versions, as uv hard codes a
> >> > >>>> > mapping of Python version and platform to a specific build and
> >> then
> >> > >>>> > updates that mapping between releases. So, updating the version
> >> of
> >> > uv
> >> > >>>> > between runs, or having two users run different versions of uv
> to
> >> > >>>> > initialize the environment can change the results.
> >> > >>>> >
> >> > >>>> > While normally such build changes are trivial, if you look at
> >> the uv
> >> > >>>> > 0.7.8 and 0.7.9 changelogs you will see that sometimes they can
> >> have
> >> > >>>> > significant
> >> > >>>> > impact: https://github.com/astral-sh/uv/releases/tag/0.7.8.
> >> Also,
> >> > >>>> this
> >> > >>>> > email finally prompted make an issue on this topic:
> >> > >>>> > https://github.com/astral-sh/uv/issues/14263.
> >> > >>>> >
> >> > >>>> > There are other ways to source python-build-standalone, such as
> >> > >>>> > pbs-installer or writing your own script, but I've not yet
> spent
> >> any
> >> > >>>> > time investigating them, so I can't comment on them.
> >> > >>>> >
> >> > >>>> > Damian
> >> > >>>> >
> >> > >>>> > -----Original Message-----
> >> > >>>> > From: Jarek Potiuk <ja...@potiuk.com>
> >> > >>>> > Sent: Wednesday, June 25, 2025 10:44 AM
> >> > >>>> > To: dev@airflow.apache.org
> >> > >>>> > Subject: [DISCUSS] Switching base Python container images to
> >> > >>>> > python-standalone ?
> >> > >>>> >
> >> > >>>> > Hello here,
> >> > >>>> >
> >> > >>>> > Together with Aritra we are looking into adding a few more
> >> things to
> >> > >>>> > our images (golang tool chain for CI image), also Shahar is
> >> > >>>> > experimenting with Rust tool chain and I also recently realized
> >> (by
> >> > >>>> > some of the issues we had) that 'Docker Official Python Image'
> >> that
> >> > is
> >> > >>>> > part of 'Official Program' [1] is not as 'Official' as I
> thought
> >> so
> >> > we
> >> > >>>> > discuss about changing the base of our images (first CI and
> then
> >> > when
> >> > >>>> > we see it works fine - PROD)
> >> > >>>> >
> >> > >>>> > Currently we are using the 'Official' image - but after some
> >> issues
> >> > >>>> > and discussions with people at PyCon and FOSS Backstage (I had
> a
> >> > >>>> > chance to talk to Python maintainers and even had a few beers
> >> with
> >> > >>>> > them) - it turned out that the official Python Image' is
> >> maintained
> >> > by
> >> > >>>> > 'a community's which really is a few pretty random people - and
> >> that
> >> > >>>> > explains for example why we have sometimes unpacked security
> >> > >>>> > vulnerabilities in setuptools etc. - because they made some
> >> > >>>> > compatibility choices and decisions that do not allow them to
> >> > upgrade
> >> > >>>> > easily, also they had some delays in releasing updated Python
> >> > >>>> > versions. And Docker does not **really** do much vetting there.
> >> > >>>> >
> >> > >>>> > So I think it would be good to switch how we build the base for
> >> our
> >> > >>>> images.
> >> > >>>> > And following the experience of `uv python` [2] - it seems that
> >> > maybe
> >> > >>>> > using "python-standalone" [3] project is a good alternative.
> It's
> >> > >>>> > managed by Astral now (so yes - another dependency on them),
> but
> >> > what
> >> > >>>> > you have with it you have practically 100% complete Python
> >> > interpreter
> >> > >>>> > installed in seconds.
> >> > >>>> > We could continue using debian-slim as a "base, base image" -
> and
> >> > >>>> > install python using "python-standalone". There are a few
> >> > >>>> > incompatibilities [4] of the distributions of Python, but there
> >> are
> >> > >>>> > very few and mostly related to some obscure systems
> >> (compatibilities
> >> > >>>> > with terminal in REPL, and gtk / UI integration that is anyhow
> >> not
> >> > >>>> > really working in "standard" Python distributions).
> >> > >>>> >
> >> > >>>> > I would love to hear what you think - happy to get any
> feedback/
> >> > >>>> > insights, suggestions and answer additional questions, provide
> >> some
> >> > >>>> > links to past "troubles" we had with Python "Official" images
> >> etc.
> >> > >>>> >
> >> > >>>> > J.
> >> > >>>> >
> >> > >>>> >
> >> > >>>> > [1] Official Python Images - https://hub.docker.com/_/python
> >> [2] UV
> >> > >>>> > Python installation -
> >> > >>>> https://docs.astral.sh/uv/guides/install-python/
> >> > >>>> > [3] Python Standalone project -
> >> > >>>> > https://github.com/astral-sh/python-build-standalone
> >> > >>>> > [4] Python Standalone incompatibilities -
> >> > >>>> >
> >> > >>>>
> >> >
> https://gregoryszorc.com/docs/python-build-standalone/main/quirks.html
> >> > >>>> > ________________________________
> >> > >>>> > Strike Technologies, LLC (“Strike”) is part of the GTS family
> of
> >> > >>>> > companies. Strike is a technology solutions provider, and is
> not
> >> a
> >> > >>>> > broker or dealer and does not transact any securities related
> >> > business
> >> > >>>> > directly whatsoever. This communication is the property of
> Strike
> >> > and
> >> > >>>> > its affiliates, and does not constitute an offer to sell or the
> >> > >>>> > solicitation of an offer to buy any security in any
> >> jurisdiction. It
> >> > >>>> > is intended only for the person to whom it is addressed and may
> >> > >>>> > contain information that is privileged, confidential, or
> >> otherwise
> >> > >>>> protected from disclosure.
> >> > >>>> > Distribution or copying of this communication, or the
> information
> >> > >>>> > contained herein, by anyone other than the intended recipient
> is
> >> > >>>> > prohibited. If you have received this communication in error,
> >> please
> >> > >>>> > immediately notify Strike at info@striketechnologies.com, and
> >> > delete
> >> > >>>> and destroy any copies hereof.
> >> > >>>> > ________________________________
> >> > >>>> >
> >> > >>>> > CONFIDENTIALITY / PRIVILEGE NOTICE: This transmission and any
> >> > >>>> > attachments are intended solely for the addressee. This
> >> transmission
> >> > >>>> > is covered by the Electronic Communications Privacy Act, 18
> U.S.C
> >> > >>>> > ''2510-2521. The information contained in this transmission is
> >> > >>>> > confidential in nature and protected from further use or
> >> disclosure
> >> > >>>> > under U.S. Pub. L. 106-102, 113 U.S. Stat. 1338 (1999), and may
> >> be
> >> > >>>> > subject to attorney-client or other legal privilege. Your use
> or
> >> > >>>> > disclosure of this information for any purpose other than that
> >> > >>>> > intended by its transmittal is strictly prohibited, and may
> >> subject
> >> > >>>> > you to fines and/or penalties under federal and state law. If
> you
> >> > are
> >> > >>>> > not the intended recipient of this transmission, please DESTROY
> >> ALL
> >> > >>>> > COPIES RECEIVED and confirm destruction to the sender via
> return
> >> > >>>> transmittal.
> >> > >>>> >
> >> > >>>> ________________________________
> >> > >>>> Strike Technologies, LLC (“Strike”) is part of the GTS family of
> >> > >>>> companies. Strike is a technology solutions provider, and is not
> a
> >> > broker
> >> > >>>> or dealer and does not transact any securities related business
> >> > directly
> >> > >>>> whatsoever. This communication is the property of Strike and its
> >> > >>>> affiliates, and does not constitute an offer to sell or the
> >> > solicitation of
> >> > >>>> an offer to buy any security in any jurisdiction. It is intended
> >> only
> >> > for
> >> > >>>> the person to whom it is addressed and may contain information
> >> that is
> >> > >>>> privileged, confidential, or otherwise protected from disclosure.
> >> > >>>> Distribution or copying of this communication, or the information
> >> > contained
> >> > >>>> herein, by anyone other than the intended recipient is
> prohibited.
> >> If
> >> > you
> >> > >>>> have received this communication in error, please immediately
> >> notify
> >> > Strike
> >> > >>>> at info@striketechnologies.com, and delete and destroy any
> copies
> >> > >>>> hereof.
> >> > >>>> ________________________________
> >> > >>>>
> >> > >>>> CONFIDENTIALITY / PRIVILEGE NOTICE: This transmission and any
> >> > >>>> attachments are intended solely for the addressee. This
> >> transmission
> >> > is
> >> > >>>> covered by the Electronic Communications Privacy Act, 18 U.S.C
> >> > ''2510-2521.
> >> > >>>> The information contained in this transmission is confidential in
> >> > nature
> >> > >>>> and protected from further use or disclosure under U.S. Pub. L.
> >> > 106-102,
> >> > >>>> 113 U.S. Stat. 1338 (1999), and may be subject to attorney-client
> >> or
> >> > other
> >> > >>>> legal privilege. Your use or disclosure of this information for
> any
> >> > purpose
> >> > >>>> other than that intended by its transmittal is strictly
> prohibited,
> >> > and may
> >> > >>>> subject you to fines and/or penalties under federal and state
> law.
> >> If
> >> > you
> >> > >>>> are not the intended recipient of this transmission, please
> DESTROY
> >> > ALL
> >> > >>>> COPIES RECEIVED and confirm destruction to the sender via return
> >> > >>>> transmittal.
> >> > >>>>
> >> > >>>>
> >> ---------------------------------------------------------------------
> >> > >>>> To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
> >> > >>>> For additional commands, e-mail: dev-help@airflow.apache.org
> >> > >>>>
> >> > >>>
> >> >
> >>
> >
>
Re: [DISCUSS] Switching base Python container images to python-standalone ?
Posted by Jarek Potiuk <ja...@potiuk.com>.
Did anyone experience any suspicious problems with the breeze CI image
prepared this way ?
I did not - python works as usual, all airflow features are working, I have
not seen any "negative" side effect of changing python to source-built one
- except slightly longer build time occasionally.
If ther are no other observations or issues - Aritra, I tink it might be
good time to look at trasplanting it to PROD image, Happy to either just
help and review or do it and have you review it - up to you :)
J.
On Fri, Jul 11, 2025 at 6:54 PM Jarek Potiuk <ja...@potiuk.com> wrote:
> And now I have to rebase my Python 3.13 PR to see if 3.13 also compiles
> well :D
>
> On Fri, Jul 11, 2025 at 6:53 PM Aritra Basu <ar...@gmail.com>
> wrote:
>
>> Yay! 🙌
>> --
>> Regards,
>> Aritra Basu
>>
>> On Fri, 11 Jul 2025, 9:48 pm Jarek Potiuk, <ja...@potiuk.com> wrote:
>>
>> > Ok. We got it merged again !
>> https://github.com/apache/airflow/pull/53150
>> > is merged. Once you build breeze images after rebase, python there will
>> be
>> > built directly from Python official sources (latest released patchlevel
>> for
>> > each version - and we have automation to upgrade them soon after new
>> > patchlevel are released).
>> >
>> > We are going to try it for a week or so and see if it all looks good, we
>> > will proceed to apply it to PROD images.
>> >
>> > J.
>> >
>> >
>> > On Sat, Jul 5, 2025 at 10:24 AM Jarek Potiuk <ja...@potiuk.com> wrote:
>> >
>> > > I merget it too early - reverting here
>> > > https://github.com/apache/airflow/pull/52900 as I found an issue with
>> > > other python versions that I overlooked
>> > >
>> > > On Sat, Jul 5, 2025 at 10:06 AM Jarek Potiuk <ja...@potiuk.com>
>> wrote:
>> > >
>> > >> FYI - we just merged Aritra's change to build Python in CI image from
>> > >> sources https://github.com/apache/airflow/pull/52265 -> we will run
>> it
>> > >> for a while in CI, i also want to test some self-upgrade scenarios.
>> > >>
>> > >> Then we will look at applying it to the PROD image if we find no
>> > >> surprises.
>> > >>
>> > >> It looks really good and we might have slightly more secure images
>> > >> produced (i.e. getting read of some libraries CVEs faster than the
>> > >> "official" images.
>> > >>
>> > >> Good job Aritra!
>> > >> J.
>> > >>
>> > >>
>> > >> On Thu, Jun 26, 2025 at 9:16 PM Jarek Potiuk <ja...@potiuk.com>
>> wrote:
>> > >>
>> > >>> It looks like building from sources on our CI using base debian
>> image
>> > >>> (with enabled optimizations) "just works" and takes 1m36s. -> and
>> > taking
>> > >>> into account that we already have pretty sophisticated CI-level
>> remote
>> > >>> caching that will allow to rebuild it only when either base image is
>> > >>> released or python is released - building sounds like a very good
>> idea.
>> > >>>
>> > >>> J.
>> > >>>
>> > >>> On Thu, Jun 26, 2025 at 7:02 PM Damian Shaw <
>> > >>> dshaw@striketechnologies.com> wrote:
>> > >>>
>> > >>>> Going from source is certainly going to give the most control, I
>> > >>>> personally choose python-build-standalone for my own images rather
>> > than
>> > >>>> building from source to outsource all the build choices, such as
>> what
>> > >>>> optimization flags to enable, what compiler tool chains to use,
>> etc.
>> > >>>>
>> > >>>> Damian
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>> -----Original Message-----
>> > >>>> From: Jarek Potiuk <ja...@potiuk.com>
>> > >>>> Sent: Thursday, June 26, 2025 10:15 AM
>> > >>>> To: dev@airflow.apache.org
>> > >>>> Subject: Re: [DISCUSS] Switching base Python container images to
>> > >>>> python-standalone ?
>> > >>>>
>> > >>>> Thanks Damian - very insightful. And yes - I did not really want to
>> > >>>> "diminish" the value of community images and work of the
>> maintainers,
>> > it
>> > >>>> was really more on the "we base our image security on the
>> assumptions
>> > that
>> > >>>> it's coming from "official" sources and surely there are some
>> > guardrails" -
>> > >>>> which turned out to be not really true.
>> > >>>>
>> > >>>> But.... We might not have to even use python standalone.
>> > >>>>
>> > >>>> Aritra just tested installation of Python straight from sources ->
>> > >>>> https://github.com/apache/airflow/pull/52265 and I was actually
>> > pretty
>> > >>>> surprised how non-problematic and fast it was - and it seems it
>> > passes all
>> > >>>> our CI tests. We just need to add a little pre-commit magic to get
>> > >>>> notified when we should update patchlevel version of Python when a
>> > new one
>> > >>>> is released, and we should be able to try it out in CI image - once
>> > it gets
>> > >>>> battle-tested with CI/breeze etc. we can transfer this to PROD
>> image
>> > as
>> > >>>> well. We already have an idea how to do it - our PROD images are
>> > optimized
>> > >>>> for size and do not contain "build essentials" - but I think we
>> > should be
>> > >>>> able to build Python in the "build" segment and simply copy
>> resulting
>> > >>>> binaries to the "main" segment - since in both cases we use the
>> same
>> > base
>> > >>>> image, such 1-1 copy should **just work** - we already do the same
>> > with
>> > >>>> installed python packages - we install them (including building
>> when
>> > >>>> needed) in build segment and we copy-over the installed .venv to
>> the
>> > >>>> main segment.
>> > >>>>
>> > >>>> So ... we might even not need a discussion - installing from Python
>> > >>>> sources is THE BEST
>> > >>>>
>> > >>>> J.
>> > >>>>
>> > >>>>
>> > >>>> On Wed, Jun 25, 2025 at 6:25 PM Damian Shaw <
>> > >>>> dshaw@striketechnologies.com>
>> > >>>> wrote:
>> > >>>>
>> > >>>> > First, I would like to thank the community members who have been
>> > >>>> > maintaining the Python docker images, it's one of those thankless
>> > >>>> > opensource infrastructure volunteer roles that they've been doing
>> > for
>> > >>>> > a long time. Unfortunately Docker assigns the title "Official
>> Image"
>> > >>>> > for various community run images, which creates a misconception
>> on
>> > the
>> > >>>> > guarantees being provided, and if I were a suspicious person I
>> would
>> > >>>> > say Docker creates this misconception on purpose to both get free
>> > work
>> > >>>> > from community members and make Docker seem more supported by
>> third
>> > >>>> > party organizations than it actually is.
>> > >>>> >
>> > >>>> > On the topic of python-build-standalone, I've been using it in
>> > >>>> > production for several months now and I'm fairly happy with it.
>> > >>>> >
>> > >>>> > However, one minor reproducibility issue I have when installing
>> > >>>> > python-build-standalone via uv, is that uv does not have an
>> ability
>> > to
>> > >>>> > pin to a specific build between uv versions, as uv hard codes a
>> > >>>> > mapping of Python version and platform to a specific build and
>> then
>> > >>>> > updates that mapping between releases. So, updating the version
>> of
>> > uv
>> > >>>> > between runs, or having two users run different versions of uv to
>> > >>>> > initialize the environment can change the results.
>> > >>>> >
>> > >>>> > While normally such build changes are trivial, if you look at
>> the uv
>> > >>>> > 0.7.8 and 0.7.9 changelogs you will see that sometimes they can
>> have
>> > >>>> > significant
>> > >>>> > impact: https://github.com/astral-sh/uv/releases/tag/0.7.8.
>> Also,
>> > >>>> this
>> > >>>> > email finally prompted make an issue on this topic:
>> > >>>> > https://github.com/astral-sh/uv/issues/14263.
>> > >>>> >
>> > >>>> > There are other ways to source python-build-standalone, such as
>> > >>>> > pbs-installer or writing your own script, but I've not yet spent
>> any
>> > >>>> > time investigating them, so I can't comment on them.
>> > >>>> >
>> > >>>> > Damian
>> > >>>> >
>> > >>>> > -----Original Message-----
>> > >>>> > From: Jarek Potiuk <ja...@potiuk.com>
>> > >>>> > Sent: Wednesday, June 25, 2025 10:44 AM
>> > >>>> > To: dev@airflow.apache.org
>> > >>>> > Subject: [DISCUSS] Switching base Python container images to
>> > >>>> > python-standalone ?
>> > >>>> >
>> > >>>> > Hello here,
>> > >>>> >
>> > >>>> > Together with Aritra we are looking into adding a few more
>> things to
>> > >>>> > our images (golang tool chain for CI image), also Shahar is
>> > >>>> > experimenting with Rust tool chain and I also recently realized
>> (by
>> > >>>> > some of the issues we had) that 'Docker Official Python Image'
>> that
>> > is
>> > >>>> > part of 'Official Program' [1] is not as 'Official' as I thought
>> so
>> > we
>> > >>>> > discuss about changing the base of our images (first CI and then
>> > when
>> > >>>> > we see it works fine - PROD)
>> > >>>> >
>> > >>>> > Currently we are using the 'Official' image - but after some
>> issues
>> > >>>> > and discussions with people at PyCon and FOSS Backstage (I had a
>> > >>>> > chance to talk to Python maintainers and even had a few beers
>> with
>> > >>>> > them) - it turned out that the official Python Image' is
>> maintained
>> > by
>> > >>>> > 'a community's which really is a few pretty random people - and
>> that
>> > >>>> > explains for example why we have sometimes unpacked security
>> > >>>> > vulnerabilities in setuptools etc. - because they made some
>> > >>>> > compatibility choices and decisions that do not allow them to
>> > upgrade
>> > >>>> > easily, also they had some delays in releasing updated Python
>> > >>>> > versions. And Docker does not **really** do much vetting there.
>> > >>>> >
>> > >>>> > So I think it would be good to switch how we build the base for
>> our
>> > >>>> images.
>> > >>>> > And following the experience of `uv python` [2] - it seems that
>> > maybe
>> > >>>> > using "python-standalone" [3] project is a good alternative. It's
>> > >>>> > managed by Astral now (so yes - another dependency on them), but
>> > what
>> > >>>> > you have with it you have practically 100% complete Python
>> > interpreter
>> > >>>> > installed in seconds.
>> > >>>> > We could continue using debian-slim as a "base, base image" - and
>> > >>>> > install python using "python-standalone". There are a few
>> > >>>> > incompatibilities [4] of the distributions of Python, but there
>> are
>> > >>>> > very few and mostly related to some obscure systems
>> (compatibilities
>> > >>>> > with terminal in REPL, and gtk / UI integration that is anyhow
>> not
>> > >>>> > really working in "standard" Python distributions).
>> > >>>> >
>> > >>>> > I would love to hear what you think - happy to get any feedback/
>> > >>>> > insights, suggestions and answer additional questions, provide
>> some
>> > >>>> > links to past "troubles" we had with Python "Official" images
>> etc.
>> > >>>> >
>> > >>>> > J.
>> > >>>> >
>> > >>>> >
>> > >>>> > [1] Official Python Images - https://hub.docker.com/_/python
>> [2] UV
>> > >>>> > Python installation -
>> > >>>> https://docs.astral.sh/uv/guides/install-python/
>> > >>>> > [3] Python Standalone project -
>> > >>>> > https://github.com/astral-sh/python-build-standalone
>> > >>>> > [4] Python Standalone incompatibilities -
>> > >>>> >
>> > >>>>
>> > https://gregoryszorc.com/docs/python-build-standalone/main/quirks.html
>> > >>>> > ________________________________
>> > >>>> > Strike Technologies, LLC (“Strike”) is part of the GTS family of
>> > >>>> > companies. Strike is a technology solutions provider, and is not
>> a
>> > >>>> > broker or dealer and does not transact any securities related
>> > business
>> > >>>> > directly whatsoever. This communication is the property of Strike
>> > and
>> > >>>> > its affiliates, and does not constitute an offer to sell or the
>> > >>>> > solicitation of an offer to buy any security in any
>> jurisdiction. It
>> > >>>> > is intended only for the person to whom it is addressed and may
>> > >>>> > contain information that is privileged, confidential, or
>> otherwise
>> > >>>> protected from disclosure.
>> > >>>> > Distribution or copying of this communication, or the information
>> > >>>> > contained herein, by anyone other than the intended recipient is
>> > >>>> > prohibited. If you have received this communication in error,
>> please
>> > >>>> > immediately notify Strike at info@striketechnologies.com, and
>> > delete
>> > >>>> and destroy any copies hereof.
>> > >>>> > ________________________________
>> > >>>> >
>> > >>>> > CONFIDENTIALITY / PRIVILEGE NOTICE: This transmission and any
>> > >>>> > attachments are intended solely for the addressee. This
>> transmission
>> > >>>> > is covered by the Electronic Communications Privacy Act, 18 U.S.C
>> > >>>> > ''2510-2521. The information contained in this transmission is
>> > >>>> > confidential in nature and protected from further use or
>> disclosure
>> > >>>> > under U.S. Pub. L. 106-102, 113 U.S. Stat. 1338 (1999), and may
>> be
>> > >>>> > subject to attorney-client or other legal privilege. Your use or
>> > >>>> > disclosure of this information for any purpose other than that
>> > >>>> > intended by its transmittal is strictly prohibited, and may
>> subject
>> > >>>> > you to fines and/or penalties under federal and state law. If you
>> > are
>> > >>>> > not the intended recipient of this transmission, please DESTROY
>> ALL
>> > >>>> > COPIES RECEIVED and confirm destruction to the sender via return
>> > >>>> transmittal.
>> > >>>> >
>> > >>>> ________________________________
>> > >>>> Strike Technologies, LLC (“Strike”) is part of the GTS family of
>> > >>>> companies. Strike is a technology solutions provider, and is not a
>> > broker
>> > >>>> or dealer and does not transact any securities related business
>> > directly
>> > >>>> whatsoever. This communication is the property of Strike and its
>> > >>>> affiliates, and does not constitute an offer to sell or the
>> > solicitation of
>> > >>>> an offer to buy any security in any jurisdiction. It is intended
>> only
>> > for
>> > >>>> the person to whom it is addressed and may contain information
>> that is
>> > >>>> privileged, confidential, or otherwise protected from disclosure.
>> > >>>> Distribution or copying of this communication, or the information
>> > contained
>> > >>>> herein, by anyone other than the intended recipient is prohibited.
>> If
>> > you
>> > >>>> have received this communication in error, please immediately
>> notify
>> > Strike
>> > >>>> at info@striketechnologies.com, and delete and destroy any copies
>> > >>>> hereof.
>> > >>>> ________________________________
>> > >>>>
>> > >>>> CONFIDENTIALITY / PRIVILEGE NOTICE: This transmission and any
>> > >>>> attachments are intended solely for the addressee. This
>> transmission
>> > is
>> > >>>> covered by the Electronic Communications Privacy Act, 18 U.S.C
>> > ''2510-2521.
>> > >>>> The information contained in this transmission is confidential in
>> > nature
>> > >>>> and protected from further use or disclosure under U.S. Pub. L.
>> > 106-102,
>> > >>>> 113 U.S. Stat. 1338 (1999), and may be subject to attorney-client
>> or
>> > other
>> > >>>> legal privilege. Your use or disclosure of this information for any
>> > purpose
>> > >>>> other than that intended by its transmittal is strictly prohibited,
>> > and may
>> > >>>> subject you to fines and/or penalties under federal and state law.
>> If
>> > you
>> > >>>> are not the intended recipient of this transmission, please DESTROY
>> > ALL
>> > >>>> COPIES RECEIVED and confirm destruction to the sender via return
>> > >>>> transmittal.
>> > >>>>
>> > >>>>
>> ---------------------------------------------------------------------
>> > >>>> To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
>> > >>>> For additional commands, e-mail: dev-help@airflow.apache.org
>> > >>>>
>> > >>>
>> >
>>
>
Re: [DISCUSS] Switching base Python container images to python-standalone ?
Posted by Jarek Potiuk <ja...@potiuk.com>.
And now I have to rebase my Python 3.13 PR to see if 3.13 also compiles
well :D
On Fri, Jul 11, 2025 at 6:53 PM Aritra Basu <ar...@gmail.com>
wrote:
> Yay! 🙌
> --
> Regards,
> Aritra Basu
>
> On Fri, 11 Jul 2025, 9:48 pm Jarek Potiuk, <ja...@potiuk.com> wrote:
>
> > Ok. We got it merged again !
> https://github.com/apache/airflow/pull/53150
> > is merged. Once you build breeze images after rebase, python there will
> be
> > built directly from Python official sources (latest released patchlevel
> for
> > each version - and we have automation to upgrade them soon after new
> > patchlevel are released).
> >
> > We are going to try it for a week or so and see if it all looks good, we
> > will proceed to apply it to PROD images.
> >
> > J.
> >
> >
> > On Sat, Jul 5, 2025 at 10:24 AM Jarek Potiuk <ja...@potiuk.com> wrote:
> >
> > > I merget it too early - reverting here
> > > https://github.com/apache/airflow/pull/52900 as I found an issue with
> > > other python versions that I overlooked
> > >
> > > On Sat, Jul 5, 2025 at 10:06 AM Jarek Potiuk <ja...@potiuk.com> wrote:
> > >
> > >> FYI - we just merged Aritra's change to build Python in CI image from
> > >> sources https://github.com/apache/airflow/pull/52265 -> we will run
> it
> > >> for a while in CI, i also want to test some self-upgrade scenarios.
> > >>
> > >> Then we will look at applying it to the PROD image if we find no
> > >> surprises.
> > >>
> > >> It looks really good and we might have slightly more secure images
> > >> produced (i.e. getting read of some libraries CVEs faster than the
> > >> "official" images.
> > >>
> > >> Good job Aritra!
> > >> J.
> > >>
> > >>
> > >> On Thu, Jun 26, 2025 at 9:16 PM Jarek Potiuk <ja...@potiuk.com>
> wrote:
> > >>
> > >>> It looks like building from sources on our CI using base debian image
> > >>> (with enabled optimizations) "just works" and takes 1m36s. -> and
> > taking
> > >>> into account that we already have pretty sophisticated CI-level
> remote
> > >>> caching that will allow to rebuild it only when either base image is
> > >>> released or python is released - building sounds like a very good
> idea.
> > >>>
> > >>> J.
> > >>>
> > >>> On Thu, Jun 26, 2025 at 7:02 PM Damian Shaw <
> > >>> dshaw@striketechnologies.com> wrote:
> > >>>
> > >>>> Going from source is certainly going to give the most control, I
> > >>>> personally choose python-build-standalone for my own images rather
> > than
> > >>>> building from source to outsource all the build choices, such as
> what
> > >>>> optimization flags to enable, what compiler tool chains to use, etc.
> > >>>>
> > >>>> Damian
> > >>>>
> > >>>>
> > >>>>
> > >>>> -----Original Message-----
> > >>>> From: Jarek Potiuk <ja...@potiuk.com>
> > >>>> Sent: Thursday, June 26, 2025 10:15 AM
> > >>>> To: dev@airflow.apache.org
> > >>>> Subject: Re: [DISCUSS] Switching base Python container images to
> > >>>> python-standalone ?
> > >>>>
> > >>>> Thanks Damian - very insightful. And yes - I did not really want to
> > >>>> "diminish" the value of community images and work of the
> maintainers,
> > it
> > >>>> was really more on the "we base our image security on the
> assumptions
> > that
> > >>>> it's coming from "official" sources and surely there are some
> > guardrails" -
> > >>>> which turned out to be not really true.
> > >>>>
> > >>>> But.... We might not have to even use python standalone.
> > >>>>
> > >>>> Aritra just tested installation of Python straight from sources ->
> > >>>> https://github.com/apache/airflow/pull/52265 and I was actually
> > pretty
> > >>>> surprised how non-problematic and fast it was - and it seems it
> > passes all
> > >>>> our CI tests. We just need to add a little pre-commit magic to get
> > >>>> notified when we should update patchlevel version of Python when a
> > new one
> > >>>> is released, and we should be able to try it out in CI image - once
> > it gets
> > >>>> battle-tested with CI/breeze etc. we can transfer this to PROD image
> > as
> > >>>> well. We already have an idea how to do it - our PROD images are
> > optimized
> > >>>> for size and do not contain "build essentials" - but I think we
> > should be
> > >>>> able to build Python in the "build" segment and simply copy
> resulting
> > >>>> binaries to the "main" segment - since in both cases we use the same
> > base
> > >>>> image, such 1-1 copy should **just work** - we already do the same
> > with
> > >>>> installed python packages - we install them (including building when
> > >>>> needed) in build segment and we copy-over the installed .venv to the
> > >>>> main segment.
> > >>>>
> > >>>> So ... we might even not need a discussion - installing from Python
> > >>>> sources is THE BEST
> > >>>>
> > >>>> J.
> > >>>>
> > >>>>
> > >>>> On Wed, Jun 25, 2025 at 6:25 PM Damian Shaw <
> > >>>> dshaw@striketechnologies.com>
> > >>>> wrote:
> > >>>>
> > >>>> > First, I would like to thank the community members who have been
> > >>>> > maintaining the Python docker images, it's one of those thankless
> > >>>> > opensource infrastructure volunteer roles that they've been doing
> > for
> > >>>> > a long time. Unfortunately Docker assigns the title "Official
> Image"
> > >>>> > for various community run images, which creates a misconception on
> > the
> > >>>> > guarantees being provided, and if I were a suspicious person I
> would
> > >>>> > say Docker creates this misconception on purpose to both get free
> > work
> > >>>> > from community members and make Docker seem more supported by
> third
> > >>>> > party organizations than it actually is.
> > >>>> >
> > >>>> > On the topic of python-build-standalone, I've been using it in
> > >>>> > production for several months now and I'm fairly happy with it.
> > >>>> >
> > >>>> > However, one minor reproducibility issue I have when installing
> > >>>> > python-build-standalone via uv, is that uv does not have an
> ability
> > to
> > >>>> > pin to a specific build between uv versions, as uv hard codes a
> > >>>> > mapping of Python version and platform to a specific build and
> then
> > >>>> > updates that mapping between releases. So, updating the version of
> > uv
> > >>>> > between runs, or having two users run different versions of uv to
> > >>>> > initialize the environment can change the results.
> > >>>> >
> > >>>> > While normally such build changes are trivial, if you look at the
> uv
> > >>>> > 0.7.8 and 0.7.9 changelogs you will see that sometimes they can
> have
> > >>>> > significant
> > >>>> > impact: https://github.com/astral-sh/uv/releases/tag/0.7.8. Also,
> > >>>> this
> > >>>> > email finally prompted make an issue on this topic:
> > >>>> > https://github.com/astral-sh/uv/issues/14263.
> > >>>> >
> > >>>> > There are other ways to source python-build-standalone, such as
> > >>>> > pbs-installer or writing your own script, but I've not yet spent
> any
> > >>>> > time investigating them, so I can't comment on them.
> > >>>> >
> > >>>> > Damian
> > >>>> >
> > >>>> > -----Original Message-----
> > >>>> > From: Jarek Potiuk <ja...@potiuk.com>
> > >>>> > Sent: Wednesday, June 25, 2025 10:44 AM
> > >>>> > To: dev@airflow.apache.org
> > >>>> > Subject: [DISCUSS] Switching base Python container images to
> > >>>> > python-standalone ?
> > >>>> >
> > >>>> > Hello here,
> > >>>> >
> > >>>> > Together with Aritra we are looking into adding a few more things
> to
> > >>>> > our images (golang tool chain for CI image), also Shahar is
> > >>>> > experimenting with Rust tool chain and I also recently realized
> (by
> > >>>> > some of the issues we had) that 'Docker Official Python Image'
> that
> > is
> > >>>> > part of 'Official Program' [1] is not as 'Official' as I thought
> so
> > we
> > >>>> > discuss about changing the base of our images (first CI and then
> > when
> > >>>> > we see it works fine - PROD)
> > >>>> >
> > >>>> > Currently we are using the 'Official' image - but after some
> issues
> > >>>> > and discussions with people at PyCon and FOSS Backstage (I had a
> > >>>> > chance to talk to Python maintainers and even had a few beers with
> > >>>> > them) - it turned out that the official Python Image' is
> maintained
> > by
> > >>>> > 'a community's which really is a few pretty random people - and
> that
> > >>>> > explains for example why we have sometimes unpacked security
> > >>>> > vulnerabilities in setuptools etc. - because they made some
> > >>>> > compatibility choices and decisions that do not allow them to
> > upgrade
> > >>>> > easily, also they had some delays in releasing updated Python
> > >>>> > versions. And Docker does not **really** do much vetting there.
> > >>>> >
> > >>>> > So I think it would be good to switch how we build the base for
> our
> > >>>> images.
> > >>>> > And following the experience of `uv python` [2] - it seems that
> > maybe
> > >>>> > using "python-standalone" [3] project is a good alternative. It's
> > >>>> > managed by Astral now (so yes - another dependency on them), but
> > what
> > >>>> > you have with it you have practically 100% complete Python
> > interpreter
> > >>>> > installed in seconds.
> > >>>> > We could continue using debian-slim as a "base, base image" - and
> > >>>> > install python using "python-standalone". There are a few
> > >>>> > incompatibilities [4] of the distributions of Python, but there
> are
> > >>>> > very few and mostly related to some obscure systems
> (compatibilities
> > >>>> > with terminal in REPL, and gtk / UI integration that is anyhow not
> > >>>> > really working in "standard" Python distributions).
> > >>>> >
> > >>>> > I would love to hear what you think - happy to get any feedback/
> > >>>> > insights, suggestions and answer additional questions, provide
> some
> > >>>> > links to past "troubles" we had with Python "Official" images etc.
> > >>>> >
> > >>>> > J.
> > >>>> >
> > >>>> >
> > >>>> > [1] Official Python Images - https://hub.docker.com/_/python [2]
> UV
> > >>>> > Python installation -
> > >>>> https://docs.astral.sh/uv/guides/install-python/
> > >>>> > [3] Python Standalone project -
> > >>>> > https://github.com/astral-sh/python-build-standalone
> > >>>> > [4] Python Standalone incompatibilities -
> > >>>> >
> > >>>>
> > https://gregoryszorc.com/docs/python-build-standalone/main/quirks.html
> > >>>> > ________________________________
> > >>>> > Strike Technologies, LLC (“Strike”) is part of the GTS family of
> > >>>> > companies. Strike is a technology solutions provider, and is not a
> > >>>> > broker or dealer and does not transact any securities related
> > business
> > >>>> > directly whatsoever. This communication is the property of Strike
> > and
> > >>>> > its affiliates, and does not constitute an offer to sell or the
> > >>>> > solicitation of an offer to buy any security in any jurisdiction.
> It
> > >>>> > is intended only for the person to whom it is addressed and may
> > >>>> > contain information that is privileged, confidential, or otherwise
> > >>>> protected from disclosure.
> > >>>> > Distribution or copying of this communication, or the information
> > >>>> > contained herein, by anyone other than the intended recipient is
> > >>>> > prohibited. If you have received this communication in error,
> please
> > >>>> > immediately notify Strike at info@striketechnologies.com, and
> > delete
> > >>>> and destroy any copies hereof.
> > >>>> > ________________________________
> > >>>> >
> > >>>> > CONFIDENTIALITY / PRIVILEGE NOTICE: This transmission and any
> > >>>> > attachments are intended solely for the addressee. This
> transmission
> > >>>> > is covered by the Electronic Communications Privacy Act, 18 U.S.C
> > >>>> > ''2510-2521. The information contained in this transmission is
> > >>>> > confidential in nature and protected from further use or
> disclosure
> > >>>> > under U.S. Pub. L. 106-102, 113 U.S. Stat. 1338 (1999), and may be
> > >>>> > subject to attorney-client or other legal privilege. Your use or
> > >>>> > disclosure of this information for any purpose other than that
> > >>>> > intended by its transmittal is strictly prohibited, and may
> subject
> > >>>> > you to fines and/or penalties under federal and state law. If you
> > are
> > >>>> > not the intended recipient of this transmission, please DESTROY
> ALL
> > >>>> > COPIES RECEIVED and confirm destruction to the sender via return
> > >>>> transmittal.
> > >>>> >
> > >>>> ________________________________
> > >>>> Strike Technologies, LLC (“Strike”) is part of the GTS family of
> > >>>> companies. Strike is a technology solutions provider, and is not a
> > broker
> > >>>> or dealer and does not transact any securities related business
> > directly
> > >>>> whatsoever. This communication is the property of Strike and its
> > >>>> affiliates, and does not constitute an offer to sell or the
> > solicitation of
> > >>>> an offer to buy any security in any jurisdiction. It is intended
> only
> > for
> > >>>> the person to whom it is addressed and may contain information that
> is
> > >>>> privileged, confidential, or otherwise protected from disclosure.
> > >>>> Distribution or copying of this communication, or the information
> > contained
> > >>>> herein, by anyone other than the intended recipient is prohibited.
> If
> > you
> > >>>> have received this communication in error, please immediately notify
> > Strike
> > >>>> at info@striketechnologies.com, and delete and destroy any copies
> > >>>> hereof.
> > >>>> ________________________________
> > >>>>
> > >>>> CONFIDENTIALITY / PRIVILEGE NOTICE: This transmission and any
> > >>>> attachments are intended solely for the addressee. This transmission
> > is
> > >>>> covered by the Electronic Communications Privacy Act, 18 U.S.C
> > ''2510-2521.
> > >>>> The information contained in this transmission is confidential in
> > nature
> > >>>> and protected from further use or disclosure under U.S. Pub. L.
> > 106-102,
> > >>>> 113 U.S. Stat. 1338 (1999), and may be subject to attorney-client or
> > other
> > >>>> legal privilege. Your use or disclosure of this information for any
> > purpose
> > >>>> other than that intended by its transmittal is strictly prohibited,
> > and may
> > >>>> subject you to fines and/or penalties under federal and state law.
> If
> > you
> > >>>> are not the intended recipient of this transmission, please DESTROY
> > ALL
> > >>>> COPIES RECEIVED and confirm destruction to the sender via return
> > >>>> transmittal.
> > >>>>
> > >>>>
> ---------------------------------------------------------------------
> > >>>> To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
> > >>>> For additional commands, e-mail: dev-help@airflow.apache.org
> > >>>>
> > >>>
> >
>
Re: [DISCUSS] Switching base Python container images to python-standalone ?
Posted by Aritra Basu <ar...@gmail.com>.
Yay! 🙌
--
Regards,
Aritra Basu
On Fri, 11 Jul 2025, 9:48 pm Jarek Potiuk, <ja...@potiuk.com> wrote:
> Ok. We got it merged again ! https://github.com/apache/airflow/pull/53150
> is merged. Once you build breeze images after rebase, python there will be
> built directly from Python official sources (latest released patchlevel for
> each version - and we have automation to upgrade them soon after new
> patchlevel are released).
>
> We are going to try it for a week or so and see if it all looks good, we
> will proceed to apply it to PROD images.
>
> J.
>
>
> On Sat, Jul 5, 2025 at 10:24 AM Jarek Potiuk <ja...@potiuk.com> wrote:
>
> > I merget it too early - reverting here
> > https://github.com/apache/airflow/pull/52900 as I found an issue with
> > other python versions that I overlooked
> >
> > On Sat, Jul 5, 2025 at 10:06 AM Jarek Potiuk <ja...@potiuk.com> wrote:
> >
> >> FYI - we just merged Aritra's change to build Python in CI image from
> >> sources https://github.com/apache/airflow/pull/52265 -> we will run it
> >> for a while in CI, i also want to test some self-upgrade scenarios.
> >>
> >> Then we will look at applying it to the PROD image if we find no
> >> surprises.
> >>
> >> It looks really good and we might have slightly more secure images
> >> produced (i.e. getting read of some libraries CVEs faster than the
> >> "official" images.
> >>
> >> Good job Aritra!
> >> J.
> >>
> >>
> >> On Thu, Jun 26, 2025 at 9:16 PM Jarek Potiuk <ja...@potiuk.com> wrote:
> >>
> >>> It looks like building from sources on our CI using base debian image
> >>> (with enabled optimizations) "just works" and takes 1m36s. -> and
> taking
> >>> into account that we already have pretty sophisticated CI-level remote
> >>> caching that will allow to rebuild it only when either base image is
> >>> released or python is released - building sounds like a very good idea.
> >>>
> >>> J.
> >>>
> >>> On Thu, Jun 26, 2025 at 7:02 PM Damian Shaw <
> >>> dshaw@striketechnologies.com> wrote:
> >>>
> >>>> Going from source is certainly going to give the most control, I
> >>>> personally choose python-build-standalone for my own images rather
> than
> >>>> building from source to outsource all the build choices, such as what
> >>>> optimization flags to enable, what compiler tool chains to use, etc.
> >>>>
> >>>> Damian
> >>>>
> >>>>
> >>>>
> >>>> -----Original Message-----
> >>>> From: Jarek Potiuk <ja...@potiuk.com>
> >>>> Sent: Thursday, June 26, 2025 10:15 AM
> >>>> To: dev@airflow.apache.org
> >>>> Subject: Re: [DISCUSS] Switching base Python container images to
> >>>> python-standalone ?
> >>>>
> >>>> Thanks Damian - very insightful. And yes - I did not really want to
> >>>> "diminish" the value of community images and work of the maintainers,
> it
> >>>> was really more on the "we base our image security on the assumptions
> that
> >>>> it's coming from "official" sources and surely there are some
> guardrails" -
> >>>> which turned out to be not really true.
> >>>>
> >>>> But.... We might not have to even use python standalone.
> >>>>
> >>>> Aritra just tested installation of Python straight from sources ->
> >>>> https://github.com/apache/airflow/pull/52265 and I was actually
> pretty
> >>>> surprised how non-problematic and fast it was - and it seems it
> passes all
> >>>> our CI tests. We just need to add a little pre-commit magic to get
> >>>> notified when we should update patchlevel version of Python when a
> new one
> >>>> is released, and we should be able to try it out in CI image - once
> it gets
> >>>> battle-tested with CI/breeze etc. we can transfer this to PROD image
> as
> >>>> well. We already have an idea how to do it - our PROD images are
> optimized
> >>>> for size and do not contain "build essentials" - but I think we
> should be
> >>>> able to build Python in the "build" segment and simply copy resulting
> >>>> binaries to the "main" segment - since in both cases we use the same
> base
> >>>> image, such 1-1 copy should **just work** - we already do the same
> with
> >>>> installed python packages - we install them (including building when
> >>>> needed) in build segment and we copy-over the installed .venv to the
> >>>> main segment.
> >>>>
> >>>> So ... we might even not need a discussion - installing from Python
> >>>> sources is THE BEST
> >>>>
> >>>> J.
> >>>>
> >>>>
> >>>> On Wed, Jun 25, 2025 at 6:25 PM Damian Shaw <
> >>>> dshaw@striketechnologies.com>
> >>>> wrote:
> >>>>
> >>>> > First, I would like to thank the community members who have been
> >>>> > maintaining the Python docker images, it's one of those thankless
> >>>> > opensource infrastructure volunteer roles that they've been doing
> for
> >>>> > a long time. Unfortunately Docker assigns the title "Official Image"
> >>>> > for various community run images, which creates a misconception on
> the
> >>>> > guarantees being provided, and if I were a suspicious person I would
> >>>> > say Docker creates this misconception on purpose to both get free
> work
> >>>> > from community members and make Docker seem more supported by third
> >>>> > party organizations than it actually is.
> >>>> >
> >>>> > On the topic of python-build-standalone, I've been using it in
> >>>> > production for several months now and I'm fairly happy with it.
> >>>> >
> >>>> > However, one minor reproducibility issue I have when installing
> >>>> > python-build-standalone via uv, is that uv does not have an ability
> to
> >>>> > pin to a specific build between uv versions, as uv hard codes a
> >>>> > mapping of Python version and platform to a specific build and then
> >>>> > updates that mapping between releases. So, updating the version of
> uv
> >>>> > between runs, or having two users run different versions of uv to
> >>>> > initialize the environment can change the results.
> >>>> >
> >>>> > While normally such build changes are trivial, if you look at the uv
> >>>> > 0.7.8 and 0.7.9 changelogs you will see that sometimes they can have
> >>>> > significant
> >>>> > impact: https://github.com/astral-sh/uv/releases/tag/0.7.8. Also,
> >>>> this
> >>>> > email finally prompted make an issue on this topic:
> >>>> > https://github.com/astral-sh/uv/issues/14263.
> >>>> >
> >>>> > There are other ways to source python-build-standalone, such as
> >>>> > pbs-installer or writing your own script, but I've not yet spent any
> >>>> > time investigating them, so I can't comment on them.
> >>>> >
> >>>> > Damian
> >>>> >
> >>>> > -----Original Message-----
> >>>> > From: Jarek Potiuk <ja...@potiuk.com>
> >>>> > Sent: Wednesday, June 25, 2025 10:44 AM
> >>>> > To: dev@airflow.apache.org
> >>>> > Subject: [DISCUSS] Switching base Python container images to
> >>>> > python-standalone ?
> >>>> >
> >>>> > Hello here,
> >>>> >
> >>>> > Together with Aritra we are looking into adding a few more things to
> >>>> > our images (golang tool chain for CI image), also Shahar is
> >>>> > experimenting with Rust tool chain and I also recently realized (by
> >>>> > some of the issues we had) that 'Docker Official Python Image' that
> is
> >>>> > part of 'Official Program' [1] is not as 'Official' as I thought so
> we
> >>>> > discuss about changing the base of our images (first CI and then
> when
> >>>> > we see it works fine - PROD)
> >>>> >
> >>>> > Currently we are using the 'Official' image - but after some issues
> >>>> > and discussions with people at PyCon and FOSS Backstage (I had a
> >>>> > chance to talk to Python maintainers and even had a few beers with
> >>>> > them) - it turned out that the official Python Image' is maintained
> by
> >>>> > 'a community's which really is a few pretty random people - and that
> >>>> > explains for example why we have sometimes unpacked security
> >>>> > vulnerabilities in setuptools etc. - because they made some
> >>>> > compatibility choices and decisions that do not allow them to
> upgrade
> >>>> > easily, also they had some delays in releasing updated Python
> >>>> > versions. And Docker does not **really** do much vetting there.
> >>>> >
> >>>> > So I think it would be good to switch how we build the base for our
> >>>> images.
> >>>> > And following the experience of `uv python` [2] - it seems that
> maybe
> >>>> > using "python-standalone" [3] project is a good alternative. It's
> >>>> > managed by Astral now (so yes - another dependency on them), but
> what
> >>>> > you have with it you have practically 100% complete Python
> interpreter
> >>>> > installed in seconds.
> >>>> > We could continue using debian-slim as a "base, base image" - and
> >>>> > install python using "python-standalone". There are a few
> >>>> > incompatibilities [4] of the distributions of Python, but there are
> >>>> > very few and mostly related to some obscure systems (compatibilities
> >>>> > with terminal in REPL, and gtk / UI integration that is anyhow not
> >>>> > really working in "standard" Python distributions).
> >>>> >
> >>>> > I would love to hear what you think - happy to get any feedback/
> >>>> > insights, suggestions and answer additional questions, provide some
> >>>> > links to past "troubles" we had with Python "Official" images etc.
> >>>> >
> >>>> > J.
> >>>> >
> >>>> >
> >>>> > [1] Official Python Images - https://hub.docker.com/_/python [2] UV
> >>>> > Python installation -
> >>>> https://docs.astral.sh/uv/guides/install-python/
> >>>> > [3] Python Standalone project -
> >>>> > https://github.com/astral-sh/python-build-standalone
> >>>> > [4] Python Standalone incompatibilities -
> >>>> >
> >>>>
> https://gregoryszorc.com/docs/python-build-standalone/main/quirks.html
> >>>> > ________________________________
> >>>> > Strike Technologies, LLC (“Strike”) is part of the GTS family of
> >>>> > companies. Strike is a technology solutions provider, and is not a
> >>>> > broker or dealer and does not transact any securities related
> business
> >>>> > directly whatsoever. This communication is the property of Strike
> and
> >>>> > its affiliates, and does not constitute an offer to sell or the
> >>>> > solicitation of an offer to buy any security in any jurisdiction. It
> >>>> > is intended only for the person to whom it is addressed and may
> >>>> > contain information that is privileged, confidential, or otherwise
> >>>> protected from disclosure.
> >>>> > Distribution or copying of this communication, or the information
> >>>> > contained herein, by anyone other than the intended recipient is
> >>>> > prohibited. If you have received this communication in error, please
> >>>> > immediately notify Strike at info@striketechnologies.com, and
> delete
> >>>> and destroy any copies hereof.
> >>>> > ________________________________
> >>>> >
> >>>> > CONFIDENTIALITY / PRIVILEGE NOTICE: This transmission and any
> >>>> > attachments are intended solely for the addressee. This transmission
> >>>> > is covered by the Electronic Communications Privacy Act, 18 U.S.C
> >>>> > ''2510-2521. The information contained in this transmission is
> >>>> > confidential in nature and protected from further use or disclosure
> >>>> > under U.S. Pub. L. 106-102, 113 U.S. Stat. 1338 (1999), and may be
> >>>> > subject to attorney-client or other legal privilege. Your use or
> >>>> > disclosure of this information for any purpose other than that
> >>>> > intended by its transmittal is strictly prohibited, and may subject
> >>>> > you to fines and/or penalties under federal and state law. If you
> are
> >>>> > not the intended recipient of this transmission, please DESTROY ALL
> >>>> > COPIES RECEIVED and confirm destruction to the sender via return
> >>>> transmittal.
> >>>> >
> >>>> ________________________________
> >>>> Strike Technologies, LLC (“Strike”) is part of the GTS family of
> >>>> companies. Strike is a technology solutions provider, and is not a
> broker
> >>>> or dealer and does not transact any securities related business
> directly
> >>>> whatsoever. This communication is the property of Strike and its
> >>>> affiliates, and does not constitute an offer to sell or the
> solicitation of
> >>>> an offer to buy any security in any jurisdiction. It is intended only
> for
> >>>> the person to whom it is addressed and may contain information that is
> >>>> privileged, confidential, or otherwise protected from disclosure.
> >>>> Distribution or copying of this communication, or the information
> contained
> >>>> herein, by anyone other than the intended recipient is prohibited. If
> you
> >>>> have received this communication in error, please immediately notify
> Strike
> >>>> at info@striketechnologies.com, and delete and destroy any copies
> >>>> hereof.
> >>>> ________________________________
> >>>>
> >>>> CONFIDENTIALITY / PRIVILEGE NOTICE: This transmission and any
> >>>> attachments are intended solely for the addressee. This transmission
> is
> >>>> covered by the Electronic Communications Privacy Act, 18 U.S.C
> ''2510-2521.
> >>>> The information contained in this transmission is confidential in
> nature
> >>>> and protected from further use or disclosure under U.S. Pub. L.
> 106-102,
> >>>> 113 U.S. Stat. 1338 (1999), and may be subject to attorney-client or
> other
> >>>> legal privilege. Your use or disclosure of this information for any
> purpose
> >>>> other than that intended by its transmittal is strictly prohibited,
> and may
> >>>> subject you to fines and/or penalties under federal and state law. If
> you
> >>>> are not the intended recipient of this transmission, please DESTROY
> ALL
> >>>> COPIES RECEIVED and confirm destruction to the sender via return
> >>>> transmittal.
> >>>>
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
> >>>> For additional commands, e-mail: dev-help@airflow.apache.org
> >>>>
> >>>
>
Re: [DISCUSS] Switching base Python container images to python-standalone ?
Posted by Jarek Potiuk <ja...@potiuk.com>.
Ok. We got it merged again ! https://github.com/apache/airflow/pull/53150
is merged. Once you build breeze images after rebase, python there will be
built directly from Python official sources (latest released patchlevel for
each version - and we have automation to upgrade them soon after new
patchlevel are released).
We are going to try it for a week or so and see if it all looks good, we
will proceed to apply it to PROD images.
J.
On Sat, Jul 5, 2025 at 10:24 AM Jarek Potiuk <ja...@potiuk.com> wrote:
> I merget it too early - reverting here
> https://github.com/apache/airflow/pull/52900 as I found an issue with
> other python versions that I overlooked
>
> On Sat, Jul 5, 2025 at 10:06 AM Jarek Potiuk <ja...@potiuk.com> wrote:
>
>> FYI - we just merged Aritra's change to build Python in CI image from
>> sources https://github.com/apache/airflow/pull/52265 -> we will run it
>> for a while in CI, i also want to test some self-upgrade scenarios.
>>
>> Then we will look at applying it to the PROD image if we find no
>> surprises.
>>
>> It looks really good and we might have slightly more secure images
>> produced (i.e. getting read of some libraries CVEs faster than the
>> "official" images.
>>
>> Good job Aritra!
>> J.
>>
>>
>> On Thu, Jun 26, 2025 at 9:16 PM Jarek Potiuk <ja...@potiuk.com> wrote:
>>
>>> It looks like building from sources on our CI using base debian image
>>> (with enabled optimizations) "just works" and takes 1m36s. -> and taking
>>> into account that we already have pretty sophisticated CI-level remote
>>> caching that will allow to rebuild it only when either base image is
>>> released or python is released - building sounds like a very good idea.
>>>
>>> J.
>>>
>>> On Thu, Jun 26, 2025 at 7:02 PM Damian Shaw <
>>> dshaw@striketechnologies.com> wrote:
>>>
>>>> Going from source is certainly going to give the most control, I
>>>> personally choose python-build-standalone for my own images rather than
>>>> building from source to outsource all the build choices, such as what
>>>> optimization flags to enable, what compiler tool chains to use, etc.
>>>>
>>>> Damian
>>>>
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: Jarek Potiuk <ja...@potiuk.com>
>>>> Sent: Thursday, June 26, 2025 10:15 AM
>>>> To: dev@airflow.apache.org
>>>> Subject: Re: [DISCUSS] Switching base Python container images to
>>>> python-standalone ?
>>>>
>>>> Thanks Damian - very insightful. And yes - I did not really want to
>>>> "diminish" the value of community images and work of the maintainers, it
>>>> was really more on the "we base our image security on the assumptions that
>>>> it's coming from "official" sources and surely there are some guardrails" -
>>>> which turned out to be not really true.
>>>>
>>>> But.... We might not have to even use python standalone.
>>>>
>>>> Aritra just tested installation of Python straight from sources ->
>>>> https://github.com/apache/airflow/pull/52265 and I was actually pretty
>>>> surprised how non-problematic and fast it was - and it seems it passes all
>>>> our CI tests. We just need to add a little pre-commit magic to get
>>>> notified when we should update patchlevel version of Python when a new one
>>>> is released, and we should be able to try it out in CI image - once it gets
>>>> battle-tested with CI/breeze etc. we can transfer this to PROD image as
>>>> well. We already have an idea how to do it - our PROD images are optimized
>>>> for size and do not contain "build essentials" - but I think we should be
>>>> able to build Python in the "build" segment and simply copy resulting
>>>> binaries to the "main" segment - since in both cases we use the same base
>>>> image, such 1-1 copy should **just work** - we already do the same with
>>>> installed python packages - we install them (including building when
>>>> needed) in build segment and we copy-over the installed .venv to the
>>>> main segment.
>>>>
>>>> So ... we might even not need a discussion - installing from Python
>>>> sources is THE BEST
>>>>
>>>> J.
>>>>
>>>>
>>>> On Wed, Jun 25, 2025 at 6:25 PM Damian Shaw <
>>>> dshaw@striketechnologies.com>
>>>> wrote:
>>>>
>>>> > First, I would like to thank the community members who have been
>>>> > maintaining the Python docker images, it's one of those thankless
>>>> > opensource infrastructure volunteer roles that they've been doing for
>>>> > a long time. Unfortunately Docker assigns the title "Official Image"
>>>> > for various community run images, which creates a misconception on the
>>>> > guarantees being provided, and if I were a suspicious person I would
>>>> > say Docker creates this misconception on purpose to both get free work
>>>> > from community members and make Docker seem more supported by third
>>>> > party organizations than it actually is.
>>>> >
>>>> > On the topic of python-build-standalone, I've been using it in
>>>> > production for several months now and I'm fairly happy with it.
>>>> >
>>>> > However, one minor reproducibility issue I have when installing
>>>> > python-build-standalone via uv, is that uv does not have an ability to
>>>> > pin to a specific build between uv versions, as uv hard codes a
>>>> > mapping of Python version and platform to a specific build and then
>>>> > updates that mapping between releases. So, updating the version of uv
>>>> > between runs, or having two users run different versions of uv to
>>>> > initialize the environment can change the results.
>>>> >
>>>> > While normally such build changes are trivial, if you look at the uv
>>>> > 0.7.8 and 0.7.9 changelogs you will see that sometimes they can have
>>>> > significant
>>>> > impact: https://github.com/astral-sh/uv/releases/tag/0.7.8. Also,
>>>> this
>>>> > email finally prompted make an issue on this topic:
>>>> > https://github.com/astral-sh/uv/issues/14263.
>>>> >
>>>> > There are other ways to source python-build-standalone, such as
>>>> > pbs-installer or writing your own script, but I've not yet spent any
>>>> > time investigating them, so I can't comment on them.
>>>> >
>>>> > Damian
>>>> >
>>>> > -----Original Message-----
>>>> > From: Jarek Potiuk <ja...@potiuk.com>
>>>> > Sent: Wednesday, June 25, 2025 10:44 AM
>>>> > To: dev@airflow.apache.org
>>>> > Subject: [DISCUSS] Switching base Python container images to
>>>> > python-standalone ?
>>>> >
>>>> > Hello here,
>>>> >
>>>> > Together with Aritra we are looking into adding a few more things to
>>>> > our images (golang tool chain for CI image), also Shahar is
>>>> > experimenting with Rust tool chain and I also recently realized (by
>>>> > some of the issues we had) that 'Docker Official Python Image' that is
>>>> > part of 'Official Program' [1] is not as 'Official' as I thought so we
>>>> > discuss about changing the base of our images (first CI and then when
>>>> > we see it works fine - PROD)
>>>> >
>>>> > Currently we are using the 'Official' image - but after some issues
>>>> > and discussions with people at PyCon and FOSS Backstage (I had a
>>>> > chance to talk to Python maintainers and even had a few beers with
>>>> > them) - it turned out that the official Python Image' is maintained by
>>>> > 'a community's which really is a few pretty random people - and that
>>>> > explains for example why we have sometimes unpacked security
>>>> > vulnerabilities in setuptools etc. - because they made some
>>>> > compatibility choices and decisions that do not allow them to upgrade
>>>> > easily, also they had some delays in releasing updated Python
>>>> > versions. And Docker does not **really** do much vetting there.
>>>> >
>>>> > So I think it would be good to switch how we build the base for our
>>>> images.
>>>> > And following the experience of `uv python` [2] - it seems that maybe
>>>> > using "python-standalone" [3] project is a good alternative. It's
>>>> > managed by Astral now (so yes - another dependency on them), but what
>>>> > you have with it you have practically 100% complete Python interpreter
>>>> > installed in seconds.
>>>> > We could continue using debian-slim as a "base, base image" - and
>>>> > install python using "python-standalone". There are a few
>>>> > incompatibilities [4] of the distributions of Python, but there are
>>>> > very few and mostly related to some obscure systems (compatibilities
>>>> > with terminal in REPL, and gtk / UI integration that is anyhow not
>>>> > really working in "standard" Python distributions).
>>>> >
>>>> > I would love to hear what you think - happy to get any feedback/
>>>> > insights, suggestions and answer additional questions, provide some
>>>> > links to past "troubles" we had with Python "Official" images etc.
>>>> >
>>>> > J.
>>>> >
>>>> >
>>>> > [1] Official Python Images - https://hub.docker.com/_/python [2] UV
>>>> > Python installation -
>>>> https://docs.astral.sh/uv/guides/install-python/
>>>> > [3] Python Standalone project -
>>>> > https://github.com/astral-sh/python-build-standalone
>>>> > [4] Python Standalone incompatibilities -
>>>> >
>>>> https://gregoryszorc.com/docs/python-build-standalone/main/quirks.html
>>>> > ________________________________
>>>> > Strike Technologies, LLC (“Strike”) is part of the GTS family of
>>>> > companies. Strike is a technology solutions provider, and is not a
>>>> > broker or dealer and does not transact any securities related business
>>>> > directly whatsoever. This communication is the property of Strike and
>>>> > its affiliates, and does not constitute an offer to sell or the
>>>> > solicitation of an offer to buy any security in any jurisdiction. It
>>>> > is intended only for the person to whom it is addressed and may
>>>> > contain information that is privileged, confidential, or otherwise
>>>> protected from disclosure.
>>>> > Distribution or copying of this communication, or the information
>>>> > contained herein, by anyone other than the intended recipient is
>>>> > prohibited. If you have received this communication in error, please
>>>> > immediately notify Strike at info@striketechnologies.com, and delete
>>>> and destroy any copies hereof.
>>>> > ________________________________
>>>> >
>>>> > CONFIDENTIALITY / PRIVILEGE NOTICE: This transmission and any
>>>> > attachments are intended solely for the addressee. This transmission
>>>> > is covered by the Electronic Communications Privacy Act, 18 U.S.C
>>>> > ''2510-2521. The information contained in this transmission is
>>>> > confidential in nature and protected from further use or disclosure
>>>> > under U.S. Pub. L. 106-102, 113 U.S. Stat. 1338 (1999), and may be
>>>> > subject to attorney-client or other legal privilege. Your use or
>>>> > disclosure of this information for any purpose other than that
>>>> > intended by its transmittal is strictly prohibited, and may subject
>>>> > you to fines and/or penalties under federal and state law. If you are
>>>> > not the intended recipient of this transmission, please DESTROY ALL
>>>> > COPIES RECEIVED and confirm destruction to the sender via return
>>>> transmittal.
>>>> >
>>>> ________________________________
>>>> Strike Technologies, LLC (“Strike”) is part of the GTS family of
>>>> companies. Strike is a technology solutions provider, and is not a broker
>>>> or dealer and does not transact any securities related business directly
>>>> whatsoever. This communication is the property of Strike and its
>>>> affiliates, and does not constitute an offer to sell or the solicitation of
>>>> an offer to buy any security in any jurisdiction. It is intended only for
>>>> the person to whom it is addressed and may contain information that is
>>>> privileged, confidential, or otherwise protected from disclosure.
>>>> Distribution or copying of this communication, or the information contained
>>>> herein, by anyone other than the intended recipient is prohibited. If you
>>>> have received this communication in error, please immediately notify Strike
>>>> at info@striketechnologies.com, and delete and destroy any copies
>>>> hereof.
>>>> ________________________________
>>>>
>>>> CONFIDENTIALITY / PRIVILEGE NOTICE: This transmission and any
>>>> attachments are intended solely for the addressee. This transmission is
>>>> covered by the Electronic Communications Privacy Act, 18 U.S.C ''2510-2521.
>>>> The information contained in this transmission is confidential in nature
>>>> and protected from further use or disclosure under U.S. Pub. L. 106-102,
>>>> 113 U.S. Stat. 1338 (1999), and may be subject to attorney-client or other
>>>> legal privilege. Your use or disclosure of this information for any purpose
>>>> other than that intended by its transmittal is strictly prohibited, and may
>>>> subject you to fines and/or penalties under federal and state law. If you
>>>> are not the intended recipient of this transmission, please DESTROY ALL
>>>> COPIES RECEIVED and confirm destruction to the sender via return
>>>> transmittal.
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
>>>> For additional commands, e-mail: dev-help@airflow.apache.org
>>>>
>>>
Re: [DISCUSS] Switching base Python container images to python-standalone ?
Posted by Jarek Potiuk <ja...@potiuk.com>.
I merget it too early - reverting here
https://github.com/apache/airflow/pull/52900 as I found an issue with other
python versions that I overlooked
On Sat, Jul 5, 2025 at 10:06 AM Jarek Potiuk <ja...@potiuk.com> wrote:
> FYI - we just merged Aritra's change to build Python in CI image from
> sources https://github.com/apache/airflow/pull/52265 -> we will run it
> for a while in CI, i also want to test some self-upgrade scenarios.
>
> Then we will look at applying it to the PROD image if we find no surprises.
>
> It looks really good and we might have slightly more secure images
> produced (i.e. getting read of some libraries CVEs faster than the
> "official" images.
>
> Good job Aritra!
> J.
>
>
> On Thu, Jun 26, 2025 at 9:16 PM Jarek Potiuk <ja...@potiuk.com> wrote:
>
>> It looks like building from sources on our CI using base debian image
>> (with enabled optimizations) "just works" and takes 1m36s. -> and taking
>> into account that we already have pretty sophisticated CI-level remote
>> caching that will allow to rebuild it only when either base image is
>> released or python is released - building sounds like a very good idea.
>>
>> J.
>>
>> On Thu, Jun 26, 2025 at 7:02 PM Damian Shaw <ds...@striketechnologies.com>
>> wrote:
>>
>>> Going from source is certainly going to give the most control, I
>>> personally choose python-build-standalone for my own images rather than
>>> building from source to outsource all the build choices, such as what
>>> optimization flags to enable, what compiler tool chains to use, etc.
>>>
>>> Damian
>>>
>>>
>>>
>>> -----Original Message-----
>>> From: Jarek Potiuk <ja...@potiuk.com>
>>> Sent: Thursday, June 26, 2025 10:15 AM
>>> To: dev@airflow.apache.org
>>> Subject: Re: [DISCUSS] Switching base Python container images to
>>> python-standalone ?
>>>
>>> Thanks Damian - very insightful. And yes - I did not really want to
>>> "diminish" the value of community images and work of the maintainers, it
>>> was really more on the "we base our image security on the assumptions that
>>> it's coming from "official" sources and surely there are some guardrails" -
>>> which turned out to be not really true.
>>>
>>> But.... We might not have to even use python standalone.
>>>
>>> Aritra just tested installation of Python straight from sources ->
>>> https://github.com/apache/airflow/pull/52265 and I was actually pretty
>>> surprised how non-problematic and fast it was - and it seems it passes all
>>> our CI tests. We just need to add a little pre-commit magic to get
>>> notified when we should update patchlevel version of Python when a new one
>>> is released, and we should be able to try it out in CI image - once it gets
>>> battle-tested with CI/breeze etc. we can transfer this to PROD image as
>>> well. We already have an idea how to do it - our PROD images are optimized
>>> for size and do not contain "build essentials" - but I think we should be
>>> able to build Python in the "build" segment and simply copy resulting
>>> binaries to the "main" segment - since in both cases we use the same base
>>> image, such 1-1 copy should **just work** - we already do the same with
>>> installed python packages - we install them (including building when
>>> needed) in build segment and we copy-over the installed .venv to the
>>> main segment.
>>>
>>> So ... we might even not need a discussion - installing from Python
>>> sources is THE BEST
>>>
>>> J.
>>>
>>>
>>> On Wed, Jun 25, 2025 at 6:25 PM Damian Shaw <
>>> dshaw@striketechnologies.com>
>>> wrote:
>>>
>>> > First, I would like to thank the community members who have been
>>> > maintaining the Python docker images, it's one of those thankless
>>> > opensource infrastructure volunteer roles that they've been doing for
>>> > a long time. Unfortunately Docker assigns the title "Official Image"
>>> > for various community run images, which creates a misconception on the
>>> > guarantees being provided, and if I were a suspicious person I would
>>> > say Docker creates this misconception on purpose to both get free work
>>> > from community members and make Docker seem more supported by third
>>> > party organizations than it actually is.
>>> >
>>> > On the topic of python-build-standalone, I've been using it in
>>> > production for several months now and I'm fairly happy with it.
>>> >
>>> > However, one minor reproducibility issue I have when installing
>>> > python-build-standalone via uv, is that uv does not have an ability to
>>> > pin to a specific build between uv versions, as uv hard codes a
>>> > mapping of Python version and platform to a specific build and then
>>> > updates that mapping between releases. So, updating the version of uv
>>> > between runs, or having two users run different versions of uv to
>>> > initialize the environment can change the results.
>>> >
>>> > While normally such build changes are trivial, if you look at the uv
>>> > 0.7.8 and 0.7.9 changelogs you will see that sometimes they can have
>>> > significant
>>> > impact: https://github.com/astral-sh/uv/releases/tag/0.7.8. Also, this
>>> > email finally prompted make an issue on this topic:
>>> > https://github.com/astral-sh/uv/issues/14263.
>>> >
>>> > There are other ways to source python-build-standalone, such as
>>> > pbs-installer or writing your own script, but I've not yet spent any
>>> > time investigating them, so I can't comment on them.
>>> >
>>> > Damian
>>> >
>>> > -----Original Message-----
>>> > From: Jarek Potiuk <ja...@potiuk.com>
>>> > Sent: Wednesday, June 25, 2025 10:44 AM
>>> > To: dev@airflow.apache.org
>>> > Subject: [DISCUSS] Switching base Python container images to
>>> > python-standalone ?
>>> >
>>> > Hello here,
>>> >
>>> > Together with Aritra we are looking into adding a few more things to
>>> > our images (golang tool chain for CI image), also Shahar is
>>> > experimenting with Rust tool chain and I also recently realized (by
>>> > some of the issues we had) that 'Docker Official Python Image' that is
>>> > part of 'Official Program' [1] is not as 'Official' as I thought so we
>>> > discuss about changing the base of our images (first CI and then when
>>> > we see it works fine - PROD)
>>> >
>>> > Currently we are using the 'Official' image - but after some issues
>>> > and discussions with people at PyCon and FOSS Backstage (I had a
>>> > chance to talk to Python maintainers and even had a few beers with
>>> > them) - it turned out that the official Python Image' is maintained by
>>> > 'a community's which really is a few pretty random people - and that
>>> > explains for example why we have sometimes unpacked security
>>> > vulnerabilities in setuptools etc. - because they made some
>>> > compatibility choices and decisions that do not allow them to upgrade
>>> > easily, also they had some delays in releasing updated Python
>>> > versions. And Docker does not **really** do much vetting there.
>>> >
>>> > So I think it would be good to switch how we build the base for our
>>> images.
>>> > And following the experience of `uv python` [2] - it seems that maybe
>>> > using "python-standalone" [3] project is a good alternative. It's
>>> > managed by Astral now (so yes - another dependency on them), but what
>>> > you have with it you have practically 100% complete Python interpreter
>>> > installed in seconds.
>>> > We could continue using debian-slim as a "base, base image" - and
>>> > install python using "python-standalone". There are a few
>>> > incompatibilities [4] of the distributions of Python, but there are
>>> > very few and mostly related to some obscure systems (compatibilities
>>> > with terminal in REPL, and gtk / UI integration that is anyhow not
>>> > really working in "standard" Python distributions).
>>> >
>>> > I would love to hear what you think - happy to get any feedback/
>>> > insights, suggestions and answer additional questions, provide some
>>> > links to past "troubles" we had with Python "Official" images etc.
>>> >
>>> > J.
>>> >
>>> >
>>> > [1] Official Python Images - https://hub.docker.com/_/python [2] UV
>>> > Python installation - https://docs.astral.sh/uv/guides/install-python/
>>> > [3] Python Standalone project -
>>> > https://github.com/astral-sh/python-build-standalone
>>> > [4] Python Standalone incompatibilities -
>>> > https://gregoryszorc.com/docs/python-build-standalone/main/quirks.html
>>> > ________________________________
>>> > Strike Technologies, LLC (“Strike”) is part of the GTS family of
>>> > companies. Strike is a technology solutions provider, and is not a
>>> > broker or dealer and does not transact any securities related business
>>> > directly whatsoever. This communication is the property of Strike and
>>> > its affiliates, and does not constitute an offer to sell or the
>>> > solicitation of an offer to buy any security in any jurisdiction. It
>>> > is intended only for the person to whom it is addressed and may
>>> > contain information that is privileged, confidential, or otherwise
>>> protected from disclosure.
>>> > Distribution or copying of this communication, or the information
>>> > contained herein, by anyone other than the intended recipient is
>>> > prohibited. If you have received this communication in error, please
>>> > immediately notify Strike at info@striketechnologies.com, and delete
>>> and destroy any copies hereof.
>>> > ________________________________
>>> >
>>> > CONFIDENTIALITY / PRIVILEGE NOTICE: This transmission and any
>>> > attachments are intended solely for the addressee. This transmission
>>> > is covered by the Electronic Communications Privacy Act, 18 U.S.C
>>> > ''2510-2521. The information contained in this transmission is
>>> > confidential in nature and protected from further use or disclosure
>>> > under U.S. Pub. L. 106-102, 113 U.S. Stat. 1338 (1999), and may be
>>> > subject to attorney-client or other legal privilege. Your use or
>>> > disclosure of this information for any purpose other than that
>>> > intended by its transmittal is strictly prohibited, and may subject
>>> > you to fines and/or penalties under federal and state law. If you are
>>> > not the intended recipient of this transmission, please DESTROY ALL
>>> > COPIES RECEIVED and confirm destruction to the sender via return
>>> transmittal.
>>> >
>>> ________________________________
>>> Strike Technologies, LLC (“Strike”) is part of the GTS family of
>>> companies. Strike is a technology solutions provider, and is not a broker
>>> or dealer and does not transact any securities related business directly
>>> whatsoever. This communication is the property of Strike and its
>>> affiliates, and does not constitute an offer to sell or the solicitation of
>>> an offer to buy any security in any jurisdiction. It is intended only for
>>> the person to whom it is addressed and may contain information that is
>>> privileged, confidential, or otherwise protected from disclosure.
>>> Distribution or copying of this communication, or the information contained
>>> herein, by anyone other than the intended recipient is prohibited. If you
>>> have received this communication in error, please immediately notify Strike
>>> at info@striketechnologies.com, and delete and destroy any copies
>>> hereof.
>>> ________________________________
>>>
>>> CONFIDENTIALITY / PRIVILEGE NOTICE: This transmission and any
>>> attachments are intended solely for the addressee. This transmission is
>>> covered by the Electronic Communications Privacy Act, 18 U.S.C ''2510-2521.
>>> The information contained in this transmission is confidential in nature
>>> and protected from further use or disclosure under U.S. Pub. L. 106-102,
>>> 113 U.S. Stat. 1338 (1999), and may be subject to attorney-client or other
>>> legal privilege. Your use or disclosure of this information for any purpose
>>> other than that intended by its transmittal is strictly prohibited, and may
>>> subject you to fines and/or penalties under federal and state law. If you
>>> are not the intended recipient of this transmission, please DESTROY ALL
>>> COPIES RECEIVED and confirm destruction to the sender via return
>>> transmittal.
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
>>> For additional commands, e-mail: dev-help@airflow.apache.org
>>>
>>