You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by po...@apache.org on 2022/04/25 20:36:24 UTC
[airflow] branch main updated (1e9765b5e5 -> b43d072254)
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
from 1e9765b5e5 Mock project_id for test_delete_cluster_error unit test
new 044894d22f image building documentation: adding new provider example
new 5ae96a15fd code review fixes
new b43d072254 fix Build docs failure
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
docs/docker-stack/build.rst | 16 +++++++++++++---
.../docker-examples/extending/add-providers/Dockerfile | 11 ++++++++++-
.../{add-providers => custom-providers}/Dockerfile | 2 +-
3 files changed, 24 insertions(+), 5 deletions(-)
copy docs/docker-stack/docker-examples/extending/{add-providers => custom-providers}/Dockerfile (99%)
[airflow] 03/03: fix Build docs failure
Posted by po...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit b43d072254c94f1f53b08e6d0efecfc9b0496c79
Author: Itay Bittan <it...@gmail.com>
AuthorDate: Thu Apr 21 18:08:49 2022 +0300
fix Build docs failure
---
docs/docker-stack/build.rst | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/docs/docker-stack/build.rst b/docs/docker-stack/build.rst
index 08e70e6c36..c151c60967 100644
--- a/docs/docker-stack/build.rst
+++ b/docs/docker-stack/build.rst
@@ -292,7 +292,7 @@ Example of adding Airflow Provider package and ``apt`` package
The following example adds ``apache-spark`` airflow-providers which requires both ``java`` and
python package from PyPI.
-.. exampleinclude:: docker-examples/extending/add-provider-extend/Dockerfile
+.. exampleinclude:: docker-examples/extending/add-providers/Dockerfile
:language: Dockerfile
:start-after: [START Dockerfile]
:end-before: [END Dockerfile]
[airflow] 02/03: code review fixes
Posted by po...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 5ae96a15fd597ffdc42371156753f8045ac260e6
Author: Itay Bittan <it...@gmail.com>
AuthorDate: Thu Apr 21 16:09:30 2022 +0300
code review fixes
---
docs/docker-stack/build.rst | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/docs/docker-stack/build.rst b/docs/docker-stack/build.rst
index 588c85ac22..08e70e6c36 100644
--- a/docs/docker-stack/build.rst
+++ b/docs/docker-stack/build.rst
@@ -275,7 +275,7 @@ Examples of image extending
---------------------------
Example of customizing Airflow Provider packages
-..............................................
+................................................
The :ref:`Airflow Providers <providers:community-maintained-providers>` are released independently of core
Airflow and sometimes you might want to upgrade specific providers only to fix some problems or
@@ -286,10 +286,11 @@ use features available in that provider version. Here is an example of how you c
:start-after: [START Dockerfile]
:end-before: [END Dockerfile]
-Example of adding Airflow Provider packages
-...................................................
+Example of adding Airflow Provider package and ``apt`` package
+..............................................................
-The following example adds ``apache-spark`` airflow-providers which requires both ``java``.
+The following example adds ``apache-spark`` airflow-providers which requires both ``java`` and
+python package from PyPI.
.. exampleinclude:: docker-examples/extending/add-provider-extend/Dockerfile
:language: Dockerfile
[airflow] 01/03: image building documentation: adding new provider example
Posted by po...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 044894d22fbeb843c893a75005091ec9ca7a6c10
Author: Itay Bittan <it...@gmail.com>
AuthorDate: Wed Apr 20 22:12:29 2022 +0300
image building documentation: adding new provider example
---
docs/docker-stack/build.rst | 13 +++++++++++--
.../docker-examples/extending/add-providers/Dockerfile | 11 ++++++++++-
.../{add-providers => custom-providers}/Dockerfile | 2 +-
3 files changed, 22 insertions(+), 4 deletions(-)
diff --git a/docs/docker-stack/build.rst b/docs/docker-stack/build.rst
index 9679172a2b..588c85ac22 100644
--- a/docs/docker-stack/build.rst
+++ b/docs/docker-stack/build.rst
@@ -274,18 +274,27 @@ You should be aware, about a few things:
Examples of image extending
---------------------------
-Example of upgrading Airflow Provider packages
+Example of customizing Airflow Provider packages
..............................................
The :ref:`Airflow Providers <providers:community-maintained-providers>` are released independently of core
Airflow and sometimes you might want to upgrade specific providers only to fix some problems or
use features available in that provider version. Here is an example of how you can do it
-.. exampleinclude:: docker-examples/extending/add-providers/Dockerfile
+.. exampleinclude:: docker-examples/extending/custom-providers/Dockerfile
:language: Dockerfile
:start-after: [START Dockerfile]
:end-before: [END Dockerfile]
+Example of adding Airflow Provider packages
+...................................................
+
+The following example adds ``apache-spark`` airflow-providers which requires both ``java``.
+
+.. exampleinclude:: docker-examples/extending/add-provider-extend/Dockerfile
+ :language: Dockerfile
+ :start-after: [START Dockerfile]
+ :end-before: [END Dockerfile]
Example of adding ``apt`` package
.................................
diff --git a/docs/docker-stack/docker-examples/extending/add-providers/Dockerfile b/docs/docker-stack/docker-examples/extending/add-providers/Dockerfile
index ffea0b137e..79a7350fa6 100644
--- a/docs/docker-stack/docker-examples/extending/add-providers/Dockerfile
+++ b/docs/docker-stack/docker-examples/extending/add-providers/Dockerfile
@@ -16,5 +16,14 @@
# This is an example Dockerfile. It is not intended for PRODUCTION use
# [START Dockerfile]
FROM apache/airflow:2.3.0.dev0
-RUN pip install --no-cache-dir apache-airflow-providers-docker==2.1.0
+USER root
+RUN apt-get update \
+ && apt-get install -y --no-install-recommends \
+ openjdk-11-jre-headless \
+ && apt-get autoremove -yqq --purge \
+ && apt-get clean \
+ && rm -rf /var/lib/apt/lists/*
+USER airflow
+ENV JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
+RUN pip install --no-cache-dir apache-airflow-providers-apache-spark==2.1.3
# [END Dockerfile]
diff --git a/docs/docker-stack/docker-examples/extending/add-providers/Dockerfile b/docs/docker-stack/docker-examples/extending/custom-providers/Dockerfile
similarity index 99%
copy from docs/docker-stack/docker-examples/extending/add-providers/Dockerfile
copy to docs/docker-stack/docker-examples/extending/custom-providers/Dockerfile
index ffea0b137e..6e516b6365 100644
--- a/docs/docker-stack/docker-examples/extending/add-providers/Dockerfile
+++ b/docs/docker-stack/docker-examples/extending/custom-providers/Dockerfile
@@ -16,5 +16,5 @@
# This is an example Dockerfile. It is not intended for PRODUCTION use
# [START Dockerfile]
FROM apache/airflow:2.3.0.dev0
-RUN pip install --no-cache-dir apache-airflow-providers-docker==2.1.0
+RUN pip install --no-cache-dir apache-airflow-providers-docker==2.5.1
# [END Dockerfile]