You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/10/31 10:46:38 UTC
[GitHub] [airflow] stromal opened a new pull request, #27406: examples
stromal opened a new pull request, #27406:
URL: https://github.com/apache/airflow/pull/27406
Dockerfile Example for make this working
DAG Example Using The above Dockerfile
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of an existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed pull request #27406: examples
Posted by GitBox <gi...@apache.org>.
potiuk closed pull request #27406: examples
URL: https://github.com/apache/airflow/pull/27406
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] stromal commented on pull request #27406: examples
Posted by GitBox <gi...@apache.org>.
stromal commented on PR #27406:
URL: https://github.com/apache/airflow/pull/27406#issuecomment-1297562777
@ayushthe1 I would love to add them as files I am just most sure how. Should I just clone the whole of the airflow repository add files and try to request merge?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ayushthe1 commented on pull request #27406: examples
Posted by GitBox <gi...@apache.org>.
ayushthe1 commented on PR #27406:
URL: https://github.com/apache/airflow/pull/27406#issuecomment-1298165380
> @ayushthe1 I would love to add them as files I am just most sure how. Should I just clone the whole of the airflow repository add files and try to request merge?
yes, the best workflow would be to :
* First, fork this repo
* Clone the forked repo on your local system
* Make the required changes in the files you want.
* add & commit your changes locally
* push the local repo/branch to your forked github repo.
* Open a pull request from your forked repo to the original airflow repo.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #27406: examples
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #27406:
URL: https://github.com/apache/airflow/pull/27406#issuecomment-1311065166
replaced with https://github.com/apache/airflow/pull/27522
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ayushthe1 commented on pull request #27406: examples
Posted by GitBox <gi...@apache.org>.
ayushthe1 commented on PR #27406:
URL: https://github.com/apache/airflow/pull/27406#issuecomment-1297357790
> Thanks. Proposed some changes and made some comments.
>
> Proposal @ayushthe1 - maybe after we finish this iteration with @stromal you can take it from there and make it into executable exampleinlude as next step?
Sure @potiuk . I will do the later part 🙂
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ayushthe1 commented on pull request #27406: examples
Posted by GitBox <gi...@apache.org>.
ayushthe1 commented on PR #27406:
URL: https://github.com/apache/airflow/pull/27406#issuecomment-1304640242
> @ayushthe1 agree. thx.
Thanks .
@potiuk ,i opened a new PR #27522 for this issue .Could you please review it.
Also, i got confused in deciding which file should be added where and how many of them .please tell for any corrections to make.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] stromal commented on a diff in pull request #27406: examples
Posted by GitBox <gi...@apache.org>.
stromal commented on code in PR #27406:
URL: https://github.com/apache/airflow/pull/27406#discussion_r1009310782
##########
docs/apache-airflow/howto/operator/python.rst:
##########
@@ -116,12 +116,109 @@ pre-defined virtual environment. The virtualenv should be preinstalled in the en
Python is run and in case ``dill`` is used, it has to be preinstalled in the virtualenv (the same
version that is installed in main Airflow environment).
+
+Dockerfile Example
+
+```
+# You canswitch the airflow and python version numbers here (limited python versions to each ariflow version)
+FROM apache/airflow:2.4.2-python3.8
+
+# ExternalPythonOperator
+# Compulsory to switch parameter to use your own venve via ExternalPythonOperator
Review Comment:
@potiuk I would love to help out but I am not sure how to add new files to the project. Do I just clone it and push it with my new added files?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] stromal commented on pull request #27406: examples
Posted by GitBox <gi...@apache.org>.
stromal commented on PR #27406:
URL: https://github.com/apache/airflow/pull/27406#issuecomment-1304046372
@ayushthe1 agree. thx.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a diff in pull request #27406: examples
Posted by GitBox <gi...@apache.org>.
potiuk commented on code in PR #27406:
URL: https://github.com/apache/airflow/pull/27406#discussion_r1009294535
##########
docs/apache-airflow/howto/operator/python.rst:
##########
@@ -116,12 +116,109 @@ pre-defined virtual environment. The virtualenv should be preinstalled in the en
Python is run and in case ``dill`` is used, it has to be preinstalled in the virtualenv (the same
version that is installed in main Airflow environment).
+
+Dockerfile Example
+
+```
+# You canswitch the airflow and python version numbers here (limited python versions to each ariflow version)
+FROM apache/airflow:2.4.2-python3.8
+
+# ExternalPythonOperator
+# Compulsory to switch parameter to use your own venve via ExternalPythonOperator
Review Comment:
```suggestion
# Compulsory to switch parameter to create your own virtual environment
# in case of using Airlfow's base Image because it has `PIP_USER` enabled by default as explained in
# https://airflow.apache.org/docs/docker-stack/build.html#important-notes-for-the-base-images
```
##########
docs/apache-airflow/howto/operator/python.rst:
##########
@@ -116,12 +116,109 @@ pre-defined virtual environment. The virtualenv should be preinstalled in the en
Python is run and in case ``dill`` is used, it has to be preinstalled in the virtualenv (the same
version that is installed in main Airflow environment).
+
+Dockerfile Example
+
+```
+# You canswitch the airflow and python version numbers here (limited python versions to each ariflow version)
+FROM apache/airflow:2.4.2-python3.8
+
+# ExternalPythonOperator
+# Compulsory to switch parameter to use your own venve via ExternalPythonOperator
+ENV PIP_USER=false
+
+#python venv setup in container
+RUN python3 -m venv /opt/airflow/venv1
+
+# Install dependencies:
+COPY airflow/requirements.txt .
+# copy over folder that containes a my own local pip package
+COPY airflow/my_own_pip_package_folder /opt/airflow/my_own_pip_package_folder
+# copy over ahtever file
+COPY my_file.py /opt/airflow/my_file.py
+
+# install packages in my own docker virtual environemnt
+RUN /opt/airflow/venv1/bin/pip install -r requirements.txt
+RUN /opt/airflow/venv1/bin/pip install /opt/airflow/my_own_pip_package_folder
+RUN /opt/airflow/venv1/bin/pip install 'apache-airflow==2.4.2' --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.4.1/constraints-3.8.txt"
+```
+
+
+DAG Example Using The above Dockerfile
+```
+"""
+Example DAG demonstrating the usage of the TaskFlow API to execute Python functions natively and within a
+virtual environment.
+"""
+from __future__ import annotations
+
+import logging
+import os
+import shutil
+import sys
+import tempfile
+import time
+from pprint import pprint
+
+import pendulum
+
+from airflow import DAG
+from airflow.decorators import task
+
+log = logging.getLogger(__name__)
+PYTHON = sys.executable
+BASE_DIR = tempfile.gettempdir()
+
+
+my_default_args = {
+ 'owner': 'me',
+ 'email': ['myemail@myemail.com''],
+ 'email_on_failure': True,
+ 'email_on_retry': True,
+ 'retries': 1,
+# 'retry_delay': timedelta(minutes=1)
+}
+
+
+with DAG(
+ dag_id='sample_schedule_dag',
+ # https://crontab.guru/
+ # min HOUR DAY_OF_MONTH MONTH DAY_OF_WEEK
+ # * * * * *
+ schedule='12 11 * * *', # IT IS AT UTC. EX.: 11:12 UTC = 12:12am BST = 12:12am UK time
+ start_date=pendulum.datetime(2021, 1, 1, tz="UTC"), # this is from whre it starts counting time to run taks, NOT like cron
+ catchup=False,
+ default_args=my_default_args,
+ tags=['sample_tag', 'sample_tag2'],
+ ) as dag:
+
+ #@task.external_python(task_id="test_external_python_venv_task", python=os.fspath(sys.executable))
Review Comment:
This comment should be removed for clarity.
##########
docs/apache-airflow/howto/operator/python.rst:
##########
@@ -116,12 +116,109 @@ pre-defined virtual environment. The virtualenv should be preinstalled in the en
Python is run and in case ``dill`` is used, it has to be preinstalled in the virtualenv (the same
version that is installed in main Airflow environment).
+
+Dockerfile Example
+
+```
+# You canswitch the airflow and python version numbers here (limited python versions to each ariflow version)
Review Comment:
```suggestion
# You can switch the Airflow and Python version numbers here (check which Python versions are suppored in the :doc:`Prerequisites <prerequisites>`)
```
##########
docs/apache-airflow/howto/operator/python.rst:
##########
@@ -116,12 +116,109 @@ pre-defined virtual environment. The virtualenv should be preinstalled in the en
Python is run and in case ``dill`` is used, it has to be preinstalled in the virtualenv (the same
version that is installed in main Airflow environment).
+
+Dockerfile Example
+
+```
+# You canswitch the airflow and python version numbers here (limited python versions to each ariflow version)
+FROM apache/airflow:2.4.2-python3.8
+
+# ExternalPythonOperator
+# Compulsory to switch parameter to use your own venve via ExternalPythonOperator
+ENV PIP_USER=false
+
+#python venv setup in container
+RUN python3 -m venv /opt/airflow/venv1
+
+# Install dependencies:
+COPY airflow/requirements.txt .
+# copy over folder that containes a my own local pip package
+COPY airflow/my_own_pip_package_folder /opt/airflow/my_own_pip_package_folder
+# copy over ahtever file
+COPY my_file.py /opt/airflow/my_file.py
+
+# install packages in my own docker virtual environemnt
+RUN /opt/airflow/venv1/bin/pip install -r requirements.txt
+RUN /opt/airflow/venv1/bin/pip install /opt/airflow/my_own_pip_package_folder
+RUN /opt/airflow/venv1/bin/pip install 'apache-airflow==2.4.2' --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.4.1/constraints-3.8.txt"
+```
+
+
+DAG Example Using The above Dockerfile
+```
+"""
+Example DAG demonstrating the usage of the TaskFlow API to execute Python functions natively and within a
+virtual environment.
+"""
+from __future__ import annotations
+
+import logging
+import os
+import shutil
+import sys
+import tempfile
+import time
+from pprint import pprint
+
+import pendulum
+
+from airflow import DAG
+from airflow.decorators import task
+
+log = logging.getLogger(__name__)
+PYTHON = sys.executable
+BASE_DIR = tempfile.gettempdir()
+
+
+my_default_args = {
+ 'owner': 'me',
+ 'email': ['myemail@myemail.com''],
+ 'email_on_failure': True,
+ 'email_on_retry': True,
+ 'retries': 1,
+# 'retry_delay': timedelta(minutes=1)
+}
+
+
+with DAG(
+ dag_id='sample_schedule_dag',
+ # https://crontab.guru/
+ # min HOUR DAY_OF_MONTH MONTH DAY_OF_WEEK
+ # * * * * *
+ schedule='12 11 * * *', # IT IS AT UTC. EX.: 11:12 UTC = 12:12am BST = 12:12am UK time
+ start_date=pendulum.datetime(2021, 1, 1, tz="UTC"), # this is from whre it starts counting time to run taks, NOT like cron
+ catchup=False,
+ default_args=my_default_args,
+ tags=['sample_tag', 'sample_tag2'],
+ ) as dag:
+
+ #@task.external_python(task_id="test_external_python_venv_task", python=os.fspath(sys.executable))
+ @task.external_python(task_id="sample_task", python='/opt/airflow/venv1/bin/python3')
Review Comment:
This is different path than the one in the Dockerfile above (yet it is explained as example for that particular Dockerfile).
##########
docs/apache-airflow/howto/operator/python.rst:
##########
@@ -116,12 +116,109 @@ pre-defined virtual environment. The virtualenv should be preinstalled in the en
Python is run and in case ``dill`` is used, it has to be preinstalled in the virtualenv (the same
version that is installed in main Airflow environment).
+
+Dockerfile Example
+
+```
+# You canswitch the airflow and python version numbers here (limited python versions to each ariflow version)
+FROM apache/airflow:2.4.2-python3.8
+
+# ExternalPythonOperator
+# Compulsory to switch parameter to use your own venve via ExternalPythonOperator
+ENV PIP_USER=false
+
+#python venv setup in container
+RUN python3 -m venv /opt/airflow/venv1
+
+# Install dependencies:
+COPY airflow/requirements.txt .
+# copy over folder that containes a my own local pip package
+COPY airflow/my_own_pip_package_folder /opt/airflow/my_own_pip_package_folder
+# copy over ahtever file
+COPY my_file.py /opt/airflow/my_file.py
+
+# install packages in my own docker virtual environemnt
+RUN /opt/airflow/venv1/bin/pip install -r requirements.txt
+RUN /opt/airflow/venv1/bin/pip install /opt/airflow/my_own_pip_package_folder
+RUN /opt/airflow/venv1/bin/pip install 'apache-airflow==2.4.2' --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.4.1/constraints-3.8.txt"
+```
+
+
+DAG Example Using The above Dockerfile
+```
+"""
+Example DAG demonstrating the usage of the TaskFlow API to execute Python functions natively and within a
+virtual environment.
+"""
+from __future__ import annotations
+
+import logging
+import os
+import shutil
+import sys
+import tempfile
+import time
+from pprint import pprint
+
+import pendulum
+
+from airflow import DAG
+from airflow.decorators import task
+
+log = logging.getLogger(__name__)
+PYTHON = sys.executable
Review Comment:
I believe both PYTHON and BASE_DIR can be removed
##########
docs/apache-airflow/howto/operator/python.rst:
##########
@@ -116,12 +116,109 @@ pre-defined virtual environment. The virtualenv should be preinstalled in the en
Python is run and in case ``dill`` is used, it has to be preinstalled in the virtualenv (the same
version that is installed in main Airflow environment).
+
+Dockerfile Example
+
+```
+# You canswitch the airflow and python version numbers here (limited python versions to each ariflow version)
+FROM apache/airflow:2.4.2-python3.8
+
+# ExternalPythonOperator
+# Compulsory to switch parameter to use your own venve via ExternalPythonOperator
+ENV PIP_USER=false
+
+#python venv setup in container
+RUN python3 -m venv /opt/airflow/venv1
+
+# Install dependencies:
+COPY airflow/requirements.txt .
+# copy over folder that containes a my own local pip package
+COPY airflow/my_own_pip_package_folder /opt/airflow/my_own_pip_package_folder
+# copy over ahtever file
+COPY my_file.py /opt/airflow/my_file.py
+
+# install packages in my own docker virtual environemnt
+RUN /opt/airflow/venv1/bin/pip install -r requirements.txt
+RUN /opt/airflow/venv1/bin/pip install /opt/airflow/my_own_pip_package_folder
+RUN /opt/airflow/venv1/bin/pip install 'apache-airflow==2.4.2' --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.4.1/constraints-3.8.txt"
+```
+
+
+DAG Example Using The above Dockerfile
+```
+"""
+Example DAG demonstrating the usage of the TaskFlow API to execute Python functions natively and within a
+virtual environment.
+"""
+from __future__ import annotations
+
+import logging
+import os
+import shutil
+import sys
+import tempfile
+import time
+from pprint import pprint
+
+import pendulum
+
+from airflow import DAG
+from airflow.decorators import task
+
+log = logging.getLogger(__name__)
+PYTHON = sys.executable
+BASE_DIR = tempfile.gettempdir()
+
+
+my_default_args = {
+ 'owner': 'me',
+ 'email': ['myemail@myemail.com''],
+ 'email_on_failure': True,
+ 'email_on_retry': True,
+ 'retries': 1,
+# 'retry_delay': timedelta(minutes=1)
+}
+
+
+with DAG(
+ dag_id='sample_schedule_dag',
+ # https://crontab.guru/
+ # min HOUR DAY_OF_MONTH MONTH DAY_OF_WEEK
+ # * * * * *
+ schedule='12 11 * * *', # IT IS AT UTC. EX.: 11:12 UTC = 12:12am BST = 12:12am UK time
+ start_date=pendulum.datetime(2021, 1, 1, tz="UTC"), # this is from whre it starts counting time to run taks, NOT like cron
+ catchup=False,
+ default_args=my_default_args,
+ tags=['sample_tag', 'sample_tag2'],
+ ) as dag:
+
+ #@task.external_python(task_id="test_external_python_venv_task", python=os.fspath(sys.executable))
+ @task.external_python(task_id="sample_task", python='/opt/airflow/venv1/bin/python3')
+ def go(): # this could be any function name
+ #PUT ALL IMPORTS HERE THAT THE FILE NEEDS
+ print('Straight of the sample DAG')
+ print('Internal Processes of sample dag')
+ print('End of sample dag')
+ return ['something, something2'] #any return value becasueit doesn not deine the succes of the task
Review Comment:
This is not "anythign" - values returned here are stored in XCom so you can use them for communication with otehr tasks.
##########
docs/apache-airflow/howto/operator/python.rst:
##########
@@ -116,12 +116,109 @@ pre-defined virtual environment. The virtualenv should be preinstalled in the en
Python is run and in case ``dill`` is used, it has to be preinstalled in the virtualenv (the same
version that is installed in main Airflow environment).
+
+Dockerfile Example
+
+```
+# You canswitch the airflow and python version numbers here (limited python versions to each ariflow version)
+FROM apache/airflow:2.4.2-python3.8
+
+# ExternalPythonOperator
+# Compulsory to switch parameter to use your own venve via ExternalPythonOperator
+ENV PIP_USER=false
+
+#python venv setup in container
+RUN python3 -m venv /opt/airflow/venv1
+
+# Install dependencies:
+COPY airflow/requirements.txt .
+# copy over folder that containes a my own local pip package
+COPY airflow/my_own_pip_package_folder /opt/airflow/my_own_pip_package_folder
+# copy over ahtever file
+COPY my_file.py /opt/airflow/my_file.py
+
+# install packages in my own docker virtual environemnt
+RUN /opt/airflow/venv1/bin/pip install -r requirements.txt
+RUN /opt/airflow/venv1/bin/pip install /opt/airflow/my_own_pip_package_folder
+RUN /opt/airflow/venv1/bin/pip install 'apache-airflow==2.4.2' --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.4.1/constraints-3.8.txt"
+```
+
+
+DAG Example Using The above Dockerfile
+```
+"""
+Example DAG demonstrating the usage of the TaskFlow API to execute Python functions natively and within a
+virtual environment.
+"""
+from __future__ import annotations
+
+import logging
+import os
+import shutil
+import sys
+import tempfile
+import time
+from pprint import pprint
+
+import pendulum
+
+from airflow import DAG
+from airflow.decorators import task
+
+log = logging.getLogger(__name__)
+PYTHON = sys.executable
+BASE_DIR = tempfile.gettempdir()
+
+
+my_default_args = {
+ 'owner': 'me',
+ 'email': ['myemail@myemail.com''],
+ 'email_on_failure': True,
+ 'email_on_retry': True,
+ 'retries': 1,
+# 'retry_delay': timedelta(minutes=1)
+}
+
+
+with DAG(
+ dag_id='sample_schedule_dag',
+ # https://crontab.guru/
+ # min HOUR DAY_OF_MONTH MONTH DAY_OF_WEEK
+ # * * * * *
+ schedule='12 11 * * *', # IT IS AT UTC. EX.: 11:12 UTC = 12:12am BST = 12:12am UK time
+ start_date=pendulum.datetime(2021, 1, 1, tz="UTC"), # this is from whre it starts counting time to run taks, NOT like cron
+ catchup=False,
+ default_args=my_default_args,
+ tags=['sample_tag', 'sample_tag2'],
+ ) as dag:
+
+ #@task.external_python(task_id="test_external_python_venv_task", python=os.fspath(sys.executable))
+ @task.external_python(task_id="sample_task", python='/opt/airflow/venv1/bin/python3')
+ def go(): # this could be any function name
+ #PUT ALL IMPORTS HERE THAT THE FILE NEEDS
Review Comment:
```suggestion
#PUT ALL LOCAL IMPORTS HERE THAT THE METHOD NEEDS
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a diff in pull request #27406: examples
Posted by GitBox <gi...@apache.org>.
potiuk commented on code in PR #27406:
URL: https://github.com/apache/airflow/pull/27406#discussion_r1009315291
##########
docs/apache-airflow/howto/operator/python.rst:
##########
@@ -116,12 +116,109 @@ pre-defined virtual environment. The virtualenv should be preinstalled in the en
Python is run and in case ``dill`` is used, it has to be preinstalled in the virtualenv (the same
version that is installed in main Airflow environment).
+
+Dockerfile Example
+
+```
+# You canswitch the airflow and python version numbers here (limited python versions to each ariflow version)
+FROM apache/airflow:2.4.2-python3.8
+
+# ExternalPythonOperator
+# Compulsory to switch parameter to use your own venve via ExternalPythonOperator
Review Comment:
Yes. We have the whole contributing quite in https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst (and one of the first things there you will find quick contribution guides with screenshots and step-by-step guidelines for different IDEs - IntelliJ, VScode, even codespaces and gitpod in the cloud if you prefer to work in the cloud.
But it's ok to just edit docs in this PR and extract it to an example included in a later follow-up (and that's fine if you share the task wuth @ayushthe1 :D, cooperation in OSS is a good thing).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on pull request #27406: examples
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on PR #27406:
URL: https://github.com/apache/airflow/pull/27406#issuecomment-1296909200
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst)
Here are some useful points:
- Pay attention to the quality of your code (flake8, mypy and type annotations). Our [pre-commits]( https://github.com/apache/airflow/blob/main/STATIC_CODE_CHECKS.rst#prerequisites-for-pre-commit-hooks) will help you with that.
- In case of a new feature add useful documentation (in docstrings or in `docs/` directory). Adding a new operator? Check this short [guide](https://github.com/apache/airflow/blob/main/docs/apache-airflow/howto/custom-operator.rst) Consider adding an example DAG that shows how users should use it.
- Consider using [Breeze environment](https://github.com/apache/airflow/blob/main/BREEZE.rst) for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
- Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
- Please follow [ASF Code of Conduct](https://www.apache.org/foundation/policies/conduct) for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
- Be sure to read the [Airflow Coding style]( https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#coding-style-and-best-practices).
Apache Airflow is a community-driven project and together we are making it better 🚀.
In case of doubts contact the developers at:
Mailing List: dev@airflow.apache.org
Slack: https://s.apache.org/airflow-slack
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ayushthe1 commented on pull request #27406: examples
Posted by GitBox <gi...@apache.org>.
ayushthe1 commented on PR #27406:
URL: https://github.com/apache/airflow/pull/27406#issuecomment-1303932078
hey @stromal ,this issue looks stale .
If u like, i will love to work on this issue now and solve it further.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org