You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/04/22 10:22:02 UTC
[GitHub] [airflow] Bowrna opened a new pull request, #23167: Migrate Docker example DAGs to new design
Bowrna opened a new pull request, #23167:
URL: https://github.com/apache/airflow/pull/23167
closes: #22444
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a newsfragement file, named `{pr_number}.significant.rst`, in [newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #23167: Migrate Docker example DAGs to new design
Posted by GitBox <gi...@apache.org>.
potiuk merged PR #23167:
URL: https://github.com/apache/airflow/pull/23167
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Bowrna commented on pull request #23167: Migrate Docker example DAGs to new design
Posted by GitBox <gi...@apache.org>.
Bowrna commented on PR #23167:
URL: https://github.com/apache/airflow/pull/23167#issuecomment-1146791152
@potiuk I made the changes you suggested and now I don't see the command images in the file diff. Thanks :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #23167: Migrate Docker example DAGs to new design
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #23167:
URL: https://github.com/apache/airflow/pull/23167#issuecomment-1146766954
I think you should run `breeze self-update` :) The images generated are from an old rich version and by self-upgrade you will get the latest one.
Then what you need to do to regenerate them - delete `images/breeze/output-commands-hash.txt` and run `breeze static-checks -t update-breeze-files`.
BTW. I will add a separate command to regnerate those images a bit more easily.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #23167: Migrate Docker example DAGs to new design
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #23167:
URL: https://github.com/apache/airflow/pull/23167#issuecomment-1146769740
The "regenerate-breeze-command" command is added here: https://github.com/apache/airflow/pull/24216
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Bowrna commented on a diff in pull request #23167: Migrate Docker example DAGs to new design
Posted by GitBox <gi...@apache.org>.
Bowrna commented on code in PR #23167:
URL: https://github.com/apache/airflow/pull/23167#discussion_r863534896
##########
tests/system/providers/docker/example_docker.py:
##########
@@ -0,0 +1,63 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+import os
+from datetime import datetime
+
+from airflow import models
+from airflow.operators.bash import BashOperator
+from airflow.providers.docker.operators.docker import DockerOperator
+
+ENV_ID = os.environ.get("SYSTEM_TESTS_ENV_ID")
+DAG_ID = 'docker_test'
+
+with models.DAG(
+ DAG_ID,
+ schedule_interval="@once",
+ start_date=datetime(2021, 1, 1),
+ catchup=False,
+ tags=["example", "docker"],
+) as dag:
+ t1 = BashOperator(task_id='print_date', bash_command='date', dag=dag)
+ t2 = BashOperator(task_id='sleep', bash_command='sleep 5', retries=3, dag=dag)
+ # [START howto_operator_docker]
+ t3 = DockerOperator(
+ docker_url='unix://var/run/docker.sock', # Set your docker URL
+ command='/bin/sleep 30',
+ image='centos:latest',
+ network_mode='bridge',
+ task_id='docker_op_tester',
+ dag=dag,
+ )
+ # [END howto_operator_docker]
+
+ t4 = BashOperator(task_id='print_hello', bash_command='echo "hello world!!!"', dag=dag)
+ # t1 >> t2
+ # t1 >> t3
+ # t3 >> t4
+
+ (
+ # TEST BODY
+ t1
+ >> [t2, t3]
+ >> t4
+ )
+
Review Comment:
Is the task present inside above parenthesis is the right representation of
# t1 >> t2
# t1 >> t3
# t3 >> t4 ?
If not can you tell me how to represent it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Bowrna commented on a diff in pull request #23167: Migrate Docker example DAGs to new design
Posted by GitBox <gi...@apache.org>.
Bowrna commented on code in PR #23167:
URL: https://github.com/apache/airflow/pull/23167#discussion_r863534896
##########
tests/system/providers/docker/example_docker.py:
##########
@@ -0,0 +1,63 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+import os
+from datetime import datetime
+
+from airflow import models
+from airflow.operators.bash import BashOperator
+from airflow.providers.docker.operators.docker import DockerOperator
+
+ENV_ID = os.environ.get("SYSTEM_TESTS_ENV_ID")
+DAG_ID = 'docker_test'
+
+with models.DAG(
+ DAG_ID,
+ schedule_interval="@once",
+ start_date=datetime(2021, 1, 1),
+ catchup=False,
+ tags=["example", "docker"],
+) as dag:
+ t1 = BashOperator(task_id='print_date', bash_command='date', dag=dag)
+ t2 = BashOperator(task_id='sleep', bash_command='sleep 5', retries=3, dag=dag)
+ # [START howto_operator_docker]
+ t3 = DockerOperator(
+ docker_url='unix://var/run/docker.sock', # Set your docker URL
+ command='/bin/sleep 30',
+ image='centos:latest',
+ network_mode='bridge',
+ task_id='docker_op_tester',
+ dag=dag,
+ )
+ # [END howto_operator_docker]
+
+ t4 = BashOperator(task_id='print_hello', bash_command='echo "hello world!!!"', dag=dag)
+ # t1 >> t2
+ # t1 >> t3
+ # t3 >> t4
+
+ (
+ # TEST BODY
+ t1
+ >> [t2, t3]
+ >> t4
+ )
+
Review Comment:
Is the task present inside above parenthesis is the right representation of
# t1 >> t2
# t1 >> t3
# t3 >> t4 ?
If not can you tell me how to represent it?
cc: @potiuk @bhirsz @mnojek
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org