You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/04/15 10:08:29 UTC
[GitHub] [airflow] turbaszek opened a new pull request #8376: Use Github
Actions to run CI
turbaszek opened a new pull request #8376: Use Github Actions to run CI
URL: https://github.com/apache/airflow/pull/8376
Let's try use Github Actions to run some of our CI workloads!
---
Make sure to mark the boxes below before creating PR: [x]
- [x] Description above provides context of the change
- [x] Unit tests coverage for changes (not needed for documentation changes)
- [x] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)"
- [x] Relevant documentation is updated including usage instructions.
- [x] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
---
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] kaxil commented on issue #8376: Use Github Actions to run
CI
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #8376: Use Github Actions to run CI
URL: https://github.com/apache/airflow/pull/8376#issuecomment-614298435
Very happy to see this. Good work @turbaszek and @dimberman
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] potiuk edited a comment on issue #8376: Use Github
Actions to run CI
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #8376: Use Github Actions to run CI
URL: https://github.com/apache/airflow/pull/8376#issuecomment-614252959
k8s tests can remain on Travis. I think it's not an issue for now - if we just have 2 Travis jobs runnnig per PR, this should be no problem even if static check and others fail on GA
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] turbaszek commented on issue #8376: Use Github Actions to
run CI
Posted by GitBox <gi...@apache.org>.
turbaszek commented on issue #8376: Use Github Actions to run CI
URL: https://github.com/apache/airflow/pull/8376#issuecomment-614257597
Oh no, I forgot about updating badge in README.md
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #8376: Use
Github Actions to run CI
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #8376: Use Github Actions to run CI
URL: https://github.com/apache/airflow/pull/8376#discussion_r409051786
##########
File path: .github/workflows/ci.yml
##########
@@ -0,0 +1,238 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+---
+name: Airflow
+on:
+ push:
+ branches: [master]
+ pull_request:
+ branches: [master]
+env:
+ BUILD_ID: ${{github.sha }}
+ MOUNT_LOCAL_SOURCES: "false"
+ MOUNT_HOST_AIRFLOW_VOLUME: "true"
+ FORCE_ANSWER_TO_QUESTIONS: "yes"
+ SKIP_CHECK_REMOTE_IMAGE: "true"
+ DB_RESET: "true"
+ VERBOSE: "true"
+ CI: "true"
+ # Should be a target branch
+ TRAVIS_BRANCH: "master"
+ TRAVIS: "true"
+jobs:
+ statics:
+ name: Static checks
+ runs-on: ubuntu-latest
+ env:
+ PYTHON_VERSION: 3.6
+ AIRFLOW_MOUNT_SOURCE_DIR_FOR_STATIC_CHECKS: "true"
+ TRAVIS_JOB_NAME: "Static"
+ steps:
+ - uses: actions/checkout@master
+ - uses: actions/setup-python@v1
+ with:
+ python-version: ${{ env.PYTHON_VERSION }}
+ - name: "Before install"
+ run: ./scripts/ci/ci_before_install.sh
+ - name: "Static checks"
+ if: success()
+ env:
+ PYTHON_VERSION: 3.6
+ run: ./scripts/ci/ci_run_all_static_checks.sh
+
+ statics-tests:
+ name: Pylint for tests
+ runs-on: ubuntu-latest
+ env:
+ PYTHON_VERSION: 3.6
+ AIRFLOW_MOUNT_SOURCE_DIR_FOR_STATIC_CHECKS: "true"
+ TRAVIS_JOB_NAME: "Static"
+ steps:
+ - uses: actions/checkout@master
+ - uses: actions/setup-python@v1
+ with:
+ python-version: ${{ env.PYTHON_VERSION }}
+ - name: "Before install"
+ run: ./scripts/ci/ci_before_install.sh
+ - name: "Static checks tests"
+ if: success()
+ env:
+ PYTHON_VERSION: 3.6
+ run: ./scripts/ci/ci_run_static_checks_pylint_tests.sh
+
+ docs:
+ name: Build docs
+ runs-on: ubuntu-latest
+ env:
+ TRAVIS_JOB_NAME: "Build documentation"
+ PYTHON_VERSION: 3.6
+ steps:
+ - uses: actions/checkout@master
+ - name: "Build documentation"
+ run: ./scripts/ci/ci_docs.sh
+
+ tests-p36-postgres-integrations:
+ name: "Tests [Postgres9.6][Py3.6][integrations]"
+ runs-on: ubuntu-latest
+ needs: [statics, statics-tests]
+ env:
+ TRAVIS_JOB_NAME: "Tests [Postgres9.6][Py3.6][integrations]"
+ BACKEND: postgres
+ PYTHON_VERSION: 3.6
+ POSTGRES_VERSION: 9.6
+ ENABLED_INTEGRATIONS: "cassandra kerberos mongo openldap rabbitmq redis"
+ RUN_INTEGRATION_TESTS: all
+ steps:
+ - uses: actions/checkout@master
+ - name: "Tests [Postgres9.6][Py3.6][integrations]"
+ run: ./scripts/ci/ci_run_airflow_testing.sh
+
+ tests-p36-postgres-providers:
+ name: "Tests [Postgres10][Py3.6][providers]"
+ runs-on: ubuntu-latest
+ needs: [statics, statics-tests]
+ env:
+ TRAVIS_JOB_NAME: "Tests [Postgres10][Py3.6][providers]"
+ BACKEND: postgres
+ POSTGRES_VERSION: 10
+ PYTHON_VERSION: 3.6
+ steps:
+ - uses: actions/checkout@master
+ - name: "Tests [Postgres10][Py3.6][providers]"
+ run: ./scripts/ci/ci_run_airflow_testing.sh tests/providers
+
+ tests-p36-postgres-core:
+ name: "Tests [Postgres9.6][Py3.6][core]"
+ runs-on: ubuntu-latest
+ needs: [statics, statics-tests]
+ env:
+ TRAVIS_JOB_NAME: "Tests [Postgres9.6][Py3.6][core]"
+ BACKEND: postgres
+ POSTGRES_VERSION: 9.6
+ PYTHON_VERSION: 3.6
+ steps:
+ - uses: actions/checkout@master
+ - name: "Tests [Postgres9.6][Py3.6][core]"
+ run: ./scripts/ci/ci_run_airflow_testing.sh --ignore=tests/providers
+
+
+ tests-p37-sqlite-integrations:
+ name: "Tests [Sqlite][3.7][integrations]"
+ runs-on: ubuntu-latest
+ needs: [statics, statics-tests]
+ env:
+ TRAVIS_JOB_NAME: "Tests [Sqlite][3.7][integrations]"
+ BACKEND: sqlite
+ PYTHON_VERSION: 3.7
+ ENABLED_INTEGRATIONS: "cassandra kerberos mongo openldap rabbitmq redis"
+ RUN_INTEGRATION_TESTS: all
+ steps:
+ - uses: actions/checkout@master
+ - name: "Tests [Sqlite][3.7][integrations]"
+ run: ./scripts/ci/ci_run_airflow_testing.sh
+
+ tests-p36-sqlite:
+ name: "Tests [Sqlite][Py3.6]"
+ runs-on: ubuntu-latest
+ needs: [statics, statics-tests]
+ env:
+ TRAVIS_JOB_NAME: "Tests [Sqlite][Py3.6]"
+ BACKEND: sqlite
+ PYTHON_VERSION: 3.6
+ steps:
+ - uses: actions/checkout@master
+ - name: "Tests [Sqlite][Py3.6]"
+ run: ./scripts/ci/ci_run_airflow_testing.sh
+
+ tests-p36-mysql-integrations:
+ name: "Tests [MySQL][Py3.6][integrations]"
+ runs-on: ubuntu-latest
+ needs: [statics, statics-tests]
+ env:
+ TRAVIS_JOB_NAME: "Tests [MySQL][Py3.6][integrations]"
+ BACKEND: sqlite
+ PYTHON_VERSION: 3.6
+ MYSQL_VERSION: 5.7
+ ENABLED_INTEGRATIONS: "cassandra kerberos mongo openldap rabbitmq redis"
+ RUN_INTEGRATION_TESTS: all
+ steps:
+ - uses: actions/checkout@master
+ - name: "Tests [MySQL][Py3.6][integrations]"
+ run: ./scripts/ci/ci_run_airflow_testing.sh
+
+ tests-p36-mysql-providers:
+ name: "Tests [MySQL5.7][Py3.7][providers][kerberos]"
+ runs-on: ubuntu-latest
+ needs: [statics, statics-tests]
+ env:
+ TRAVIS_JOB_NAME: "Tests [MySQL5.7][Py3.7][providers][kerberos]"
+ BACKEND: mysql
+ PYTHON_VERSION: 3.7
+ MYSQL_VERSION: 5.7
+ ENABLED_INTEGRATIONS: "kerberos"
+ steps:
+ - uses: actions/checkout@master
+ - name: "Tests [MySQL5.7][Py3.7][providers][kerberos]"
+ run: ./scripts/ci/ci_run_airflow_testing.sh tests/providers
+
+ tests-p37-mysql-kerberos:
+ name: "Tests [MySQL5.7][Py3.7][core][kerberos]"
+ runs-on: ubuntu-latest
+ needs: [statics, statics-tests]
+ env:
+ TRAVIS_JOB_NAME: "Tests [MySQL5.7][Py3.7][core][kerberos]"
+ BACKEND: mysql
+ PYTHON_VERSION: 3.7
+ MYSQL_VERSION: 5.7
+ ENABLED_INTEGRATIONS: "kerberos"
+ steps:
+ - uses: actions/checkout@master
+ - name: "Tests [MySQL5.7][Py3.7][core][kerberos]"
+ run: ./scripts/ci/ci_run_airflow_testing.sh --ignore=tests/providers
+
+ requirements:
+ name: "Generate requirements"
+ runs-on: ubuntu-latest
+ needs: [statics, statics-tests]
+ strategy:
+ matrix:
+ python-version: [3.6, 3.7]
Review comment:
Maybe we can make the [3.6, 3.7] into variable as well? That would be great if we could :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] dimberman commented on issue #8376: Use Github Actions to
run CI
Posted by GitBox <gi...@apache.org>.
dimberman commented on issue #8376: Use Github Actions to run CI
URL: https://github.com/apache/airflow/pull/8376#issuecomment-614248841
@turbaszek @potiuk seeing how CRITICALLY clogged travis is right now I think it makes sense to move forward with this and then we can improve as needed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] dimberman commented on issue #8376: Use Github Actions to
run CI
Posted by GitBox <gi...@apache.org>.
dimberman commented on issue #8376: Use Github Actions to run CI
URL: https://github.com/apache/airflow/pull/8376#issuecomment-614272009
@potiuk once we merge https://github.com/apache/airflow/pull/8219 I should be able to move over k8s tests pretty quickly as I'll be able to make the astronomer chart compliant with the official prod image
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #8376: Use
Github Actions to run CI
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #8376: Use Github Actions to run CI
URL: https://github.com/apache/airflow/pull/8376#discussion_r409055666
##########
File path: .github/workflows/ci.yml
##########
@@ -0,0 +1,238 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+---
+name: Airflow
+on:
+ push:
+ branches: [master]
+ pull_request:
+ branches: [master]
+env:
+ BUILD_ID: ${{github.sha }}
+ MOUNT_LOCAL_SOURCES: "false"
+ MOUNT_HOST_AIRFLOW_VOLUME: "true"
+ FORCE_ANSWER_TO_QUESTIONS: "yes"
+ SKIP_CHECK_REMOTE_IMAGE: "true"
+ DB_RESET: "true"
+ VERBOSE: "true"
+ CI: "true"
+ # Should be a target branch
+ TRAVIS_BRANCH: "master"
+ TRAVIS: "true"
+jobs:
+ statics:
+ name: Static checks
+ runs-on: ubuntu-latest
+ env:
+ PYTHON_VERSION: 3.6
+ AIRFLOW_MOUNT_SOURCE_DIR_FOR_STATIC_CHECKS: "true"
+ TRAVIS_JOB_NAME: "Static"
+ steps:
+ - uses: actions/checkout@master
+ - uses: actions/setup-python@v1
+ with:
+ python-version: ${{ env.PYTHON_VERSION }}
+ - name: "Before install"
+ run: ./scripts/ci/ci_before_install.sh
+ - name: "Static checks"
+ if: success()
+ env:
+ PYTHON_VERSION: 3.6
+ run: ./scripts/ci/ci_run_all_static_checks.sh
+
+ statics-tests:
+ name: Pylint for tests
+ runs-on: ubuntu-latest
+ env:
+ PYTHON_VERSION: 3.6
+ AIRFLOW_MOUNT_SOURCE_DIR_FOR_STATIC_CHECKS: "true"
+ TRAVIS_JOB_NAME: "Static"
+ steps:
+ - uses: actions/checkout@master
+ - uses: actions/setup-python@v1
+ with:
+ python-version: ${{ env.PYTHON_VERSION }}
+ - name: "Before install"
+ run: ./scripts/ci/ci_before_install.sh
Review comment:
Idea about getting rid of the unnecessary logs. ... I think in the current GA the "ci_before_install.sh" produces unnecessary logs that are not needed in 99% of cases.. And I think there is an easy way to fix them: https://help.github.com/en/actions/configuring-and-managing-workflows/persisting-workflow-data-using-artifacts
We could redirect both stdout and stderr (> file.txt 2>&1) to a file and upload it as artifact of the build . This way we have access to it if needed but it will not clutter the log of the "real" build.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] potiuk commented on issue #8376: Use Github Actions to
run CI
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #8376: Use Github Actions to run CI
URL: https://github.com/apache/airflow/pull/8376#issuecomment-614251679
I think the only remaining one that can be moved is the backport packages 1.9 build
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] potiuk commented on issue #8376: Use Github Actions to
run CI
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #8376: Use Github Actions to run CI
URL: https://github.com/apache/airflow/pull/8376#issuecomment-614251002
Agree @turbaszek @dimberman -> the cmments were just comments in this case and we can improve later. As lon as we are decreasing pressure from Travis rather than increasing, it's fine
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] dimberman merged pull request #8376: Use Github Actions
to run CI
Posted by GitBox <gi...@apache.org>.
dimberman merged pull request #8376: Use Github Actions to run CI
URL: https://github.com/apache/airflow/pull/8376
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] potiuk commented on issue #8376: Use Github Actions to
run CI
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #8376: Use Github Actions to run CI
URL: https://github.com/apache/airflow/pull/8376#issuecomment-614252959
k8s tests can remain on Travis. I think it's not an issue for now - if we jus have 2 jobs runnnig per PR, this should be no problem even if static check and others fail on GA
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] potiuk commented on issue #8376: Use Github Actions to
run CI
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #8376: Use Github Actions to run CI
URL: https://github.com/apache/airflow/pull/8376#issuecomment-613952226
Let's also add the non-kubernetes tests to it and remove them already from travis.yml ? WDYT @turbaszek ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] turbaszek commented on a change in pull request #8376:
Use Github Actions to run CI
Posted by GitBox <gi...@apache.org>.
turbaszek commented on a change in pull request #8376: Use Github Actions to run CI
URL: https://github.com/apache/airflow/pull/8376#discussion_r409078420
##########
File path: .github/workflows/ci.yml
##########
@@ -0,0 +1,238 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+---
+name: Airflow
+on:
+ push:
+ branches: [master]
+ pull_request:
+ branches: [master]
+env:
+ BUILD_ID: ${{github.sha }}
+ MOUNT_LOCAL_SOURCES: "false"
+ MOUNT_HOST_AIRFLOW_VOLUME: "true"
+ FORCE_ANSWER_TO_QUESTIONS: "yes"
+ SKIP_CHECK_REMOTE_IMAGE: "true"
+ DB_RESET: "true"
+ VERBOSE: "true"
+ CI: "true"
+ # Should be a target branch
+ TRAVIS_BRANCH: "master"
+ TRAVIS: "true"
+jobs:
+ statics:
+ name: Static checks
+ runs-on: ubuntu-latest
+ env:
+ PYTHON_VERSION: 3.6
+ AIRFLOW_MOUNT_SOURCE_DIR_FOR_STATIC_CHECKS: "true"
+ TRAVIS_JOB_NAME: "Static"
+ steps:
+ - uses: actions/checkout@master
+ - uses: actions/setup-python@v1
+ with:
+ python-version: ${{ env.PYTHON_VERSION }}
+ - name: "Before install"
+ run: ./scripts/ci/ci_before_install.sh
+ - name: "Static checks"
+ if: success()
+ env:
+ PYTHON_VERSION: 3.6
+ run: ./scripts/ci/ci_run_all_static_checks.sh
+
+ statics-tests:
+ name: Pylint for tests
+ runs-on: ubuntu-latest
+ env:
+ PYTHON_VERSION: 3.6
+ AIRFLOW_MOUNT_SOURCE_DIR_FOR_STATIC_CHECKS: "true"
+ TRAVIS_JOB_NAME: "Static"
+ steps:
+ - uses: actions/checkout@master
+ - uses: actions/setup-python@v1
+ with:
+ python-version: ${{ env.PYTHON_VERSION }}
+ - name: "Before install"
+ run: ./scripts/ci/ci_before_install.sh
+ - name: "Static checks tests"
+ if: success()
+ env:
+ PYTHON_VERSION: 3.6
+ run: ./scripts/ci/ci_run_static_checks_pylint_tests.sh
+
+ docs:
+ name: Build docs
+ runs-on: ubuntu-latest
+ env:
+ TRAVIS_JOB_NAME: "Build documentation"
+ PYTHON_VERSION: 3.6
+ steps:
+ - uses: actions/checkout@master
+ - name: "Build documentation"
+ run: ./scripts/ci/ci_docs.sh
+
+ tests-p36-postgres-integrations:
+ name: "Tests [Postgres9.6][Py3.6][integrations]"
+ runs-on: ubuntu-latest
+ needs: [statics, statics-tests]
+ env:
+ TRAVIS_JOB_NAME: "Tests [Postgres9.6][Py3.6][integrations]"
+ BACKEND: postgres
+ PYTHON_VERSION: 3.6
+ POSTGRES_VERSION: 9.6
+ ENABLED_INTEGRATIONS: "cassandra kerberos mongo openldap rabbitmq redis"
+ RUN_INTEGRATION_TESTS: all
+ steps:
+ - uses: actions/checkout@master
+ - name: "Tests [Postgres9.6][Py3.6][integrations]"
+ run: ./scripts/ci/ci_run_airflow_testing.sh
+
+ tests-p36-postgres-providers:
+ name: "Tests [Postgres10][Py3.6][providers]"
+ runs-on: ubuntu-latest
+ needs: [statics, statics-tests]
+ env:
+ TRAVIS_JOB_NAME: "Tests [Postgres10][Py3.6][providers]"
+ BACKEND: postgres
+ POSTGRES_VERSION: 10
+ PYTHON_VERSION: 3.6
+ steps:
+ - uses: actions/checkout@master
+ - name: "Tests [Postgres10][Py3.6][providers]"
+ run: ./scripts/ci/ci_run_airflow_testing.sh tests/providers
+
+ tests-p36-postgres-core:
+ name: "Tests [Postgres9.6][Py3.6][core]"
+ runs-on: ubuntu-latest
+ needs: [statics, statics-tests]
+ env:
+ TRAVIS_JOB_NAME: "Tests [Postgres9.6][Py3.6][core]"
+ BACKEND: postgres
+ POSTGRES_VERSION: 9.6
+ PYTHON_VERSION: 3.6
+ steps:
+ - uses: actions/checkout@master
+ - name: "Tests [Postgres9.6][Py3.6][core]"
+ run: ./scripts/ci/ci_run_airflow_testing.sh --ignore=tests/providers
+
+
+ tests-p37-sqlite-integrations:
+ name: "Tests [Sqlite][3.7][integrations]"
+ runs-on: ubuntu-latest
+ needs: [statics, statics-tests]
+ env:
+ TRAVIS_JOB_NAME: "Tests [Sqlite][3.7][integrations]"
+ BACKEND: sqlite
+ PYTHON_VERSION: 3.7
+ ENABLED_INTEGRATIONS: "cassandra kerberos mongo openldap rabbitmq redis"
+ RUN_INTEGRATION_TESTS: all
+ steps:
+ - uses: actions/checkout@master
+ - name: "Tests [Sqlite][3.7][integrations]"
+ run: ./scripts/ci/ci_run_airflow_testing.sh
+
+ tests-p36-sqlite:
+ name: "Tests [Sqlite][Py3.6]"
+ runs-on: ubuntu-latest
+ needs: [statics, statics-tests]
+ env:
+ TRAVIS_JOB_NAME: "Tests [Sqlite][Py3.6]"
+ BACKEND: sqlite
+ PYTHON_VERSION: 3.6
+ steps:
+ - uses: actions/checkout@master
+ - name: "Tests [Sqlite][Py3.6]"
+ run: ./scripts/ci/ci_run_airflow_testing.sh
+
+ tests-p36-mysql-integrations:
+ name: "Tests [MySQL][Py3.6][integrations]"
+ runs-on: ubuntu-latest
+ needs: [statics, statics-tests]
+ env:
+ TRAVIS_JOB_NAME: "Tests [MySQL][Py3.6][integrations]"
+ BACKEND: sqlite
+ PYTHON_VERSION: 3.6
+ MYSQL_VERSION: 5.7
+ ENABLED_INTEGRATIONS: "cassandra kerberos mongo openldap rabbitmq redis"
+ RUN_INTEGRATION_TESTS: all
+ steps:
+ - uses: actions/checkout@master
+ - name: "Tests [MySQL][Py3.6][integrations]"
+ run: ./scripts/ci/ci_run_airflow_testing.sh
+
+ tests-p36-mysql-providers:
+ name: "Tests [MySQL5.7][Py3.7][providers][kerberos]"
+ runs-on: ubuntu-latest
+ needs: [statics, statics-tests]
+ env:
+ TRAVIS_JOB_NAME: "Tests [MySQL5.7][Py3.7][providers][kerberos]"
+ BACKEND: mysql
+ PYTHON_VERSION: 3.7
+ MYSQL_VERSION: 5.7
+ ENABLED_INTEGRATIONS: "kerberos"
+ steps:
+ - uses: actions/checkout@master
+ - name: "Tests [MySQL5.7][Py3.7][providers][kerberos]"
+ run: ./scripts/ci/ci_run_airflow_testing.sh tests/providers
+
+ tests-p37-mysql-kerberos:
+ name: "Tests [MySQL5.7][Py3.7][core][kerberos]"
+ runs-on: ubuntu-latest
+ needs: [statics, statics-tests]
+ env:
+ TRAVIS_JOB_NAME: "Tests [MySQL5.7][Py3.7][core][kerberos]"
+ BACKEND: mysql
+ PYTHON_VERSION: 3.7
+ MYSQL_VERSION: 5.7
+ ENABLED_INTEGRATIONS: "kerberos"
+ steps:
+ - uses: actions/checkout@master
+ - name: "Tests [MySQL5.7][Py3.7][core][kerberos]"
+ run: ./scripts/ci/ci_run_airflow_testing.sh --ignore=tests/providers
+
+ requirements:
+ name: "Generate requirements"
+ runs-on: ubuntu-latest
+ needs: [statics, statics-tests]
+ strategy:
+ matrix:
+ python-version: [3.6, 3.7]
Review comment:
The best option would be to extract this logic to yaml template but Github Actions doesn't support it yet...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] turbaszek commented on issue #8376: Use Github Actions to
run CI
Posted by GitBox <gi...@apache.org>.
turbaszek commented on issue #8376: Use Github Actions to run CI
URL: https://github.com/apache/airflow/pull/8376#issuecomment-614248440
@potiuk @dimberman still waiting for travis but everything else is green. I know that Daniel is working on further improvements so I would suggest to move this PR forward and address any suggestion as follow up PRs. WDYT?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] dimberman commented on issue #8376: Use Github Actions to
run CI
Posted by GitBox <gi...@apache.org>.
dimberman commented on issue #8376: Use Github Actions to run CI
URL: https://github.com/apache/airflow/pull/8376#issuecomment-614252179
@turbaszek @potiuk I think we can merge this without the k8s tests and revert if it breaks anything (which it shouldn't). Breaking normal conventions but right now this is a pretty desperate situation.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] potiuk commented on issue #8376: Use Github Actions to
run CI
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #8376: Use Github Actions to run CI
URL: https://github.com/apache/airflow/pull/8376#issuecomment-613951343
I hope it will work :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #8376: Use
Github Actions to run CI
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #8376: Use Github Actions to run CI
URL: https://github.com/apache/airflow/pull/8376#discussion_r409050055
##########
File path: .github/workflows/ci.yml
##########
@@ -0,0 +1,238 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+---
+name: Airflow
+on:
+ push:
+ branches: [master]
+ pull_request:
+ branches: [master]
+env:
+ BUILD_ID: ${{github.sha }}
+ MOUNT_LOCAL_SOURCES: "false"
+ MOUNT_HOST_AIRFLOW_VOLUME: "true"
+ FORCE_ANSWER_TO_QUESTIONS: "yes"
+ SKIP_CHECK_REMOTE_IMAGE: "true"
+ DB_RESET: "true"
+ VERBOSE: "true"
+ CI: "true"
+ # Should be a target branch
+ TRAVIS_BRANCH: "master"
+ TRAVIS: "true"
+jobs:
+ statics:
+ name: Static checks
+ runs-on: ubuntu-latest
+ env:
+ PYTHON_VERSION: 3.6
Review comment:
Is it possible to extract python version to variable ? when we cherry-pick to 1.10 we use 3.5 as base version and it would be great to be able to change it in one place only. It was not possible in Travis, but maybe we can do it in GA? Maybe a global DEFAULT_PYTHON_VERSION env will do the job?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services