You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/12/08 21:30:41 UTC

[GitHub] [airflow] ashb commented on a change in pull request #12930: Upgrading to Airflow 2.0 doc

ashb commented on a change in pull request #12930:
URL: https://github.com/apache/airflow/pull/12930#discussion_r538821665



##########
File path: docs/apache-airflow/upgrading-to-2.rst
##########
@@ -0,0 +1,1164 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+
+Upgrading to Airflow 2.0+
+-------------------------
+
+.. contents:: :local:
+
+Apache Airflow 2.0 is a major release and the purpose of this document is to assist
+users to migrate from Airflow 1.10.x to Airflow 2.0.
+
+Step 1: Upgrade to Python 3
+'''''''''''''''''''''''''''
+
+Airflow 1.10 will be the last release series to support Python 2. Airflow 2.0.0 will require Python 3.6+.
+
+If you have a specific task that still requires Python 2 then you can use the :class:`~airflow.operators.python.PythonVirtualenvOperator` for this.
+
+For a list of breaking changes between Python 2 and Python 3, please refer to this
+[handy blog](https://blog.couchbase.com/tips-and-tricks-for-upgrading-from-python-2-to-python-3/)
+from the CouchBaseDB team.
+
+
+Step 2: Upgrade to Airflow 1.10.14 (a.k.a our "bridge" release)
+'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
+
+To minimize friction for users upgrading from Airflow 1.10 to Airflow 2.0 and beyond, a "bridge"
+release and final 1.10 version will be made available. Airflow 1.10.14 includes support for various critical features
+that make it easy for users to test DAGs and make sure they are as compatible with Airflow 2.0, so that the . We strongly recommend that all users upgrading to Airflow 2.0 first
+upgrade to Airflow 1.10.14 is straight forward.
+
+Features in 1.10.14 include:
+
+1. Most breaking DAG and architecture changes of Airflow 2.0 have been backported to Airflow 1.10.14. This backward-compatibility does not mean
+that 1.10.14 will process these DAGs the same way as Airflow 2.0. What this does mean is that most Airflow 2.0
+compatible DAGs will work in Airflow 1.10.14. This backport will give users time to modify their DAGs over time
+without any service disruption.
+
+2. We have also backported the updated Airflow 2.0 CLI commands to Airflow 1.10.4, so that users can modify their scripts
+to be compatible with Airflow 2.0 before the upgrade.
+
+3. For users of the KubernetesExecutor, we have backported the ``pod_template_file`` capability for the KubernetesExecutor
+as well as a script that will generate a ``pod_template_file`` based on your ``airflow.cfg`` settings. To generate this file
+simply run the following command:
+
+.. code-block:: bash
+
+     airflow generate_pod_template -o <output file path>
+
+Once you have performed this step, simply write out the file path to this file in the ``pod_template_file`` section of the ``kubernetes``
+section of your ``airflow.cfg``
+
+Step 3: Install and run the Upgrade check scripts
+'''''''''''''''''''''''''''''''''''''''''''''''''
+
+After upgrading to Airflow 1.10.14, we recommend that you install the "upgrade check" scripts. These scripts will read through your ``airflow.cfg`` and all of your Dags and will give a detailed report of all changes required before upgrading. We are testing this script diligently, and our goal is that any Airflow setup that can pass these tests will be able to upgrade to 2.0 without any issues.

Review comment:
       ```suggestion
   After upgrading to Airflow 1.10.14, we recommend that you install the "upgrade check" scripts. These scripts will read through your ``airflow.cfg`` and all of your DAGs and will give a detailed report of all changes required before upgrading. We are testing this script diligently, and our goal is that any Airflow setup that can pass these tests will be able to upgrade to 2.0 without any issues.
   ```




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org