[jira] [Created] (AIRFLOW-3036) Upgrading to Airflow 1.10 not possible using GCP Cloud SQL for MYSQL

["Smith Mathieu (JIRA)" <ji...@apache.org>]

Smith Mathieu created AIRFLOW-3036:
--------------------------------------

             Summary: Upgrading to Airflow 1.10 not possible using GCP Cloud SQL for MYSQL
                 Key: AIRFLOW-3036
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-3036
             Project: Apache Airflow
          Issue Type: Bug
          Components: core, db
    Affects Versions: 1.10.0
         Environment: Google Cloud Platform, Google Kubernetes Engine, Airflow 1.10 on Debian Stretch, Google Cloud SQL MySQL
            Reporter: Smith Mathieu


The upgrade path to airflow 1.10 seems impossible for users of MySQL in Google's Cloud SQL service given new mysql requirements for 1.10.

 

When executing "airflow upgradedb"

```
INFO [alembic.runtime.migration] Running upgrade d2ae31099d61 -> 0e2a74e0fc9f, Add time zone awareness
Traceback (most recent call last):
 File "/usr/local/bin/airflow", line 32, in <module>
 args.func(args)
 File "/usr/local/lib/python3.6/site-packages/airflow/bin/cli.py", line 1002, in initdb
 db_utils.initdb(settings.RBAC)
 File "/usr/local/lib/python3.6/site-packages/airflow/utils/db.py", line 92, in initdb
 upgradedb()
 File "/usr/local/lib/python3.6/site-packages/airflow/utils/db.py", line 346, in upgradedb
 command.upgrade(config, 'heads')
 File "/usr/local/lib/python3.6/site-packages/alembic/command.py", line 174, in upgrade
 script.run_env()
 File "/usr/local/lib/python3.6/site-packages/alembic/script/base.py", line 416, in run_env
 util.load_python_file(self.dir, 'env.py')
 File "/usr/local/lib/python3.6/site-packages/alembic/util/pyfiles.py", line 93, in load_python_file
 module = load_module_py(module_id, path)
 File "/usr/local/lib/python3.6/site-packages/alembic/util/compat.py", line 68, in load_module_py
 module_id, path).load_module(module_id)
 File "<frozen importlib._bootstrap_external>", line 399, in _check_name_wrapper
 File "<frozen importlib._bootstrap_external>", line 823, in load_module
 File "<frozen importlib._bootstrap_external>", line 682, in load_module
 File "<frozen importlib._bootstrap>", line 265, in _load_module_shim
 File "<frozen importlib._bootstrap>", line 684, in _load
 File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
 File "<frozen importlib._bootstrap_external>", line 678, in exec_module
 File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
 File "/usr/local/lib/python3.6/site-packages/airflow/migrations/env.py", line 91, in <module>
 run_migrations_online()
 File "/usr/local/lib/python3.6/site-packages/airflow/migrations/env.py", line 86, in run_migrations_online
 context.run_migrations()
 File "<string>", line 8, in run_migrations
 File "/usr/local/lib/python3.6/site-packages/alembic/runtime/environment.py", line 807, in run_migrations
 self.get_context().run_migrations(**kw)
 File "/usr/local/lib/python3.6/site-packages/alembic/runtime/migration.py", line 321, in run_migrations
 step.migration_fn(**kw)
 File "/usr/local/lib/python3.6/site-packages/airflow/migrations/versions/0e2a74e0fc9f_add_time_zone_awareness.py", line 46, in upgrade
 raise Exception("Global variable explicit_defaults_for_timestamp needs to be on (1) for mysql")
Exception: Global variable explicit_defaults_for_timestamp needs to be on (1) for mysql
```
 

Reading documentation for upgrading to airflow 1.10, it seems the requirement for explicit_defaults_for_timestamp=1 was intentional. 

 

However,  MySQL on Google Cloud SQL does not support configuring this variable and it is off by default. Users of MySQL and Cloud SQL do not have an upgrade path to 1.10

In GCP, Cloud SQL is _the_ hosted MySQL solution. 

https://cloud.google.com/sql/docs/mysql/flags



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)