You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/12/03 01:57:52 UTC
[GitHub] [airflow] vikramkoka opened a new issue #12769: Documentation needed for DB upgrade as part of 2.0
vikramkoka opened a new issue #12769:
URL: https://github.com/apache/airflow/issues/12769
Following up on the dev call on 30th of November, there was a clear desire expressed for documentation around the database upgrade process from Airflow 1.10.14 (or equivalent) to Airflow 2.0. Though the upgrade process is conceptually no different from a regular 1.10.x to a 1.10.x+1 release, the fact that there are significant known database changes may raise concerns in the minds of Airflow users as part of the upgrade.
To ease their concerns, the following questions should ideally be answered as part of the documentation specifically either as part of the "Upgrading to 2.0 document" or linked from there.
Q 1. Is there anything "special" which I need to be done to upgrade from 1.10.x to 2.0 with respect to the database?
Ans. I don't believe so, other than the normal upgrade checks.
Q 2. How long should I expect this database upgrade expected to take?
Ans. I am not quite sure how to answer this since it depends on the data. We can possibly share sample times based on tested data sets.
Q 3. Can I do something to reduce the database upgrade time?
Ans. A couple of options here. One possibility is to recommend the maintenance DAGs to be run to archive / delete older task history, xcom data, and equivalent. Another possibility is to provide a script for them to run as part of the Airflow project distribution, possibly part of upgrade check scripts.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on issue #12769: Documentation needed for DB upgrade as part of 2.0
Posted by GitBox <gi...@apache.org>.
ashb commented on issue #12769:
URL: https://github.com/apache/airflow/issues/12769#issuecomment-738736577
> After that, I think it is worth improving the descriptions in UPDATING.md and the rules for airflow upgrade-check commands. I miss the SQL command that will show the problematic rows.
The rules in upgrade_check will print the problematic conn_ids:
- https://github.com/apache/airflow/blob/v1-10-stable/airflow/upgrade/rules/conn_id_is_unique.py
- https://github.com/apache/airflow/blob/v1-10-stable/airflow/upgrade/rules/conn_type_is_not_nullable.py#L41-L46
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb closed issue #12769: Documentation needed for DB upgrade as part of 2.0
Posted by GitBox <gi...@apache.org>.
ashb closed issue #12769:
URL: https://github.com/apache/airflow/issues/12769
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on issue #12769: Documentation needed for DB upgrade as part of 2.0
Posted by GitBox <gi...@apache.org>.
mik-laj commented on issue #12769:
URL: https://github.com/apache/airflow/issues/12769#issuecomment-737717346
>. Q 1. Is there anything "special" which I need to be done to upgrade from 1.10.x to 2.0 with respect to the database?
We had to do some database migrations which may not be compatible with your data. You must manually update the records in the database for the migrations to be completed. For more information, see: [UPDATIING.md](https://github.com/apache/airflow/blob/master/UPDATING.md#database-schema-changes)
You can also detect these problems via [airflow upgrade_check](https://pypi.org/project/apache-airflow-upgrade-check/).
After that, I think it is worth improving the descriptions in UPDATING.md and the rules for airflow upgrade-check commands. I miss the SQL command that will show the problematic rows.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #12769: Documentation needed for DB upgrade as part of 2.0
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #12769:
URL: https://github.com/apache/airflow/issues/12769#issuecomment-737609456
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org