You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Timelessprod (via GitHub)" <gi...@apache.org> on 2024/08/29 09:42:49 UTC
[I] Alembic migration failed when updating from 2.8.3 to 2.10.0 (version file ec3471c1e067) [airflow]
Timelessprod opened a new issue, #41853:
URL: https://github.com/apache/airflow/issues/41853
### Apache Airflow version
2.10.0
### If "Other Airflow 2 version" selected, which one?
_No response_
### What happened?
When upgrading Airflow from version 2.8.3 to 2.10.0, Airflow fails to restart. Indeed the Alembic migration fails with the error:
```
alembic.util.exc.CommandError: Can't locate revision identified by 'ec3471c1e067'
```
The missing version file is located here : https://github.com/apache/airflow/blob/e001b88f5875cfd7e295891a0bbdbc75a3dccbfb/airflow/migrations/versions/0148_2_10_0_dataset_alias_dataset_event.py
It's no longer on the master branch but it is on the last commit with tag [`2.10.0`](https://github.com/apache/airflow/tree/2.10.0)
In a attempt to fix the problem I tried to revert the version back to 2.8.3 but Alembic is still looking for the `ec3471c1e067` version file when restarting the instance, and thus Airflow is unable to start anymore.
### What you think should happen instead?
The migration should have go through without error and Airflow start.
### How to reproduce
- Install Airflow 2.8.3 with a persistent database
- Update the Airflow version to 2.10.0
- Restart the instance
- See the error
### Operating System
Debian 12.6
### Versions of Apache Airflow Providers
From the official constraints file at https://raw.githubusercontent.com/apache/airflow/constraints-2.10.0/constraints-3.11.txt
```
apache-airflow-providers-airbyte==3.8.1
apache-airflow-providers-alibaba==2.8.1
apache-airflow-providers-amazon==8.27.0
apache-airflow-providers-apache-beam==5.7.2
apache-airflow-providers-apache-cassandra==3.5.1
apache-airflow-providers-apache-drill==2.7.3
apache-airflow-providers-apache-druid==3.10.2
apache-airflow-providers-apache-flink==1.4.2
apache-airflow-providers-apache-hdfs==4.4.2
apache-airflow-providers-apache-hive==8.1.2
apache-airflow-providers-apache-iceberg==1.0.0
apache-airflow-providers-apache-impala==1.4.2
apache-airflow-providers-apache-kafka==1.5.0
apache-airflow-providers-apache-kylin==3.6.2
apache-airflow-providers-apache-livy==3.8.1
apache-airflow-providers-apache-pig==4.4.1
apache-airflow-providers-apache-pinot==4.4.2
apache-airflow-providers-apache-spark==4.9.0
apache-airflow-providers-apprise==1.3.2
apache-airflow-providers-arangodb==2.5.1
apache-airflow-providers-asana==2.5.1
apache-airflow-providers-atlassian-jira==2.6.1
apache-airflow-providers-celery==3.7.3
apache-airflow-providers-cloudant==3.5.2
apache-airflow-providers-cncf-kubernetes==8.3.4
apache-airflow-providers-cohere==1.2.1
apache-airflow-providers-common-compat==1.1.0
apache-airflow-providers-common-io==1.4.0
apache-airflow-providers-common-sql==1.15.0
apache-airflow-providers-databricks==6.8.0
apache-airflow-providers-datadog==3.6.1
apache-airflow-providers-dbt-cloud==3.9.0
apache-airflow-providers-dingding==3.5.1
apache-airflow-providers-discord==3.7.1
apache-airflow-providers-docker==3.12.3
apache-airflow-providers-elasticsearch==5.4.2
apache-airflow-providers-exasol==4.5.3
apache-airflow-providers-fab==1.2.2
apache-airflow-providers-facebook==3.5.2
apache-airflow-providers-ftp==3.10.1
apache-airflow-providers-github==2.6.2
apache-airflow-providers-google==10.21.1
apache-airflow-providers-grpc==3.5.2
apache-airflow-providers-hashicorp==3.7.1
apache-airflow-providers-http==4.12.0
apache-airflow-providers-imap==3.6.1
apache-airflow-providers-influxdb==2.6.0
apache-airflow-providers-jdbc==4.4.0
apache-airflow-providers-jenkins==3.6.1
apache-airflow-providers-microsoft-azure==10.3.0
apache-airflow-providers-microsoft-mssql==3.8.0
apache-airflow-providers-microsoft-psrp==2.7.1
apache-airflow-providers-microsoft-winrm==3.5.1
apache-airflow-providers-mongo==4.1.2
apache-airflow-providers-mysql==5.6.3
apache-airflow-providers-neo4j==3.6.1
apache-airflow-providers-odbc==4.6.3
apache-airflow-providers-openai==1.2.2
apache-airflow-providers-openfaas==3.5.1
apache-airflow-providers-openlineage==1.10.0
apache-airflow-providers-opensearch==1.3.0
apache-airflow-providers-opsgenie==5.6.1
apache-airflow-providers-oracle==3.10.3
apache-airflow-providers-pagerduty==3.7.2
apache-airflow-providers-papermill==3.7.2
apache-airflow-providers-pgvector==1.2.2
apache-airflow-providers-pinecone==2.0.1
apache-airflow-providers-postgres==5.11.3
apache-airflow-providers-presto==5.5.2
apache-airflow-providers-qdrant==1.1.2
apache-airflow-providers-redis==3.7.1
apache-airflow-providers-salesforce==5.7.2
apache-airflow-providers-samba==4.7.1
apache-airflow-providers-segment==3.5.1
apache-airflow-providers-sendgrid==3.5.1
apache-airflow-providers-sftp==4.10.3
apache-airflow-providers-singularity==3.5.1
apache-airflow-providers-slack==8.8.0
apache-airflow-providers-smtp==1.7.1
apache-airflow-providers-snowflake==5.6.1
apache-airflow-providers-sqlite==3.8.2
apache-airflow-providers-ssh==3.12.0
apache-airflow-providers-tableau==4.5.2
apache-airflow-providers-tabular==1.5.1
apache-airflow-providers-telegram==4.5.2
apache-airflow-providers-teradata==2.5.0
apache-airflow-providers-trino==5.7.2
apache-airflow-providers-vertica==3.8.2
apache-airflow-providers-weaviate==2.0.0
apache-airflow-providers-yandex==3.11.2
apache-airflow-providers-ydb==1.2.0
apache-airflow-providers-zendesk==4.7.1
```
### Deployment
Other 3rd-party Helm chart
### Deployment details
Custom Helm chart deployment.
### Anything else?
Currently my instance of Airflow is unable to start, even after trying to revert to the last used version (2.8.3).
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Alembic migration failed when updating from 2.8.3 to 2.10.0 (version file ec3471c1e067) [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk closed issue #41853: Alembic migration failed when updating from 2.8.3 to 2.10.0 (version file ec3471c1e067)
URL: https://github.com/apache/airflow/issues/41853
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Alembic migration failed when updating from 2.8.3 to 2.10.0 (version file ec3471c1e067) [airflow]
Posted by "Timelessprod (via GitHub)" <gi...@apache.org>.
Timelessprod commented on issue #41853:
URL: https://github.com/apache/airflow/issues/41853#issuecomment-2317724421
@tirkarthi Why have those version files been removed from the 2.10.0 release ? Currently Alembic cannot find them so there's something wrong with the release I think.
Is this "v2-10-test" version among the pre-releases available on PIP ? My libraries are managed by PIP and I'd prefer no to manually clone the repo if possible.
Thank you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Alembic migration failed when updating from 2.8.3 to 2.10.0 (version file ec3471c1e067) [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #41853:
URL: https://github.com/apache/airflow/issues/41853#issuecomment-2317919336
You have not mentioned it but my guess is that you are dynamically installing packages and after this dynamic installation - ikely your image actually runs lower version of Airlfow.
The revision is in the alembic table so the migration worked - but the airflow command you run uses a downgraded version of airflow after your dynamic `pip install` command.
Please take a look on what version of the image you are using and whether your customization / extending the image has some problems. For example when you are installing dynamically new packages - the `pip install` command might automatically downgrade airflow. It solely depends on packages you install and resolution of pip which might decide that downgrading airflow fulfills other dependencies better.
That's why in the offiical installation description of upgrade scenarios, you will find that you always have to specify `apache-airflow=${AIRFLOW_VERSION}` https://airflow.apache.org/docs/apache-airflow/stable/installation/installing-from-pypi.html#installation-and-upgrade-scenarios - so that `pip` won't downgrade airflow.
This is one of the reasons why we always discouraged installing packages dynamically when your image starts - because for example airlfow version might change - but also other packages might and recommend users to build their own image - this way they can be sure what version of packages they have there. With dynamic package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Alembic migration failed when updating from 2.8.3 to 2.10.0 (version file ec3471c1e067) [airflow]
Posted by "tirkarthi (via GitHub)" <gi...@apache.org>.
tirkarthi commented on issue #41853:
URL: https://github.com/apache/airflow/issues/41853#issuecomment-2317643918
main branch is now under development for Airflow 3 where migration files were pruned. You should probably try using v2-10-test if fixes for 2.10 are needed and use 2.10.0 tag.
https://github.com/apache/airflow/tree/v2-10-test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org