You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/12/01 20:32:20 UTC
[GitHub] [airflow] halkstar opened a new issue #19945: Running alembic migration downgrade on database causes error
halkstar opened a new issue #19945:
URL: https://github.com/apache/airflow/issues/19945
### Apache Airflow version
2.2.2 (latest released)
### Operating System
PRETTY_NAME="Debian GNU/Linux 10 (buster)" NAME="Debian GNU/Linux" VERSION_ID="10" VERSION="10 (buster)" VERSION_CODENAME=buster ID=debian HOME_URL="https://www.debian.org/" SUPPORT_URL="https://www.debian.org/support" BUG_REPORT_URL="https://bugs.debian.org/"
### Versions of Apache Airflow Providers
apache-airflow-providers-amazon==2.4.0
apache-airflow-providers-celery==2.1.0
apache-airflow-providers-cncf-kubernetes==2.1.0
apache-airflow-providers-datadog==2.0.1
apache-airflow-providers-docker==2.3.0
apache-airflow-providers-elasticsearch==2.1.0
apache-airflow-providers-ftp==2.0.1
apache-airflow-providers-google==6.1.0
apache-airflow-providers-grpc==2.0.1
apache-airflow-providers-hashicorp==2.1.1
apache-airflow-providers-http==2.0.1
apache-airflow-providers-imap==2.0.1
apache-airflow-providers-microsoft-azure==3.3.0
apache-airflow-providers-mysql==2.1.1
apache-airflow-providers-odbc==2.0.1
apache-airflow-providers-postgres==2.3.0
apache-airflow-providers-redis==2.0.1
apache-airflow-providers-sendgrid==2.0.1
apache-airflow-providers-sftp==2.2.0
apache-airflow-providers-slack==4.1.0
apache-airflow-providers-sqlite==2.0.1
apache-airflow-providers-ssh==2.3.0
### Deployment
Docker-Compose
### Deployment details
_No response_
### What happened
My current database alembic_version is set at e9304a3141f0
While running the command `alembic -c alembic.ini downgrade a13f7613ad25` I received the following error:
INFO [alembic.runtime.migration] Running downgrade e9304a3141f0 -> 83f031fd9f1c, make xcom pkey columns non-nullable
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1277, in _execute_context
cursor, statement, parameters, context
File "/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
cursor.execute(statement, parameters)
psycopg2.errors.InvalidTableDefinition: column "key" is in a primary key
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/airflow/.local/bin/alembic", line 8, in <module>
sys.exit(main())
File "/home/airflow/.local/lib/python3.7/site-packages/alembic/config.py", line 588, in main
CommandLine(prog=prog).main(argv=argv)
File "/home/airflow/.local/lib/python3.7/site-packages/alembic/config.py", line 582, in main
self.run_cmd(cfg, options)
File "/home/airflow/.local/lib/python3.7/site-packages/alembic/config.py", line 562, in run_cmd
**dict((k, getattr(options, k, None)) for k in kwarg)
File "/home/airflow/.local/lib/python3.7/site-packages/alembic/command.py", line 366, in downgrade
script.run_env()
File "/home/airflow/.local/lib/python3.7/site-packages/alembic/script/base.py", line 563, in run_env
util.load_python_file(self.dir, "env.py")
File "/home/airflow/.local/lib/python3.7/site-packages/alembic/util/pyfiles.py", line 92, in load_python_file
module = load_module_py(module_id, path)
File "/home/airflow/.local/lib/python3.7/site-packages/alembic/util/pyfiles.py", line 108, in load_module_py
spec.loader.exec_module(module) # type: ignore
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "migrations/env.py", line 107, in <module>
run_migrations_online()
File "migrations/env.py", line 101, in run_migrations_online
context.run_migrations()
File "<string>", line 8, in run_migrations
File "/home/airflow/.local/lib/python3.7/site-packages/alembic/runtime/environment.py", line 851, in run_migrations
self.get_context().run_migrations(**kw)
File "/home/airflow/.local/lib/python3.7/site-packages/alembic/runtime/migration.py", line 620, in run_migrations
step.migration_fn(**kw)
File "/home/airflow/.local/lib/python3.7/site-packages/airflow/migrations/versions/e9304a3141f0_make_xcom_pkey_columns_non_nullable.py", line 76, in downgrade
bop.alter_column("execution_date", type_=_get_timestamp(conn), nullable=True)
File "/usr/local/lib/python3.7/contextlib.py", line 119, in __exit__
next(self.gen)
File "/home/airflow/.local/lib/python3.7/site-packages/alembic/operations/base.py", line 374, in batch_alter_table
impl.flush()
File "/home/airflow/.local/lib/python3.7/site-packages/alembic/operations/batch.py", line 107, in flush
fn(*arg, **kw)
File "/home/airflow/.local/lib/python3.7/site-packages/alembic/ddl/postgresql.py", line 185, in alter_column
**kw
File "/home/airflow/.local/lib/python3.7/site-packages/alembic/ddl/impl.py", line 240, in alter_column
existing_comment=existing_comment,
File "/home/airflow/.local/lib/python3.7/site-packages/alembic/ddl/impl.py", line 197, in _exec
return conn.execute(construct, multiparams)
File "/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
return meth(self, multiparams, params)
File "/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
return connection._execute_ddl(self, multiparams, params)
File "/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1073, in _execute_ddl
compiled,
File "/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1317, in _execute_context
e, statement, parameters, cursor, context
File "/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1511, in _handle_dbapi_exception
sqlalchemy_exception, with_traceback=exc_info[2], from_=e
File "/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1277, in _execute_context
cursor, statement, parameters, context
File "/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.InvalidTableDefinition) column "key" is in a primary key
[SQL: ALTER TABLE xcom ALTER COLUMN key DROP NOT NULL]
It seems the DOWN migration step for revision `e9304a3141f0 - Make XCom primary key columns non-nullable` is not backwards compatible as it is attempting to make a Primary Key nullable, which is not possible.
I suspect the revision of `bbf4a7ad0465 - Remove id column from xcom` has something to do with this - id perhaps being the old primary key for the table.
Revision list: https://airflow.apache.org/docs/apache-airflow/stable/migrations-ref.html
### What you expected to happen
I expected the alembic version to downgrade to the appropriate version
### How to reproduce
On airflow image 2.2.2 attempt to run `alembic -c alembic.ini downgrade {any version lower than revision id e9304a3141f0}`
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #19945: Running alembic migration downgrade on database causes error
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #19945:
URL: https://github.com/apache/airflow/issues/19945#issuecomment-984031247
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jedcunningham closed issue #19945: Running alembic migration downgrade on database causes error
Posted by GitBox <gi...@apache.org>.
jedcunningham closed issue #19945:
URL: https://github.com/apache/airflow/issues/19945
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org