You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/08/13 02:37:21 UTC
[GitHub] [airflow] vitaly-krugl opened a new issue #10309: initdb crashes when using non-empty sql_alchemy_schema in configuration
vitaly-krugl opened a new issue #10309:
URL: https://github.com/apache/airflow/issues/10309
**Apache Airflow version**: 1.10.11
**Environment**:
- **OS** (e.g. from /etc/os-release): Debian GNU/Linux 9
- **Kernel** (e.g. `uname -a`): Linux b15edd925a5a 4.19.76-linuxkit #1 SMP Tue May 26 11:42:35 UTC 2020 x86_64 GNU/Linux
**What happened**:
I specified `sql_alchemy_schema = af` in the `[core]` section of airflow.cfg and then ran `airflow initdb`.
It crashed with the following exception:
```
empower_wf_postgres | 2020-08-13 01:53:41.062 UTC [73] ERROR: relation "af.slot_pool" does not exist at character 185
empower_wf_postgres | 2020-08-13 01:53:41.062 UTC [73] STATEMENT: SELECT af.slot_pool.id AS af_slot_pool_id, af.slot_pool.pool AS af_slot_pool_pool, af.slot_pool.slots AS af_slot_pool_slots, af.slot_pool.description AS af_slot_pool_description
empower_wf_postgres | FROM af.slot_pool
empower_wf_postgres | WHERE af.slot_pool.pool = 'default_pool'
empower_wf_postgres | LIMIT 1
empower_wf_app | Traceback (most recent call last):
empower_wf_app | File "/tmp/empower-wf-venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1277, in _execute_context
empower_wf_app | self.dialect.do_execute(
empower_wf_app | File "/tmp/empower-wf-venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 593, in do_execute
empower_wf_app | cursor.execute(statement, parameters)
empower_wf_app | psycopg2.errors.UndefinedTable: relation "af.slot_pool" does not exist
empower_wf_app | LINE 2: FROM af.slot_pool
empower_wf_app | ^
empower_wf_app |
empower_wf_app |
empower_wf_app | The above exception was the direct cause of the following exception:
empower_wf_app |
empower_wf_app | Traceback (most recent call last):
empower_wf_app | File "/a/bin/airflow", line 37, in <module>
empower_wf_app | args.func(args)
empower_wf_app | File "/tmp/empower-wf-venv/lib/python3.8/site-packages/airflow/bin/cli.py", line 1329, in initdb
empower_wf_app | db.initdb(settings.RBAC)
empower_wf_app | File "/tmp/empower-wf-venv/lib/python3.8/site-packages/airflow/utils/db.py", line 323, in initdb
empower_wf_app | upgradedb()
empower_wf_app | File "/tmp/empower-wf-venv/lib/python3.8/site-packages/airflow/utils/db.py", line 387, in upgradedb
empower_wf_app | add_default_pool_if_not_exists()
empower_wf_app | File "/tmp/empower-wf-venv/lib/python3.8/site-packages/airflow/utils/db.py", line 74, in wrapper
empower_wf_app | return func(*args, **kwargs)
empower_wf_app | File "/tmp/empower-wf-venv/lib/python3.8/site-packages/airflow/utils/db.py", line 90, in add_default_pool_if_not_exists
empower_wf_app | if not Pool.get_pool(Pool.DEFAULT_POOL_NAME, session=session):
empower_wf_app | File "/tmp/empower-wf-venv/lib/python3.8/site-packages/airflow/utils/db.py", line 70, in wrapper
empower_wf_app | return func(*args, **kwargs)
empower_wf_app | File "/tmp/empower-wf-venv/lib/python3.8/site-packages/airflow/models/pool.py", line 45, in get_pool
empower_wf_app | return session.query(Pool).filter(Pool.pool == pool_name).first()
empower_wf_app | File "/tmp/empower-wf-venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3397, in first
empower_wf_app | ret = list(self[0:1])
empower_wf_app | File "/tmp/empower-wf-venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3171, in __getitem__
empower_wf_app | return list(res)
empower_wf_app | File "/tmp/empower-wf-venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3503, in __iter__
empower_wf_app | return self._execute_and_instances(context)
empower_wf_app | File "/tmp/empower-wf-venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3528, in _execute_and_instances
empower_wf_app | result = conn.execute(querycontext.statement, self._params)
empower_wf_app | File "/tmp/empower-wf-venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1014, in execute
empower_wf_app | return meth(self, multiparams, params)
empower_wf_app | File "/tmp/empower-wf-venv/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
empower_wf_app | return connection._execute_clauseelement(self, multiparams, params)
empower_wf_app | File "/tmp/empower-wf-venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1127, in _execute_clauseelement
empower_wf_app | ret = self._execute_context(
empower_wf_app | File "/tmp/empower-wf-venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1317, in _execute_context
empower_wf_app | self._handle_dbapi_exception(
empower_wf_app | File "/tmp/empower-wf-venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1511, in _handle_dbapi_exception
empower_wf_app | util.raise_(
empower_wf_app | File "/tmp/empower-wf-venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 178, in raise_
empower_wf_app | raise exception
empower_wf_app | File "/tmp/empower-wf-venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1277, in _execute_context
empower_wf_app | self.dialect.do_execute(
empower_wf_app | File "/tmp/empower-wf-venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 593, in do_execute
empower_wf_app | cursor.execute(statement, parameters)
empower_wf_app | sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedTable) relation "af.slot_pool" does not exist
empower_wf_app | LINE 2: FROM af.slot_pool
empower_wf_app | ^
empower_wf_app |
empower_wf_app | [SQL: SELECT af.slot_pool.id AS af_slot_pool_id, af.slot_pool.pool AS af_slot_pool_pool, af.slot_pool.slots AS af_slot_pool_slots, af.slot_pool.description AS af_slot_pool_description
empower_wf_app | FROM af.slot_pool
empower_wf_app | WHERE af.slot_pool.pool = %(pool_1)s
empower_wf_app | LIMIT %(param_1)s]
empower_wf_app | [parameters: {'pool_1': 'default_pool', 'param_1': 1}]
empower_wf_app | (Background on this error at: http://sqlalche.me/e/13/f405)
```
**What you expected to happen**:
I expected `airflow initdb` to succeed with a non-empty `sql_alchemy_schema`
**How to reproduce it**:
Specify `sql_alchemy_schema = af` in the `[core]` section of airflow.cfg and then run `airflow initdb` against postgresql database. I am using "postgres:10.11" docker image.
**Anything else we need to know**:
According to this stackoverflow - https://stackoverflow.com/questions/58991074/airflow-initdb-slot-pool-does-not-exists - setting sql_alchemy_schema used to work in Airflow 1.10.2, and then started failing in Airflow 1.10.6 or earlier.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] GuillaumeDesforges commented on issue #10309: initdb crashes when using non-empty sql_alchemy_schema in configuration
Posted by GitBox <gi...@apache.org>.
GuillaumeDesforges commented on issue #10309:
URL: https://github.com/apache/airflow/issues/10309#issuecomment-833468169
The revisions do not use this setting anymore, which is very upsetting. In my case, I have to get a dedicated db for Airflow because of this bug.
I think the revisions need to be modified to use the `schema` kwarg to any create_table or alter_table. That's a SQLAlchemy thing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] joshi95 removed a comment on issue #10309: initdb crashes when using non-empty sql_alchemy_schema in configuration
Posted by GitBox <gi...@apache.org>.
joshi95 removed a comment on issue #10309:
URL: https://github.com/apache/airflow/issues/10309#issuecomment-683314333
Hi I can pick this up.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal closed issue #10309: initdb crashes when using non-empty sql_alchemy_schema in configuration
Posted by GitBox <gi...@apache.org>.
eladkal closed issue #10309:
URL: https://github.com/apache/airflow/issues/10309
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] joshi95 commented on issue #10309: initdb crashes when using non-empty sql_alchemy_schema in configuration
Posted by GitBox <gi...@apache.org>.
joshi95 commented on issue #10309:
URL: https://github.com/apache/airflow/issues/10309#issuecomment-683314333
Hi I can pick this up.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org