You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/12/11 02:48:32 UTC

[GitHub] [airflow] jinyeluo opened a new issue #13004: unable to open database file with run_as_user set to a different user

jinyeluo opened a new issue #13004:
URL: https://github.com/apache/airflow/issues/13004


   **Apache Airflow version**:
   1.10.12
   
   **Kubernetes version (if you are using kubernetes)** (use `kubectl version`):
   
   **Environment**:
   
   - **Cloud provider or hardware configuration**:
   - **OS** (e.g. from /etc/os-release): Red Hat 7.9
   - **Kernel** (e.g. `uname -a`):  Linux hqhddev84 3.10.0-1160.2.2.el7.x86_64
   - **Install tools**: pip install
   - **Others**:
   
   **What happened**:
   Whenever I use 'run_as_user',  the system switches the thread to be on a different user, then not able to update the table with status.
   The error happens on a PythonOperator in my case, but I believe it happens to all operators.
   
   <!-- (please include exact error messages if you can) -->
   
   [2020-12-10 21:25:45,344] {taskinstance.py:1150} ERROR - (sqlite3.OperationalError) unable to open database file
   [SQL: UPDATE task_instance SET start_date=?, end_date=?, state=? WHERE task_instance.dag_id = ? AND task_instance.execution_date = ? AND task_instance.task_id IN (?, ?)]
   [parameters: ('2020-12-11 02:25:45.342154', '2020-12-11 02:25:45.342154', 'skipped', 'frank_file_sensor_test_v2', '2020-12-11 02:25:18.725891', 'delete_file_op', 'do_something_useful')]
   (Background on this error at: http://sqlalche.me/e/13/e3q8)
   Traceback (most recent call last):
     File "/data/work01/venv/airflow_venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1277, in _execute_context
       cursor, statement, parameters, context
     File "/data/work01/venv/airflow_venv/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 593, in do_execute
       cursor.execute(statement, parameters)
   sqlite3.OperationalError: unable to open database file
   
   
   **What you expected to happen**:
   The python_callable should be executed as the user I defined using "run_as_user", but all airflow specific logic, like updating database status, should be executed as "airflow".
   
   <!-- What do you think went wrong? -->
   Whenever I use 'run_as_user',  the system switches the thread to be on a different user, then not able to update the table with status.
   The error happens on a PythonOperator in my case, but I believe it happens to all operators.
   **How to reproduce it**:
   
   As log as a Dag is defined with a "run_as_user", and the user doesn't have access to sqlite file, running the Dag will cause the error.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] ashb commented on issue #13004: unable to open database file whenever run_as_user set to a different user

Posted by GitBox <gi...@apache.org>.
ashb commented on issue #13004:
URL: https://github.com/apache/airflow/issues/13004#issuecomment-743069727


   This is a "WONTFIX" I'm afraid -- it's a problem with using SQLLite which is not recommended for anything other than simple tests https://airflow.apache.org/docs/apache-airflow/stable/best-practices.html?highlight=sqlite#database-backend
   
   We do have longer term plans that would fix this as a side-effect, but this is a limitation of SQLite and the linux permissions model.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on issue #13004: unable to open database file with run_as_user set to a different user

Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #13004:
URL: https://github.com/apache/airflow/issues/13004#issuecomment-742930317


   Thanks for opening your first issue here! Be sure to follow the issue template!
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] ashb closed issue #13004: unable to open database file whenever run_as_user set to a different user

Posted by GitBox <gi...@apache.org>.
ashb closed issue #13004:
URL: https://github.com/apache/airflow/issues/13004


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org