[GitHub] [airflow] potiuk commented on issue #15965: sqlite3.OperationalError: no such table: dag

[GitBox <gi...@apache.org>]

potiuk commented on issue #15965:
URL: https://github.com/apache/airflow/issues/15965#issuecomment-844875095


   The problem is that you are using sqlite database. Sqlite works in different way than postgres/mysql. Instead of having a server where different containers connect to,  sqlites writes local file where the database is kept. Unless you create a shared volume which is mounted in all the containers, the database initialization you do in in "init" container is gone at the moment the container finishes. In your case, when webserver or scheduler start - each of them have their own new, fresh, non-initialized sqlite database and it is not going to work, because all of them should share a single database.
   
   There are two ways you can approach it:
   
   1) modify the docker compose.yaml to include such a shared volume, mount it in all containers. This requires a bit more intrisic knowledge of how docker/docker-compose works. Ideally this should be a named volume that can survive stopping/starting docker containers, so that the DB is more persistent.
   
   2) use postgres database - in the way that the original docker compose of ours uses it. It starts a postgres service which then all the containers connect to, so the init container will initialize the database and other containers will be able to us it..
   
   Please let me know what worked for you and I will close the issue then.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org