[GitHub] [airflow] chrisrfu opened a new issue #20004: Airflow UI "stuck" and not showing most recent change

[GitBox <gi...@apache.org>]

chrisrfu opened a new issue #20004:
URL: https://github.com/apache/airflow/issues/20004


   ### Apache Airflow version
   
   2.2.2 (latest released)
   
   ### Operating System
   
   Mac OS X 10.15.7
   
   ### Versions of Apache Airflow Providers
   
   _No response_
   
   ### Deployment
   
   Docker-Compose
   
   ### Deployment details
   
   running it locally with docker and docker-compose version 3
   
   ### What happened
   
   1. run _build_me.sh_
   - _build_me.sh_ consists of:
   ```
   #!/bin/bash
   TAG=$1
   if [ -z ${TAG} ]; then
       read -p "tag: (latest) " response
       TAG=${response:-latest}
   fi
   docker build --progress=plain --no-cache -t {tag_name}:${TAG} -f Dockerfile .
   ```
   
   Dockerfile looks like this
   ```
   FROM apache/airflow:2.2.2
   
   USER airflow
   
   WORKDIR /opt/airflow
   
   COPY --chown=airflow dags/ /opt/airflow/dags/
   
   COPY airflow.cfg ./airflow.cfg
   COPY requirements.txt ./requirements.txt
   COPY devops ./devops
   
   EXPOSE 8082 8080 5555
   
   RUN pip install -r requirements.txt
   ```
   2. run _run_me.sh_
   - _run_me.sh_  looks like this:
   ```
   #!/bin/bash
   docker-compose --env-file ./config/.env.local up airflow-init
   docker-compose --env-file ./config/.env.local up
   ```
   
   3. go into the airflow UI, log in, and see that what is displayed is not reflective of the code. 
   Example: I deliberately create syntax error in my code but it doesn't show up as a dag import error on the UI.  Or more recently there's no tasks on the UI even though I have dag files.
   
   4. check the webserver container using  `docker exec -it {container_id} /bin/bash` to see if syntax error is there and in fact is there.
   
   ### What you expected to happen
   
   I expected the UI to be reflective of what is in the code. After implementing a change to my code, restarting the docker containers, re-build the docker image, run docker-compose, I should be able to see new changes (not something that is "stuck")
   
   It seems there's some sort of error in the scheduler:
   ModuleNotFoundError: No module named 'airflow_local_settings'
   
   This is the airflow logs:
   [airflow_all_log_files.txt](https://github.com/apache/airflow/files/7646817/airflow_all_log_files.txt)
   
   And this is the airflow scheduler logs:
   [airflow_scheduler_log_files.txt](https://github.com/apache/airflow/files/7646818/airflow_scheduler_log_files.txt)
   
   
   
   
   ### How to reproduce
   
   You can use the latest docker 4.3.0 (71786) or docker 3.3.3 (64133). I tried both and the result is the same.
   
   ### Anything else
   
   The problem occurs every time I build and run dockerfile and docker-compose. I've tried using Airflow 2.2.1, 2.2.0, 2.1.4 and so far same result.
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk commented on issue #20004: Airflow UI "stuck" and not showing most recent change

[GitBox <gi...@apache.org>]

potiuk commented on issue #20004:
URL: https://github.com/apache/airflow/issues/20004#issuecomment-986320371


   I think you need to dig deeper. The docker-compose is just a quick start and you are on your own to make any modifications.  You have your own modifications in tehre, local dags, airflow.cfg, some devops folder. There are plenty of things that could go wrong - your images might be not available to your docker-compose, you have not restarted the webserver and scheduler, you have no docker-compose files you use.  I am afraid you made some mistake that you need to debug in your deployment and you need to track it down.  
   
   Let me just comfort you that others do not see this kind of problem, so it's for sure some problem on your side, not airflow. But  from what you posted, it's not obvious what it can be so you need to dig deeper (as usual when you configure your own custom deployment). I am converting it into discussion now - I recommend you to check all the steps of rebuild, reload, restart again, make sure that all your components use the same image with the same dags (note that import errors come from scheduler, not webserver - webserver only displays them from the DB that is filled-in by the scheduler.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org