You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/01/19 16:12:32 UTC
[GitHub] [airflow] potiuk opened a new issue #20960: Enter Shell environment with Breeze
potiuk opened a new issue #20960:
URL: https://github.com/apache/airflow/issues/20960
Includes:
* port forwarding
* selecting different backends
* selecting different python version
* diagnostics information
* remember last used parameters
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Bowrna commented on issue #20960: Breeze: Enter Shell environment with Breeze
Posted by GitBox <gi...@apache.org>.
Bowrna commented on issue #20960:
URL: https://github.com/apache/airflow/issues/20960#issuecomment-1021056331
@potiuk I will pick this one next
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #20960: Breeze: Enter Shell environment with Breeze
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #20960:
URL: https://github.com/apache/airflow/issues/20960
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Bowrna commented on issue #20960: Breeze: Enter Shell environment with Breeze
Posted by GitBox <gi...@apache.org>.
Bowrna commented on issue #20960:
URL: https://github.com/apache/airflow/issues/20960#issuecomment-1021056331
@potiuk I will pick this one next
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Bowrna commented on issue #20960: Breeze: Enter Shell environment with Breeze
Posted by GitBox <gi...@apache.org>.
Bowrna commented on issue #20960:
URL: https://github.com/apache/airflow/issues/20960#issuecomment-1022063213
@potiuk can you guide me in old breeze to the part of code that gets executed when we do `./breeze shell` command? I am not able to backtrack the code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Bowrna edited a comment on issue #20960: Breeze: Enter Shell environment with Breeze
Posted by GitBox <gi...@apache.org>.
Bowrna edited a comment on issue #20960:
URL: https://github.com/apache/airflow/issues/20960#issuecomment-1022797006
initialization::create_directories()
Creates folders to mount inside the container.
initialization::check_docker_version()
This checks the docker version and instructs if failed to install docker. Check if the docker available is good to go
build_images::forget_last_answer()
To remove the forced answer given previously.
build_images::determine_docker_cache_strategy()
We use pulled docker image cache by default for CI images
build_images::rebuild_ci_image_if_needed()
Checks if rebuild CI image is needed or not. Checks if docker build is needed. If yes, confirms image rebuild.
breeze::print_badge()
This emits the airflow logo, backend(mysql, postgres etc), backend_version, checks the cache to see if we have to suppress the asciiart logo. If not, print the visual art for airflow
initialization::summarize_build_environment()
How do we get values for all env variables initialized here? Are all of them are constants or do we have to compute on-demand or pass as a param? I will have to verify this part.
breeze::print_setup_instructions()
Prints setup instruction in case we find that autocomplete is not set
resoure_check() in run_resource_check.py
breeze::prepare_command_file()
This generates script inside build_cache dir depending on the type of configuration chosen and env var for breeze and file that can be run to use the docker command.
@potiuk I tried to find the logical steps getting executed in entering the shell. Please correct me if any of my understanding is wrong.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #20960: Breeze: Enter Shell environment with Breeze
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #20960:
URL: https://github.com/apache/airflow/issues/20960#issuecomment-1021319032
Great!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #20960: Breeze: Enter Shell environment with Breeze
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #20960:
URL: https://github.com/apache/airflow/issues/20960#issuecomment-1022450596
Look for `enter_breeze` :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #20960: Breeze: Enter Shell environment with Breeze
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #20960:
URL: https://github.com/apache/airflow/issues/20960#issuecomment-1023123165
Looks good. Few comments:
> build_images::forget_last_answer()
We can skip this one. It has been used in pre-commits and shoudl have been removed in the last `buildx` refactor. I will remove it shortly. It was used to remember which answer we gave when "build" pre-commit run, so that "mypy" and "flake" would not ask the same question again. But right now I opted for simply "warning" the user and continuing with the pre-commit rather than asking the question, so this is not needed any more.
> initialization::summarize_build_environment()
We should use `BuildParams` for that (and possibly add stuff that is missing there). This one shoudl likely be different though for CI and PROD build. Right now the same "summary" is used for PROD/CI. but PROD build accept many more params and variables, so we could end up with two different summaries - one for CI and one for PROD image.
> breeze::prepare_command_file()
This one is a bit too "complex" and counter-intuitive and we **might** want to simplify it. Initially, I've done it because I wanted to have an easy way to run the final "docker-compose" command which breeze runs in the same way breeze does by running this simple "dc" script generated. This is nice to debug and test but a bit anti-pattern (you generate code that you execute later) and complex to understand. Since we are in python, I would rather now do it differently. We have the `--verbose` flag that prints the command we execute, so rather than setting the variables in the "dc" scripts, I'd pass them to docker-compose command via -e command. I.e instaad of :
What currently generated `dc_ci` script does:
```
PYTHON_MAJOR_MINOR_VERSION=3.6
COMPOSE_FILE=file1;file2....
docker-compose "${@}"
```
I would simply run:
```
docker-compose run -e PYTHON_MAJOR_MINOR_VERSION=3.6 -f <file_1> -f <file2> .....
```
which is equivalent.
One more thing the `dc_ci` script does - it checks if the docker-compose is the right version, so we should also do the check in python.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Bowrna commented on issue #20960: Breeze: Enter Shell environment with Breeze
Posted by GitBox <gi...@apache.org>.
Bowrna commented on issue #20960:
URL: https://github.com/apache/airflow/issues/20960#issuecomment-1022797006
initialization::check_docker_version()
This checks the docker version and instructs if failed to install docker. Check if the docker available is good to go
build_images::forget_last_answer()
To remove the forced answer given previously.
build_images::determine_docker_cache_strategy()
We use pulled docker image cache by default for CI images
build_images::rebuild_ci_image_if_needed()
Checks if rebuild CI image is needed or not. Checks if docker build is needed. If yes, confirms image rebuild.
breeze::print_badge()
This emits the airflow logo, backend(mysql, postgres etc), backend_version, checks the cache to see if we have to suppress the asciiart logo. If not, print the visual art for airflow
initialization::summarize_build_environment()
How do we get values for all env variables initialized here? Are all of them are constants or do we have to compute on-demand or pass as a param? I will have to verify this part.
breeze::print_setup_instructions()
Prints setup instruction in case we find that autocomplete is not set
resoure_check() in run_resource_check.py
breeze::prepare_command_file()
This generates script inside build_cache dir depending on the type of configuration chosen and env var for breeze and file that can be run to use the docker command.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #20960: Breeze: Enter Shell environment with Breeze
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #20960:
URL: https://github.com/apache/airflow/issues/20960#issuecomment-1021319032
Great!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org