You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/03/11 22:42:27 UTC
[GitHub] [airflow] JavierLopezT opened a new issue #14733: Contributors quick start for Mac
JavierLopezT opened a new issue #14733:
URL: https://github.com/apache/airflow/issues/14733
There is this great guide for Ubuntu https://github.com/apache/airflow/blob/master/CONTRIBUTORS_QUICK_START.rst
I have tried to follow it in Mac with several errors, and after some research on Google for solving them, I have not been able to make Breeze work on my mac.
Would we nice to have the adapted version for Mac
@VBhojawala
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #14733: Contributors quick start for Mac
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #14733:
URL: https://github.com/apache/airflow/issues/14733#issuecomment-879858396
I think there is no single place for MacOS: https://github.com/apache/airflow/blob/main/BREEZE.rst contains links to MacOS relevant stuff for contributors (where it is relevant) same here: https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst . there are also some mentions https://airflow.apache.org/docs/apache-airflow/stable/start/docker.html . But there is no MacOS -only guide.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] grahovam commented on issue #14733: Contributors quick start for Mac
Posted by GitBox <gi...@apache.org>.
grahovam commented on issue #14733:
URL: https://github.com/apache/airflow/issues/14733#issuecomment-879755599
I remember @kaxil mentioned a guide specific for mac users in the current summit. I don't fully remember, was it a guide on how to run airflow on docker on mac? I can't find it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #14733: Contributors quick start for Mac
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #14733:
URL: https://github.com/apache/airflow/issues/14733#issuecomment-800513337
You can use `./breeze stop` to remove all containers and restart it. I just added a fix to cleanly remove all volumes always - https://github.com/apache/airflow/pull/14825 (I hope to merge it later today) but in your case it should work already. before.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #14733: Contributors quick start for Mac
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #14733:
URL: https://github.com/apache/airflow/issues/14733#issuecomment-797101438
It would be great if you could document problems you had on your Mac. This would be tremendously helpful what you have been struggling with.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #14733: Contributors quick start for Mac
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #14733:
URL: https://github.com/apache/airflow/issues/14733#issuecomment-800515409
`./breeze stop` - is your friedn in such cases (it happens when some of the containers get some internal problem. Also there is a nice "Troubleshooting" chapter that you can follow in BREEZE.rst - and there are a number of steps you can follow to "clean up" your docker engine. Usually the problem with Docker on Mac are coming from not enough memory/space allocated (see BREEZE.rst for intstructions on how to do it). occassionally the engine needs cleaning (`pruning') but this is all covered in "Troubleshooting" chapter.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #14733: Contributors quick start for Mac
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #14733:
URL: https://github.com/apache/airflow/issues/14733#issuecomment-803295152
@Visya https://github.com/apache/airflow/pull/14908 -> PR to add the checks and resource information!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on issue #14733: Contributors quick start for Mac
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #14733:
URL: https://github.com/apache/airflow/issues/14733#issuecomment-880143094
Oh yes, this one https://github.com/apache/airflow/blob/177dfbd12a42a5c229640c6c830f43f280ea5caa/BREEZE.rst#docker-community-edition -- basically if you are on Mac, then you should increase resources for Docker otherwise the container will fail
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #14733: Contributors quick start for Mac
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #14733:
URL: https://github.com/apache/airflow/issues/14733#issuecomment-797101438
It would be great if you could document problems you have on your Mac. This would be tremendously helpful what you have been struggling with.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Visya edited a comment on issue #14733: Contributors quick start for Mac
Posted by GitBox <gi...@apache.org>.
Visya edited a comment on issue #14733:
URL: https://github.com/apache/airflow/issues/14733#issuecomment-799577069
I have a problem on my Mac too.
In my case mysql container created by Breeze constantly restarts.
```
> ./breeze --python 3.8 --backend mysql
####################################################################################################
Port forwarding:
Ports are forwarded to the running docker containers for webserver and database
* 28080 -> forwarded to Airflow webserver -> airflow:8080
* 25555 -> forwarded to Flower dashboard -> airflow:5555
* 25433 -> forwarded to Postgres database -> postgres:5432
* 23306 -> forwarded to MySQL database -> mysql:3306
* 26379 -> forwarded to Redis broker -> redis:6379
Here are links to those services that you can use on host:
* Webserver: http://127.0.0.1:28080
* Flower: http://127.0.0.1:25555
* Postgres: jdbc:postgresql://127.0.0.1:25433/airflow?user=postgres&password=airflow
* Mysql: jdbc:mysql://127.0.0.1:23306/airflow?user=root
* Redis: redis://127.0.0.1:26379/0
####################################################################################################
You can setup autocomplete by running 'breeze setup-autocomplete'
####################################################################################################
You can toggle ascii/cheatsheet by running:
* breeze toggle-suppress-cheatsheet
* breeze toggle-suppress-asciiart
####################################################################################################
ERROR: for airflow Container "068f1006a91c" is unhealthy.
ERROR: Encountered errors while bringing up the project.
ERROR: The previous step completed with error. Please take a look at output above
```
I can run `breeze restart` and `breeze start-airflow`, but I can't connect to mysql or run tests.
```
> ./breeze --backend mysql --mysql-version 5.7 --python 3.8 --db-reset --test-type All tests
Backend: mysql
MySQL version: 5.7
Python version: 3.8
Resetting the DB!
Selected test type: All
182f3d0a12609a1fec40de7f9d0de83cb893427484d236f3fab4650b2b3c5524
CI image.
Branch name: master
Docker image: apache/airflow:master-python3.8-ci
GitHub cache prefix: docker.pkg.github.com/apache/airflow
Airflow source version: 2.1.0.dev0
Python version: 3.8
DockerHub user: apache
DockerHub repo: airflow
Backend: mysql 5.7
ERROR: for airflow Container "068f1006a91c" is unhealthy.
ERROR: Encountered errors while bringing up the project.
ERROR: The previous step completed with error. Please take a look at output above
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Visya commented on issue #14733: Contributors quick start for Mac
Posted by GitBox <gi...@apache.org>.
Visya commented on issue #14733:
URL: https://github.com/apache/airflow/issues/14733#issuecomment-799577069
I have a problem on my Mac.
In my case mysql container created by Breeze constantly restarts.
```
> ./breeze --python 3.8 --backend mysql
####################################################################################################
Port forwarding:
Ports are forwarded to the running docker containers for webserver and database
* 28080 -> forwarded to Airflow webserver -> airflow:8080
* 25555 -> forwarded to Flower dashboard -> airflow:5555
* 25433 -> forwarded to Postgres database -> postgres:5432
* 23306 -> forwarded to MySQL database -> mysql:3306
* 26379 -> forwarded to Redis broker -> redis:6379
Here are links to those services that you can use on host:
* Webserver: http://127.0.0.1:28080
* Flower: http://127.0.0.1:25555
* Postgres: jdbc:postgresql://127.0.0.1:25433/airflow?user=postgres&password=airflow
* Mysql: jdbc:mysql://127.0.0.1:23306/airflow?user=root
* Redis: redis://127.0.0.1:26379/0
####################################################################################################
You can setup autocomplete by running 'breeze setup-autocomplete'
####################################################################################################
You can toggle ascii/cheatsheet by running:
* breeze toggle-suppress-cheatsheet
* breeze toggle-suppress-asciiart
####################################################################################################
ERROR: for airflow Container "068f1006a91c" is unhealthy.
ERROR: Encountered errors while bringing up the project.
ERROR: The previous step completed with error. Please take a look at output above
```
I can run `breeze restart` and `breeze start-airflow`, but I can't connect to mysql or run tests.
```
> ./breeze --backend mysql --mysql-version 5.7 --python 3.8 --db-reset --test-type All tests
Backend: mysql
MySQL version: 5.7
Python version: 3.8
Resetting the DB!
Selected test type: All
182f3d0a12609a1fec40de7f9d0de83cb893427484d236f3fab4650b2b3c5524
CI image.
Branch name: master
Docker image: apache/airflow:master-python3.8-ci
GitHub cache prefix: docker.pkg.github.com/apache/airflow
Airflow source version: 2.1.0.dev0
Python version: 3.8
DockerHub user: apache
DockerHub repo: airflow
Backend: mysql 5.7
ERROR: for airflow Container "068f1006a91c" is unhealthy.
ERROR: Encountered errors while bringing up the project.
ERROR: The previous step completed with error. Please take a look at output above
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] JavierLopezT commented on issue #14733: Contributors quick start for Mac
Posted by GitBox <gi...@apache.org>.
JavierLopezT commented on issue #14733:
URL: https://github.com/apache/airflow/issues/14733#issuecomment-810945307
> It would be great if you could document problems you had on your Mac. This would be tremendously helpful what you have been struggling with.
You're right. In my case, I had some problems installing things that I have to solve searching on google (can't recall right now). After that, the breeze command didn't work, neither on the working directory where I installed everything nor in any other directory. Also, I have zsh and something I had to modify to solve an error started to raise a question every time I start the terminal:
```
zsh compinit: insecure directories, run compaudit for list.
Ignore insecure directories and continue [y] or abort compinit [n]?
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] JavierLopezT edited a comment on issue #14733: Contributors quick start for Mac
Posted by GitBox <gi...@apache.org>.
JavierLopezT edited a comment on issue #14733:
URL: https://github.com/apache/airflow/issues/14733#issuecomment-810945307
> It would be great if you could document problems you had on your Mac. This would be tremendously helpful what you have been struggling with.
You're right. In my case, I had some problems installing things that I had to solve searching on google (can't recall right now). After that, the breeze command didn't work, neither on the working directory where I installed everything nor in any other directory, even though I repeated the instructions several times. Also, I have zsh and something I had to modify to solve an error messed up something, because every time I start a console I see this question:
```
zsh compinit: insecure directories, run compaudit for list.
Ignore insecure directories and continue [y] or abort compinit [n]?
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #14733: Contributors quick start for Mac
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #14733:
URL: https://github.com/apache/airflow/issues/14733#issuecomment-803001151
BTW. We might want to add some more sophisticated mechanisms to warn the users. I already detect the CPUs and memory available for docker as part of the incoming PR for CI tests, i can add disk monitoring and raise a warning or even fail Breeze if I found those unaccepteably low. I created #14899 and will implement it shortly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] marcosmarxm commented on issue #14733: Contributors quick start for Mac
Posted by GitBox <gi...@apache.org>.
marcosmarxm commented on issue #14733:
URL: https://github.com/apache/airflow/issues/14733#issuecomment-797752236
@JavierLopezT take a look at #14231
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #14733: Contributors quick start for Mac
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #14733:
URL: https://github.com/apache/airflow/issues/14733#issuecomment-810966949
It looks like you had some other software mess around with your setup.
Unfortunately there is not much can be done if your local environment is broken, other thay googling and check how to fix your environment. That's really something you have to take a bit of control/responsibilty yourself when you install other software and allow it to make some changes, there is literally nothing we can do to prevent such changes.
In this case actually googling the answer is the right thing to do, because there is no way we could foresee all kind of problems can be introduced by messing with your environment. This is all in your hands, unfortunately. As developer who is not only a user of Mac you have the powers to break things but also responsibility to fix them.
For example the compinit problem has apparently an easy solution (just googled it) https://stackoverflow.com/questions/13762280/zsh-compinit-insecure-directories (but I have no guarantee it will work for you, maybe you need to dig a bit deeper).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Visya commented on issue #14733: Contributors quick start for Mac
Posted by GitBox <gi...@apache.org>.
Visya commented on issue #14733:
URL: https://github.com/apache/airflow/issues/14733#issuecomment-802610174
Thanks! Pruning and increasing the memory worked.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org