You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/05/13 16:47:35 UTC
[GitHub] [airflow] andrewgodwin opened a new pull request #15826: Add Airflow Standalone command
andrewgodwin opened a new pull request #15826:
URL: https://github.com/apache/airflow/pull/15826
Runs all parts of an airflow deployment under one main process, providing an easier level of entry than Breeze and a very handy tool for local development.
Handles the following:
* Runs all database migrations/db init steps
* Creates an admin user if one is not present (with a randomised password)
* Runs the webserver
* Runs the scheduler
* Overrides the executor to be `LocalExecutor` or `SequentialExecutor` depending on the database in use
It will also be able to run the `triggerer` process from AIP-40 with a three-line change once that PR lands.
This diff also updates the quickstart guide to use this rather than the separate commands, since it makes the set of commands significantly shorter.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on pull request #15826: Add Airflow Standalone command
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on pull request #15826:
URL: https://github.com/apache/airflow/pull/15826#issuecomment-841039451
> I think we should just keep the "standalone" as "Quick Start" for Users-only not for devs. And ask Users to use `./breeze` for development/testing (Contributors)
Fine for me :).
Though I think it might be worth mentioning this one as a "quick" way to test changes locally. It might be super-useful during development of some core changes I think. It's not even a "third option" really - it's more mentioning this command in the "local development environment" and actually even in Breeze :). I am already thinking about the contributor's workshop for the summit and adding a "standalone" command to the workshop would be really great.
There are a few places in our docs (Quick contributors start etc.) that could benefit from explicit mentioning of that command.
Might be worth adding it after #14176 change from @paolaperaza
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #15826: Add Airflow Standalone command
Posted by GitBox <gi...@apache.org>.
kaxil commented on pull request #15826:
URL: https://github.com/apache/airflow/pull/15826#issuecomment-840765037
> > Ah, perfect! It wasn't super clear to me when I was new quite what Breeze was for, so glad we can now have a definitive Thing to point people towards when they just want a quick setup.
>
> Yeah. BTW for testing DAGs/Running airflow itself we also have https://airflow.apache.org/docs/apache-airflow/stable/start/docker.html - this is mostly targeted at the users and it does not require airflow sources or package (just download docker-compose file and run it the image)
>
> I think the "standalone" airflow could be (?) used for both developers (if they add something) or users (but this require some more setup from the users I guess). I am thinking about the Docs and where to add them (except the usual command line docs), and I think it's good add it as both a "quick run airflow" option for development and a "quick run" for the users?:
>
> * https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#development-environments - for developers
> * https://airflow.apache.org/docs/apache-airflow/stable/start/ - for users
>
> WDYT?
I think we should just keep the "standalone" as "Quick Start" for Users-only not for devs. And ask Users to use `./breeze` for development/testing (Contributors)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] andrewgodwin commented on pull request #15826: Add Airflow Standalone command
Posted by GitBox <gi...@apache.org>.
andrewgodwin commented on pull request #15826:
URL: https://github.com/apache/airflow/pull/15826#issuecomment-840756238
> So I think there is almost no overlap with Breeze :D
Ah, perfect! It wasn't super clear to me when I was new quite what Breeze was for, so glad we can now have a definitive Thing to point people towards when they just want a quick setup.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil merged pull request #15826: Add Airflow Standalone command
Posted by GitBox <gi...@apache.org>.
kaxil merged pull request #15826:
URL: https://github.com/apache/airflow/pull/15826
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #15826: Add Airflow Standalone command
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #15826:
URL: https://github.com/apache/airflow/pull/15826#issuecomment-840726430
Nice one! Really like where it goes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #15826: Add Airflow Standalone command
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #15826:
URL: https://github.com/apache/airflow/pull/15826#issuecomment-842576257
🎉 . Going to use it in the upcoming workshops @turbaszek FYI
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #15826: Add Airflow Standalone command
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #15826:
URL: https://github.com/apache/airflow/pull/15826#issuecomment-841039451
> I think we should just keep the "standalone" as "Quick Start" for Users-only not for devs. And ask Users to use `./breeze` for development/testing (Contributors)
Fine for me :).
Though I think it might be worth mentioning this one as a "quick" way to test changes locally. It might be super-useful during development of some core changes I think. It's not even a "third option" really - it's more mentioning the "local development environment" and actually even in Breeze :). I am already thinking about the contributor's workshop for the summit and adding a "standalone" command to the workshop would be really great.
There are a few places in our docs (Quick contributors start etc.) that could benefit from explicit mentioning of that command.
Might be worth adding it after #14176 change from @paolaperaza
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on pull request #15826: Add Airflow Standalone command
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on pull request #15826:
URL: https://github.com/apache/airflow/pull/15826#issuecomment-840728879
BTW. Breeze has never been a tool for running airflow (the `start-airflow` command is a recent addition). It's really an environment to run all tests (both unit and integration tests) for airflow development in the same way/environment as they are in CI so that you can locally reproduce CI failures, And the `start-airflow` command is mostly there to verify if a released airflow version works when installed from packages. So I think there is almost no overlap with Breeze :D
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #15826: Add Airflow Standalone command
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #15826:
URL: https://github.com/apache/airflow/pull/15826#issuecomment-840728879
BTW. Breeze has never been a tool for running airflow (the `start-airflow` command is a recent addition). It's really an environment to run all tests for airflow development in the same way as they are in CI, And the `start-airflow` command is mostly there to verify if a released airflow version works when installed from packages. So I think there is almost no overlap with Breeze :D
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #15826: Add Airflow Standalone command
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #15826:
URL: https://github.com/apache/airflow/pull/15826#issuecomment-840761975
> Ah, perfect! It wasn't super clear to me when I was new quite what Breeze was for, so glad we can now have a definitive Thing to point people towards when they just want a quick setup.
Yeah. BTW for testing DAGs/Running airflow itself we also have https://airflow.apache.org/docs/apache-airflow/stable/start/docker.html - this is mostly targeted at the users and it does not require airflow sources or package (just download docker-compose file and run it the image)
I think the "standalone" airflow could be (?) used for both developers (if they add something) or users (but this require some more setup from the users I guess). I am thinking about the Docs and where to add them (except the usual command line docs), and I think it's good add it as both a "quick run airflow" option for development and a "quick run" for the users?:
* https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#development-environments - for developers
* https://airflow.apache.org/docs/apache-airflow/stable/start/ - for users
WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #15826: Add Airflow Standalone command
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #15826:
URL: https://github.com/apache/airflow/pull/15826#issuecomment-840764080
The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest master or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on pull request #15826: Add Airflow Standalone command
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on pull request #15826:
URL: https://github.com/apache/airflow/pull/15826#issuecomment-840761975
> Ah, perfect! It wasn't super clear to me when I was new quite what Breeze was for, so glad we can now have a definitive Thing to point people towards when they just want a quick setup.
Yeah. BTW for testing DAGs/Running airflow itself we also have https://airflow.apache.org/docs/apache-airflow/stable/start/docker.html - this is mostly targeted at the users and it does not require airflow sources or package (just download docker-compose file and run it)
I think the "standalone" airflow could be (?) used for both developers (if they add something) or users (but this require some more setup from the users I guess). I am thinking about the Docs and where to add them (except the usual command line docs), and I think it's good add it as both a "quick run airflow" option for development and a "quick run" for the users?:
* https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#development-environments - for developers
* https://airflow.apache.org/docs/apache-airflow/stable/start/ - for users
WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org