You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/05/30 10:42:06 UTC
[GitHub] [airflow] ephraimbuddy opened a new pull request #9070: Shorten command help and move long help to command description
ephraimbuddy opened a new pull request #9070:
URL: https://github.com/apache/airflow/pull/9070
---
Closes: #8447
Make sure to mark the boxes below before creating PR: [x]
- [x] Description above provides context of the change
- [x] Unit tests coverage for changes (not needed for documentation changes)
- [x] Target Github ISSUE in description if exists
- [x] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)"
- [x] Relevant documentation is updated including usage instructions.
- [x] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
---
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on a change in pull request #9070: Shorten command help and move long help to command description
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #9070:
URL: https://github.com/apache/airflow/pull/9070#discussion_r432888705
##########
File path: airflow/cli/cli_parser.py
##########
@@ -1326,8 +1347,11 @@ def _add_command(
subparsers: argparse._SubParsersAction, # pylint: disable=protected-access
sub: CLICommand
) -> None:
+ description = sub.description
+ if not description:
+ description = sub.help
sub_proc = subparsers.add_parser(
- sub.name, help=sub.help
+ sub.name, help=sub.help, description=description,
Review comment:
```suggestion
sub.name, help=sub.help, description=sub.description or self.help,
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj edited a comment on pull request #9070: Shorten command help and move long help to command description
Posted by GitBox <gi...@apache.org>.
mik-laj edited a comment on pull request #9070:
URL: https://github.com/apache/airflow/pull/9070#issuecomment-636314422
You could also split the description of the `airflow rotate_fernet_key` and `airflow celery` command. Links should be in extended help.
```
usage: airflow [-h]
{celery,config,connections,dags,db,info,kerberos,pools,roles,rotate_fernet_key,scheduler,sync_perm,tasks,users,variables,version,webserver}
...
positional arguments:
Groups:
celery Start celery components. Works only when using
CeleryExecutor. For more information, see https://airf
low.readthedocs.io/en/stable/executor/celery.html
connections List/Add/Delete connections
dags List and manage DAGs
db Database operations
pools CRUD operations on pools
roles Create/List roles
tasks List and manage tasks
users CRUD operations on users
variables CRUD operations on variables
Commands:
config Show current application configuration
info Show information about current Airflow and environment
kerberos Start a kerberos ticket renewer
rotate_fernet_key Rotate all encrypted connection credentials and
variables; see
https://airflow.readthedocs.io/en/stable/howto/secure-
connections.html#rotating-encryption-keys
scheduler Start a scheduler instance
sync_perm Update permissions for existing roles and DAGs
version Show the version
webserver Start a Airflow webserver instance
optional arguments:
-h, --help show this help message and exit
airflow command error: the following arguments are required: subcommand, see help above.
```
```
usage: airflow [-h]
{celery,config,connections,dags,db,info,kerberos,pools,roles,rotate_fernet_key,scheduler,sync_perm,tasks,users,variables,version,webserver}
...
positional arguments:
Groups:
celery [DISABLED] Start celery components.
connections List/Add/Delete connections
dags List and manage DAGs
db Database operations
pools CRUD operations on pools
roles Create/List roles
tasks List and manage tasks
users CRUD operations on users
variables CRUD operations on variables
Commands:
config Show current application configuration
info Show information about current Airflow and environment
kerberos Start a kerberos ticket renewer
rotate_fernet_key Rotate encrypted connection credentials and variables
scheduler Start a scheduler instance
sync_perm Update permissions for existing roles and DAGs
version Show the version
webserver Start a Airflow webserver instance
optional arguments:
-h, --help show this help message and exit
airflow command error: the following arguments are required: subcommand, see help above.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ephraimbuddy commented on pull request #9070: Shorten command help and move long help to command description
Posted by GitBox <gi...@apache.org>.
ephraimbuddy commented on pull request #9070:
URL: https://github.com/apache/airflow/pull/9070#issuecomment-636326235
How about the --subdir argument? Should I work on it?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on pull request #9070: Shorten command help and move long help to command description
Posted by GitBox <gi...@apache.org>.
mik-laj commented on pull request #9070:
URL: https://github.com/apache/airflow/pull/9070#issuecomment-636369265
> How about the --subdir argument? Should I work on it?
It's not perfect, but I have no idea.
Could you look at `airflow dags test --help` command? The argument descriptions are too long.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on a change in pull request #9070: Shorten command help and move long help to command description
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #9070:
URL: https://github.com/apache/airflow/pull/9070#discussion_r432888933
##########
File path: airflow/cli/cli_parser.py
##########
@@ -742,13 +722,15 @@ class ActionCommand(NamedTuple):
help: str
func: Callable
args: Iterable[Arg]
+ description: str = ''
Review comment:
```suggestion
description: Optional[st]r = None
```
This makes mypy more aware of the types. Then it can suggest more.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on pull request #9070: Shorten command help and move long help to command description
Posted by GitBox <gi...@apache.org>.
mik-laj commented on pull request #9070:
URL: https://github.com/apache/airflow/pull/9070#issuecomment-636313866
I think we should also improve the description of some arguments.
For example:
```
airflow dags show --help
```
Before
```
usage: airflow dags show [-h] [--imgcat] [-s SAVE] [-S SUBDIR] dag_id
positional arguments:
dag_id The id of the dag
optional arguments:
-h, --help show this help message and exit
--imgcat Displays graph using the imgcat tool.
For more information, see: https://www.iterm2.com/documentation-images.html
-s SAVE, --save SAVE Saves the result to the indicated file.
The file format is determined by the file extension. For more information about supported format, see: https://www.graphviz.org/doc/info/output.html
If you want to create a PNG file then you should execute the following command:
airflow dags show <DAG_ID> --save output.png
If you want to create a DOT file then you should execute the following command:
airflow dags show <DAG_ID> --save output.dot
-S SUBDIR, --subdir SUBDIR
File location or directory from which to look for the dag. Defaults to '[AIRFLOW_HOME]/dags' where [AIRFLOW_HOME] is the value you set for 'AIRFLOW_HOME' config you set in 'airflow.cfg'
```
My proposition:
```
usage: airflow dags show [-h] [--imgcat] [-s SAVE] [-S SUBDIR] dag_id
The --imgcat option only works in iTerm.
For more information, see: https://www.iterm2.com/documentation-images.html
Saving the graph
The file format is determined by the file extension. For more information about supported format, see: https://www.graphviz.org/doc/info/output.html
If you want to create a PNG file then you should execute the following command:
airflow dags show <DAG_ID> --save output.png
If you want to create a DOT file then you should execute the following command:
airflow dags show <DAG_ID> --save output.dot
positional arguments:
dag_id The id of the dag
optional arguments:
-h, --help show this help message and exit
--imgcat Displays graph using the imgcat tool.
-s SAVE, --save SAVE Saves the result to the indicated file.
-S SUBDIR, --subdir SUBDIR
File location or directory from which to look for the dag. Defaults to '[AIRFLOW_HOME]/dags' where [AIRFLOW_HOME] is the value you set for 'AIRFLOW_HOME' config you set in 'airflow.cfg'
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #9070: Shorten command help and move long help to command description
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #9070:
URL: https://github.com/apache/airflow/pull/9070#issuecomment-636321374
Agree in general @mik-laj But should not we do it in separate PRs to avoid one BIG PR with many many changes ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on pull request #9070: Shorten command help and move long help to command description
Posted by GitBox <gi...@apache.org>.
mik-laj commented on pull request #9070:
URL: https://github.com/apache/airflow/pull/9070#issuecomment-636314422
You could also split the description of the `airflow rotate_fernet_key` and `airflow celery` command. Links should be in extended help.
```usage: airflow [-h]
{celery,config,connections,dags,db,info,kerberos,pools,roles,rotate_fernet_key,scheduler,sync_perm,tasks,users,variables,version,webserver}
...
positional arguments:
Groups:
celery Start celery components. Works only when using
CeleryExecutor. For more information, see https://airf
low.readthedocs.io/en/stable/executor/celery.html
connections List/Add/Delete connections
dags List and manage DAGs
db Database operations
pools CRUD operations on pools
roles Create/List roles
tasks List and manage tasks
users CRUD operations on users
variables CRUD operations on variables
Commands:
config Show current application configuration
info Show information about current Airflow and environment
kerberos Start a kerberos ticket renewer
rotate_fernet_key Rotate all encrypted connection credentials and
variables; see
https://airflow.readthedocs.io/en/stable/howto/secure-
connections.html#rotating-encryption-keys
scheduler Start a scheduler instance
sync_perm Update permissions for existing roles and DAGs
version Show the version
webserver Start a Airflow webserver instance
optional arguments:
-h, --help show this help message and exit
airflow command error: the following arguments are required: subcommand, see help above.
```
```
usage: airflow [-h]
{celery,config,connections,dags,db,info,kerberos,pools,roles,rotate_fernet_key,scheduler,sync_perm,tasks,users,variables,version,webserver}
...
positional arguments:
Groups:
celery [DISABLED] Start celery components.
connections List/Add/Delete connections
dags List and manage DAGs
db Database operations
pools CRUD operations on pools
roles Create/List roles
tasks List and manage tasks
users CRUD operations on users
variables CRUD operations on variables
Commands:
config Show current application configuration
info Show information about current Airflow and environment
kerberos Start a kerberos ticket renewer
rotate_fernet_key Rotate encrypted connection credentials and variables
scheduler Start a scheduler instance
sync_perm Update permissions for existing roles and DAGs
version Show the version
webserver Start a Airflow webserver instance
optional arguments:
-h, --help show this help message and exit
airflow command error: the following arguments are required: subcommand, see help above.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj edited a comment on pull request #9070: Shorten command help and move long help to command description
Posted by GitBox <gi...@apache.org>.
mik-laj edited a comment on pull request #9070:
URL: https://github.com/apache/airflow/pull/9070#issuecomment-636313866
I think we should also improve the description of some arguments.
For example:
```
airflow dags show --help
```
Before
```
usage: airflow dags show [-h] [--imgcat] [-s SAVE] [-S SUBDIR] dag_id
positional arguments:
dag_id The id of the dag
optional arguments:
-h, --help show this help message and exit
--imgcat Displays graph using the imgcat tool.
For more information, see: https://www.iterm2.com/documentation-images.html
-s SAVE, --save SAVE Saves the result to the indicated file.
The file format is determined by the file extension. For more information about supported format, see: https://www.graphviz.org/doc/info/output.html
If you want to create a PNG file then you should execute the following command:
airflow dags show <DAG_ID> --save output.png
If you want to create a DOT file then you should execute the following command:
airflow dags show <DAG_ID> --save output.dot
-S SUBDIR, --subdir SUBDIR
File location or directory from which to look for the dag. Defaults to '[AIRFLOW_HOME]/dags' where [AIRFLOW_HOME] is the value you set for 'AIRFLOW_HOME' config you set in 'airflow.cfg'
```
My proposition:
```
usage: airflow dags show [-h] [--imgcat] [-s SAVE] [-S SUBDIR] dag_id
The --imgcat option only works in iTerm.
For more information, see: https://www.iterm2.com/documentation-images.html
Saving the graph
The file format is determined by the file extension. For more information about supported format, see: https://www.graphviz.org/doc/info/output.html
If you want to create a PNG file then you should execute the following command:
airflow dags show <DAG_ID> --save output.png
If you want to create a DOT file then you should execute the following command:
airflow dags show <DAG_ID> --save output.dot
positional arguments:
dag_id The id of the dag
optional arguments:
-h, --help show this help message and exit
--imgcat Displays graph using the imgcat tool.
-s SAVE, --save SAVE Saves the result to the indicated file.
-S SUBDIR, --subdir SUBDIR
File location or directory from which to look for the dag. Defaults to '[AIRFLOW_HOME]/dags' where [AIRFLOW_HOME] is the value you set for 'AIRFLOW_HOME' config you set in 'airflow.cfg'
``
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #9070: Shorten command help and move long help to command description
Posted by GitBox <gi...@apache.org>.
potiuk merged pull request #9070:
URL: https://github.com/apache/airflow/pull/9070
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on a change in pull request #9070: Shorten command help and move long help to command description
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #9070:
URL: https://github.com/apache/airflow/pull/9070#discussion_r432888964
##########
File path: airflow/cli/cli_parser.py
##########
@@ -742,13 +722,15 @@ class ActionCommand(NamedTuple):
help: str
func: Callable
args: Iterable[Arg]
+ description: str = ''
class GroupCommand(NamedTuple):
"""ClI command with subcommands"""
name: str
help: str
subcommands: Iterable
+ description: str = ''
Review comment:
```suggestion
description: Optional[str] = None
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on pull request #9070: Shorten command help and move long help to command description
Posted by GitBox <gi...@apache.org>.
mik-laj commented on pull request #9070:
URL: https://github.com/apache/airflow/pull/9070#issuecomment-636315116
I think it is also worth adding the default value for the epilogue based on the current description.
```
airflow db shell --help
```
Before
```
usage: airflow db shell [-h]
optional arguments:
-h, --help show this help message and exit
```
After:
```
usage: airflow db shell [-h]
Runs a shell to access the database
optional arguments:
-h, --help show this help message and exit
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on a change in pull request #9070: Shorten command help and move long help to command description
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #9070:
URL: https://github.com/apache/airflow/pull/9070#discussion_r432888705
##########
File path: airflow/cli/cli_parser.py
##########
@@ -1326,8 +1347,11 @@ def _add_command(
subparsers: argparse._SubParsersAction, # pylint: disable=protected-access
sub: CLICommand
) -> None:
+ description = sub.description
+ if not description:
+ description = sub.help
sub_proc = subparsers.add_parser(
- sub.name, help=sub.help
+ sub.name, help=sub.help, description=description,
Review comment:
```suggestion
sub.name, help=sub.help, description=sub.description or sub.help,
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on a change in pull request #9070: Shorten command help and move long help to command description
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #9070:
URL: https://github.com/apache/airflow/pull/9070#discussion_r432888705
##########
File path: airflow/cli/cli_parser.py
##########
@@ -1326,8 +1347,11 @@ def _add_command(
subparsers: argparse._SubParsersAction, # pylint: disable=protected-access
sub: CLICommand
) -> None:
+ description = sub.description
+ if not description:
+ description = sub.help
sub_proc = subparsers.add_parser(
- sub.name, help=sub.help
+ sub.name, help=sub.help, description=description,
Review comment:
```suggestion
sub.name, help=sub.help, description=sub.description or sub.help,
```
It's much shorter.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on a change in pull request #9070: Shorten command help and move long help to command description
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #9070:
URL: https://github.com/apache/airflow/pull/9070#discussion_r432874518
##########
File path: airflow/cli/cli_parser.py
##########
@@ -1226,7 +1235,8 @@ class GroupCommand(NamedTuple):
ActionCommand(
name='rotate_fernet_key',
func=lazy_load_command('airflow.cli.commands.rotate_fernet_key_command.rotate_fernet_key'),
- help=(
+ help='Rotate all encrypted connection credentials and variables',
Review comment:
```suggestion
help='Rotate encrypted connection credentials and variables',
```
Before:
<img width="741" alt="Screenshot 2020-05-30 at 19 59 40" src="https://user-images.githubusercontent.com/12058428/83335906-36780300-a2b0-11ea-9a97-dd0b8ba00c08.png">
After:
<img width="879" alt="Screenshot 2020-05-30 at 19 59 25" src="https://user-images.githubusercontent.com/12058428/83335903-3415a900-a2b0-11ea-9190-1328404ea31d.png">
This is a small thing, but it will cause sadness to many people.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on pull request #9070: Shorten command help and move long help to command description
Posted by GitBox <gi...@apache.org>.
mik-laj commented on pull request #9070:
URL: https://github.com/apache/airflow/pull/9070#issuecomment-636325975
This change only one file. I doubt that this change could drastically increase its size. On the other hand, it will allow us to standardize texts. One big PR will facilitate communication and allow us to avoid merge conflicts.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org