You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "ephraimbuddy (via GitHub)" <gi...@apache.org> on 2023/02/02 12:24:55 UTC
[GitHub] [airflow] ephraimbuddy opened a new pull request, #29309: Add `airflow db drop-archived` command
ephraimbuddy opened a new pull request, #29309:
URL: https://github.com/apache/airflow/pull/29309
This command drops the archive tables directly
As part of this, the _confirm_drop_archives function was made more interactive
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ephraimbuddy commented on pull request #29309: Add `airflow db drop-archived` command
Posted by "ephraimbuddy (via GitHub)" <gi...@apache.org>.
ephraimbuddy commented on PR #29309:
URL: https://github.com/apache/airflow/pull/29309#issuecomment-1423270546
> > Overall looks good. I don't like that we use "cleaned" for one command and "archived" for another, we should pick one term and use it in both places. "cleaned" might be better, as it matches "db clean"?
>
> On the other hand, they can refer to different things. Clean is what we do to the table. Archive is what we do with the data. We cleaned the table, by archiving the data. And we can purge the archived data by dropping the tables, optionally exporting them, etc.
Yes. We also have `--skip-archived` as part of `db clean`. That was my thought in naming it archived
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dstandish commented on pull request #29309: Add `airflow db drop-archived` command
Posted by "dstandish (via GitHub)" <gi...@apache.org>.
dstandish commented on PR #29309:
URL: https://github.com/apache/airflow/pull/29309#issuecomment-1423216750
> Overall looks good. I don't like that we use "cleaned" for one command and "archived" for another, we should pick one term and use it in both places. "cleaned" might be better, as it matches "db clean"?
On the other hand, they can refer to different things. Clean is what we do to the table. Archive is what we do with the data. We cleaned the table, by archiving the data. And we can purge the archived data by dropping the tables, optionally exporting them, etc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ephraimbuddy commented on pull request #29309: Add `airflow db drop-archived` command
Posted by "ephraimbuddy (via GitHub)" <gi...@apache.org>.
ephraimbuddy commented on PR #29309:
URL: https://github.com/apache/airflow/pull/29309#issuecomment-1414029554
> (He repeated himself. :)
Haha :) 🤦‍♂️
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] norm commented on pull request #29309: Add `airflow db drop-archived` command
Posted by "norm (via GitHub)" <gi...@apache.org>.
norm commented on PR #29309:
URL: https://github.com/apache/airflow/pull/29309#issuecomment-1413846077
Think docs/apache-airflow/howto/usage-cli.rst needs to be updated too.
(He repeated himself. :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] norm commented on a diff in pull request #29309: Add `airflow db drop-archived` command
Posted by "norm (via GitHub)" <gi...@apache.org>.
norm commented on code in PR #29309:
URL: https://github.com/apache/airflow/pull/29309#discussion_r1094533045
##########
airflow/cli/commands/db_command.py:
##########
@@ -218,3 +218,12 @@ def export_cleaned(args):
table_names=args.tables,
drop_archives=args.drop_archives,
)
+
+
+@cli_utils.action_cli(check_db=False)
+def drop_archived(args):
+ """Drops archived tables from metadata database."""
+ drop_archived_tables(
+ table_names=args.tables,
+ confirm=not args.yes,
Review Comment:
Can you rename this to something like `needs_confirm` or `manual_confirm` — because in my head using `-y` means you have confirmed it and it just reads … wrong.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jedcunningham commented on a diff in pull request #29309: Add `airflow db drop-archived` command
Posted by "jedcunningham (via GitHub)" <gi...@apache.org>.
jedcunningham commented on code in PR #29309:
URL: https://github.com/apache/airflow/pull/29309#discussion_r1101637903
##########
docs/apache-airflow/howto/usage-cli.rst:
##########
@@ -232,6 +232,18 @@ location must exist.
Other options include: ``--tables`` to specify the tables to export, ``--drop-archives`` to drop the archive tables after
exporting.
+Dropping the archived tables
+----------------------------
+If during the ``db clean`` process, you did not use the ``--skip-archive`` option which drops the archived table, you can
+still drop the archive tables using the ``db drop-archived`` command. This operation is irreversible and you are encouraged
+to use ``db export-cleaned`` command to backup the tables to disk before dropping them.
+
+You can specify the tables to drop using the ``--tables`` option. If no tables are specified, all archive tables will be
+dropped.
+
+For help on the ``db drop-archived`` command, run: ``airflow db drop-archived --help``.
+
+
Review Comment:
```suggestion
```
I don't think we should add this, `--help` is a thing for every command.
##########
docs/apache-airflow/howto/usage-cli.rst:
##########
@@ -232,6 +232,18 @@ location must exist.
Other options include: ``--tables`` to specify the tables to export, ``--drop-archives`` to drop the archive tables after
exporting.
+Dropping the archived tables
+----------------------------
Review Comment:
```suggestion
----------------------------
```
##########
docs/apache-airflow/howto/usage-cli.rst:
##########
@@ -232,6 +232,18 @@ location must exist.
Other options include: ``--tables`` to specify the tables to export, ``--drop-archives`` to drop the archive tables after
exporting.
+Dropping the archived tables
+----------------------------
+If during the ``db clean`` process, you did not use the ``--skip-archive`` option which drops the archived table, you can
+still drop the archive tables using the ``db drop-archived`` command. This operation is irreversible and you are encouraged
+to use ``db export-cleaned`` command to backup the tables to disk before dropping them.
Review Comment:
```suggestion
to use the ``db export-cleaned`` command to backup the tables to disk before dropping them.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ephraimbuddy merged pull request #29309: Add `airflow db drop-archived` command
Posted by "ephraimbuddy (via GitHub)" <gi...@apache.org>.
ephraimbuddy merged PR #29309:
URL: https://github.com/apache/airflow/pull/29309
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org