You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "pierrejeambrun (via GitHub)" <gi...@apache.org> on 2025/01/24 16:42:10 UTC
[PR] AIP-84 Add patch task_instance dry_run endpoint [airflow]
pierrejeambrun opened a new pull request, #46018:
URL: https://github.com/apache/airflow/pull/46018
## What it does
Similarly to the `POST backfills` in dry_run mode. This adds a dry_run mode endpoint for the patch task instances and patch mapped task instances. Removed the `dry_run` param of the default patch TI and mapped TI endpoint, because it was only affecting the `state`.
## Why
While working on the front-end to mark task instances as success or failed, I realized that there was not way to see affected task instances for a specific set state with settings (past / future / etc...). We could also just improve the existing patch endpoint to fully support a dry_run mode, but for the same reasons as we did for backfill, different endpoint is just clearer. (Same endpoint would have different return type based on the `dry_run` value. `dry_run: False` would return a TI, but `dry_run: True` would return a list of affected tasks. That's confusing on the client side, and force manual casting or overloading).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] AIP-84 Add patch task_instance dry_run endpoint [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun merged PR #46018:
URL: https://github.com/apache/airflow/pull/46018
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] AIP-84 Add patch task_instance dry_run endpoint [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on code in PR #46018:
URL: https://github.com/apache/airflow/pull/46018#discussion_r1930252933
##########
airflow/api_fastapi/core_api/routes/public/task_instances.py:
##########
@@ -717,34 +705,123 @@ def patch_task_instance(
fields_to_update = body.model_fields_set
if update_mask:
fields_to_update = fields_to_update.intersection(update_mask)
- data = body.model_dump(include=fields_to_update, by_alias=True)
else:
try:
PatchTaskInstanceBody.model_validate(body)
except ValidationError as e:
raise RequestValidationError(errors=e.errors())
- data = body.model_dump(by_alias=True)
+
+ return dag, ti, body.model_dump(include=fields_to_update, by_alias=True)
Review Comment:
PR here https://github.com/apache/airflow/pull/46118
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] AIP-84 Add patch task_instance dry_run endpoint [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on code in PR #46018:
URL: https://github.com/apache/airflow/pull/46018#discussion_r1931888041
##########
airflow/ui/openapi-gen/requests/types.gen.ts:
##########
@@ -2159,6 +2158,28 @@ export type PostClearTaskInstancesData = {
export type PostClearTaskInstancesResponse = TaskInstanceCollectionResponse;
+export type PatchTaskInstanceDryRunData = {
+ dagId: string;
+ dagRunId: string;
+ mapIndex: number;
+ requestBody: PatchTaskInstanceBody;
+ taskId: string;
+ updateMask?: Array<string> | null;
+};
+
+export type PatchTaskInstanceDryRunResponse = TaskInstanceCollectionResponse;
+
+export type PatchTaskInstanceDryRun1Data = {
Review Comment:
That's because we have two endpoints, the one for classic task instance, and the one for 'mapped task' instance. For mapped task instance `map_index` is required and in the path parameter.
For non mapped TI, `map_index` is optional in the query parameters and defaulting to -1.
So the API is very flexible and we can call:
- PATH dag/<dag_id>/dagRuns/<dag_run/tasks/<task_id>/dry_run (map_index defaults to -1)
- PATH dag/<dag_id>/dagRuns/<dag_run/tasks/<task_id>/dry_run?map_index=<map_index> (map_index is optional, defaulting to -1, explicitely provided as a query parameter)
- PATH dag/<dag_id>/dagRuns/<dag_run/tasks/<task_id>/<map_index>/dry_run (map_index is required as a path parameter) (that form is to comply with how legacy map_index are passed usually, for instance to get tries, or setNote, dependencies, etc...)
Also both `@router.patch` are registered on 1 function (1 controller), so it end ups conflicting in names and we endup with `PatchTaskInstanceDryRun1Data` and `PatchTaskInstanceDryRunData` in the front-end. But we will most likely only use one of the two forms.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] AIP-84 Add patch task_instance dry_run endpoint [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on code in PR #46018:
URL: https://github.com/apache/airflow/pull/46018#discussion_r1929026181
##########
airflow/api_fastapi/core_api/routes/public/task_instances.py:
##########
@@ -717,34 +705,123 @@ def patch_task_instance(
fields_to_update = body.model_fields_set
if update_mask:
fields_to_update = fields_to_update.intersection(update_mask)
- data = body.model_dump(include=fields_to_update, by_alias=True)
else:
try:
PatchTaskInstanceBody.model_validate(body)
except ValidationError as e:
raise RequestValidationError(errors=e.errors())
- data = body.model_dump(by_alias=True)
+
+ return dag, ti, body.model_dump(include=fields_to_update, by_alias=True)
Review Comment:
Also @ajitg25
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] AIP-84 Add patch task_instance dry_run endpoint [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on code in PR #46018:
URL: https://github.com/apache/airflow/pull/46018#discussion_r1931888041
##########
airflow/ui/openapi-gen/requests/types.gen.ts:
##########
@@ -2159,6 +2158,28 @@ export type PostClearTaskInstancesData = {
export type PostClearTaskInstancesResponse = TaskInstanceCollectionResponse;
+export type PatchTaskInstanceDryRunData = {
+ dagId: string;
+ dagRunId: string;
+ mapIndex: number;
+ requestBody: PatchTaskInstanceBody;
+ taskId: string;
+ updateMask?: Array<string> | null;
+};
+
+export type PatchTaskInstanceDryRunResponse = TaskInstanceCollectionResponse;
+
+export type PatchTaskInstanceDryRun1Data = {
Review Comment:
That's because we have two endpoints on the same 'controller', the one for classic task instance, and the one for 'maped task' instance. For mapped task instance `map_index` is required and in the path parameter.
For non mapped TI, `map_index` is optional in the query parameters and defaulting to -1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] AIP-84 Add patch task_instance dry_run endpoint [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on PR #46018:
URL: https://github.com/apache/airflow/pull/46018#issuecomment-2615109566
cc: @atul-astronomer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] AIP-84 Add patch task_instance dry_run endpoint [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on code in PR #46018:
URL: https://github.com/apache/airflow/pull/46018#discussion_r1928975110
##########
airflow/api_fastapi/core_api/routes/public/task_instances.py:
##########
@@ -717,34 +705,123 @@ def patch_task_instance(
fields_to_update = body.model_fields_set
if update_mask:
fields_to_update = fields_to_update.intersection(update_mask)
- data = body.model_dump(include=fields_to_update, by_alias=True)
else:
try:
PatchTaskInstanceBody.model_validate(body)
except ValidationError as e:
raise RequestValidationError(errors=e.errors())
- data = body.model_dump(by_alias=True)
+
+ return dag, ti, body.model_dump(include=fields_to_update, by_alias=True)
Review Comment:
I think we should always only reqly on `fields_to_update`, i.e `body.model_fields_set` field actually set by the user.
For instance It feels weird when sending `{"note": "something"}` to have `state` filled in as None, and then setting the state as None, while this does not look the intend of the user. Problem appears when patching an instance with a partial payload but without specifying the `update_mask`, maybe we should error in this case. @rawwar what do you think, should I update other `patch` as well to use `fields_to_update` to avoid this edge case ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] AIP-84 Add patch task_instance dry_run endpoint [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on code in PR #46018:
URL: https://github.com/apache/airflow/pull/46018#discussion_r1931888041
##########
airflow/ui/openapi-gen/requests/types.gen.ts:
##########
@@ -2159,6 +2158,28 @@ export type PostClearTaskInstancesData = {
export type PostClearTaskInstancesResponse = TaskInstanceCollectionResponse;
+export type PatchTaskInstanceDryRunData = {
+ dagId: string;
+ dagRunId: string;
+ mapIndex: number;
+ requestBody: PatchTaskInstanceBody;
+ taskId: string;
+ updateMask?: Array<string> | null;
+};
+
+export type PatchTaskInstanceDryRunResponse = TaskInstanceCollectionResponse;
+
+export type PatchTaskInstanceDryRun1Data = {
Review Comment:
That's because we have two endpoints, the one for classic task instance, and the one for 'mapped task' instance. For mapped task instance `map_index` is required and in the path parameter.
For non mapped TI, `map_index` is optional in the query parameters and defaulting to -1.
So the API is very flexible and we can call:
- PATH dag/<dag_id>/dagRuns/<dag_run/tasks/<task_id>/dry_run (map_index defaults to -1)
- PATH dag/<dag_id>/dagRuns/<dag_run/tasks/<task_id>/dry_run?map_index=<map_index> (map_index is optional, defaulting to -1, explicitely provided as a query parameter)
- PATH dag/<dag_id>/dagRuns/<dag_run/tasks/<task_id>/<map_index>/dry_run (map_index is required) (that form is to comply with how legacy map_index are passed usually, for instance to get tries, or setNote, dependencies, etc...)
Also both `@router.patch` are registered on 1 function (1 controller), so it end ups conflicting in names and we endup with `PatchTaskInstanceDryRun1Data` and `PatchTaskInstanceDryRunData` in the front-end. But we will most likely only use 1.
##########
airflow/ui/openapi-gen/requests/types.gen.ts:
##########
@@ -2159,6 +2158,28 @@ export type PostClearTaskInstancesData = {
export type PostClearTaskInstancesResponse = TaskInstanceCollectionResponse;
+export type PatchTaskInstanceDryRunData = {
+ dagId: string;
+ dagRunId: string;
+ mapIndex: number;
+ requestBody: PatchTaskInstanceBody;
+ taskId: string;
+ updateMask?: Array<string> | null;
+};
+
+export type PatchTaskInstanceDryRunResponse = TaskInstanceCollectionResponse;
+
+export type PatchTaskInstanceDryRun1Data = {
Review Comment:
That's because we have two endpoints, the one for classic task instance, and the one for 'mapped task' instance. For mapped task instance `map_index` is required and in the path parameter.
For non mapped TI, `map_index` is optional in the query parameters and defaulting to -1.
So the API is very flexible and we can call:
- PATH dag/<dag_id>/dagRuns/<dag_run/tasks/<task_id>/dry_run (map_index defaults to -1)
- PATH dag/<dag_id>/dagRuns/<dag_run/tasks/<task_id>/dry_run?map_index=<map_index> (map_index is optional, defaulting to -1, explicitely provided as a query parameter)
- PATH dag/<dag_id>/dagRuns/<dag_run/tasks/<task_id>/<map_index>/dry_run (map_index is required as a path parameter) (that form is to comply with how legacy map_index are passed usually, for instance to get tries, or setNote, dependencies, etc...)
Also both `@router.patch` are registered on 1 function (1 controller), so it end ups conflicting in names and we endup with `PatchTaskInstanceDryRun1Data` and `PatchTaskInstanceDryRunData` in the front-end. But we will most likely only use 1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] AIP-84 Add patch task_instance dry_run endpoint [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on code in PR #46018:
URL: https://github.com/apache/airflow/pull/46018#discussion_r1931888041
##########
airflow/ui/openapi-gen/requests/types.gen.ts:
##########
@@ -2159,6 +2158,28 @@ export type PostClearTaskInstancesData = {
export type PostClearTaskInstancesResponse = TaskInstanceCollectionResponse;
+export type PatchTaskInstanceDryRunData = {
+ dagId: string;
+ dagRunId: string;
+ mapIndex: number;
+ requestBody: PatchTaskInstanceBody;
+ taskId: string;
+ updateMask?: Array<string> | null;
+};
+
+export type PatchTaskInstanceDryRunResponse = TaskInstanceCollectionResponse;
+
+export type PatchTaskInstanceDryRun1Data = {
Review Comment:
That's because we have two endpoints, the one for classic task instance, and the one for 'mapped task' instance. For mapped task instance `map_index` is required and in the path parameter.
For non mapped TI, `map_index` is optional in the query parameters and defaulting to -1.
So the API is very flexible and we can call:
- PATH dag/<dag_id>/dagRuns/<dag_run/tasks/<task_id>/dry_run (map_index defaults to -1)
- PATH dag/<dag_id>/dagRuns/<dag_run/tasks/<task_id>/dry_run?map_index=<map_index> (map_index is explicitely provided as a query parameter)
- PATH dag/<dag_id>/dagRuns/<dag_run/tasks/<task_id>/<map_index>/dry_run (map_index is required) (that form is to comply with how legacy map_index are passed usually, for instance to get tries, or setNote, dependencies, etc...)
Also both `@router.patch` are registered on 1 function (1 controller), so it end ups conflicting in names and we endup with `PatchTaskInstanceDryRun1Data` and `PatchTaskInstanceDryRunData` in the front-end. But we will most likely only use 1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] AIP-84 Add patch task_instance dry_run endpoint [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on code in PR #46018:
URL: https://github.com/apache/airflow/pull/46018#discussion_r1931888041
##########
airflow/ui/openapi-gen/requests/types.gen.ts:
##########
@@ -2159,6 +2158,28 @@ export type PostClearTaskInstancesData = {
export type PostClearTaskInstancesResponse = TaskInstanceCollectionResponse;
+export type PatchTaskInstanceDryRunData = {
+ dagId: string;
+ dagRunId: string;
+ mapIndex: number;
+ requestBody: PatchTaskInstanceBody;
+ taskId: string;
+ updateMask?: Array<string> | null;
+};
+
+export type PatchTaskInstanceDryRunResponse = TaskInstanceCollectionResponse;
+
+export type PatchTaskInstanceDryRun1Data = {
Review Comment:
That's because we have two endpoints, the one for classic task instance, and the one for 'mapped task' instance. For mapped task instance `map_index` is required and in the path parameter.
For non mapped TI, `map_index` is optional in the query parameters and defaulting to -1.
So the API is very flexible and we can call:
- PATH dag/<dag_id>/dagRuns/<dag_run/tasks/<task_id>/dry_run (map_index defaults to -1)
- PATH dag/<dag_id>/dagRuns/<dag_run/tasks/<task_id>/dry_run?map_index=<map_index> (map_index is explicitely provided as a query parameter)
- PATH dag/<dag_id>/dagRuns/<dag_run/tasks/<task_id>/<map_index>/dry_run (map_index is required) (that form is to comply with how legacy map_index are passed.
Also both `@router.patch` are registered on 1 function (1 controller), so it end ups conflicting in names and we endup with `PatchTaskInstanceDryRun1Data` and `PatchTaskInstanceDryRunData` in the front-end. But we will most likely only use 1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] AIP-84 Add patch task_instance dry_run endpoint [airflow]
Posted by "bbovenzi (via GitHub)" <gi...@apache.org>.
bbovenzi commented on PR #46018:
URL: https://github.com/apache/airflow/pull/46018#issuecomment-2612977840
Good catch! I forgot that we weren't using the public rest api for this in the old UI.
What if we made this a GET endpoint? Since its a dry_run, we're not actually changing anything and then it saves us the time of building a custom query wrapper like I just did for clearing in https://github.com/apache/airflow/pull/45987.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] AIP-84 Add patch task_instance dry_run endpoint [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on code in PR #46018:
URL: https://github.com/apache/airflow/pull/46018#discussion_r1931888041
##########
airflow/ui/openapi-gen/requests/types.gen.ts:
##########
@@ -2159,6 +2158,28 @@ export type PostClearTaskInstancesData = {
export type PostClearTaskInstancesResponse = TaskInstanceCollectionResponse;
+export type PatchTaskInstanceDryRunData = {
+ dagId: string;
+ dagRunId: string;
+ mapIndex: number;
+ requestBody: PatchTaskInstanceBody;
+ taskId: string;
+ updateMask?: Array<string> | null;
+};
+
+export type PatchTaskInstanceDryRunResponse = TaskInstanceCollectionResponse;
+
+export type PatchTaskInstanceDryRun1Data = {
Review Comment:
That's because we have two endpoints, the one for classic task instance, and the one for 'mapped task' instance. For mapped task instance `map_index` is required and in the path parameter.
For non mapped TI, `map_index` is optional in the query parameters and defaulting to -1.
So we can call either:
- PATH dag/<dag_id>/dagRuns/<dag_run/tasks/<task_id>/dry_run (map_index defaults to -1)
- PATH dag/<dag_id>/dagRuns/<dag_run/tasks/<task_id>/<map_index>/dry_run?map_index=<map_index> (map_index is explicitely provided)
- PATH dag/<dag_id>/dagRuns/<dag_run/tasks/<task_id>/<map_index>/dry_run (map_index is required) (that form is to comply with how legacy map_index are passed.
Also both `@router.patch` are registered on 1 function (1 controller), so it end ups conflicting in names and we endup with `PatchTaskInstanceDryRun1Data` and `PatchTaskInstanceDryRunData` in the front-end. But we will most likely only use 1 of the both.
##########
airflow/ui/openapi-gen/requests/types.gen.ts:
##########
@@ -2159,6 +2158,28 @@ export type PostClearTaskInstancesData = {
export type PostClearTaskInstancesResponse = TaskInstanceCollectionResponse;
+export type PatchTaskInstanceDryRunData = {
+ dagId: string;
+ dagRunId: string;
+ mapIndex: number;
+ requestBody: PatchTaskInstanceBody;
+ taskId: string;
+ updateMask?: Array<string> | null;
+};
+
+export type PatchTaskInstanceDryRunResponse = TaskInstanceCollectionResponse;
+
+export type PatchTaskInstanceDryRun1Data = {
Review Comment:
That's because we have two endpoints, the one for classic task instance, and the one for 'mapped task' instance. For mapped task instance `map_index` is required and in the path parameter.
For non mapped TI, `map_index` is optional in the query parameters and defaulting to -1.
So we can call either:
- PATH dag/<dag_id>/dagRuns/<dag_run/tasks/<task_id>/dry_run (map_index defaults to -1)
- PATH dag/<dag_id>/dagRuns/<dag_run/tasks/<task_id>/<map_index>/dry_run?map_index=<map_index> (map_index is explicitely provided)
- PATH dag/<dag_id>/dagRuns/<dag_run/tasks/<task_id>/<map_index>/dry_run (map_index is required) (that form is to comply with how legacy map_index are passed.
Also both `@router.patch` are registered on 1 function (1 controller), so it end ups conflicting in names and we endup with `PatchTaskInstanceDryRun1Data` and `PatchTaskInstanceDryRunData` in the front-end. But we will most likely only use 1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] AIP-84 Add patch task_instance dry_run endpoint [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on PR #46018:
URL: https://github.com/apache/airflow/pull/46018#issuecomment-2612998968
> Good catch! I forgot that we weren't using the public rest api for this in the old UI.
>What if we made this a GET endpoint? Since its a dry_run, we're not actually changing anything and then it saves us the time of building a custom query wrapper like I just did for clearing in https://github.com/apache/airflow/pull/45987.
We could to that, and indeed that will save some time on the front-end part. Transforming the current body into query parameters I suppose ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] AIP-84 Add patch task_instance dry_run endpoint [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on PR #46018:
URL: https://github.com/apache/airflow/pull/46018#issuecomment-2613100643
> What if we made this a GET endpoint? Since its a dry_run, we're not actually changing anything and then it saves us the time of building a custom query wrapper like I just did for clearing in https://github.com/apache/airflow/pull/45987.
We could do that to save an extra custom query wrapper in the front-end. We also have the create backfill endpoint that currently implements a `dry_run` version as well. If we are to put `dry_run` versions of endpoints on `GET` method, we most likely need to rework that create backfill dry_run endpoint to comply with that. Also we need to convert payloads used for the non dry_run version of POST / PATCH into query parameters and perform the same validation most likely instantiating back those data models. I think the work we save on the front-end is balanced out by the extra work we need on the backend. (though all of that are minor efforts)
Also that's just me but as a user I wouldn't expect the method to change from a dry_run call to a non dry_run one.
I would lean towards keeping the same method when implementing a dry_run version of an endpoint. Considering the elements above, let me know if that makes sense 😄
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] AIP-84 Add patch task_instance dry_run endpoint [airflow]
Posted by "bbovenzi (via GitHub)" <gi...@apache.org>.
bbovenzi commented on code in PR #46018:
URL: https://github.com/apache/airflow/pull/46018#discussion_r1930905267
##########
airflow/ui/openapi-gen/requests/types.gen.ts:
##########
@@ -2159,6 +2158,28 @@ export type PostClearTaskInstancesData = {
export type PostClearTaskInstancesResponse = TaskInstanceCollectionResponse;
+export type PatchTaskInstanceDryRunData = {
+ dagId: string;
+ dagRunId: string;
+ mapIndex: number;
+ requestBody: PatchTaskInstanceBody;
+ taskId: string;
+ updateMask?: Array<string> | null;
+};
+
+export type PatchTaskInstanceDryRunResponse = TaskInstanceCollectionResponse;
+
+export type PatchTaskInstanceDryRun1Data = {
Review Comment:
Why does this type exist?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] AIP-84 Add patch task_instance dry_run endpoint [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on code in PR #46018:
URL: https://github.com/apache/airflow/pull/46018#discussion_r1931888041
##########
airflow/ui/openapi-gen/requests/types.gen.ts:
##########
@@ -2159,6 +2158,28 @@ export type PostClearTaskInstancesData = {
export type PostClearTaskInstancesResponse = TaskInstanceCollectionResponse;
+export type PatchTaskInstanceDryRunData = {
+ dagId: string;
+ dagRunId: string;
+ mapIndex: number;
+ requestBody: PatchTaskInstanceBody;
+ taskId: string;
+ updateMask?: Array<string> | null;
+};
+
+export type PatchTaskInstanceDryRunResponse = TaskInstanceCollectionResponse;
+
+export type PatchTaskInstanceDryRun1Data = {
Review Comment:
That's because we have two endpoints, the one for classic task instance, and the one for 'mapped task' instance. For mapped task instance `map_index` is required and in the path parameter.
For non mapped TI, `map_index` is optional in the query parameters and defaulting to -1.
So the API is very flexible and we can call:
- PATH dag/<dag_id>/dagRuns/<dag_run/tasks/<task_id>/dry_run (map_index defaults to -1)
- PATH dag/<dag_id>/dagRuns/<dag_run/tasks/<task_id>/<map_index>/dry_run?map_index=<map_index> (map_index is explicitely provided)
- PATH dag/<dag_id>/dagRuns/<dag_run/tasks/<task_id>/<map_index>/dry_run (map_index is required) (that form is to comply with how legacy map_index are passed.
Also both `@router.patch` are registered on 1 function (1 controller), so it end ups conflicting in names and we endup with `PatchTaskInstanceDryRun1Data` and `PatchTaskInstanceDryRunData` in the front-end. But we will most likely only use 1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] AIP-84 Add patch task_instance dry_run endpoint [airflow]
Posted by "rawwar (via GitHub)" <gi...@apache.org>.
rawwar commented on code in PR #46018:
URL: https://github.com/apache/airflow/pull/46018#discussion_r1929662602
##########
airflow/api_fastapi/core_api/routes/public/task_instances.py:
##########
@@ -717,34 +705,123 @@ def patch_task_instance(
fields_to_update = body.model_fields_set
if update_mask:
fields_to_update = fields_to_update.intersection(update_mask)
- data = body.model_dump(include=fields_to_update, by_alias=True)
else:
try:
PatchTaskInstanceBody.model_validate(body)
except ValidationError as e:
raise RequestValidationError(errors=e.errors())
- data = body.model_dump(by_alias=True)
+
+ return dag, ti, body.model_dump(include=fields_to_update, by_alias=True)
Review Comment:
Yeah. +1 to use `fields_to_update`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org