You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/07/13 00:04:47 UTC
[GitHub] [airflow] jedcunningham opened a new pull request, #25011: Add dataset events to dataset api
jedcunningham opened a new pull request, #25011:
URL: https://github.com/apache/airflow/pull/25011
Add api endpoint to get events for a dataset.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] bbovenzi commented on a diff in pull request #25011: Add dataset events to dataset api
Posted by GitBox <gi...@apache.org>.
bbovenzi commented on code in PR #25011:
URL: https://github.com/apache/airflow/pull/25011#discussion_r920537668
##########
airflow/api_connexion/endpoints/dataset_endpoint.py:
##########
@@ -59,3 +61,28 @@ def get_datasets(
query = apply_sorting(query, order_by, {}, allowed_filter_attrs)
datasets = query.offset(offset).limit(limit).all()
return dataset_collection_schema.dump(DatasetCollection(datasets=datasets, total_entries=total_entries))
+
+
+@security.requires_access([(permissions.ACTION_CAN_READ, permissions.RESOURCE_DATASET)])
+@provide_session
+@format_parameters({'limit': check_limit})
+def get_dataset_events(
+ id, *, limit: int, offset: int = 0, order_by: str = "created_at", session: Session = NEW_SESSION
+):
+ """Get events for a Dataset"""
+ dataset = session.query(Dataset).get(id)
+ if not dataset:
+ raise NotFound(
+ "Dataset not found",
+ detail=f"The Dataset with id: `{id}` was not found",
+ )
+
+ allowed_attrs = ['task_id', 'dag_id', 'run_id', 'map_index', 'created_at']
Review Comment:
yeah, this is fine for now. I just wanted to bring it up for later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] bbovenzi commented on a diff in pull request #25011: Add dataset events to dataset api
Posted by GitBox <gi...@apache.org>.
bbovenzi commented on code in PR #25011:
URL: https://github.com/apache/airflow/pull/25011#discussion_r920501373
##########
airflow/api_connexion/endpoints/dataset_endpoint.py:
##########
@@ -59,3 +61,28 @@ def get_datasets(
query = apply_sorting(query, order_by, {}, allowed_filter_attrs)
datasets = query.offset(offset).limit(limit).all()
return dataset_collection_schema.dump(DatasetCollection(datasets=datasets, total_entries=total_entries))
+
+
+@security.requires_access([(permissions.ACTION_CAN_READ, permissions.RESOURCE_DATASET)])
+@provide_session
+@format_parameters({'limit': check_limit})
+def get_dataset_events(
+ id, *, limit: int, offset: int = 0, order_by: str = "created_at", session: Session = NEW_SESSION
+):
+ """Get events for a Dataset"""
+ dataset = session.query(Dataset).get(id)
+ if not dataset:
+ raise NotFound(
+ "Dataset not found",
+ detail=f"The Dataset with id: `{id}` was not found",
+ )
+
+ allowed_attrs = ['task_id', 'dag_id', 'run_id', 'map_index', 'created_at']
Review Comment:
should we allow sorting by `map_index` alone? Practically, I feel like really this would at least combine with the task_id, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jedcunningham commented on pull request #25011: Add dataset events to dataset api
Posted by GitBox <gi...@apache.org>.
jedcunningham commented on PR #25011:
URL: https://github.com/apache/airflow/pull/25011#issuecomment-1184900099
Closing in favor of #25039.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] bbovenzi commented on a diff in pull request #25011: Add dataset events to dataset api
Posted by GitBox <gi...@apache.org>.
bbovenzi commented on code in PR #25011:
URL: https://github.com/apache/airflow/pull/25011#discussion_r920500554
##########
airflow/api_connexion/endpoints/dataset_endpoint.py:
##########
@@ -59,3 +61,28 @@ def get_datasets(
query = apply_sorting(query, order_by, {}, allowed_filter_attrs)
datasets = query.offset(offset).limit(limit).all()
return dataset_collection_schema.dump(DatasetCollection(datasets=datasets, total_entries=total_entries))
+
+
+@security.requires_access([(permissions.ACTION_CAN_READ, permissions.RESOURCE_DATASET)])
+@provide_session
+@format_parameters({'limit': check_limit})
+def get_dataset_events(
+ id, *, limit: int, offset: int = 0, order_by: str = "created_at", session: Session = NEW_SESSION
Review Comment:
instead of just ordering, it may be useful to be able to filter by a dag_id
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jedcunningham closed pull request #25011: Add dataset events to dataset api
Posted by GitBox <gi...@apache.org>.
jedcunningham closed pull request #25011: Add dataset events to dataset api
URL: https://github.com/apache/airflow/pull/25011
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jedcunningham commented on a diff in pull request #25011: Add dataset events to dataset api
Posted by GitBox <gi...@apache.org>.
jedcunningham commented on code in PR #25011:
URL: https://github.com/apache/airflow/pull/25011#discussion_r920536028
##########
airflow/api_connexion/endpoints/dataset_endpoint.py:
##########
@@ -59,3 +61,28 @@ def get_datasets(
query = apply_sorting(query, order_by, {}, allowed_filter_attrs)
datasets = query.offset(offset).limit(limit).all()
return dataset_collection_schema.dump(DatasetCollection(datasets=datasets, total_entries=total_entries))
+
+
+@security.requires_access([(permissions.ACTION_CAN_READ, permissions.RESOURCE_DATASET)])
+@provide_session
+@format_parameters({'limit': check_limit})
+def get_dataset_events(
+ id, *, limit: int, offset: int = 0, order_by: str = "created_at", session: Session = NEW_SESSION
+):
+ """Get events for a Dataset"""
+ dataset = session.query(Dataset).get(id)
+ if not dataset:
+ raise NotFound(
+ "Dataset not found",
+ detail=f"The Dataset with id: `{id}` was not found",
+ )
+
+ allowed_attrs = ['task_id', 'dag_id', 'run_id', 'map_index', 'created_at']
Review Comment:
I agree, generally, though we probably need to allow multiple order_by's and I don't want to intermix that here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org