You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/02/17 18:04:28 UTC
[GitHub] [airflow] ephraimbuddy opened a new pull request #14280: Add plugins endpoint to the REST API
ephraimbuddy opened a new pull request #14280:
URL: https://github.com/apache/airflow/pull/14280
This PR seeks to add `/plugins` endpoint. The endpoint exposes the plugins loaded to the webserver
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ephraimbuddy commented on pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
ephraimbuddy commented on pull request #14280:
URL: https://github.com/apache/airflow/pull/14280#issuecomment-784878811
Hi @kaxil, @turbaszek, @mik-laj please re-review this as I have added pagination to the endpoint
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #14280:
URL: https://github.com/apache/airflow/pull/14280#discussion_r578537897
##########
File path: airflow/api_connexion/openapi/v1.yaml
##########
@@ -1380,7 +1380,99 @@ paths:
application/json:
schema:
$ref: '#/components/schemas/VersionInfo'
-
+ /plugins:
+ get:
+ summary: Get a list of loaded plugins
+ x-openapi-router-controller: airflow.api_connexion.endpoints.plugin_endpoint
+ operationId: get_plugins
+ tags: [Plugin]
+ responses:
+ '200':
+ description: Success
+ content:
+ application/json:
+ schema:
+ type: object
+ properties:
+ number:
+ type: string
+ description: The plugin number
+ name:
+ type: string
+ description: The name of the plugin
+ attrs:
+ type: object
+ properties:
+ hooks:
+ type: array
+ items:
+ type: string
+ nullable: true
+ description: The plugin hooks
+ executors:
+ type: array
+ items:
+ type: string
+ nullable: true
+ description: The plugin executors
+ macros:
+ type: array
+ items:
+ type: string
+ nullable: true
+ description: The plugin macros
+ admin_views:
+ type: array
+ items:
+ type: string
+ nullable: true
+ description: The plugins admin views
+ flask_blueprints:
+ type: array
+ items:
+ type: string
+ nullable: true
+ description: The flask blueprints
+ menu_links:
+ type: array
+ items:
+ type: string
+ nullable: true
+ description: The menu links
+ appbuilder_views:
+ type: array
+ items:
+ type: string
+ nullable: true
+ description: The appuilder views
+ appbuilder_menu_items:
+ type: array
+ items:
+ type: string
+ nullable: true
+ description: The appbuilder menu items
Review comment:
```suggestion
description: The Flask Appbuilder menu items
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #14280:
URL: https://github.com/apache/airflow/pull/14280#discussion_r578537336
##########
File path: airflow/api_connexion/endpoints/plugin_endpoint.py
##########
@@ -0,0 +1,65 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from airflow import plugins_manager
+from airflow.api_connexion import security
+from airflow.api_connexion.exceptions import NotFound
+from airflow.api_connexion.schemas.plugin_schema import PluginCollection, plugin_collection_schema
+from airflow.security import permissions
+
+
+@security.requires_access(
+ [
+ (permissions.ACTION_CAN_READ, permissions.RESOURCE_PLUGIN),
+ (permissions.ACTION_CAN_ACCESS_MENU, permissions.RESOURCE_PLUGIN),
+ ]
+)
+def get_plugins():
+ """Get plugins endpoint"""
+ plugins_manager.ensure_plugins_loaded()
+ plugins_manager.integrate_executor_plugins()
+ plugins_manager.initialize_extra_operators_links_plugins()
+ plugins_manager.initialize_web_ui_plugins()
+ plugins_attributes_to_dump = [
+ "hooks",
+ "executors",
+ "macros",
+ "admin_views",
+ "flask_blueprints",
+ "menu_links",
+ "appbuilder_views",
+ "appbuilder_menu_items",
+ "global_operator_extra_links",
+ "operator_extra_links",
+ "source",
+ ]
+
+ plugins = []
+ for plugin_no, plugin in enumerate(plugins_manager.plugins, 1):
+ plugin_data = {
+ 'number': plugin_no,
+ 'name': plugin.name,
+ 'attrs': {},
+ }
+ for attr_name in plugins_attributes_to_dump:
+ attr_value = getattr(plugin, attr_name)
+ plugin_data['attrs'][attr_name] = attr_value
+
+ plugins.append(plugin_data)
Review comment:
+1
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #14280:
URL: https://github.com/apache/airflow/pull/14280#issuecomment-781316047
[The Workflow run](https://github.com/apache/airflow/actions/runs/578127275) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ephraimbuddy commented on a change in pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
ephraimbuddy commented on a change in pull request #14280:
URL: https://github.com/apache/airflow/pull/14280#discussion_r581040478
##########
File path: airflow/api_connexion/endpoints/plugin_endpoint.py
##########
@@ -0,0 +1,33 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from airflow.api_connexion import security
+from airflow.api_connexion.exceptions import NotFound
+from airflow.api_connexion.schemas.plugin_schema import PluginCollection, plugin_collection_schema
+from airflow.plugins_manager import get_plugin_info
+from airflow.security import permissions
+
+
+@security.requires_access([(permissions.ACTION_CAN_READ, permissions.RESOURCE_PLUGIN)])
+def get_plugins():
+ """Get plugins endpoint"""
+ plugins_info = get_plugin_info()
+ if plugins_info:
+ return plugin_collection_schema.dump(
+ PluginCollection(plugins=plugins_info, total_entries=len(plugins_info))
+ )
+ raise NotFound(detail="Plugin not found")
Review comment:
I'm also thinking if we should have `limit` and `offset` on this endpoint? That's pagination? @kaxil @mik-laj
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ephraimbuddy commented on a change in pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
ephraimbuddy commented on a change in pull request #14280:
URL: https://github.com/apache/airflow/pull/14280#discussion_r581040478
##########
File path: airflow/api_connexion/endpoints/plugin_endpoint.py
##########
@@ -0,0 +1,33 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from airflow.api_connexion import security
+from airflow.api_connexion.exceptions import NotFound
+from airflow.api_connexion.schemas.plugin_schema import PluginCollection, plugin_collection_schema
+from airflow.plugins_manager import get_plugin_info
+from airflow.security import permissions
+
+
+@security.requires_access([(permissions.ACTION_CAN_READ, permissions.RESOURCE_PLUGIN)])
+def get_plugins():
+ """Get plugins endpoint"""
+ plugins_info = get_plugin_info()
+ if plugins_info:
+ return plugin_collection_schema.dump(
+ PluginCollection(plugins=plugins_info, total_entries=len(plugins_info))
+ )
+ raise NotFound(detail="Plugin not found")
Review comment:
I'm also thinking if we should have `limit` and `offset` on this endpoint? That's pagination. @kaxil @mik-laj
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #14280:
URL: https://github.com/apache/airflow/pull/14280#discussion_r578537580
##########
File path: airflow/api_connexion/openapi/v1.yaml
##########
@@ -1380,7 +1380,99 @@ paths:
application/json:
schema:
$ref: '#/components/schemas/VersionInfo'
-
+ /plugins:
+ get:
+ summary: Get a list of loaded plugins
+ x-openapi-router-controller: airflow.api_connexion.endpoints.plugin_endpoint
+ operationId: get_plugins
+ tags: [Plugin]
+ responses:
+ '200':
+ description: Success
+ content:
+ application/json:
+ schema:
+ type: object
+ properties:
+ number:
+ type: string
+ description: The plugin number
+ name:
+ type: string
+ description: The name of the plugin
+ attrs:
+ type: object
+ properties:
+ hooks:
+ type: array
+ items:
+ type: string
+ nullable: true
+ description: The plugin hooks
+ executors:
+ type: array
+ items:
+ type: string
+ nullable: true
+ description: The plugin executors
+ macros:
+ type: array
+ items:
+ type: string
+ nullable: true
+ description: The plugin macros
+ admin_views:
+ type: array
+ items:
+ type: string
+ nullable: true
+ description: The plugins admin views
+ flask_blueprints:
+ type: array
+ items:
+ type: string
+ nullable: true
+ description: The flask blueprints
+ menu_links:
+ type: array
+ items:
+ type: string
+ nullable: true
+ description: The menu links
+ appbuilder_views:
+ type: array
+ items:
+ type: string
+ nullable: true
+ description: The appuilder views
+ appbuilder_menu_items:
+ type: array
+ items:
+ type: string
+ nullable: true
+ description: The appbuilder menu items
+ global_operator_extra_links:
+ type: array
+ items:
+ type: string
+ nullable: true
+ description: The global operator extra links
+ operator_extra_links:
+ type: array
+ items:
+ type: string
+ nullable: true
+ description: Operator extra link
Review comment:
```suggestion
description: Operator extra links
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ephraimbuddy commented on a change in pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
ephraimbuddy commented on a change in pull request #14280:
URL: https://github.com/apache/airflow/pull/14280#discussion_r579241318
##########
File path: airflow/api_connexion/endpoints/plugin_endpoint.py
##########
@@ -0,0 +1,45 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from airflow import plugins_manager
+from airflow.api_connexion import security
+from airflow.api_connexion.exceptions import NotFound
+from airflow.api_connexion.schemas.plugin_schema import PluginCollection, plugin_collection_schema
+from airflow.cli.commands.plugins_command import PLUGINS_ATTRIBUTES_TO_DUMP
+from airflow.security import permissions
+
+
+@security.requires_access(
+ [
+ (permissions.ACTION_CAN_READ, permissions.RESOURCE_PLUGIN),
+ (permissions.ACTION_CAN_ACCESS_MENU, permissions.RESOURCE_PLUGIN),
+ ]
+)
+def get_plugins():
+ """Get plugins endpoint"""
+ plugins_manager.ensure_plugins_loaded()
+ plugins_manager.integrate_macros_plugins()
+ plugins_manager.integrate_executor_plugins()
+ plugins_manager.initialize_extra_operators_links_plugins()
+ plugins_manager.initialize_web_ui_plugins()
+ plugins = []
+ for plugin in plugins_manager.plugins:
+ info = {"name": plugin.name}
+ info.update({n: getattr(plugin, n) for n in PLUGINS_ATTRIBUTES_TO_DUMP})
+ plugins.append(info)
Review comment:
Ok. Should I fix it with this PR?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ephraimbuddy commented on a change in pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
ephraimbuddy commented on a change in pull request #14280:
URL: https://github.com/apache/airflow/pull/14280#discussion_r578745245
##########
File path: airflow/api_connexion/endpoints/plugin_endpoint.py
##########
@@ -0,0 +1,45 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from airflow import plugins_manager
+from airflow.api_connexion import security
+from airflow.api_connexion.exceptions import NotFound
+from airflow.api_connexion.schemas.plugin_schema import PluginCollection, plugin_collection_schema
+from airflow.cli.commands.plugins_command import PLUGINS_ATTRIBUTES_TO_DUMP
+from airflow.security import permissions
+
+
+@security.requires_access(
+ [
+ (permissions.ACTION_CAN_READ, permissions.RESOURCE_PLUGIN),
+ (permissions.ACTION_CAN_ACCESS_MENU, permissions.RESOURCE_PLUGIN),
+ ]
+)
+def get_plugins():
+ """Get plugins endpoint"""
+ plugins_manager.ensure_plugins_loaded()
+ plugins_manager.integrate_macros_plugins()
+ plugins_manager.integrate_executor_plugins()
+ plugins_manager.initialize_extra_operators_links_plugins()
+ plugins_manager.initialize_web_ui_plugins()
+ plugins = []
+ for plugin in plugins_manager.plugins:
+ info = {"name": plugin.name}
+ info.update({n: getattr(plugin, n) for n in PLUGINS_ATTRIBUTES_TO_DUMP})
+ plugins.append(info)
Review comment:
Cool
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #14280:
URL: https://github.com/apache/airflow/pull/14280#issuecomment-783085211
[The Workflow run](https://github.com/apache/airflow/actions/runs/588018020) is cancelling this PR. Building images for the PR has failed. Follow the the workflow link to check the reason.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #14280:
URL: https://github.com/apache/airflow/pull/14280#issuecomment-782683215
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest master at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #14280:
URL: https://github.com/apache/airflow/pull/14280#issuecomment-784900049
[The Workflow run](https://github.com/apache/airflow/actions/runs/595395078) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] SamWheating commented on a change in pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
SamWheating commented on a change in pull request #14280:
URL: https://github.com/apache/airflow/pull/14280#discussion_r580672531
##########
File path: airflow/api_connexion/endpoints/plugin_endpoint.py
##########
@@ -0,0 +1,33 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from airflow.api_connexion import security
+from airflow.api_connexion.exceptions import NotFound
+from airflow.api_connexion.schemas.plugin_schema import PluginCollection, plugin_collection_schema
+from airflow.plugins_manager import get_plugin_info
+from airflow.security import permissions
+
+
+@security.requires_access([(permissions.ACTION_CAN_READ, permissions.RESOURCE_PLUGIN)])
+def get_plugins():
+ """Get plugins endpoint"""
+ plugins_info = get_plugin_info()
+ if plugins_info:
+ return plugin_collection_schema.dump(
+ PluginCollection(plugins=plugins_info, total_entries=len(plugins_info))
+ )
+ raise NotFound(detail="Plugin not found")
Review comment:
Should this return an empty (but successful) response rather than a 404 if there are no plugins? I believe this is the behaviour of other `list` endpoints.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ephraimbuddy commented on a change in pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
ephraimbuddy commented on a change in pull request #14280:
URL: https://github.com/apache/airflow/pull/14280#discussion_r580427032
##########
File path: airflow/api_connexion/openapi/v1.yaml
##########
@@ -132,6 +132,7 @@ info:
| Airflow version | Description |
|-|-|
| v2.0 | Initial release |
+ | v2.0.2 | Added /plugins endpoint |
Review comment:
Is this OK @turbaszek @mik-laj
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #14280:
URL: https://github.com/apache/airflow/pull/14280#discussion_r579448068
##########
File path: airflow/api_connexion/endpoints/plugin_endpoint.py
##########
@@ -0,0 +1,45 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from airflow import plugins_manager
+from airflow.api_connexion import security
+from airflow.api_connexion.exceptions import NotFound
+from airflow.api_connexion.schemas.plugin_schema import PluginCollection, plugin_collection_schema
+from airflow.cli.commands.plugins_command import PLUGINS_ATTRIBUTES_TO_DUMP
+from airflow.security import permissions
+
+
+@security.requires_access(
+ [
+ (permissions.ACTION_CAN_READ, permissions.RESOURCE_PLUGIN),
+ (permissions.ACTION_CAN_ACCESS_MENU, permissions.RESOURCE_PLUGIN),
+ ]
+)
+def get_plugins():
+ """Get plugins endpoint"""
+ plugins_manager.ensure_plugins_loaded()
+ plugins_manager.integrate_macros_plugins()
+ plugins_manager.integrate_executor_plugins()
+ plugins_manager.initialize_extra_operators_links_plugins()
+ plugins_manager.initialize_web_ui_plugins()
+ plugins = []
+ for plugin in plugins_manager.plugins:
+ info = {"name": plugin.name}
+ info.update({n: getattr(plugin, n) for n in PLUGINS_ATTRIBUTES_TO_DUMP})
+ plugins.append(info)
Review comment:
Fine with me too
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on a change in pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #14280:
URL: https://github.com/apache/airflow/pull/14280#discussion_r579862498
##########
File path: airflow/plugins_manager.py
##########
@@ -427,3 +427,39 @@ def integrate_macros_plugins() -> None:
# Register the newly created module on airflow.macros such that it
# can be accessed when rendering templates.
setattr(macros, plugin.name, macros_module)
+
+
+def get_plugin_info(attrs_to_dump: Optional[List[str]] = None):
+ """
+ Dump plugins attributes
+
+ :param attrs_to_dump: A list of plugin attributes to dump
+ :type attrs_to_dump: List
+ """
+ plugins_attributes_to_dump = [
+ "hooks",
+ "executors",
+ "macros",
+ "admin_views",
+ "flask_blueprints",
+ "menu_links",
+ "appbuilder_views",
+ "appbuilder_menu_items",
+ "global_operator_extra_links",
+ "operator_extra_links",
+ "source",
+ ]
Review comment:
when it is a set, the order of the elements is maintained, which might be expected when building the user interface.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on a change in pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #14280:
URL: https://github.com/apache/airflow/pull/14280#discussion_r581051324
##########
File path: airflow/api_connexion/endpoints/plugin_endpoint.py
##########
@@ -0,0 +1,33 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from airflow.api_connexion import security
+from airflow.api_connexion.exceptions import NotFound
+from airflow.api_connexion.schemas.plugin_schema import PluginCollection, plugin_collection_schema
+from airflow.plugins_manager import get_plugin_info
+from airflow.security import permissions
+
+
+@security.requires_access([(permissions.ACTION_CAN_READ, permissions.RESOURCE_PLUGIN)])
+def get_plugins():
+ """Get plugins endpoint"""
+ plugins_info = get_plugin_info()
+ if plugins_info:
+ return plugin_collection_schema.dump(
+ PluginCollection(plugins=plugins_info, total_entries=len(plugins_info))
+ )
+ raise NotFound(detail="Plugin not found")
Review comment:
Yes. Yes. we should add these parameters if it is possible to have a unified interface which will allow dynamically generating clients and applications based on Opec API spec.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
mik-laj commented on pull request #14280:
URL: https://github.com/apache/airflow/pull/14280#issuecomment-782922965
Should we update summary of changes in docs?
http://apache-airflow-docs.s3-website.eu-central-1.amazonaws.com/docs/apache-airflow/latest/stable-rest-api-ref.html#section/Overview/Versioning-and-Endpoint-Lifecycle
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #14280:
URL: https://github.com/apache/airflow/pull/14280#issuecomment-782161788
[The Workflow run](https://github.com/apache/airflow/actions/runs/581891361) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #14280:
URL: https://github.com/apache/airflow/pull/14280#discussion_r581081341
##########
File path: airflow/api_connexion/endpoints/plugin_endpoint.py
##########
@@ -0,0 +1,33 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from airflow.api_connexion import security
+from airflow.api_connexion.exceptions import NotFound
+from airflow.api_connexion.schemas.plugin_schema import PluginCollection, plugin_collection_schema
+from airflow.plugins_manager import get_plugin_info
+from airflow.security import permissions
+
+
+@security.requires_access([(permissions.ACTION_CAN_READ, permissions.RESOURCE_PLUGIN)])
+def get_plugins():
+ """Get plugins endpoint"""
+ plugins_info = get_plugin_info()
+ if plugins_info:
+ return plugin_collection_schema.dump(
+ PluginCollection(plugins=plugins_info, total_entries=len(plugins_info))
+ )
+ raise NotFound(detail="Plugin not found")
Review comment:
Yup, we should
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek commented on a change in pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
turbaszek commented on a change in pull request #14280:
URL: https://github.com/apache/airflow/pull/14280#discussion_r579239868
##########
File path: airflow/api_connexion/endpoints/plugin_endpoint.py
##########
@@ -0,0 +1,45 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from airflow import plugins_manager
+from airflow.api_connexion import security
+from airflow.api_connexion.exceptions import NotFound
+from airflow.api_connexion.schemas.plugin_schema import PluginCollection, plugin_collection_schema
+from airflow.cli.commands.plugins_command import PLUGINS_ATTRIBUTES_TO_DUMP
+from airflow.security import permissions
+
+
+@security.requires_access(
+ [
+ (permissions.ACTION_CAN_READ, permissions.RESOURCE_PLUGIN),
+ (permissions.ACTION_CAN_ACCESS_MENU, permissions.RESOURCE_PLUGIN),
+ ]
+)
+def get_plugins():
+ """Get plugins endpoint"""
+ plugins_manager.ensure_plugins_loaded()
+ plugins_manager.integrate_macros_plugins()
+ plugins_manager.integrate_executor_plugins()
+ plugins_manager.initialize_extra_operators_links_plugins()
+ plugins_manager.initialize_web_ui_plugins()
+ plugins = []
+ for plugin in plugins_manager.plugins:
+ info = {"name": plugin.name}
+ info.update({n: getattr(plugin, n) for n in PLUGINS_ATTRIBUTES_TO_DUMP})
+ plugins.append(info)
Review comment:
@ephraimbuddy I think it is possible that it was omitted
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ephraimbuddy commented on a change in pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
ephraimbuddy commented on a change in pull request #14280:
URL: https://github.com/apache/airflow/pull/14280#discussion_r578755022
##########
File path: airflow/api_connexion/endpoints/plugin_endpoint.py
##########
@@ -0,0 +1,45 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from airflow import plugins_manager
+from airflow.api_connexion import security
+from airflow.api_connexion.exceptions import NotFound
+from airflow.api_connexion.schemas.plugin_schema import PluginCollection, plugin_collection_schema
+from airflow.cli.commands.plugins_command import PLUGINS_ATTRIBUTES_TO_DUMP
+from airflow.security import permissions
+
+
+@security.requires_access(
+ [
+ (permissions.ACTION_CAN_READ, permissions.RESOURCE_PLUGIN),
+ (permissions.ACTION_CAN_ACCESS_MENU, permissions.RESOURCE_PLUGIN),
+ ]
+)
+def get_plugins():
+ """Get plugins endpoint"""
+ plugins_manager.ensure_plugins_loaded()
+ plugins_manager.integrate_macros_plugins()
+ plugins_manager.integrate_executor_plugins()
+ plugins_manager.initialize_extra_operators_links_plugins()
+ plugins_manager.initialize_web_ui_plugins()
+ plugins = []
+ for plugin in plugins_manager.plugins:
+ info = {"name": plugin.name}
+ info.update({n: getattr(plugin, n) for n in PLUGINS_ATTRIBUTES_TO_DUMP})
+ plugins.append(info)
Review comment:
What do you think about the attributes being dumped? It's different for the CLI. The cli does not include the `admin_views` and `menu_links`, while webserver views contain the two attributes. Could it be a mistake in the CLI?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek commented on a change in pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
turbaszek commented on a change in pull request #14280:
URL: https://github.com/apache/airflow/pull/14280#discussion_r578668266
##########
File path: airflow/api_connexion/endpoints/plugin_endpoint.py
##########
@@ -0,0 +1,45 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from airflow import plugins_manager
+from airflow.api_connexion import security
+from airflow.api_connexion.exceptions import NotFound
+from airflow.api_connexion.schemas.plugin_schema import PluginCollection, plugin_collection_schema
+from airflow.cli.commands.plugins_command import PLUGINS_ATTRIBUTES_TO_DUMP
+from airflow.security import permissions
+
+
+@security.requires_access(
+ [
+ (permissions.ACTION_CAN_READ, permissions.RESOURCE_PLUGIN),
+ (permissions.ACTION_CAN_ACCESS_MENU, permissions.RESOURCE_PLUGIN),
+ ]
+)
+def get_plugins():
+ """Get plugins endpoint"""
+ plugins_manager.ensure_plugins_loaded()
+ plugins_manager.integrate_macros_plugins()
+ plugins_manager.integrate_executor_plugins()
+ plugins_manager.initialize_extra_operators_links_plugins()
+ plugins_manager.initialize_web_ui_plugins()
+ plugins = []
+ for plugin in plugins_manager.plugins:
+ info = {"name": plugin.name}
+ info.update({n: getattr(plugin, n) for n in PLUGINS_ATTRIBUTES_TO_DUMP})
+ plugins.append(info)
Review comment:
I think we have already similar code, should we introduce some utility function to avoid code duplication?
https://github.com/apache/airflow/blob/8f21fb1bf77fc67e37dc13613778ff1e6fa87cea/airflow/cli/commands/plugins_command.py#L64-L68
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #14280:
URL: https://github.com/apache/airflow/pull/14280#discussion_r580983923
##########
File path: airflow/api_connexion/endpoints/plugin_endpoint.py
##########
@@ -0,0 +1,33 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from airflow.api_connexion import security
+from airflow.api_connexion.exceptions import NotFound
+from airflow.api_connexion.schemas.plugin_schema import PluginCollection, plugin_collection_schema
+from airflow.plugins_manager import get_plugin_info
+from airflow.security import permissions
+
+
+@security.requires_access([(permissions.ACTION_CAN_READ, permissions.RESOURCE_PLUGIN)])
+def get_plugins():
+ """Get plugins endpoint"""
+ plugins_info = get_plugin_info()
+ if plugins_info:
+ return plugin_collection_schema.dump(
+ PluginCollection(plugins=plugins_info, total_entries=len(plugins_info))
+ )
+ raise NotFound(detail="Plugin not found")
Review comment:
oh yeah, I think so -- we should just return an empty list as we don in the Webserver:
https://github.com/apache/airflow/blob/25fa3092323e0cd13b11bb8e809b88ff3c043e4e/airflow/www/templates/airflow/plugin.html#L28-L31
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek commented on a change in pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
turbaszek commented on a change in pull request #14280:
URL: https://github.com/apache/airflow/pull/14280#discussion_r579612349
##########
File path: airflow/plugins_manager.py
##########
@@ -427,3 +427,39 @@ def integrate_macros_plugins() -> None:
# Register the newly created module on airflow.macros such that it
# can be accessed when rendering templates.
setattr(macros, plugin.name, macros_module)
+
+
+def get_plugin_info(attrs_to_dump: Optional[List[str]] = None):
+ """
+ Dump plugins attributes
+
+ :param attrs_to_dump: A list of plugin attributes to dump
+ :type attrs_to_dump: List
+ """
+ plugins_attributes_to_dump = [
+ "hooks",
+ "executors",
+ "macros",
+ "admin_views",
+ "flask_blueprints",
+ "menu_links",
+ "appbuilder_views",
+ "appbuilder_menu_items",
+ "global_operator_extra_links",
+ "operator_extra_links",
+ "source",
+ ]
Review comment:
How about moving this value outside of the function as a constant? Then we can use it as default, WDYT? Also a nit, but should we change it to be a `set` instead of `list`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jhtimmins commented on a change in pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
jhtimmins commented on a change in pull request #14280:
URL: https://github.com/apache/airflow/pull/14280#discussion_r580391201
##########
File path: airflow/api_connexion/endpoints/plugin_endpoint.py
##########
@@ -0,0 +1,38 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from airflow.api_connexion import security
+from airflow.api_connexion.exceptions import NotFound
+from airflow.api_connexion.schemas.plugin_schema import PluginCollection, plugin_collection_schema
+from airflow.plugins_manager import get_plugin_info
+from airflow.security import permissions
+
+
+@security.requires_access(
+ [
+ (permissions.ACTION_CAN_READ, permissions.RESOURCE_PLUGIN),
+ (permissions.ACTION_CAN_ACCESS_MENU, permissions.RESOURCE_PLUGIN),
Review comment:
I dont think it makes sense to require menu access for the API endpoint, since the menu is exclusively a UI-related component.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ephraimbuddy merged pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
ephraimbuddy merged pull request #14280:
URL: https://github.com/apache/airflow/pull/14280
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek commented on a change in pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
turbaszek commented on a change in pull request #14280:
URL: https://github.com/apache/airflow/pull/14280#discussion_r579612197
##########
File path: airflow/plugins_manager.py
##########
@@ -427,3 +427,39 @@ def integrate_macros_plugins() -> None:
# Register the newly created module on airflow.macros such that it
# can be accessed when rendering templates.
setattr(macros, plugin.name, macros_module)
+
+
+def get_plugin_info(attrs_to_dump: Optional[List[str]] = None):
Review comment:
```suggestion
def get_plugin_info(attrs_to_dump: Optional[List[str]] = None) -> List[Dict]:
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ephraimbuddy commented on a change in pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
ephraimbuddy commented on a change in pull request #14280:
URL: https://github.com/apache/airflow/pull/14280#discussion_r578755022
##########
File path: airflow/api_connexion/endpoints/plugin_endpoint.py
##########
@@ -0,0 +1,45 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from airflow import plugins_manager
+from airflow.api_connexion import security
+from airflow.api_connexion.exceptions import NotFound
+from airflow.api_connexion.schemas.plugin_schema import PluginCollection, plugin_collection_schema
+from airflow.cli.commands.plugins_command import PLUGINS_ATTRIBUTES_TO_DUMP
+from airflow.security import permissions
+
+
+@security.requires_access(
+ [
+ (permissions.ACTION_CAN_READ, permissions.RESOURCE_PLUGIN),
+ (permissions.ACTION_CAN_ACCESS_MENU, permissions.RESOURCE_PLUGIN),
+ ]
+)
+def get_plugins():
+ """Get plugins endpoint"""
+ plugins_manager.ensure_plugins_loaded()
+ plugins_manager.integrate_macros_plugins()
+ plugins_manager.integrate_executor_plugins()
+ plugins_manager.initialize_extra_operators_links_plugins()
+ plugins_manager.initialize_web_ui_plugins()
+ plugins = []
+ for plugin in plugins_manager.plugins:
+ info = {"name": plugin.name}
+ info.update({n: getattr(plugin, n) for n in PLUGINS_ATTRIBUTES_TO_DUMP})
+ plugins.append(info)
Review comment:
What do you think about the attributes being dumped? It's different for the CLI. The cli does not include the `admin_views` and `menu_links`, while webserver view contains the two attributes. Could it be a mistake in the CLI?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #14280:
URL: https://github.com/apache/airflow/pull/14280#discussion_r580986170
##########
File path: airflow/api_connexion/openapi/v1.yaml
##########
@@ -1380,7 +1381,96 @@ paths:
application/json:
schema:
$ref: '#/components/schemas/VersionInfo'
-
+ /plugins:
+ get:
+ summary: Get a list of loaded plugins
+ x-openapi-router-controller: airflow.api_connexion.endpoints.plugin_endpoint
+ operationId: get_plugins
+ tags: [Plugin]
+ responses:
+ '200':
+ description: Success
+ content:
+ application/json:
+ schema:
+ type: object
+ properties:
+ number:
+ type: string
+ description: The plugin number
+ name:
+ type: string
+ description: The name of the plugin
+ hooks:
+ type: array
+ items:
+ type: string
+ nullable: true
+ description: The plugin hooks
+ executors:
+ type: array
+ items:
+ type: string
+ nullable: true
+ description: The plugin executors
+ macros:
+ type: array
+ items:
+ type: string
+ nullable: true
+ description: The plugin macros
+ admin_views:
+ type: array
+ items:
+ type: string
+ nullable: true
+ description: The plugins admin views
+ flask_blueprints:
+ type: array
+ items:
+ type: string
+ nullable: true
+ description: The flask blueprints
+ menu_links:
+ type: array
+ items:
+ type: string
+ nullable: true
+ description: The menu links
Review comment:
Or atleast add "deprecated" in the API docs
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ephraimbuddy commented on pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
ephraimbuddy commented on pull request #14280:
URL: https://github.com/apache/airflow/pull/14280#issuecomment-782931714
> Should we update summary of changes in docs?
> http://apache-airflow-docs.s3-website.eu-central-1.amazonaws.com/docs/apache-airflow/latest/stable-rest-api-ref.html#section/Overview/Versioning-and-Endpoint-Lifecycle
Sure. Will do that
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #14280:
URL: https://github.com/apache/airflow/pull/14280#discussion_r580985750
##########
File path: airflow/api_connexion/openapi/v1.yaml
##########
@@ -1380,7 +1381,96 @@ paths:
application/json:
schema:
$ref: '#/components/schemas/VersionInfo'
-
+ /plugins:
+ get:
+ summary: Get a list of loaded plugins
+ x-openapi-router-controller: airflow.api_connexion.endpoints.plugin_endpoint
+ operationId: get_plugins
+ tags: [Plugin]
+ responses:
+ '200':
+ description: Success
+ content:
+ application/json:
+ schema:
+ type: object
+ properties:
+ number:
+ type: string
+ description: The plugin number
+ name:
+ type: string
+ description: The name of the plugin
+ hooks:
+ type: array
+ items:
+ type: string
+ nullable: true
+ description: The plugin hooks
+ executors:
+ type: array
+ items:
+ type: string
+ nullable: true
+ description: The plugin executors
+ macros:
+ type: array
+ items:
+ type: string
+ nullable: true
+ description: The plugin macros
+ admin_views:
+ type: array
+ items:
+ type: string
+ nullable: true
+ description: The plugins admin views
+ flask_blueprints:
+ type: array
+ items:
+ type: string
+ nullable: true
+ description: The flask blueprints
+ menu_links:
+ type: array
+ items:
+ type: string
+ nullable: true
+ description: The menu links
Review comment:
Should we remove `menu_links` and `admin_views` -- they are for the old UI
https://github.com/apache/airflow/blob/25fa3092323e0cd13b11bb8e809b88ff3c043e4e/airflow/plugins_manager.py#L316-L328
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #14280:
URL: https://github.com/apache/airflow/pull/14280#issuecomment-781627151
[The Workflow run](https://github.com/apache/airflow/actions/runs/579402990) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek commented on a change in pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
turbaszek commented on a change in pull request #14280:
URL: https://github.com/apache/airflow/pull/14280#discussion_r578387036
##########
File path: airflow/api_connexion/endpoints/plugin_endpoint.py
##########
@@ -0,0 +1,65 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from airflow import plugins_manager
+from airflow.api_connexion import security
+from airflow.api_connexion.exceptions import NotFound
+from airflow.api_connexion.schemas.plugin_schema import PluginCollection, plugin_collection_schema
+from airflow.security import permissions
+
+
+@security.requires_access(
+ [
+ (permissions.ACTION_CAN_READ, permissions.RESOURCE_PLUGIN),
+ (permissions.ACTION_CAN_ACCESS_MENU, permissions.RESOURCE_PLUGIN),
+ ]
+)
+def get_plugins():
+ """Get plugins endpoint"""
+ plugins_manager.ensure_plugins_loaded()
+ plugins_manager.integrate_executor_plugins()
+ plugins_manager.initialize_extra_operators_links_plugins()
+ plugins_manager.initialize_web_ui_plugins()
+ plugins_attributes_to_dump = [
+ "hooks",
+ "executors",
+ "macros",
+ "admin_views",
+ "flask_blueprints",
+ "menu_links",
+ "appbuilder_views",
+ "appbuilder_menu_items",
+ "global_operator_extra_links",
+ "operator_extra_links",
+ "source",
+ ]
+
+ plugins = []
+ for plugin_no, plugin in enumerate(plugins_manager.plugins, 1):
+ plugin_data = {
+ 'number': plugin_no,
+ 'name': plugin.name,
+ 'attrs': {},
+ }
+ for attr_name in plugins_attributes_to_dump:
+ attr_value = getattr(plugin, attr_name)
+ plugin_data['attrs'][attr_name] = attr_value
+
+ plugins.append(plugin_data)
Review comment:
Should we try to unify this response with what users get using `airflow plugins` command using `json` output?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek commented on a change in pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
turbaszek commented on a change in pull request #14280:
URL: https://github.com/apache/airflow/pull/14280#discussion_r579340196
##########
File path: airflow/api_connexion/endpoints/plugin_endpoint.py
##########
@@ -0,0 +1,45 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from airflow import plugins_manager
+from airflow.api_connexion import security
+from airflow.api_connexion.exceptions import NotFound
+from airflow.api_connexion.schemas.plugin_schema import PluginCollection, plugin_collection_schema
+from airflow.cli.commands.plugins_command import PLUGINS_ATTRIBUTES_TO_DUMP
+from airflow.security import permissions
+
+
+@security.requires_access(
+ [
+ (permissions.ACTION_CAN_READ, permissions.RESOURCE_PLUGIN),
+ (permissions.ACTION_CAN_ACCESS_MENU, permissions.RESOURCE_PLUGIN),
+ ]
+)
+def get_plugins():
+ """Get plugins endpoint"""
+ plugins_manager.ensure_plugins_loaded()
+ plugins_manager.integrate_macros_plugins()
+ plugins_manager.integrate_executor_plugins()
+ plugins_manager.initialize_extra_operators_links_plugins()
+ plugins_manager.initialize_web_ui_plugins()
+ plugins = []
+ for plugin in plugins_manager.plugins:
+ info = {"name": plugin.name}
+ info.update({n: getattr(plugin, n) for n in PLUGINS_ATTRIBUTES_TO_DUMP})
+ plugins.append(info)
Review comment:
I'm ok with that, wdyt @kaxil ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ephraimbuddy commented on a change in pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
ephraimbuddy commented on a change in pull request #14280:
URL: https://github.com/apache/airflow/pull/14280#discussion_r581723816
##########
File path: airflow/api_connexion/openapi/v1.yaml
##########
@@ -1380,7 +1381,96 @@ paths:
application/json:
schema:
$ref: '#/components/schemas/VersionInfo'
-
+ /plugins:
+ get:
+ summary: Get a list of loaded plugins
+ x-openapi-router-controller: airflow.api_connexion.endpoints.plugin_endpoint
+ operationId: get_plugins
+ tags: [Plugin]
+ responses:
+ '200':
+ description: Success
+ content:
+ application/json:
+ schema:
+ type: object
+ properties:
+ number:
+ type: string
+ description: The plugin number
+ name:
+ type: string
+ description: The name of the plugin
+ hooks:
+ type: array
+ items:
+ type: string
+ nullable: true
+ description: The plugin hooks
+ executors:
+ type: array
+ items:
+ type: string
+ nullable: true
+ description: The plugin executors
+ macros:
+ type: array
+ items:
+ type: string
+ nullable: true
+ description: The plugin macros
+ admin_views:
+ type: array
+ items:
+ type: string
+ nullable: true
+ description: The plugins admin views
+ flask_blueprints:
+ type: array
+ items:
+ type: string
+ nullable: true
+ description: The flask blueprints
+ menu_links:
+ type: array
+ items:
+ type: string
+ nullable: true
+ description: The menu links
Review comment:
I removed it. I saw that they wasn't in plugins command and since this is new endpoint, letting it come without them looks good to me?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #14280: Add plugins endpoint to the REST API
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #14280:
URL: https://github.com/apache/airflow/pull/14280#issuecomment-785164418
[The Workflow run](https://github.com/apache/airflow/actions/runs/596486456) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org