[GitHub] [airflow] chinwobble commented on a change in pull request #19736: Add Databricks Deferrable Operators

[GitBox <gi...@apache.org>]

chinwobble commented on a change in pull request #19736:
URL: https://github.com/apache/airflow/pull/19736#discussion_r753877719



##########
File path: airflow/providers/databricks/hooks/databricks.py
##########
@@ -493,3 +504,120 @@ def __init__(self, token: str) -> None:
     def __call__(self, r: PreparedRequest) -> PreparedRequest:
         r.headers['Authorization'] = 'Bearer ' + self.token
         return r
+
+
+class DatabricksAsyncHook(DatabricksHook):
+    """
+    Async version of the ``DatabricksHook``
+    Implements only necessary methods used further in Databricks Triggers.
+    """
+
+    def __init__(self, *args: Any, **kwargs: Any) -> None:
+        super().__init__(*args, **kwargs)
+
+    async def __aenter__(self):
+        self._session = aiohttp.ClientSession()

Review comment:
       This is really good stuff.
   
   Currently you are using one Hook per trigger. That means if you had 32 concurrent triggers, then they would all have their own client sessions.
   I've had a look at the docs for aiohttp, it says:
   ```
   it is suggested you use a single session for the lifetime of your application to benefit from connection pooling.
   ```
   https://docs.aiohttp.org/en/stable/client_reference.html
   
   What do you think?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org