[GitHub] [airflow] ashb commented on a change in pull request #18470: LineageBackend is notified on pre_execute and post fail

[GitBox <gi...@apache.org>]

ashb commented on a change in pull request #18470:
URL: https://github.com/apache/airflow/pull/18470#discussion_r714841692



##########
File path: airflow/lineage/__init__.py
##########
@@ -47,20 +47,38 @@ class Metadata:
     data: Dict = attr.ib()
 
 
-def get_backend() -> Optional[LineageBackend]:
-    """Gets the lineage backend if defined in the configs"""
-    clazz = conf.getimport("lineage", "backend", fallback=None)
-
-    if clazz:
-        if not issubclass(clazz, LineageBackend):
-            raise TypeError(
-                f"Your custom Lineage class `{clazz.__name__}` "
-                f"is not a subclass of `{LineageBackend.__name__}`."
-            )
+NoBackend = ...
+
+
+class Backend:
+    _backend = None
+
+    @classmethod
+    def get_backend(cls) -> Optional[LineageBackend]:
+        """Gets the lineage backend lazily if defined in the configs"""
+        if cls._backend == NoBackend:
+            return None
+        elif cls._backend:
+            return cls._backend
+
+        clazz = conf.getimport("lineage", "backend", fallback=None)
+        if clazz:
+            if not issubclass(clazz, LineageBackend):
+                raise TypeError(
+                    f"Your custom Lineage class `{clazz.__name__}` "
+                    f"is not a subclass of `{LineageBackend.__name__}`."
+                )
+            else:
+                cls._backend = clazz()
+                return cls._backend
         else:
-            return clazz()
+            cls._backend = NoBackend
+            return None
+
 
-    return None
+def get_backend():
+    """Gets the lineage backend lazily if defined in the configs"""
+    return Backend.get_backend()

Review comment:
       This change can be made much simpler with `@cache` decorator.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org