You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "sunank200 (via GitHub)" <gi...@apache.org> on 2023/06/13 12:22:32 UTC
[GitHub] [airflow] sunank200 commented on a diff in pull request #31609: openlineage, mysql: add OpenLineage support for MySQL
sunank200 commented on code in PR #31609:
URL: https://github.com/apache/airflow/pull/31609#discussion_r1228024677
##########
airflow/providers/common/sql/hooks/sql.py:
##########
@@ -516,3 +517,40 @@ def test_connection(self):
message = str(e)
return status, message
+
+ def get_database_info(self, connection):
Review Comment:
missing doc-strings and annotations
##########
airflow/providers/mysql/hooks/mysql.py:
##########
@@ -298,3 +298,24 @@ def bulk_load_custom(
cursor.close()
conn.commit()
conn.close() # type: ignore[misc]
+
+ def get_database_info(self, connection):
+ from airflow.providers.openlineage.sqlparser import DatabaseInfo
+
+ return DatabaseInfo(
+ scheme=self.get_database_dialect(connection),
+ authority=self._get_authority(connection),
+ information_schema_columns=[
+ "table_schema",
+ "table_name",
+ "column_name",
+ "ordinal_position",
+ "column_type",
+ ],
+ normalize_name_method=lambda name: name.upper(),
Review Comment:
is the name always supposed to be upper?
##########
airflow/providers/openlineage/extractors/base.py:
##########
@@ -35,6 +37,22 @@ class OperatorLineage:
run_facets: dict[str, BaseFacet] = Factory(dict)
job_facets: dict[str, BaseFacet] = Factory(dict)
+ @staticmethod
+ def merge(base: OperatorLineage, other: OperatorLineage) -> OperatorLineage:
+ """
Review Comment:
Should this be a separate static method? Shouldn't this be done while populating the lineage?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org