[GitHub] [airflow] Lee-W commented on a diff in pull request #34517: Respect 'soft_fail' argument when running DatabricksPartitionSensor

["Lee-W (via GitHub)" <gi...@apache.org>]

Lee-W commented on code in PR #34517:
URL: https://github.com/apache/airflow/pull/34517#discussion_r1332848005


##########
airflow/providers/databricks/sensors/databricks_partition.py:
##########
@@ -182,7 +182,11 @@ def _generate_partition_query(
         partition_columns = self._sql_sensor(f"DESCRIBE DETAIL {table_name}")[0][7]
         self.log.debug("Partition columns: %s", partition_columns)
         if len(partition_columns) < 1:
-            raise AirflowException(f"Table {table_name} does not have partitions")
+            message = f"Table {table_name} does not have partitions"

Review Comment:
   We'll need to add the TODO like https://github.com/apache/airflow/blob/659d94f0ae89f47a7d4b95d6c19ab7f87bd3a60f/airflow/providers/airbyte/sensors/airbyte.py#L63 so that it's less likely for us to forget it



##########
tests/providers/databricks/sensors/test_databricks_partition.py:
##########
@@ -99,3 +99,67 @@ def test_unsupported_conn_type(self):
     def test_partition_sensor(self, patched_poke):
         patched_poke.return_value = True
         assert self.partition_sensor.poke({})
+
+    @pytest.mark.parametrize(
+        "soft_fail, expected_exception", ((False, AirflowException), (True, AirflowSkipException))
+    )
+    def test_fail__generate_partition_query(self, soft_fail, expected_exception):
+        self.partition_sensor.soft_fail = soft_fail
+        table_name = "test"
+        with pytest.raises(expected_exception) as e, patch(
+            "airflow.providers.databricks.sensors.databricks_partition.DatabricksPartitionSensor"
+            "._sql_sensor"
+        ) as _sql_sensor:
+            _sql_sensor.return_value = [[[], [], [], [], [], [], [], []]]
+            self.partition_sensor._generate_partition_query(
+                prefix="", suffix="", joiner_val="", table_name=table_name
+            )
+        assert e.match(f"Table {table_name} does not have partitions")

Review Comment:
   should we just move `match` to `pytest.raises(expected_exception, match=...)`?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org