[GitHub] [airflow] kosteev commented on a diff in pull request #27155: Metric for raw task return codes

[GitBox <gi...@apache.org>]

kosteev commented on code in PR #27155:
URL: https://github.com/apache/airflow/pull/27155#discussion_r1004255873


##########
airflow/jobs/local_task_job.py:
##########
@@ -162,6 +162,7 @@ def handle_task_exit(self, return_code: int) -> None:
         # Without setting this, heartbeat may get us
         self.terminating = True
         self.log.info("Task exited with return code %s", return_code)
+        Stats.incr(f'ti.raw_task_return_code.{self.dag_id}.{self.task_instance.task_id}.{return_code}')

Review Comment:
   Not sure about how is better design here, but I thought for a moment that maybe good to have encapsulated the logic of emitting this metric inside taskinstance.py module (as other metrics relate to task instance).



##########
tests/jobs/test_local_task_job.py:
##########
@@ -374,6 +374,29 @@ def test_localtaskjob_double_trigger(self):
 
         session.close()
 
+    @patch.object(StandardTaskRunner, 'return_code')
+    @mock.patch('airflow.jobs.scheduler_job.Stats.incr')

Review Comment:
   It is good to have autospec=True parameter set for mocked objects, that will enforce method signature checks by unittest.mock library.
   Although, I am not sure if this guideline is followed in the repo.



##########
tests/jobs/test_local_task_job.py:
##########
@@ -374,6 +374,29 @@ def test_localtaskjob_double_trigger(self):
 
         session.close()
 
+    @patch.object(StandardTaskRunner, 'return_code')
+    @mock.patch('airflow.jobs.scheduler_job.Stats.incr')
+    def test_raw_task_return_code_metric(self, mock_stats_incr, mock_return_code, create_dummy_dag):
+
+        _, task = create_dummy_dag('test_localtaskjob_double_trigger')
+        mock_stats_incr.reset_mock()

Review Comment:
   Is it really needed (reset_mock)? This is just beginning of the method.



##########
airflow/jobs/local_task_job.py:
##########
@@ -162,6 +162,7 @@ def handle_task_exit(self, return_code: int) -> None:
         # Without setting this, heartbeat may get us
         self.terminating = True
         self.log.info("Task exited with return code %s", return_code)
+        Stats.incr(f'ti.raw_task_return_code.{self.dag_id}.{self.task_instance.task_id}.{return_code}')

Review Comment:
   Or maybe metric should be more like `local_task_job.task_exit.{dag_id}.{task_id}.{return_code}`. WDYT?



##########
tests/jobs/test_local_task_job.py:
##########
@@ -374,6 +374,29 @@ def test_localtaskjob_double_trigger(self):
 
         session.close()
 
+    @patch.object(StandardTaskRunner, 'return_code')
+    @mock.patch('airflow.jobs.scheduler_job.Stats.incr')
+    def test_raw_task_return_code_metric(self, mock_stats_incr, mock_return_code, create_dummy_dag):
+
+        _, task = create_dummy_dag('test_localtaskjob_double_trigger')
+        mock_stats_incr.reset_mock()
+
+        ti_run = TaskInstance(task=task, execution_date=DEFAULT_DATE)
+        ti_run.refresh_from_db()
+        job1 = LocalTaskJob(task_instance=ti_run, executor=SequentialExecutor())
+
+        mock_return_code.side_effect = [None, -9, None]
+
+        with timeout(10):
+            job1.run()
+
+        mock_stats_incr.assert_has_calls(
+            [
+                mock.call('ti.raw_task_return_code.test_localtaskjob_double_trigger.op1.-9'),
+            ],
+            any_order=True,

Review Comment:
   You set "any_order=True", but only one call is expected, looks strange?
   Maybe good idea to actually have multiple calls expected from different tasks.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org