You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "MaksYermak (via GitHub)" <gi...@apache.org> on 2024/06/24 12:50:49 UTC

Re: [PR] Add parameter for the last number of queries to the DB to the logs [airflow]

MaksYermak commented on code in PR #40323:
URL: https://github.com/apache/airflow/pull/40323#discussion_r1650979649


##########
tests/dag_processing/test_processor.py:
##########
@@ -1008,6 +1010,14 @@ def test_nullbyte_exception_handling_when_preimporting_airflow(self, mock_contex
         )
         processor.start()
 
+    def test_counter_for_last_num_of_db_queries(self):
+        dag_filepath = TEST_DAG_FOLDER / "test_multiple_dags.py"
+
+        with create_session() as session:
+            _, _, last_num_of_db_queries = self._process_file(dag_filepath, TEST_DAG_FOLDER, session)
+
+        assert last_num_of_db_queries == 94

Review Comment:
   @kosteev @potiuk I have updated the test and created a new file for DAGs. But during updating I have noticed that we count queries in a different way in my implementation and implementation which the community is using for a test. In realization for a test we apply the `after_cursor_execute` event to the `engine` object. In my realization I apply the `do_orm_execute` event to the `session`. As a result we have different count numbers.
   For example these are numbers for a file with 5 DAGs with `EmptyOperator`:
   - session: 26 queries for parsing a file with one DAG, 17 queries more for each new DAG. 94 for file with 5 DAGs; 
   - engine: 32 queries for parsing a file with one DAG, 22 queries more for each new DAG; 120 for file with 5 DAGs; 
   In that case what do you think is better to use as a counting implementation?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org