[jira] [Commented] (AIRFLOW-6974) Using MS SQL Server 17 as a backend, Migration cc1e65623dc7_add_max_tries_column_to_task_instance.py fails if load_examples = True

["Sam T (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17088134#comment-17088134 ] 

Sam T commented on AIRFLOW-6974:
--------------------------------

I think this issue happened before AIRFLOW-1452.

i used the SQL utility ([https://raw.githubusercontent.com/amachanic/sp_whoisactive/master/who_is_active.sql])

and confirmed locking issue.

!image-2020-04-20-18-14-10-275.png!
 
+*First Query: (session_id: 53)*+
 
(@P1 NVARCHAR(6),@P2 NVARCHAR(20))SELECT [INFORMATION_SCHEMA].[TABLES].[TABLE_NAME]
FROM [INFORMATION_SCHEMA].[TABLES]
WHERE [INFORMATION_SCHEMA].[TABLES].[TABLE_SCHEMA] = CAST(@P1 AS NVARCHAR(max)) AND [INFORMATION_SCHEMA].[TABLES].[TABLE_TYPE] = CAST(@P2 AS NVARCHAR(max)) ORDER BY [INFORMATION_SCHEMA].[TABLES].[TABLE_NAME]
 
*+Second Query (session_id:54 , blocked by session 53)+* 
(@P1 INT,@P2 NVARCHAR(24))SELECT TOP 1 slot_pool.id AS slot_pool_id, slot_pool.pool AS slot_pool_pool, slot_pool.slots AS slot_pool_slots, slot_pool.description AS slot_pool_description
FROM slot_pool
WHERE slot_pool.slots = @P1 AND slot_pool.pool = @P2
 

> Using MS SQL Server 17 as a backend, Migration cc1e65623dc7_add_max_tries_column_to_task_instance.py fails if load_examples = True
> ----------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: AIRFLOW-6974
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-6974
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: database
>    Affects Versions: 1.10.9
>            Reporter: Tony Brookes
>            Priority: Minor
>         Attachments: airflow-mssql-stack-trace.txt, image-2020-04-20-18-14-10-275.png
>
>
> This took me a while to figure out as there was another issue with the migration in question which is cc1e65623dc7_add_max_tries_column_to_task_instance.py
> This file USED to have an issue where it would sit there forever during an initdb on MS SQL Server, essentially deadlocked with itself.
> I couldn't figure out why it was still sitting there for me, given that I was using the version of the migration where this had been fixed, so I went looking at the locks on the DB.  I found TWO processes running on the DB both originating inside the airflow initdb Python instance.
> The first was happily sitting there trying to query the max_retries column on a table, but the other was attempting to query the table "slot_pool" from within example_subdag_operator.py .  I killed the session which was querying that table and of course my Python process crashed, helpfully with a stack trace.
> The session I killed was interacting with the DB running in EXAMPLES and was actually complaining that the table was not a valid object name.  As soon as I set load_examples = False, the initdb process ran through in a few seconds and all was well.  But with load_examples = True it would reliably hang on this specific migration every single time.
> I have attached a full stack trace from when I terminated the second DB session.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)