[GitHub] [airflow] V0lantis commented on issue #23361: Scheduler crashes with psycopg2.errors.DeadlockDetected exception

[GitBox <gi...@apache.org>]

V0lantis commented on issue #23361:
URL: https://github.com/apache/airflow/issues/23361#issuecomment-1133120999

   I experience exactly the same issue. Reducing the number of schedulers to one seems to have resolved the issue for my deployment, but now, tasks are scheduling a lot slower. I initially had 3 schedulers. Here are my new config variables:
   
   ```cfg
   [scheduler]
   job_heartbeat_sec = 30
   scheduler_heartbeat_sec = 5
   num_runs = -1
   scheduler_idle_sleep_time = 1
   min_file_process_interval = 30
   dag_dir_list_interval = 120
   print_stats_interval = 240
   pool_metrics_interval = 5
   scheduler_health_check_threshold = 60
   orphaned_tasks_check_interval = 300.0
   scheduler_zombie_task_threshold = 300
   catchup_by_default = True
   max_tis_per_query = 512
   use_row_level_locking = False
   max_dagruns_to_create_per_loop = 100
   max_dagruns_per_loop_to_schedule = 200
   schedule_after_task_execution = True
   parsing_processes = 2
   file_parsing_sort_mode = modified_time
   use_job_schedule = True
   allow_trigger_in_future = False
   dependency_detector = airflow.serialization.serialized_objects.DependencyDetector
   trigger_timeout_check_interval = 15
   run_duration = 41460
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org