[GitHub] [druid] didip commented on issue #11397: Too many PENDING tasks even though middle managers have plenty of capacity.

[GitBox <gi...@apache.org>]

didip commented on issue #11397:
URL: https://github.com/apache/druid/issues/11397#issuecomment-871092508


   I see a lot of these:
   
   ```
   2021-06-30T04:38:24,834 INFO [TaskQueue-Manager] org.apache.druid.indexing.overlord.RemoteTaskRunner - Assigned a task[single_phase_sub_task_mydata_dcodaahg_2021-06-30T03:51:48.334Z] that is already pending!
   ```
   and a lot of these:
   ```
   2021-06-30T04:37:36,965 INFO [TaskQueue-Manager] org.apache.druid.indexing.overlord.TaskQueue - Asking taskRunner to run: index_parallel_mydata_gcjhdhak_2021-06-30T04:37:36.950Z
   2021-06-30T04:37:36,965 INFO [TaskQueue-Manager] org.apache.druid.indexing.overlord.RemoteTaskRunner - Added pending task index_parallel_mydata_gcjhdhak_2021-06-30T04:37:36.950Z
   2021-06-30T04:37:36,965 INFO [TaskQueue-Manager] org.apache.druid.indexing.overlord.TaskQueue - Asking taskRunner to clean up 4 tasks.
   2021-06-30T04:37:36,965 INFO [TaskQueue-Manager] org.apache.druid.indexing.overlord.RemoteTaskRunner - Shutdown [single_phase_sub_task_mydata_cfdbeihn_2021-06-26T04:14:06.444Z] because: [task is not in knownTaskIds[[single_phase_sub_task_mydata_keijolhd_2021-06-30T04:19:48.958Z, single_phase_sub_task_mydata_ofdfacjh_2021-06-30T04:05:38.429Z, single_phase_sub_task_mydata_igjfljdb_2021-06-30T04:05:57.470Z, index_parallel_mydata_kklpkeci_2021-06-30T04:37:27.541Z, single_phase_sub_task_mydata_gbfbmdga_2021-06-30T04:20:14.880Z, single_phase_sub_task_mydata_gdcpnmok_2021-06-30T04:20:14.921Z, single_phase_sub_task_mydata_acdlabpn_2021-06-30T04:33:49.050Z, single_phase_sub_task_mydata_lniojeic_2021-06-30T03:51:53.935Z, single_phase_sub_task_mydata_bpjnfifd_2021-06-30T04:20:15.904Z, single_phase_sub_task_mydata_moehkdgm_2021-06-30T04:05:58.143Z, single_phase_sub_task_mydata_abkgdhfp_2021-06-30T04:20:13.947Z, single_phase_sub_task_mydata_naffkmcj_2021-06-30T04:06:01.414Z, single_phase_sub_t
 ask_mydata_haninkha_2021-06-30T04:33:42.148Z, single_phase_sub_task_mydata_ifmfjggm_2021-06-30T03:51:52.049Z, single_phase_sub_task_mydata_gncnpkih_2021-06-30T04:33:55.047Z, single_phase_sub_task_mydata_gboogald_2021-06-30T04:05:36.201Z, single_phase_sub_task_mydata_dhfdeaid_2021-06-30T04:06:05.154Z, single_phase_sub_task_mydata_hjpphmad_2021-06-30T04:19:50.672Z, single_phase_sub_task_mydata_omeocibn_2021-06-30T04:06:06.087Z, index_parallel_mydata_dkfjnnjn_2021-06-30T04:37:07.248Z, single_phase_sub_task_mydata_bonbjnal_2021-06-30T04:05:46.466Z, single_phase_sub_task_mydata_kjdlmiel_2021-06-30T04:19:50.860Z, single_phase_sub_task_mydata_kdpffoii_2021-06-30T04:06:06.371Z, single_phase_sub_task_mydata_cipjohga_2021-06-30T04:20:14.940Z, single_phase_sub_task_mydata_hbaapnhl_2021-06-30T04:01:58.700Z, single_phase_sub_task_mydata_ldamdnkd_2021-06-30T04:19:51.025Z, single_phase_sub_task_mydata_npdkjgbk_2021-06-30T04:19:55.099Z, single_phase_sub_task_mydata_nkgnoppo_2021-06-30T04:05:46.354Z
 , single_phase_sub_task_mydata_lppkleoo_2021-06-30T04:05:58.727Z, single_phase_sub_task_mydata_ioieblfa_2021-06-30T04:05:30.444Z, single_phase_sub_task_mydata_ammamngf_2021-06-30T04:33:51.003Z, single_phase_sub_task_mydata_nhdffopk_2021-06-30T04:20:07.927Z, single_phase_sub_task_mydata_bggndljg_2021-06-30T03:51:57.302Z, single_phase_sub_task_mydata_bbfjjdkc_2021-06-30T04:19:42.814Z, single_phase_sub_task_mydata_gleimmha_2021-06-30T04:20:10.879Z, single_phase_sub_task_mydata_eagnhgnc_2021-06-30T04:25:23.962Z, single_phase_sub_task_mydata_jfdagdli_2021-06-30T03:51:57.931Z, single_phase_sub_task_mydata_icbdciaf_2021-06-30T04:05:56.520Z, single_phase_sub_task_mydata_jaeidggd_2021-06-30T04:05:38.193Z, single_phase_sub_task_mydata_hifieong_2021-06-30T04:05:42.510Z, single_phase_sub_task_mydata_lmkgidog_2021-06-30T04:06:05.466Z, single_phase_sub_task_mydata_lbnkncpk_2021-06-30T04:16:11.084Z, single_phase_sub_task_mydata_aekfflnn_2021-06-30T04:20:17.680Z, single_phase_sub_task_mydata_cfplho
 jj_2021-06-30T04:05:47.160Z, single_phase_sub_task_mydata_ookaddkk_2021-06-30T04:33:42.951Z, single_phase_sub_task_mydata_nbmpmjcc_2021-06-30T04:20:17.876Z, single_phase_sub_task_mydata_khnhdlho_2021-06-30T04:05:44.221Z, single_phase_sub_task_mydata_dpngdcjl_2021-06-30T04:19:52.780Z, single_phase_sub_task_mydata_iefddgjk_2021-06-30T03:51:52.947Z, single_phase_sub_task_mydata_fgdphjil_2021-06-30T03:56:58.268Z, single_phase_sub_task_mydata_cdljkeee_2021-06-30T04:30:24.134Z, single_phase_sub_task_mydata_kbmlkcbh_2021-06-30T04:33:28.911Z, single_phase_sub_task_mydata_emjdoeeg_2021-06-30T04:20:14.901Z, single_phase_sub_task_mydata_bjjeofkn_2021-06-30T04:33:58.078Z, single_phase_sub_task_mydata_bnmboccb_2021-06-30T04:33:58.014Z, single_phase_sub_task_mydata_bobpajkb_2021-06-30T04:19:45.683Z, single_phase_sub_task_mydata_bmgbcppj_2021-06-30T04:19:55.001Z, single_phase_sub_task_mydata_pnghlhpe_2021-06-30T04:20:12.892Z, single_phase_sub_task_mydata_odfelbbh_2021-06-30T04:33:42.030Z, single_p
 hase_sub_task_mydata_gkmbemnj_2021-06-30T04:19:44.958Z, single_phase_sub_task_mydata_gaaoicfp_2021-06-30T04:20:03.884Z, single_phase_sub_task_mydata_dcodaahg_2021-06-30T03:51:48.334Z, single_phase_sub_task_mydata_lmpcigfi_2021-06-30T04:05:50.176Z, single_phase_sub_task_mydata_agdleoed_2021-06-30T04:33:53.077Z, single_phase_sub_task_mydata_lainegjl_2021-06-30T04:33:29.181Z, single_phase_sub_task_mydata_jfdmhncn_2021-06-30T04:19:43.680Z, single_phase_sub_task_mydata_gglajoaj_2021-06-30T04:06:01.346Z, single_phase_sub_task_mydata_dfllkggb_2021-06-30T04:20:15.924Z, single_phase_sub_task_mydata_klbonbfl_2021-06-30T04:05:54.289Z, single_phase_sub_task_mydata_cgbgomoi_2021-06-30T03:51:48.075Z, single_phase_sub_task_mydata_oiplicel_2021-06-30T04:33:47.979Z, single_phase_sub_task_mydata_oghhkhnc_2021-06-30T04:19:45.815Z, index_parallel_mydata_idpakmne_2021-06-30T03:16:11.810Z, single_phase_sub_task_mydata_jjddmakb_2021-06-30T04:05:26.667Z, single_phase_sub_task_mydata_eobikjeo_2021-06-30T04:
 20:20.795Z, single_phase_sub_task_mydata_nfkoildc_2021-06-30T04:05:31.550Z, single_phase_sub_task_mydata_cekcajic_2021-06-30T04:19:42.793Z, single_phase_sub_task_mydata_cjgnefem_2021-06-30T04:20:19.788Z, single_phase_sub_task_mydata_bkgdlnbg_2021-06-30T04:33:56.018Z, single_phase_sub_task_mydata_obeiammk_2021-06-30T04:20:22.881Z, single_phase_sub_task_mydata_hincjmbk_2021-06-30T04:05:25.451Z, single_phase_sub_task_mydata_ogacpbok_2021-06-30T04:11:09.776Z, single_phase_sub_task_mydata_hcjpefhj_2021-06-30T04:06:05.339Z, single_phase_sub_task_mydata_ofbgipie_2021-06-30T04:33:55.017Z, single_phase_sub_task_mydata_dfejdlba_2021-06-30T04:20:13.881Z, single_phase_sub_task_mydata_jbggacnd_2021-06-30T04:06:09.263Z, single_phase_sub_task_mydata_kkeenabe_2021-06-30T04:05:50.450Z, single_phase_sub_task_mydata_jngphnhl_2021-06-30T04:05:31.529Z, single_phase_sub_task_mydata_pnnhaggc_2021-06-30T04:06:08.356Z, single_phase_sub_task_mydata_ioibbfnh_2021-06-30T04:33:38.080Z, single_phase_sub_task_myd
 ata_cjafcalb_2021-06-30T04:06:06.758Z, single_phase_sub_task_mydata_hdcmkapl_2021-06-30T04:20:21.886Z, index_parallel_mydata_gcjhdhak_2021-06-30T04:37:36.950Z, single_phase_sub_task_mydata_mpjcncli_2021-06-30T04:05:58.275Z, single_phase_sub_task_mydata_pinefnhc_2021-06-30T04:33:48.986Z, single_phase_sub_task_mydata_nfphglaa_2021-06-30T04:05:29.436Z, single_phase_sub_task_mydata_mhdebkhc_2021-06-30T04:05:38.173Z, single_phase_sub_task_mydata_nflpjfdo_2021-06-30T04:20:23.894Z, single_phase_sub_task_mydata_mgehkgmg_2021-06-30T04:20:15.881Z, single_phase_sub_task_mydata_gnfbjapf_2021-06-30T04:05:36.443Z, single_phase_sub_task_mydata_pbjnnnnn_2021-06-30T04:20:08.875Z, single_phase_sub_task_mydata_adjmggom_2021-06-30T04:19:56.860Z, single_phase_sub_task_mydata_keeolmna_2021-06-30T04:33:53.056Z, single_phase_sub_task_mydata_mgannkma_2021-06-30T04:05:26.328Z]]]
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org