You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by "DaanHoogland (via GitHub)" <gi...@apache.org> on 2026/01/14 12:40:05 UTC

[I] bogus usage jobs may be created when usage is behind in processing [cloudstack]

DaanHoogland opened a new issue, #12424:
URL: https://github.com/apache/cloudstack/issues/12424

   ### problem
   
   During trouble shooting missing data, when usage is behind it was found that a lot of usage_jobs are created that won’t have a period to process:
   
   ```
   2026-01-13 01:00:38,566 DEBUG [cloud.usage.UsageManagerImpl] (Usage-HB-1:null) (logid:) it's been 86438558 ms since last usage job and 66261443 ms until next job, scheduling an immediate job to catch up (aggregation duration is 1440 minutes)
   2026-01-13 01:00:38,568 DEBUG [cloud.usage.UsageManagerImpl] (Usage-HB-1:null) (logid:) Scheduling Usage job...
   2026-01-13 01:00:38,570 INFO  [cloud.usage.UsageManagerImpl] (Usage-Job-1:null) (logid:) starting usage job...
   2026-01-13 01:00:38,606 INFO  [cloud.usage.UsageManagerImpl] (Usage-Job-1:null) (logid:) Parsing usage records between Mon Jan 12 01:00:00 CET 2026 and Tue Jan 13 00:59:59 CET 2026
   2026-01-13 01:01:38,581 DEBUG [cloud.usage.UsageManagerImpl] (Usage-HB-1:null) (logid:) it's been 86498581 ms since last usage job and 66201420 ms until next job, scheduling an immediate job to catch up (aggregation duration is 1440 minutes)
   2026-01-13 01:01:38,581 DEBUG [cloud.usage.UsageManagerImpl] (Usage-HB-1:null) (logid:) Scheduling Usage job...
   2026-01-13 01:02:38,559 DEBUG [cloud.usage.UsageManagerImpl] (Usage-HB-1:null) (logid:) it's been 86558559 ms since last usage job and 66141442 ms until next job, scheduling an immediate job to catch up (aggregation duration is 1440 minutes)
   2026-01-13 01:02:38,559 DEBUG [cloud.usage.UsageManagerImpl] (Usage-HB-1:null) (logid:) Scheduling Usage job...
   2026-01-13 01:03:38,558 DEBUG [cloud.usage.UsageManagerImpl] (Usage-HB-1:null) (logid:) it's been 86618558 ms since last usage job and 66081443 ms until next job, scheduling an immediate job to catch up (aggregation duration is 1440 minutes)
   2026-01-13 01:03:38,558 DEBUG [cloud.usage.UsageManagerImpl] (Usage-HB-1:null) (logid:) Scheduling Usage job...
   2026-01-13 01:04:38,559 DEBUG [cloud.usage.UsageManagerImpl] (Usage-HB-1:null) (logid:) it's been 86678559 ms since last usage job and 66021442 ms until next job, scheduling an immediate job to catch up (aggregation duration is 1440 minutes)
   2026-01-13 01:04:38,559 DEBUG [cloud.usage.UsageManagerImpl] (Usage-HB-1:null) (logid:) Scheduling Usage job...
   2026-01-13 01:05:38,558 DEBUG [cloud.usage.UsageManagerImpl] (Usage-HB-1:null) (logid:) it's been 86738559 ms since last usage job and 65961442 ms until next job, scheduling an immediate job to catch up (aggregation duration is 1440 minutes)
   2026-01-13 01:05:38,558 DEBUG [cloud.usage.UsageManagerImpl] (Usage-HB-1:null) (logid:) Scheduling Usage job...
   2026-01-13 01:06:02,952 DEBUG [cloud.usage.UsageManagerImpl] (Usage-Job-1:null) (logid:) Creating networking offering: 18 for Vm: 75432 for account: 239496
   2026-01-13 01:06:03,028 DEBUG [cloud.usage.UsageManagerImpl] (Usage-Job-1:null) (logid:) create volume with id : 163459 for account: 239496
   2026-01-13 01:06:03,497 DEBUG [cloud.usage.UsageManagerImpl] (Usage-Job-1:null) (logid:) deleting network offering: 18 from Vm: 96981
   ```
   the HeartBeat thread should be clever about existing jobs and not create the extra jobs.
   
   
   ### versions
   
   ACS 4.19
   
   ### The steps to reproduce the bug
   
   1. have an environment running for a while
   2. prune intermediate and usage tables a couple of months
   3. set the events of those months to unprocessed
   4. start usage
   5. observe the a job starts and multiple other jobs are created.
   6. the extra jobs seem to have an interval of -1 second and fail (of course)
   
   
   ### What to do about it?
   
   make sure the heartbeat job does not create jobs while jobs are being processed/scheduled already.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] bogus usage jobs may be created when usage is behind in processing [cloudstack]

Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
DaanHoogland closed issue #12424: bogus usage jobs may be created when usage is behind in processing
URL: https://github.com/apache/cloudstack/issues/12424


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] bogus usage jobs may be created when usage is behind in processing [cloudstack]

Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on issue #12424:
URL: https://github.com/apache/cloudstack/issues/12424#issuecomment-3817173448

   I was able to reproduce the issue (With some code orchestration to speed up the process)
   
   Steps:
   1. Set `usage.stats.job.aggregation.range` to **5 minutes**
   <img width="1032" height="85" alt="Image" src="https://github.com/user-attachments/assets/52abdfbc-88bf-4d65-9d81-ed03632eb0f1" />
   
   2. Put a sleep in `UsageManagerImpl.parse(UsageJobVO job, long startDateMillis, long endDateMillis)` for **5 minutes** (we need a long running usage parsing job) 
   3. stop the cloudstack-usage service.
   4. start it after **~12 minutes**
   5. The heartbeat code sees that it has been long since last job was scheduled, so it schedules it.
   6. Usage job scheduled sleeps for **5 minutes**
   7. Heartbeat code runs every minute and logs the Usage job starting every one minute (It doesn't actually start new jobs, just updates the last running job with the new heartbeat)
   ```
   2026-01-29 11:31:58,846 INFO  [cloud.usage.UsageServer] (main:[]) (logid:) UsageServer ready...                                                                                                                                                                                                                                     
   2026-01-29 11:32:13,827 DEBUG [usage.UsageManagerImpl.Heartbeat] (Usage-HB-1:[]) (logid:) it's been 733825 ms since last usage job and 63166176 ms until next job, scheduling an immediate job to catch up (aggregation duration is 5 minutes)                                                                                      
   2026-01-29 11:32:13,828 DEBUG [cloud.usage.UsageManagerImpl_EnhancerByCloudStack_7373c433] (Usage-HB-1:[]) (logid:) Scheduling Usage job...                                                                                                                                                                                         
   2026-01-29 11:32:13,829 INFO  [cloud.usage.UsageManagerImpl_EnhancerByCloudStack_7373c433] (Usage-Job-1:[]) (logid:) starting usage job...                                                                                                                                                                                          
   2026-01-29 11:32:13,881 INFO  [cloud.usage.UsageManagerImpl_EnhancerByCloudStack_7373c433] (Usage-Job-1:[]) (logid:) Parsing usage records between [2026-01-29T11:20:00+0000] and [2026-01-29T11:31:59+0000].                                                                                                                       
   2026-01-29 11:33:13,808 DEBUG [usage.UsageManagerImpl.Heartbeat] (Usage-HB-1:[]) (logid:) it's been 793808 ms since last usage job and 63106193 ms until next job, scheduling an immediate job to catch up (aggregation duration is 5 minutes)                                                                                      
   2026-01-29 11:33:13,808 DEBUG [cloud.usage.UsageManagerImpl_EnhancerByCloudStack_7373c433] (Usage-HB-1:[]) (logid:) Scheduling Usage job...                                                                                                                                                                                         
   2026-01-29 11:34:13,806 DEBUG [usage.UsageManagerImpl.Heartbeat] (Usage-HB-1:[]) (logid:) it's been 853806 ms since last usage job and 63046195 ms until next job, scheduling an immediate job to catch up (aggregation duration is 5 minutes)                                                                                      
   2026-01-29 11:34:13,806 DEBUG [cloud.usage.UsageManagerImpl_EnhancerByCloudStack_7373c433] (Usage-HB-1:[]) (logid:) Scheduling Usage job...                                                                                                                                                                                         
   2026-01-29 11:35:13,816 DEBUG [usage.UsageManagerImpl.Heartbeat] (Usage-HB-1:[]) (logid:) it's been 913816 ms since last usage job and 62986185 ms until next job, scheduling an immediate job to catch up (aggregation duration is 5 minutes)                                                                                      
   2026-01-29 11:35:13,816 DEBUG [cloud.usage.UsageManagerImpl_EnhancerByCloudStack_7373c433] (Usage-HB-1:[]) (logid:) Scheduling Usage job...                                                                                                                                                                                         
   2026-01-29 11:36:13,804 DEBUG [usage.UsageManagerImpl.Heartbeat] (Usage-HB-1:[]) (logid:) it's been 973804 ms since last usage job and 62926197 ms until next job, scheduling an immediate job to catch up (aggregation duration is 5 minutes)                                                                                      
   2026-01-29 11:36:13,805 DEBUG [cloud.usage.UsageManagerImpl_EnhancerByCloudStack_7373c433] (Usage-HB-1:[]) (logid:) Scheduling Usage job...                                                                                                                                                                                         
   2026-01-29 11:37:13,804 DEBUG [usage.UsageManagerImpl.Heartbeat] (Usage-HB-1:[]) (logid:) it's been 1033804 ms since last usage job and 62866197 ms until next job, scheduling an immediate job to catch up (aggregation duration is 5 minutes)                                                                                     
   2026-01-29 11:37:13,804 DEBUG [cloud.usage.UsageManagerImpl_EnhancerByCloudStack_7373c433] (Usage-HB-1:[]) (logid:) Scheduling Usage job...      
   ```
   
   ```
   mysql> mysql> select * from usage_job where end_millis=0;
   +------+------------------------------------------------------+--------+----------+-----------+--------------+------------+-----------+------------+----------+---------+---------------------+
   | id   | host                                                 | pid    | job_type | scheduled | start_millis | end_millis | exec_time | start_date | end_date | success | heartbeat           |
   +------+------------------------------------------------------+--------+----------+-----------+--------------+------------+-----------+------------+----------+---------+---------------------+
   | 1811 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 393299 |        0 |         0 |            0 |          0 |         0 | NULL       | NULL     |    NULL | 2026-01-29 11:34:13 |
   +------+------------------------------------------------------+--------+----------+-----------+--------------+------------+-----------+------------+----------+---------+---------------------+
   1 row in set (0.00 sec)
   
   mysql> select * from usage_job where end_millis=0;
   +------+------------------------------------------------------+--------+----------+-----------+--------------+------------+-----------+------------+----------+---------+---------------------+
   | id   | host                                                 | pid    | job_type | scheduled | start_millis | end_millis | exec_time | start_date | end_date | success | heartbeat           |
   +------+------------------------------------------------------+--------+----------+-----------+--------------+------------+-----------+------------+----------+---------+---------------------+
   | 1811 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 393299 |        0 |         0 |            0 |          0 |         0 | NULL       | NULL     |    NULL | 2026-01-29 11:34:13 |
   +------+------------------------------------------------------+--------+----------+-----------+--------------+------------+-----------+------------+----------+---------+---------------------+
   1 row in set (0.00 sec)
   
   mysql> 
   mysql> select * from usage_job where end_millis=0;
   +------+------------------------------------------------------+--------+----------+-----------+--------------+------------+-----------+------------+----------+---------+---------------------+
   | id   | host                                                 | pid    | job_type | scheduled | start_millis | end_millis | exec_time | start_date | end_date | success | heartbeat           |
   +------+------------------------------------------------------+--------+----------+-----------+--------------+------------+-----------+------------+----------+---------+---------------------+
   | 1811 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 393299 |        0 |         0 |            0 |          0 |         0 | NULL       | NULL     |    NULL | 2026-01-29 11:35:13 |
   +------+------------------------------------------------------+--------+----------+-----------+--------------+------------+-----------+------------+----------+---------+---------------------+
   1 row in set (0.00 sec)
   
   mysql> select * from usage_job where end_millis=0;
   +------+------------------------------------------------------+--------+----------+-----------+--------------+------------+-----------+------------+----------+---------+---------------------+
   | id   | host                                                 | pid    | job_type | scheduled | start_millis | end_millis | exec_time | start_date | end_date | success | heartbeat           |
   +------+------------------------------------------------------+--------+----------+-----------+--------------+------------+-----------+------------+----------+---------+---------------------+
   | 1811 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 393299 |        0 |         0 |            0 |          0 |         0 | NULL       | NULL     |    NULL | 2026-01-29 11:36:13 |
   +------+------------------------------------------------------+--------+----------+-----------+--------------+------------+-----------+------------+----------+---------+---------------------+
   1 row in set (0.00 sec)
   
   mysql> select * from usage_job where end_millis=0;
   +------+------------------------------------------------------+--------+----------+-----------+--------------+------------+-----------+------------+----------+---------+---------------------+
   | id   | host                                                 | pid    | job_type | scheduled | start_millis | end_millis | exec_time | start_date | end_date | success | heartbeat           |
   +------+------------------------------------------------------+--------+----------+-----------+--------------+------------+-----------+------------+----------+---------+---------------------+
   | 1812 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 393299 |        0 |         0 |            0 |          0 |         0 | NULL       | NULL     |    NULL | 2026-01-29 11:37:16 |
   +------+------------------------------------------------------+--------+----------+-----------+--------------+------------+-----------+------------+----------+---------+---------------------+
   1 row in set (0.00 sec)
   
   ```
   
   Please note that the new jobs are not actually being created. The last running job is updated.
   
   As a fix, Heartbeat code should check if there is a job already running and not try to schedule it again.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] bogus usage jobs may be created when usage is behind in processing [cloudstack]

Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on issue #12424:
URL: https://github.com/apache/cloudstack/issues/12424#issuecomment-3845914872

   I could also see lot's of failed -1 second jobs as a side-effect of the above
   
   ```
   mysql> select * from usage_job where id>3140;       
   +------+------------------------------------------------------+--------+----------+-----------+---------------+---------------+-----------+---------------------+---------------------+---------+---------------------+
   | id   | host                                                 | pid    | job_type | scheduled | start_millis  | end_millis    | exec_time | start_date          | end_date            | success | heartbeat           |
   +------+------------------------------------------------------+--------+----------+-----------+---------------+---------------+-----------+---------------------+---------------------+---------+---------------------+
   | 3141 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 393299 |        0 |         0 | 1770085920000 | 1770086219999 |    300810 | 2026-02-03 02:32:00 | 2026-02-03 02:36:59 |       1 | 2026-02-03 02:42:13 |
   | 3142 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 393299 |        0 |         0 | 1770086220000 | 1770086519999 |    300870 | 2026-02-03 02:37:00 | 2026-02-03 02:41:59 |       1 | 2026-02-03 02:47:13 |
   | 3143 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 393299 |        0 |         0 | 1770086520000 | 1770086819999 |    300879 | 2026-02-03 02:42:00 | 2026-02-03 02:46:59 |       1 | 2026-02-03 02:52:13 |
   | 3144 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 393299 |        0 |         0 | 1770086820000 | 1770087119999 |    300785 | 2026-02-03 02:47:00 | 2026-02-03 02:51:59 |       1 | 2026-02-03 02:57:13 |
   | 3146 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770087120000 | 1770087659999 |   3602755 | 2026-02-03 02:52:00 | 2026-02-03 03:00:59 |       1 | 2026-02-03 04:01:30 |
   | 3147 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770087660000 | 1770091259999 |     11428 | 2026-02-03 03:01:00 | 2026-02-03 04:00:59 |       1 | 2026-02-03 04:01:33 |
   | 3148 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         2 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:44 |
   | 3149 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:44 |
   | 3150 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:44 |
   | 3151 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:44 |
   | 3152 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         2 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:44 |
   | 3153 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:44 |
   | 3154 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         2 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:45 |
   | 3155 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:45 |
   | 3156 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:45 |
   | 3157 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:45 |
   | 3158 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:45 |
   | 3159 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:45 |
   | 3160 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:45 |
   | 3161 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:45 |
   | 3162 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:45 |
   | 3163 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:45 |
   | 3164 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:45 |
   | 3165 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:45 |
   | 3166 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:45 |
   | 3167 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:45 |
   | 3168 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:45 |
   | 3169 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:45 |
   | 3170 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         2 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:45 |
   | 3171 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:45 |
   | 3172 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         2 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:45 |
   | 3173 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:45 |
   | 3174 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:45 |
   | 3175 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:45 |
   | 3176 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:45 |
   | 3177 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:45 |
   | 3178 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         2 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:45 |
   | 3179 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:45 |
   | 3180 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:45 |
   | 3181 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:45 |
   | 3182 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         2 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:45 |
   | 3183 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:45 |
   | 3184 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:45 |
   | 3185 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:45 |
   | 3186 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:45 |
   | 3187 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         2 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:45 |
   | 3188 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:45 |
   | 3189 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:45 |
   | 3190 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:46 |
   | 3191 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:46 |
   | 3192 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         2 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:46 |
   | 3193 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:46 |
   | 3194 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:46 |
   | 3195 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         2 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:46 |
   | 3196 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:46 |
   | 3197 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:46 |
   | 3198 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:46 |
   | 3199 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:46 |
   | 3200 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:46 |
   | 3201 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:46 |
   | 3202 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:46 |
   | 3203 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:46 |
   | 3204 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:46 |
   | 3205 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:46 |
   | 3206 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 | 1770091260000 | 1770091259999 |         1 | 2026-02-03 04:01:00 | 2026-02-03 04:00:59 |       0 | 2026-02-03 04:01:46 |
   | 3207 | ref-trl-10702-k-Mol8-abhisar-sinha-mgmt2/10.0.35.225 | 672936 |        0 |         0 |             0 |             0 |         0 | NULL                | NULL                |    NULL | 2026-02-03 04:01:46 |
   +------+------------------------------------------------------+--------+----------+-----------+---------------+---------------+-----------+---------------------+---------------------+---------+---------------------+
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] bogus usage jobs may be created when usage is behind in processing [cloudstack]

Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
DaanHoogland commented on issue #12424:
URL: https://github.com/apache/cloudstack/issues/12424#issuecomment-3896191051

   fixed in #12616 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org