You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Matheus Fontes <ma...@ghz.com.br> on 2022/07/26 20:30:10 UTC
Usage computing removed items
Hi,
Is anyone having problems with usage computing removed items?
We have a user that reported the problem.
The volume was deleted since 2022-05-06
mysql> select id,account_id,created,removed,state from volumes where id=3246;
+------+------------+---------------------+---------------------+----------+
| id | account_id | created | removed | state |
+------+------------+---------------------+---------------------+----------+
| 3246 | 545 | 2021-04-08 17:03:16 | 2022-05-06 15:06:52 | Expunged |
+------+------------+---------------------+---------------------+----------+
In usage process we can see it on parsing volume call:
2022-07-26 00:18:17,471 DEBUG [usage.parser.VolumeUsageParser] (Usage-Job-1:null) (logid:) Parsing all Volume usage events for account: 545
2022-07-26 00:18:17,472 DEBUG [usage.parser.VolumeUsageParser] (Usage-Job-1:null) (logid:) Total running time 86400000ms
2022-07-26 00:18:17,472 DEBUG [usage.parser.VolumeUsageParser] (Usage-Job-1:null) (logid:) Creating Volume usage record for vol: 3246, usage: 24, startDate: Mon Jul 25 00:00:00 BRT 2022, endDate: Mon Jul 25 23:59:59 BRT 2022, for account: 545
2022-07-26 00:18:17,484 DEBUG [usage.parser.VolumeUsageParser] (Usage-Job-1:null) (logid:) Total running time 86400000ms
2022-07-26 00:18:17,484 DEBUG [usage.parser.VolumeUsageParser] (Usage-Job-1:null) (logid:) Creating Volume usage record for vol: 3246, usage: 24, startDate: Mon Jul 25 00:00:00 BRT 2022, endDate: Mon Jul 25 23:59:59 BRT 2022, for account: 545
And it are being computed to account:
(ascenty) # > list usagerecords domainid=XXXXXXXXXXXXXXXXXXX accountid=d14c8cb9-fd92-43c8-9ddb-f2ed4e9af5a8 type=6 startdate=2022-07-25 enddate=2022-07-25 filter=account,rawusage,size,startdate,usage,usagetype,
{
"count": 2,
"usagerecord": [
{
"accountid": "d14c8cb9-fd92-43c8-9ddb-f2ed4e9af5a8",
"rawusage": "24",
"size": 53687091200,
"startdate": "2022-07-25'T'00:00:00-03:00",
"usage": "24 Hrs",
"usagetype": 6
},
{
"accountid": "d14c8cb9-fd92-43c8-9ddb-f2ed4e9af5a8",
"rawusage": "24",
"size": 53687091200,
"startdate": "2022-07-25'T'00:00:00-03:00",
"usage": "24 Hrs",
"usagetype": 6
}
]
}
Re: Usage computing removed items
Posted by Matheus Fontes <ma...@ghz.com.br>.
I found a query that maybe a problem:
When usage job start it founds a VOLUME.DELETE in usage_event table and after that it look for a register in usage_volume table with that query:
SELECT usage_volume.id, usage_volume.zone_id, usage_volume.account_id, usage_volume.domain_id, usage_volume.volume_id, usage_volume.disk_offering_id, usage_volume.template_id, usage_volume.size, usage_volume.created, usage_volume.deleted FROM usage_volume WHERE usage_volume.account_id = 9 AND usage_volume.id = 110 AND usage_volume.deleted IS NULL;
Empty set (0.00 sec)
but its return empty because the field filter is wrong. I think the query needs to look for a volume_id field.
SELECT usage_volume.id, usage_volume.zone_id, usage_volume.account_id, usage_volume.domain_id, usage_volume.volume_id, usage_volume.disk_offering_id, usage_volume.template_id, usage_volume.size, usage_volume.created, usage_volume.deleted FROM usage_volume WHERE usage_volume.account_id = 9 AND usage_volume.volume_id = 110 AND usage_volume.deleted IS NULL;
+----+---------+------------+-----------+-----------+------------------+-------------+-------------+---------------------+---------+
| id | zone_id | account_id | domain_id | volume_id | disk_offering_id | template_id | size | created | deleted |
+----+---------+------------+-----------+-----------+------------------+-------------+-------------+---------------------+---------+
| 74 | 1 | 9 | 2 | 110 | 78 | NULL | 10737418240 | 2021-06-08 19:19:38 | NULL |
+----+---------+------------+-----------+-----------+------------------+-------------+-------------+---------------------+---------+
1 row in set (0.00 sec)
WIth this wrong query, the usage_volume line will never be found and will never be update with correct deleted field.
Is there someone else experiencing this problem?
> On 26 Jul 2022, at 17:30, Matheus Fontes <ma...@ghz.com.br> wrote:
>
> Hi,
> Is anyone having problems with usage computing removed items?
> We have a user that reported the problem.
> The volume was deleted since 2022-05-06
>
> mysql> select id,account_id,created,removed,state from volumes where id=3246;
> +------+------------+---------------------+---------------------+----------+
> | id | account_id | created | removed | state |
> +------+------------+---------------------+---------------------+----------+
> | 3246 | 545 | 2021-04-08 17:03:16 | 2022-05-06 15:06:52 | Expunged |
> +------+------------+---------------------+---------------------+----------+
>
>
> In usage process we can see it on parsing volume call:
> 2022-07-26 00:18:17,471 DEBUG [usage.parser.VolumeUsageParser] (Usage-Job-1:null) (logid:) Parsing all Volume usage events for account: 545
> 2022-07-26 00:18:17,472 DEBUG [usage.parser.VolumeUsageParser] (Usage-Job-1:null) (logid:) Total running time 86400000ms
> 2022-07-26 00:18:17,472 DEBUG [usage.parser.VolumeUsageParser] (Usage-Job-1:null) (logid:) Creating Volume usage record for vol: 3246, usage: 24, startDate: Mon Jul 25 00:00:00 BRT 2022, endDate: Mon Jul 25 23:59:59 BRT 2022, for account: 545
> 2022-07-26 00:18:17,484 DEBUG [usage.parser.VolumeUsageParser] (Usage-Job-1:null) (logid:) Total running time 86400000ms
> 2022-07-26 00:18:17,484 DEBUG [usage.parser.VolumeUsageParser] (Usage-Job-1:null) (logid:) Creating Volume usage record for vol: 3246, usage: 24, startDate: Mon Jul 25 00:00:00 BRT 2022, endDate: Mon Jul 25 23:59:59 BRT 2022, for account: 545
>
>
> And it are being computed to account:
> (ascenty) # > list usagerecords domainid=XXXXXXXXXXXXXXXXXXX accountid=d14c8cb9-fd92-43c8-9ddb-f2ed4e9af5a8 type=6 startdate=2022-07-25 enddate=2022-07-25 filter=account,rawusage,size,startdate,usage,usagetype,
> {
> "count": 2,
> "usagerecord": [
> {
> "accountid": "d14c8cb9-fd92-43c8-9ddb-f2ed4e9af5a8",
> "rawusage": "24",
> "size": 53687091200,
> "startdate": "2022-07-25'T'00:00:00-03:00",
> "usage": "24 Hrs",
> "usagetype": 6
> },
> {
> "accountid": "d14c8cb9-fd92-43c8-9ddb-f2ed4e9af5a8",
> "rawusage": "24",
> "size": 53687091200,
> "startdate": "2022-07-25'T'00:00:00-03:00",
> "usage": "24 Hrs",
> "usagetype": 6
> }
> ]
> }
Re: Usage computing removed items
Posted by Jorge Luiz Correa <jo...@embrapa.br.INVALID>.
Same here, exactly the same problem with volumes.
I've looked in github source for the SQL query that generates the
aggregated registries to try to understand but I couldn't find.
:/
Appreciate any help too.
Em ter., 26 de jul. de 2022 17:31, Matheus Fontes <ma...@ghz.com.br>
escreveu:
> Hi,
> Is anyone having problems with usage computing removed items?
> We have a user that reported the problem.
> The volume was deleted since 2022-05-06
>
> mysql> select id,account_id,created,removed,state from volumes where
> id=3246;
>
> +------+------------+---------------------+---------------------+----------+
> | id | account_id | created | removed | state
> |
>
> +------+------------+---------------------+---------------------+----------+
> | 3246 | 545 | 2021-04-08 17:03:16 | 2022-05-06 15:06:52 | Expunged
> |
>
> +------+------------+---------------------+---------------------+----------+
>
>
> In usage process we can see it on parsing volume call:
> 2022-07-26 00:18:17,471 DEBUG [usage.parser.VolumeUsageParser]
> (Usage-Job-1:null) (logid:) Parsing all Volume usage events for account: 545
> 2022-07-26 00:18:17,472 DEBUG [usage.parser.VolumeUsageParser]
> (Usage-Job-1:null) (logid:) Total running time 86400000ms
> 2022-07-26 00:18:17,472 DEBUG [usage.parser.VolumeUsageParser]
> (Usage-Job-1:null) (logid:) Creating Volume usage record for vol: 3246,
> usage: 24, startDate: Mon Jul 25 00:00:00 BRT 2022, endDate: Mon Jul 25
> 23:59:59 BRT 2022, for account: 545
> 2022-07-26 00:18:17,484 DEBUG [usage.parser.VolumeUsageParser]
> (Usage-Job-1:null) (logid:) Total running time 86400000ms
> 2022-07-26 00:18:17,484 DEBUG [usage.parser.VolumeUsageParser]
> (Usage-Job-1:null) (logid:) Creating Volume usage record for vol: 3246,
> usage: 24, startDate: Mon Jul 25 00:00:00 BRT 2022, endDate: Mon Jul 25
> 23:59:59 BRT 2022, for account: 545
>
>
> And it are being computed to account:
> (ascenty) # > list usagerecords domainid=XXXXXXXXXXXXXXXXXXX
> accountid=d14c8cb9-fd92-43c8-9ddb-f2ed4e9af5a8 type=6 startdate=2022-07-25
> enddate=2022-07-25 filter=account,rawusage,size,startdate,usage,usagetype,
> {
> "count": 2,
> "usagerecord": [
> {
> "accountid": "d14c8cb9-fd92-43c8-9ddb-f2ed4e9af5a8",
> "rawusage": "24",
> "size": 53687091200,
> "startdate": "2022-07-25'T'00:00:00-03:00",
> "usage": "24 Hrs",
> "usagetype": 6
> },
> {
> "accountid": "d14c8cb9-fd92-43c8-9ddb-f2ed4e9af5a8",
> "rawusage": "24",
> "size": 53687091200,
> "startdate": "2022-07-25'T'00:00:00-03:00",
> "usage": "24 Hrs",
> "usagetype": 6
> }
> ]
> }
--
__________________________
Aviso de confidencialidade
Esta mensagem da
Empresa Brasileira de Pesquisa Agropecuaria (Embrapa), empresa publica
federal regida pelo disposto na Lei Federal no. 5.851, de 7 de dezembro
de 1972, e enviada exclusivamente a seu destinatario e pode conter
informacoes confidenciais, protegidas por sigilo profissional. Sua
utilizacao desautorizada e ilegal e sujeita o infrator as penas da lei.
Se voce a recebeu indevidamente, queira, por gentileza, reenvia-la ao
emitente, esclarecendo o equivoco.
Confidentiality note
This message from
Empresa Brasileira de Pesquisa Agropecuaria (Embrapa), a government
company established under Brazilian law (5.851/72), is directed
exclusively to its addressee and may contain confidential data,
protected under professional secrecy rules. Its unauthorized use is
illegal and may subject the transgressor to the law's penalties. If you
are not the addressee, please send it back, elucidating the failure.