You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by "winterhazel (via GitHub)" <gi...@apache.org> on 2024/01/23 19:44:17 UTC
Re: [PR] Normalize dates in Usage and Quota APIs [cloudstack]
winterhazel commented on PR #8243:
URL: https://github.com/apache/cloudstack/pull/8243#issuecomment-1906806312
Hey @DaanHoogland, I tried to replicate the scenario you described here, with a MS in Singapore (MS and `usage.execution.timezone` set to GMT+8) and a DC in Australia (`usage.aggregation.timezone` set to GMT+11), and did some tests for this PR. Everything seems to be working as intended; in fact, considering dates as they were informed even fixes some issues in this scenario. Below are my results.
Some problems I have identified which this PR fixes:
<details>
<summary>
Unable to add credits with quotaCredits
</summary>
**Before the changes:**
The time in Singapore was `2024-01-23T00:22:55+0800`. I tried to execute `quotaCredits` to add credits to an account:
```bash
(admin) 🐱 > quota credits account=admin domainid=72b54c4a-771d-11ee-8e59-5254003754dc value=5
🙈 Error: (HTTP 431, error code 4350) Incorrect deposit date: Mon Jan 22 21:22:55 SGT 2024 there are balance entries after this date
```
But the MS would not let me add credits to any account, since it incorrectly calculated the current date (deposit date) as being 3 hours before the actual date, and therefore detected balance entries after it.
**After:**
The time in Singapore was `2024-01-19T21:26:06+0800`. I executed the same command:
```bash
(admin) 🐱 > quota credits account=admin domainid=72b54c4a-771d-11ee-8e59-5254003754dc value=5
{
"quotacredits": {
"credits": 5,
"currency": "$",
"updated_by": "admin",
"updated_on": "2024-01-19T21:26:06+0800"
}
}
```
And was able to successfully add credits to the account.
`updated_on` also had the expected value. Just to ensure that the date was indeed correct, I verified it through `quotaBalance`.
```bash
(admin) 🐱 > quota balance account=admin domainid=72b54c4a-771d-11ee-8e59-5254003754dc startdate=2024-01-18 enddate=2024-01-19
{
"balance": {
"credits": [
{
"credits": 5,
"updated_on": "2024-01-19T21:26:06+0800"
}
],
"currency": "$",
"enddate": "2024-01-19T23:59:59+0800",
"endquota": 5,
"startdate": "2024-01-18T00:00:00+0800",
"startquota": 0
}
}
```
</details>
<details>
<summary>
listUsageRecords does not retrieve the requested data
</summary>
**Before the changes:**
The current time in Australia was `2023-01-23T03:11:26+1100`. I executed the command:
```bash
(admin) 🐱 > list usagerecords startdate='2024-01-22T19:00:00+1100' enddate='2024-01-23T04:00:00+1100'
```
```bash
2024-01-23 00:11:26,343 DEBUG [c.c.u.UsageServiceImpl] (qtp391630194-23:ctx-acf58df9 ctx-d3f654e9) (logid:ba781cb9) getting usage records for account: 2, domainId: null, between Mon Jan 22 13:00:00 SGT 2024 and Mon Jan 22 22:00:00 SGT 2024, using pageSize: 500 and startIndex: 0
```
MS queries for usage records between `2024-01-22T16:00:00+1100` and `2024-01-23T01:00:00+1100`, which is not the provided range. I also verified that the last returned usage record had the `startdate` of `2024-01-23T00:00:00+1100`.
**After the changes:**
The current time in Australia was `2023-01-23T05:10:21+1100`. I executed the following command:
```bash
(admin) 🐱 > list usagerecords startdate='2024-01-22T19:00:00+1100' enddate='2024-01-23T07:59:00+1100'
```
```bash
2024-01-23 02:10:21,078 DEBUG [c.c.u.UsageServiceImpl] (qtp391630194-19:ctx-9e97b4a7 ctx-395a7f59) (logid:38101bda) Getting usage records for account [2] in domain [null], between [Mon Jan 22 16:00:00 SGT 2024] and [Tue Jan 23 04:59:00 SGT 2024], using pageSize [500] and startIndex [0].
```
MS queries for usage records between `2024-01-22T19:00:00+1100` and `2024-01-23T07:59:00+1100`, which is the expected range. I also verified that the last returned usage record had the `startdate` of `2024-01-23T04:00:00+1100` and the `enddate` of `2024-01-23T04:59:59+1100`.
</details>
<details>
<summary>
Wrong dates in quotaTariffCreate
</summary>
**Before:**
```bash
(admin) 🐱 > quota tariffcreate name="testA" value=2 usagetype=1 startdate="2024-01-24T01:00:00+1100"
{
"quotatariff": {
"currency": "$",
"effectiveDate": "2024-01-23T19:00:00+0800",
"name": "testA",
"tariffValue": 2,
"usageDiscriminator": "None",
"usageName": "RUNNING_VM",
"usageType": 1,
"usageTypeDescription": "Running Vm Usage",
"usageUnit": "Compute*Month",
"uuid": "b3074921-fd3d-46bd-a5e6-19c445c9a7d8"
}
}
```
When creating a Quota tariff, `effectiveDate` does not get set to the provided `startDate`.
```bash
(admin) 🐱 > quota tariffcreate name="test2A" value=2 usagetype=1 startdate="2024-01-25T01:00:00+1100" enddate="2024-01-25T04:00:00+1100"
{
"quotatariff": {
"currency": "$",
"effectiveDate": "2024-01-24T19:00:00+0800",
"endDate": "2024-01-24T22:00:00+0800",
"name": "test2A",
"tariffValue": 2,
"usageDiscriminator": "None",
"usageName": "RUNNING_VM",
"usageType": 1,
"usageTypeDescription": "Running Vm Usage",
"usageUnit": "Compute*Month",
"uuid": "c22ac24b-45ea-4ba2-8dc2-533592ac46ca"
}
}
```
Both `effectiveDate` and `endDate` do not get set to the expected value.
Just to ensure that this is not a problem only with the API's response, I verified that they are indeed wrong in the database:
```bash
+----+------------------------+---------------------+---------------------+
| id | name | effective_on | end_date |
+----+------------------------+---------------------+---------------------+
| 26 | testA | 2024-01-23 11:00:00 | NULL |
| 27 | test2A | 2024-01-24 11:00:00 | 2024-01-24 14:00:00 |
+----+------------------------+---------------------+---------------------+
```
**After:**
```bash
(admin) 🐱 > quota tariffcreate name="testB" value=2 usagetype=1 startdate="2024-01-24T01:00:00+1100"
{
"quotatariff": {
"currency": "$",
"effectiveDate": "2024-01-23T22:00:00+0800",
"id": "201b2163-8194-48ff-a048-c36453677538",
"name": "testB",
"tariffValue": 2,
"usageDiscriminator": "None",
"usageName": "RUNNING_VM",
"usageType": 1,
"usageTypeDescription": "Running Vm Usage",
"usageUnit": "Compute*Month"
}
}
```
`effectiveDate` gets set to the correct value.
```bash
(admin) 🐱 > quota tariffcreate name="test2B" value=2 usagetype=1 startdate="2024-01-25T01:00:00+1100" enddate="2024-01-25T04:00:00+1100"
{
"quotatariff": {
"currency": "$",
"effectiveDate": "2024-01-24T22:00:00+0800",
"endDate": "2024-01-25T01:00:00+0800",
"id": "6f7f9250-425b-493d-8cc0-7b5158d73ea4",
"name": "test2B",
"tariffValue": 2,
"usageDiscriminator": "None",
"usageName": "RUNNING_VM",
"usageType": 1,
"usageTypeDescription": "Running Vm Usage",
"usageUnit": "Compute*Month"
}
}
```
Both `effectiveDate` and `endDate` get set to the provided values.
In the database:
```bash
+----+------------------------+---------------------+---------------------+
| id | name | effective_on | end_date |
+----+------------------------+---------------------+---------------------+
| 30 | testB | 2024-01-23 14:00:00 | NULL |
| 31 | test2B | 2024-01-24 14:00:00 | 2024-01-24 17:00:00 |
+----+------------------------+---------------------+---------------------+
```
I also verified that this issue was present in quotaTariffUpdate, and that it has been fixed.
</details>
Other tests:
<details>
<summary>
listUsageRecords
</summary>
```bash
(admin) 🐱 > list usagerecords startdate='2024-01-22 19:00:00' enddate='2024-01-23 07:59:00'
...
```
```bash
2024-01-24 00:07:06,185 DEBUG [c.c.u.UsageServiceImpl] (qtp391630194-15:ctx-49d437aa ctx-40c137c4) (logid:a6160874) Getting usage records for account [2] in domain [null], between [Mon Jan 22 19:00:00 SGT 2024] and [Tue Jan 23 07:59:00 SGT 2024], using pageSize [500] and startIndex [0].
```
When not adding a timezone, `startdate` and `enddate` are considered in the MS's timezone. I verified that the last usage record had the `startdate` of `2024-01-23T06:00:00+0800` and `enddate` of `2024-01-23T06:59:59+0800`.
</details>
<details>
<summary>
quotaStatement
</summary>
```bash
(admin) 🐱 > quota statement account=admin domainid=72b54c4a-771d-11ee-8e59-5254003754dc startdate=2024-01-19T13:00:00+1100 enddate=2024-01-20
{
"statement": {
"currency": "$",
"enddate": "2024-01-20T23:59:59+0800",
"quotausage": [
{
"accountid": 2,
"domain": 1,
"name": "RUNNING_VM",
"quota": 0,
"type": 1,
"unit": "Compute*Month"
},
{
"accountid": 2,
"domain": 1,
"name": "ALLOCATED_VM",
"quota": 0,
"type": 2,
"unit": "Compute*Month"
},
{
"accountid": 2,
"domain": 1,
"name": "IP_ADDRESS",
"quota": 0,
"type": 3,
"unit": "IP*Month"
},
{
"accountid": 2,
"domain": 1,
"name": "NETWORK_BYTES_SENT",
"quota": 0,
"type": 4,
"unit": "GB"
},
{
"accountid": 2,
"domain": 1,
"name": "NETWORK_BYTES_RECEIVED",
"quota": 0,
"type": 5,
"unit": "GB"
},
{
"accountid": 2,
"domain": 1,
"name": "VOLUME",
"quota": 0,
"type": 6,
"unit": "GB*Month"
},
{
"accountid": 2,
"domain": 1,
"name": "TEMPLATE",
"quota": 0,
"type": 7,
"unit": "GB*Month"
},
{
"accountid": 2,
"domain": 1,
"name": "ISO",
"quota": 0,
"type": 8,
"unit": "GB*Month"
},
{
"accountid": 2,
"domain": 1,
"name": "SNAPSHOT",
"quota": 0,
"type": 9,
"unit": "GB*Month"
},
{
"accountid": 2,
"domain": 1,
"name": "SECURITY_GROUP",
"quota": 0,
"type": 10,
"unit": "Policy*Month"
},
{
"accountid": 2,
"domain": 1,
"name": "LOAD_BALANCER_POLICY",
"quota": 0,
"type": 11,
"unit": "Policy*Month"
},
{
"accountid": 2,
"domain": 1,
"name": "PORT_FORWARDING_RULE",
"quota": 0,
"type": 12,
"unit": "Policy*Month"
},
{
"accountid": 2,
"domain": 1,
"name": "NETWORK_OFFERING",
"quota": 0,
"type": 13,
"unit": "Policy*Month"
},
{
"accountid": 2,
"domain": 1,
"name": "VPN_USERS",
"quota": 0,
"type": 14,
"unit": "Policy*Month"
},
{
"accountid": 2,
"domain": 1,
"name": "VM_DISK_IO_READ",
"quota": 0,
"type": 21,
"unit": "IOPS"
},
{
"accountid": 2,
"domain": 1,
"name": "VM_DISK_IO_WRITE",
"quota": 0,
"type": 22,
"unit": "IOPS"
},
{
"accountid": 2,
"domain": 1,
"name": "VM_DISK_BYTES_READ",
"quota": 0,
"type": 23,
"unit": "Bytes"
},
{
"accountid": 2,
"domain": 1,
"name": "VM_DISK_BYTES_WRITE",
"quota": 0,
"type": 24,
"unit": "Bytes"
},
{
"accountid": 2,
"domain": 1,
"name": "VM_SNAPSHOT",
"quota": 0,
"type": 25,
"unit": "GB*Month"
},
{
"accountid": 2,
"domain": 1,
"name": "VOLUME_SECONDARY",
"quota": 0,
"type": 26,
"unit": "GB*Month"
},
{
"accountid": 2,
"domain": 1,
"name": "VM_SNAPSHOT_ON_PRIMARY",
"quota": 0,
"type": 27,
"unit": "GB*Month"
},
{
"accountid": 2,
"domain": 1,
"name": "BACKUP",
"quota": 0,
"type": 28,
"unit": "GB*Month"
}
],
"startdate": "2024-01-19T10:00:00+0800",
"totalquota": 0
}
}
```
`startdate` and `enddate` use the provided values (`enddate` gets set to the end of a day according to the MS's timezone).
</details>
<details>
<summary>
quotaTariffCreate
</summary>
```bash
(admin) 🐱 > quota tariffcreate name="testtimezonenotprovided" value=2 usagetype=1 startdate="2024-01-24 10:00:00" enddate="2024-01-25 12:00:00"
{
"quotatariff": {
"currency": "$",
"effectiveDate": "2024-01-24T10:00:00+0800",
"endDate": "2024-01-25T12:00:00+0800",
"id": "ef836a1d-958a-4113-a9da-3121986628a4",
"name": "testtimezonenotprovided",
"tariffValue": 2,
"usageDiscriminator": "None",
"usageName": "RUNNING_VM",
"usageType": 1,
"usageTypeDescription": "Running Vm Usage",
"usageUnit": "Compute*Month"
}
}
```
The tariff is created with the provided dates in the MS's timezone.
</details>
<details>
<summary>
quotaTariffUpdate
</summary>
```bash
(admin) 🐱 > quota tariffupdate name=testB enddate=2024-01-26
{
"quotatariff": {
"currency": "$",
"effectiveDate": "2024-01-23T22:00:00+0800",
"endDate": "2024-01-26T23:59:59+0800",
"id": "7fc5f42a-bc0e-4848-89d7-35eacc64504d",
"name": "testB",
"tariffValue": 2,
"usageDiscriminator": "None",
"usageName": "RUNNING_VM",
"usageType": 1,
"usageTypeDescription": "Running Vm Usage",
"usageUnit": "Compute*Month"
}
}
(admin) 🐱 > quota tariffupdate name=testB enddate=2024-01-26T23:59:59+1100
{
"quotatariff": {
"currency": "$",
"effectiveDate": "2024-01-23T22:00:00+0800",
"endDate": "2024-01-26T20:59:59+0800",
"id": "a2e323b7-10b5-4018-aa9d-e50bbee079ff",
"name": "testB",
"tariffValue": 2,
"usageDiscriminator": "None",
"usageName": "RUNNING_VM",
"usageType": 1,
"usageTypeDescription": "Running Vm Usage",
"usageUnit": "Compute*Month"
}
}
```
Dates are updated according to the provided values.
</details>
<details>
<summary>
quotaTariffList
</summary>
```bash
(admin) 🐱 > quota tarifflist startdate=2024-01-25 enddate=2024-01-26
{
"count": 1,
"quotatariff": [
{
"currency": "$",
"effectiveDate": "2024-01-25T00:00:00+0800",
"endDate": "2024-01-25T23:59:59+0800",
"id": "473451f8-b895-49d5-801c-c4eeea242ec0",
"name": "test3B",
"tariffValue": 2,
"usageDiscriminator": "None",
"usageName": "RUNNING_VM",
"usageType": 1,
"usageTypeDescription": "Running Vm Usage",
"usageUnit": "Compute*Month"
}
]
}
(admin) 🐱 > quota tarifflist startdate=2024-01-25 enddate=2024-01-25T23:59:59+0800
{
"count": 1,
"quotatariff": [
{
"currency": "$",
"effectiveDate": "2024-01-25T00:00:00+0800",
"endDate": "2024-01-25T23:59:59+0800",
"id": "473451f8-b895-49d5-801c-c4eeea242ec0",
"name": "test3B",
"tariffValue": 2,
"usageDiscriminator": "None",
"usageName": "RUNNING_VM",
"usageType": 1,
"usageTypeDescription": "Running Vm Usage",
"usageUnit": "Compute*Month"
}
]
}
(admin) 🐱 > quota tarifflist startdate=2024-01-25 enddate=2024-01-25T23:59:58+0800
<nothing>
(admin) 🐱 > quota tarifflist startdate=2024-01-24T01:00:00+1100 enddate=2024-01-26
{
"count": 3,
"quotatariff": [
{
"currency": "$",
"effectiveDate": "2024-01-25T00:00:00+0800",
"endDate": "2024-01-25T23:59:59+0800",
"id": "473451f8-b895-49d5-801c-c4eeea242ec0",
"name": "test3B",
"tariffValue": 2,
"usageDiscriminator": "None",
"usageName": "RUNNING_VM",
"usageType": 1,
"usageTypeDescription": "Running Vm Usage",
"usageUnit": "Compute*Month"
},
{
"currency": "$",
"effectiveDate": "2024-01-24T22:00:00+0800",
"endDate": "2024-01-25T01:00:00+0800",
"id": "6f7f9250-425b-493d-8cc0-7b5158d73ea4",
"name": "test2B",
"tariffValue": 2,
"usageDiscriminator": "None",
"usageName": "RUNNING_VM",
"usageType": 1,
"usageTypeDescription": "Running Vm Usage",
"usageUnit": "Compute*Month"
},
{
"currency": "$",
"effectiveDate": "2024-01-23T22:00:00+0800",
"endDate": "2024-01-26T23:59:59+0800",
"id": "aa615bcb-a9af-4b2c-8806-6711b8c13623",
"name": "testB",
"tariffValue": 2,
"usageDiscriminator": "None",
"usageName": "RUNNING_VM",
"usageType": 1,
"usageTypeDescription": "Running Vm Usage",
"usageUnit": "Compute*Month"
}
]
}
(admin) 🐱 > quota tarifflist startdate=2024-01-24T02:00:00+1100 enddate=2024-01-26
{
"count": 2,
"quotatariff": [
{
"currency": "$",
"effectiveDate": "2024-01-25T00:00:00+0800",
"endDate": "2024-01-25T23:59:59+0800",
"id": "473451f8-b895-49d5-801c-c4eeea242ec0",
"name": "test3B",
"tariffValue": 2,
"usageDiscriminator": "None",
"usageName": "RUNNING_VM",
"usageType": 1,
"usageTypeDescription": "Running Vm Usage",
"usageUnit": "Compute*Month"
},
{
"currency": "$",
"effectiveDate": "2024-01-24T22:00:00+0800",
"endDate": "2024-01-25T01:00:00+0800",
"id": "6f7f9250-425b-493d-8cc0-7b5158d73ea4",
"name": "test2B",
"tariffValue": 2,
"usageDiscriminator": "None",
"usageName": "RUNNING_VM",
"usageType": 1,
"usageTypeDescription": "Running Vm Usage",
"usageUnit": "Compute*Month"
}
]
}
```
Tariffs are filtered by `startdate` and `enddate` as expected.
</details>
<details>
<summary>
quotaTariffDelete
</summary>
```bash
(admin) 🐱 > quota tariffdelete id=ef836a1d-958a-4113-a9da-3121986628a4
{
"success": true
}
```
Working as expected.
</details>
---
Could someone else also take a look at this PR? @shwstppr @rohityadavcloud
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org