You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by GitBox <gi...@apache.org> on 2021/08/16 07:56:10 UTC
[GitHub] [cloudstack] borisstoyanov opened a new issue #5316: paged listServiceOffering response returns duplicated records in different pages
borisstoyanov opened a new issue #5316:
URL: https://github.com/apache/cloudstack/issues/5316
<!--
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release and main branch are affected too.
Always add information AFTER of these HTML comments, but no need to delete the comments.
-->
##### ISSUE TYPE
<!-- Pick one below and delete the rest -->
* Bug Report
##### COMPONENT NAME
<!--
Categorize the issue, e.g. API, VR, VPN, UI, etc.
-->
~~~
API
~~~
##### CLOUDSTACK VERSION
<!--
New line separated list of affected versions, commit ID for issues on main branch.
-->
~~~
Main and older
~~~
##### SUMMARY
<!-- Explain the problem/feature briefly -->
Create a large number of offerings, in my case about 250. list the offerings and divide the response into 3 pages of 100
While comparing the records from page1 and 2 I was able to find an id on the both pages. Checkout the attached file.
##### STEPS TO REPRODUCE
<!--
For bugs, show exactly how to reproduce the problem, using a minimal test-case. Use Screenshots if accurate.
For new features, show how the feature would be used.
-->
<!-- Paste example playbooks or commands between quotes below -->
~~~
See attachment for details
~~~
<!-- You can also paste gist.github.com links for larger files -->
##### EXPECTED RESULTS
<!-- What did you expect to happen when running the steps above? -->
~~~
Pages should contain unique ids
~~~
##### ACTUAL RESULTS
<!-- What actually happened? -->
<!-- Paste verbatim command output between quotes below -->
~~~
pages contained a duplicated id
~~~
[compare paged api response.txt](https://github.com/apache/cloudstack/files/6990673/compare.paged.api.response.txt)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] shwstppr commented on issue #5316: paged listServiceOffering response returns duplicated records in different pages
Posted by GitBox <gi...@apache.org>.
shwstppr commented on issue #5316:
URL: https://github.com/apache/cloudstack/issues/5316#issuecomment-899520598
@weizhouapache @nvazquez
By default value for `sort_key` field remains `0`. Maybe @borisstoyanov can confirm but this issue is seen with older versions as well 4.9 probably.
I've created a PR to use `id` field as secondary sort criteria which fixes the issue in my tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd closed issue #5316: paged listServiceOffering response returns duplicated records in different pages
Posted by GitBox <gi...@apache.org>.
rhtyd closed issue #5316:
URL: https://github.com/apache/cloudstack/issues/5316
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] weizhouapache commented on issue #5316: paged listServiceOffering response returns duplicated records in different pages
Posted by GitBox <gi...@apache.org>.
weizhouapache commented on issue #5316:
URL: https://github.com/apache/cloudstack/issues/5316#issuecomment-899500663
@shwstppr @borisstoyanov
is it similar as #3934 which has been fixed by #4494 ?
try
```
UPDATE `cloud`.`disk_offering` JOIN (SELECT COUNT(1) AS count FROM `cloud`.`disk_offering` WHERE `sort_key` != 0) AS tbl_tmp SET `sort_key` = `id` WHERE count = 0;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] nvazquez commented on issue #5316: paged listServiceOffering response returns duplicated records in different pages
Posted by GitBox <gi...@apache.org>.
nvazquez commented on issue #5316:
URL: https://github.com/apache/cloudstack/issues/5316#issuecomment-899450325
Thanks for testing @shwstppr, I checked the code for listDiskOfferings and the `QueryManager` is simply obtaining data from the DAO layer using a search criteria and a filter based on the start index and the page size value. Perhaps the filter can be replaced by this method: `StringUtils.applyPagination`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd closed issue #5316: paged listServiceOffering response returns duplicated records in different pages
Posted by GitBox <gi...@apache.org>.
rhtyd closed issue #5316:
URL: https://github.com/apache/cloudstack/issues/5316
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] weizhouapache commented on issue #5316: paged listServiceOffering response returns duplicated records in different pages
Posted by GitBox <gi...@apache.org>.
weizhouapache commented on issue #5316:
URL: https://github.com/apache/cloudstack/issues/5316#issuecomment-899523230
> @weizhouapache @nvazquez
> By default value for `sort_key` field remains `0`. Maybe @borisstoyanov can confirm but this issue is seen with older versions as well 4.9 probably.
> I've created a PR to use `id` field as secondary sort criteria which fixes the issue in my tests.
@shwstppr sounds like a better idea.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] shwstppr commented on issue #5316: paged listServiceOffering response returns duplicated records in different pages
Posted by GitBox <gi...@apache.org>.
shwstppr commented on issue #5316:
URL: https://github.com/apache/cloudstack/issues/5316#issuecomment-899383497
I am able to reproduce this with a Simulator env for both compute and disk offerings. Not sure if the problem happens with other resources as well.
I created 1000 compute and disk offerings. Tried listing pages for disk offering with `pagesize=500`. It resulted in many duplicates. Logs for disk offerings:
[diskoffering.log](https://github.com/apache/cloudstack/files/6991419/diskoffering.log)
While setting global config `default.page.size` to 1200 and listing 1000 items at once there were no duplicates,
```
shwstppr@shwstppr-ThinkPad:~/Desktop/temp|
⇒ cmk list diskofferings filter=id page=1 pagesize=1000 > ./dpage3
shwstppr@shwstppr-ThinkPad:~/Desktop/temp|
⇒ sort ./dpage3 | uniq -d
+--------------------------------------+
```
@rhtyd @weizhouapache @nvazquez could this be a caching issue with ApiServer?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] weizhouapache edited a comment on issue #5316: paged listServiceOffering response returns duplicated records in different pages
Posted by GitBox <gi...@apache.org>.
weizhouapache edited a comment on issue #5316:
URL: https://github.com/apache/cloudstack/issues/5316#issuecomment-899500663
@shwstppr @borisstoyanov
is it similar as #3934 which has been fixed by #4494 ?
try ?
```
UPDATE `cloud`.`disk_offering` JOIN (SELECT COUNT(1) AS count FROM `cloud`.`disk_offering` WHERE `sort_key` != 0) AS tbl_tmp SET `sort_key` = `id` WHERE count = 0;
```
what should the default value of sort_key be, 0 or id ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org