[GitHub] [lucene-solr] bruno-roustant commented on issue #1395: SOLR-14365: CollapsingQParser - Avoiding always allocate int[] and float[] with size equals to number of unique values (WIP)

[GitBox <gi...@apache.org>]

bruno-roustant commented on issue #1395: SOLR-14365: CollapsingQParser - Avoiding always allocate int[] and float[] with size equals to number of unique values (WIP)
URL: https://github.com/apache/lucene-solr/pull/1395#issuecomment-610481938
 
 
   > so in total we pay 2^22 + 2^23 bytes
   Why are sizes before and after rehash added?
   
   I take an example of a max number of elements of 1 M known ahead.
   The array size will be exactly 1M, so memory = 1M * element size B.
   
   For a load of 1.5%, the map memory = 2^16 * element size B.
   (nextPowerOf2(15000/0.75)=2^15)
   So I get a ratio of array mem / map mem = 15.25
   
   For a load of 15%, the map memory = 2^19 * element size.
   (nextPowerOf2(150000/0.75)=2^18)
   So I get a ratio of array mem / map mem = 1.9
   
   For a load of 20%, the map memory = 2^20 * element size.
   (nextPowerOf2(200000/0.75)=2^19)
   So I get a ratio of array mem / map mem = 0.95
   
   Above 20% load, the array memory is less than the map memory.
   So if we want to focus on the memory benefit, we can set a threshold at 20% load. If we want the performance we can set the threshold lower.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org