[GitHub] [incubator-pegasus] hycdong opened a new issue #808: Multiget and scan iteration count bug

[GitBox <gi...@apache.org>]

hycdong opened a new issue #808:
URL: https://github.com/apache/incubator-pegasus/issues/808


   ## Feature enhancemant and bug report
   As https://github.com/apache/incubator-pegasus/issues/486 shows, `Pegasus version 1.12.3` adds iteration count and iteration time check while calling rocksdb range read. 
   
   For example, client use multiget interface and set `max_kv_count` option to control the expected value count this round would like to get. If this round don't get completed data, server will return false and the last read key-value pair, the next round can specify that key to start next round. 
   
   ### Case1. When total_count < max_kv_count <= max_iteration_count
   > Assuming: `total_count = 50`, `max_kv_count=100`, server `max_iteration_count=3000`
   1. [1, 50] value are expired -> can't get any value, return true
   2. [1, 20] value are expired, [21, 50] is valid -> get value [21, 50], count = 30, return true
   3. [1, 50] value are all valid -> get value [1, 50], count = 50, return true
   
   ### Case2. When total_count > max_kv_count >= max_iteration
   > Assuming: `total_count = 4000`, `max_kv_count=3500`, server `max_iteration_count=3000`
   1. [1, 3000] value are all expired, [3001, 4000] are valid -> can't get any value, return false
   2. [1, 500] value are expired, [501, 4000] are valid -> get value [501,3500], count = 3000, return false
   3. [1, 4000] value are all valid -> get value [1, 3000], count = 3000, return false
   
   ### Case3. When the total_count > max_iteration_count > max_kv_count
   > Assuming: `total_count = 4000`, `max_kv_count=100`, server `max_iteration_count=3000`
   1. [1, 3000] value are all expired, [3001, 4000] are valid -> can't get any value, return false
   2. **[1, 2950] value are expired, [2951, 4000] are valid -> can't get any value, return false**
   3. **[1, 100] value are all expired, [101, 4000] are valid -> can't get any value, return false**
   4. **[1, 20] value are expired, [21, 4000] is valid -> get value [21,100], count = 80, return false**
   5. [1, 4000] value are all valid -> get value [1, 100], count = 100, return false
   
   However, the **Bold** cases above don't meet the client expectation, the expected result are:
   - [1, 2950] value are expired, [2951, 4000] are valid -> value[2951, 3000], count = 50, return false
   - [1, 100] value are all expired, [101, 4000] are valid -> value [101, 200], count = 100, return false
   - [1, 20] value are expired, [21, 4000] is valid -> value[21, 120], count = 100, return false
   
   In particular, for case2-1 and case3-1, if too many value expired or filtered (exceed the max_iteration_count), server **SHOULD NOT** iterate more to get a valid value.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org

[GitHub] [incubator-pegasus] hycdong closed issue #808: Multiget and scan iteration count bug

[GitBox <gi...@apache.org>]

hycdong closed issue #808:
URL: https://github.com/apache/incubator-pegasus/issues/808


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org