[jira] Commented: (CAY-999) Scaling paginated list

["Ari Maniatis (JIRA)" <de...@cayenne.apache.org>]

    [ https://issues.apache.org/cayenne/browse/CAY-999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12764#action_12764 ] 

Ari Maniatis commented on CAY-999:
----------------------------------

1. Very useful.
2. Sounds like it could get complicated fast. Algorithms would be similar to virtual memory paging within an operating system (wikipaedia page with good links here: http://en.wikipedia.org/wiki/Page_replacement_algorithm). The danger is that performance regressions could be easily possible under certain workloads. Maybe OSCache has already solved some of these problems.

> Scaling paginated list
> ----------------------
>
>                 Key: CAY-999
>                 URL: https://issues.apache.org/cayenne/browse/CAY-999
>             Project: Cayenne
>          Issue Type: Improvement
>          Components: Cayenne Core Library
>    Affects Versions: 3.0
>            Reporter: Andrus Adamchik
>            Assignee: Andrus Adamchik
>
> An idea for scaling IncrementalFaultList to store massive amount of objects, like hundreds of thousands.This pertains to the server-side IncrementalFaultList. The problems to solve are the speed of the initial list initialization and overall memory use.
> 1. Simplify ID representation:
> Even unresolved lists can take significant amount of memory... Each unresolved object slot currently stores a DataRow with N number of entries, where N is the number of PK columns for the entity. I.e. most often than not - 1 entry. Here is a memory use calculation for various representations of an unresolved entry, based on a single int PK DbEntity.
> a. DataRow - 120 bytes, 
> b. HashMap - 104 bytes,
> c. Object[] - 32 bytes,
> d java.lang.Integer - 16 bytes
> [primitive int is even better, but it complicates the implementation, as we'd need a parallel int[]  (long[], double[], etc.) , so all in all we may get no gain]
> 2. Swap out LRU pages
> For the very large lists, it would make sense to un-fault resolved pages when more pages are resolved , so that the list size doesn't grow beyond a certain fixed amount, no matter how many pages are resolved. These parameters will have to be configurable per query, as some users would prefer to keep the entire thing...

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.