You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cayenne.apache.org by "Andrus Adamchik (JIRA)" <de...@cayenne.apache.org> on 2008/03/06 14:15:17 UTC
[jira] Updated: (CAY-999) Scaling paginated list
[ https://issues.apache.org/cayenne/browse/CAY-999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrus Adamchik updated CAY-999:
--------------------------------
Description:
An idea for scaling IncrementalFaultList to store massive amount of objects, like hundreds of thousands.This pertains to the server-side IncrementalFaultList. The problems to solve are the speed of the initial list initialization and overall memory use.
1. Simplify ID representation:
Even unresolved lists can take significant amount of memory... Each unresolved object slot currently stores a DataRow with N number of entries, where N is the number of PK columns for the entity. I.e. most often than not - 1 entry. Here is a memory use calculation for various representations of an unresolved entry, based on a single int PK DbEntity.
a. DataRow - 120 bytes,
b. HashMap - 104 bytes,
c. Object[] - 32 bytes,
d java.lang.Integer - 16 bytes
[primitive int is even better, but it complicates the implementation, as we'd need a parallel int[] (long[], double[], etc.) , so all in all we may get no gain]
was:
An idea for scaling IncrementalFaultList to store massive amount of objects, like hundreds of thousands.This pertains to the server-side IncrementalFaultList. The problems to solve are the speed of the initial list initialization and overall memory use.
1. Simplify ID representation:
Even unresolved lists can take significant amount of memory... Each unresolved object slot currently stores a DataRow with N number of entries, where N is the number of PK columns for the entity. I.e. most often than not - 1 entry. Here is a memory use calculation for various representations of an unresolved entry, based on a single int PK DbEntity.
a. DataRow - 120 bytes,
b. HashMap - 104 bytes,
c. Object[] - 32 bytes,
d java.lang.Integer - 16 bytes
[primitive int is even better, but it complicates the implementation, as we'd need a parallel int[] (long[], double[], etc.) , so all in all we may get no gain]
2. Swap out LRU pages
For the very large lists, it would make sense to un-fault resolved pages when more pages are resolved , so that the list size doesn't grow beyond a certain fixed amount, no matter how many pages are resolved. These parameters will have to be configurable per query, as some users would prefer to keep the entire thing...
Splitting second optimization in a separate Jira
> Scaling paginated list
> ----------------------
>
> Key: CAY-999
> URL: https://issues.apache.org/cayenne/browse/CAY-999
> Project: Cayenne
> Issue Type: Improvement
> Components: Cayenne Core Library
> Affects Versions: 3.0
> Reporter: Andrus Adamchik
> Assignee: Andrus Adamchik
>
> An idea for scaling IncrementalFaultList to store massive amount of objects, like hundreds of thousands.This pertains to the server-side IncrementalFaultList. The problems to solve are the speed of the initial list initialization and overall memory use.
> 1. Simplify ID representation:
> Even unresolved lists can take significant amount of memory... Each unresolved object slot currently stores a DataRow with N number of entries, where N is the number of PK columns for the entity. I.e. most often than not - 1 entry. Here is a memory use calculation for various representations of an unresolved entry, based on a single int PK DbEntity.
> a. DataRow - 120 bytes,
> b. HashMap - 104 bytes,
> c. Object[] - 32 bytes,
> d java.lang.Integer - 16 bytes
> [primitive int is even better, but it complicates the implementation, as we'd need a parallel int[] (long[], double[], etc.) , so all in all we may get no gain]
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.