[Myfaces Wiki] Update of "WorkingWithLargeTables" by SimonKitching

[Apache Wiki <wi...@apache.org>]

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Myfaces Wiki" for change notification.

The following page has been changed by SimonKitching:
http://wiki.apache.org/myfaces/WorkingWithLargeTables

New page:
Components t:dataModel and t:dataScroller work together nicely to allow a user to "page" through a set of a few dozen to a few hundred records.
However the implementation does assume that the entire set of available records are in memory and wrapped up inside a !ListDataModel or !ArrayDataModel.

When the available dataset is quite large, and the application can have many users, this can lead to memory usage problems.

This page contains discussions on how to handle this scenario.

=== On-demand loading ===

A custom DataModel can be used to allow data to be loaded "on demand".

First, a class needs to be defined which your "business methods" (eg EJBs) can use to pass "pages" of data back to the UI.
This class needs to be defined in your project, as the "business" level shouldn't be extending !MyFaces classes:

{{{
package example;

import java.util.List;

/**
 * A simple class that represents a "page" of data out of a longer set, ie
 * a list of objects together with info to indicate the starting row and
 * the full size of the dataset. EJBs can return instances of this type
 * when returning subsets of available data.
 */
public class DataPage<T> {

  private int datasetSize;
  private int startRow;
  private List<T> data;
	
  /**
   * Create an object representing a sublist of a dataset.
   * 
   * @param datasetSize is the total number of matching rows
   * available.
   * 
   * @param startRow is the index within the complete dataset
   * of the first element in the data list.
   * 
   * @param data is a list of consecutive objects from the
   * dataset.
   */
  public DataPage(int datasetSize, int startRow, List<T> data) {
    this.datasetSize = datasetSize;
    this.startRow = startRow;
    this.data = data;
  }

  /**
   * Return the number of items in the full dataset.
   */
  public int getDatasetSize() {
    return datasetSize;
  }

  /**
   * Return the offset within the full dataset of the first
   * element in the list held by this object.
   */
  public int getStartRow() {
    return startRow;
  }

  /**
   * Return the list of objects held by this object, which
   * is a continuous subset of the full dataset.
   */
  public List<T> getData() {
    return data;
  }
}
}}}

Now a custom !DataModel can use this !DataPage stuff. Again, it is recommended that you copy this code into your project and change the package name appropriately. This class can't go in the !MyFaces libraries as it depends on !DataPage, and as noted above, !DataPage is accessed by your business level code so it really can't be in the !MyFaces libs:

{{{
package example;

import javax.faces.model.DataModel;

import example.DataPage;

/**
 * A special type of JSF DataModel to allow a datatable and datascroller
 * to page through a large set of data without having to hold the entire
 * set of data in memory at once.
 * <p>
 * Any time a managed bean wants to avoid holding an entire dataset,
 * the managed bean should declare an inner class which extends this
 * class and implements the fetchData method. This method is called
 * as needed when the table requires data that isn't available in the
 * current data page held by this object.
 * <p>
 * This does require the managed bean (and in general the business
 * method that the managed bean uses) to provide the data wrapped in
 * a DataPage object that provides info on the full size of the dataset.
 */
public abstract class PagedListDataModel<T> extends DataModel {

    int pageSize;
    int rowIndex;
    DataPage<T> page;
    
    /*
     * Create a datamodel that pages through the data showing the specified
     * number of rows on each page.
     */
    public PagedListDataModel(int pageSize) {
        super();
        this.pageSize = pageSize;
        this.rowIndex = -1;
        this.page = null;
    }

    /**
     * Not used in this class; data is fetched via a callback to the
     * fetchData method rather than by explicitly assigning a list.
     */
    @Override
    public void setWrappedData(Object o) {
        throw new UnsupportedOperationException("setWrappedData");
    }

    @Override
    public int getRowIndex() {
        return rowIndex;
    }

    /**
     * Specify what the "current row" within the dataset is. Note that
     * the UIData component will repeatedly call this method followed
     * by getRowData to obtain the objects to render in the table.
     */
    @Override
    public void setRowIndex(int index) {
        rowIndex = index;
    }

    /**
     * Return the total number of rows of data available (not just the
     * number of rows in the current page!).
     */
    @Override
    public int getRowCount() {
        return getPage().getDatasetSize();
    }
    
    /**
     * Return a DataPage object; if one is not currently available then
     * fetch one. Note that this doesn't ensure that the datapage
     * returned includes the current rowIndex row; see getRowData.
     */
    private DataPage<T> getPage() {
        if (page != null)
            return page;
        
        int rowIndex = getRowIndex();
        int startRow = rowIndex;
        if (rowIndex == -1) {
            // even when no row is selected, we still need a page
            // object so that we know the amount of data available.
           startRow = 0;
        }

        // invoke method on enclosing class
        return fetchPage(startRow, pageSize);
    }

    /**
     * Return the object corresponding to the current rowIndex.
     * If the DataPage object currently cached doesn't include that
     * index then fetchPage is called to retrieve the appropriate page.
     */
    @Override
    public Object getRowData(){
        DataPage<T> page = getPage();
        int datasetSize = page.getDatasetSize();
        int startRow = page.getStartRow();
        int nRows = page.getData().size();
        int endRow = startRow + nRows;
        
        if (rowIndex < 0) {
            throw new IllegalArgumentException(
                "Invalid rowIndex for PagedListDataModel; not within page");
        }

        if (rowIndex >= datasetSize) {
            throw new IllegalArgumentException("Invalid rowIndex");
        }
        
        if (rowIndex < startRow) {
            page = fetchPage(rowIndex, pageSize);
            startRow = page.getStartRow();
        } else if (rowIndex >= endRow) {
            page = fetchPage(rowIndex, pageSize);
            startRow = page.getStartRow();
        }
        
        return page.getData().get(rowIndex - startRow);
    }

    @Override
    public Object getWrappedData() {
        return page.getData();
    }

    /**
     * Return true if the rowIndex value is currently set to a
     * value that matches some element in the dataset. Note that
     * it may match a row that is not in the currently cached 
     * DataPage; if so then when getRowData is called the
     * required DataPage will be fetched by calling fetchData.
     */
    @Override
    public boolean isRowAvailable() {
        DataPage<T> page = getPage();
        if (page == null)
            return false;
        
        int rowIndex = getRowIndex();
        if (rowIndex < 0) {
            return false;
        } else if (rowIndex >= page.getDatasetSize()) {
            return false;
        } else {
            return true;
        }
    }

    /**
     * Method which must be implemented in cooperation with the
     * managed bean class to fetch data on demand.
     */
    public abstract DataPage<T> fetchPage(int startRow, int pageSize);
}
}}}


Finally, the managed bean needs to provide a simple inner class that provides the fetchPage implementation:

{{{
  public SomeManagedBean {
    ....


    private DataPage<SomeRowObject> getDataPage(int startRow, int pageSize) {
      // access database here, or call EJB to do so
    }

    public DataModel getDataModel() {
        if (dataModel == null) {
            dataModel = new LocalDataModel(getRowsPerPage());
        }

        return dataModel;
    }

    private class LocalDataModel extends PagedListDataModel {
        public LocalDataModel(int pageSize) {
            super(pageSize);
        }
        
        public DataPage<SomeRowObject> fetchPage(int startRow, int pageSize) {
            // call enclosing managed bean method to fetch the data
            return getDataPage(startRow, pageSize);
        }
    }

}}}

The jsp pages are then trivial; by default the t:dataScroller will update the t:dataTable's "first" property, and that's all that is needed because when the table
asks the custom DataModel for the necessary rows callbacks to fetchPage will be made which will fetch exactly the data required. No event listeners, action methods,
or anything else is required as glue.

=== Other approaches ===

In an email thread on this topic, an alternative was decribed:
   http://sebcsaba.fw.hu/scrollablelist.html

And so was this:
  http://www.mail-archive.com/users%40myfaces.apache.org/msg12597.html