Query on PojoContext seems to be too slow

[Alessandro Portosa <al...@eng.it>]

Dear all,
I'm trying to implement MetaModel as component to allows aggregation and 
filtering on a Pojo object used in out project as data store.
My first integration does something like the following code snippet:

/SimpleTableDef dataStoreDef = new SimpleTableDef(uniqueTableName, 
columnNames, columnTypes);//
//ArrayTableDataProvider dataStoreTableProvider = new 
ArrayTableDataProvider(dataStoreDef, arrays);//
// DataContext dataContext = new PojoDataContext(DEFAULT_SCHEMA_NAME, 
dataStoreTableProvider);//
//
//// Change table name to be concurrency-safe//
//String newSqlQuery = sqlQuery.replace(DEFAULT_TABLE_NAME, 
uniqueTableName);//
//Query query = dataContext.parseQuery(newSqlQuery);//
//DataSet dataSet = dataContext.executeQuery(query);/

Basically, I extract the metadata (/uniqueTableName, columnNames, 
columnTypes/) and data (/arrays/). The problem is the query execution 
time, which seems to be soo long even for medium dataset (150.000 rows). 
It takes *minutes *to process the query extracting the resulting 
dataset. All the information are in-memory, so I would expect a better 
performance. The query involves only one DataContext.

Is there anything wrong in the above operation? Am I missing something? 
Or such long time is a known detail?
Best regards,
Alessandro
//

Re: Query on PojoContext seems to be too slow

[Kasper Sørensen <i....@gmail.com>]

Hi Alessandro,

I'm not immediately sure why this might be taking that long. Can you share
an example query that you use?

I wouldn't expect POJO datacontext to ever be the fastest implementation.
It's mostly just a convenient thing to use for testing etc. where the
footprint is very very small. I have myself always only used it for
unittesting - to replace a "real" datacontext with a PojoDataContext that
thereby stubs all external I/O. But there is certainly also no indexing or
other performance tweaks in the PojoDataContext that could make it proper
fast. If you want that kind of thing I would maybe consider something like
a JdbcDataContext on H2 or another embedded database.

Best regards,
Kasper

2015-10-26 14:07 GMT+01:00 Alessandro Portosa <al...@eng.it>:

> Dear all,
> I'm trying to implement MetaModel as component to allows aggregation and
> filtering on a Pojo object used in out project as data store.
> My first integration does something like the following code snippet:
>
> /SimpleTableDef dataStoreDef = new SimpleTableDef(uniqueTableName,
> columnNames, columnTypes);//
> //ArrayTableDataProvider dataStoreTableProvider = new
> ArrayTableDataProvider(dataStoreDef, arrays);//
> // DataContext dataContext = new PojoDataContext(DEFAULT_SCHEMA_NAME,
> dataStoreTableProvider);//
> //
> //// Change table name to be concurrency-safe//
> //String newSqlQuery = sqlQuery.replace(DEFAULT_TABLE_NAME,
> uniqueTableName);//
> //Query query = dataContext.parseQuery(newSqlQuery);//
> //DataSet dataSet = dataContext.executeQuery(query);/
>
> Basically, I extract the metadata (/uniqueTableName, columnNames,
> columnTypes/) and data (/arrays/). The problem is the query execution time,
> which seems to be soo long even for medium dataset (150.000 rows). It takes
> *minutes *to process the query extracting the resulting dataset. All the
> information are in-memory, so I would expect a better performance. The
> query involves only one DataContext.
>
> Is there anything wrong in the above operation? Am I missing something? Or
> such long time is a known detail?
> Best regards,
> Alessandro
> //
>