[jira] [Created] (METAMODEL-105) Reactive DataSet consumption

["Kasper Sørensen (JIRA)" <ji...@apache.org>]

Kasper Sørensen created METAMODEL-105:
-----------------------------------------

             Summary: Reactive DataSet consumption
                 Key: METAMODEL-105
                 URL: https://issues.apache.org/jira/browse/METAMODEL-105
             Project: Apache MetaModel
          Issue Type: New Feature
            Reporter: Kasper Sørensen


Today our query API returns a DataSet which we can blockingly iterate a la this:

{code}
DataSet ds = ...
while (ds.next()) {
  Row r = ds.getRow();
  // do something with the row
}
ds.close();
{code}

This works well for many cases, but many databases also support a leaner way of consuming data - a reactive way - which publishes the data to the consumer bit by bit. This means that we don't need blocking .next() calls and it also means that the publisher may introduce multithreading, buffering or other techniques in a smarter way.

It would require a big API change though. We could consider doing this in a few different ways.

1) Introduce it on the DataSet, a la:

{code}
DataSet ds = ...
ds.consume(new Consumer<Row> {
  // some method here that takes a row
});
{code}

The down-side of this is that it may be more complex to figure out at query execution time what strategy (reactive or not) to apply. Another small issue is that DataSet is Closeable and the act of closing it should ideally be done behind the covers one our reactive mechanism knows when it can do it.

2) Introduce it on the DataContext, a la:

{code}
Query q = ...
DataContext dc = ...
dc.executeQuery(q, new Consumer<Row> {
  // some method here that takes a row
});
{code}

This approach is probably better because it doesn't have the pitfalls of approach 1. The downside is that it takes a Query and by now we also have a query builder API and even a String-based QueryParser. But I guess plugging in reactive support for those would just be a few more additional methods.

To minimize the impact we could consider doing this in a sub-interface of DataContext. On the other hand I do think it would be nice if every DataContext would have the method(s) available so that you could polymorphically use reactive data consumption. We would just need a naive implementation in AbstractDataContext, but that's also pretty easy to do I guess.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)