Fluo use-case question: cross row/table updates

["St. Amour, Leo - 0553 - MITLL" <Le...@ll.mit.edu>]

Hello,

I have an Accumulo scenario that is presenting issues and was wondering if
Fluo would help:

Suppose I have two tables (Main, Other). As I insert information into the
Main table, I want to store and manage a data structure in the Other table.
This data structure is a self-balancing binary tree. Therefore, some data
inserts will be simple and others will cause updates to propagate across
nodes of the Other tree. This would cause cross-row updates, which are not
guaranteed to be atomic.

This scenario does not necessarily require two tables, I can always combine
the two. Ultimately, I¹m looking for a way to make cross row updates to
dynamically update a data structure that is embedded in a table.

At a high-level, it seems like Fluo could help with this problem. Any
information would be greatly appreciated.

Thank you,

Leo

Re: Fluo use-case question: cross row/table updates

[Christopher <ct...@apache.org>]

We should consider supporting cross-table updates in 2.0.

On Fri, Jul 29, 2016 at 1:52 PM Mike Walch <mw...@apache.org> wrote:

> Hi Leo,
>
> I think your problem is a good use case for Fluo.
>
> A couple of notes:
>
> * Fluo supports cross-row updates. However, a Fluo application has only one
> table.  Cross-table updates are not supported at this time.  However, we
> just typically combine data that would be in two tables into one by using
> row prefixes (i.e Main data would be prefixed with 'm:' and Other data
> would be prefixed with 'o:').
> * While cross-table updates in Fluo are not possible, you can export data
> from Fluo to Accumulo or even external systems (like Solr, Elastic Search)
> using the export queue recipe (see
> https://fluo.apache.org/docs/fluo-recipes/1.0.0-beta-2/export-queue/ ). We
> have an example Fluo application called Webindex that maintains inbound
> link counts in Fluo but exports its indexes to Accumulo (
> https://github.com/fluo-io/webindex).
> * This is a good time to take a look at Fluo as we are about to release
> 1.0.0.  We are following semantic versioning (http://semver.org) so any
> thing written using 1.0.0 will be compatible with any 1.* release.
>
> Let us know how it goes!
>
> Best,
> Mike
>
> On Fri, Jul 29, 2016 at 10:22 AM St. Amour, Leo - 0553 - MITLL <
> Leo.St.Amour@ll.mit.edu> wrote:
>
> > Hello,
> >
> > I have an Accumulo scenario that is presenting issues and was wondering
> > if Fluo would help:
> >
> > Suppose I have two tables (Main, Other). As I insert information into the
> > Main table, I want to store and manage a data structure in the Other
> table.
> > This data structure is a self-balancing binary tree. Therefore, some data
> > inserts will be simple and others will cause updates to propagate across
> > nodes of the Other tree. This would cause cross-row updates, which are
> not
> > guaranteed to be atomic.
> >
> > This scenario does not necessarily require two tables, I can always
> > combine the two. Ultimately, I’m looking for a way to make cross row
> > updates to dynamically update a data structure that is embedded in a
> table.
> >
> > At a high-level, it seems like Fluo could help with this problem. Any
> > information would be greatly appreciated.
> >
> > Thank you,
> >
> > Leo
> >
>

Re: Fluo use-case question: cross row/table updates

[Mike Walch <mw...@apache.org>]

Hi Leo,

I think your problem is a good use case for Fluo.

A couple of notes:

* Fluo supports cross-row updates. However, a Fluo application has only one
table.  Cross-table updates are not supported at this time.  However, we
just typically combine data that would be in two tables into one by using
row prefixes (i.e Main data would be prefixed with 'm:' and Other data
would be prefixed with 'o:').
* While cross-table updates in Fluo are not possible, you can export data
from Fluo to Accumulo or even external systems (like Solr, Elastic Search)
using the export queue recipe (see
https://fluo.apache.org/docs/fluo-recipes/1.0.0-beta-2/export-queue/ ). We
have an example Fluo application called Webindex that maintains inbound
link counts in Fluo but exports its indexes to Accumulo (
https://github.com/fluo-io/webindex).
* This is a good time to take a look at Fluo as we are about to release
1.0.0.  We are following semantic versioning (http://semver.org) so any
thing written using 1.0.0 will be compatible with any 1.* release.

Let us know how it goes!

Best,
Mike

On Fri, Jul 29, 2016 at 10:22 AM St. Amour, Leo - 0553 - MITLL <
Leo.St.Amour@ll.mit.edu> wrote:

> Hello,
>
> I have an Accumulo scenario that is presenting issues and was wondering
> if Fluo would help:
>
> Suppose I have two tables (Main, Other). As I insert information into the
> Main table, I want to store and manage a data structure in the Other table.
> This data structure is a self-balancing binary tree. Therefore, some data
> inserts will be simple and others will cause updates to propagate across
> nodes of the Other tree. This would cause cross-row updates, which are not
> guaranteed to be atomic.
>
> This scenario does not necessarily require two tables, I can always
> combine the two. Ultimately, I’m looking for a way to make cross row
> updates to dynamically update a data structure that is embedded in a table.
>
> At a high-level, it seems like Fluo could help with this problem. Any
> information would be greatly appreciated.
>
> Thank you,
>
> Leo
>