Scalable/Fault-tolerant Large Cache Operator

[Chandni Singh <ch...@datatorrent.com>]

We have a requirement where large amount of data (key/value)  is kept in
memory for a long time (few hours).

I am in need of a scalable operator which can be backed by a
database/filesystem but provides this functionality in a fault-tolerant
manner.

So I wanted to see what option I have in Malhar library to achieve this and
the only option I could find is -
com.datatorrent.lib.db.cache.AbstractDBLookupCacheBasedOperator.

The above expects a BackupStore but there is only one such implementation
of it - JDBCLookupCacheBackedOperator and I don't know whether that is
fault-tolerant.

Do we have other options in Malhar?
It will be good to have an operator that buckets tuples and load/offload
buckets in memory in Malhar.

Please recommend alternatives.

Thanks,
Chandni