Are Dynamic tables backed by rocksdb?

[kant kodali <ka...@gmail.com>]

Hi All,

Are Dynamic tables backed by Rocksdb or in memory? if they are backed by
RocksDB can I use SQL to query the state?

Thanks!

Re: Are Dynamic tables backed by rocksdb?

[Fabian Hueske <fh...@gmail.com>]

Hi,

Dynamic tables might not be persisted at all but only when it is necessary
for the computation of a query.
For example a simple "SELECT * FROM t WHERE a = 1" query on an append only
table t does not require to persist t.

However, there are a bunch of operations that require to store some parts
of a dynamic table but not necessarily the full table.
All operators that need to store data put it into regular Flink state. This
means that all state is stored in the configured state backend.
You cannot directly query the data from there, but could use the state
processor API. However, this also assumes that you know the internal data
representation that the operators use.

Best, Fabian


Am Do., 31. Okt. 2019 um 09:51 Uhr schrieb kant kodali <ka...@gmail.com>:

> Hi All,
>
> Are Dynamic tables backed by Rocksdb or in memory? if they are backed by
> RocksDB can I use SQL to query the state?
>
> Thanks!
>