RocksDB

[Timothy Victor <vi...@gmail.com>]

Can the RocksDB state backend used by Flink be queries from outside, e.g.
via SQL?

Or maybe a better question, is there a RocksDB SinkFunction that exists?

Thanks

Tim

Re: RocksDB

[David Anderson <da...@ververica.com>]

The State Processor API goes a bit in the direction you asking about, by
making it possible to query savepoints.

https://ci.apache.org/projects/flink/flink-docs-stable/dev/libs/state_processor_api.html
<https://ci.apache.org/projects/flink/flink-docs-release-1.10/dev/libs/state_processor_api.html>

Regards,
David


On Tue, Mar 10, 2020 at 1:05 PM Aljoscha Krettek <al...@apache.org>
wrote:

> On 10.03.20 11:36, Timothy Victor wrote:
> > Can the RocksDB state backend used by Flink be queries from outside, e.g.
> > via SQL?
>
> That's not possible, but you might be interested in queryable state:
>
> https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/queryable_state.html
>
> > Or maybe a better question, is there a RocksDB SinkFunction that exists?n
>
> I'm afraid that doesn't exist, and it probably never will because
> usually RocksDB is not a distributed system so writing to it from a
> parallel stream processing application would most likely not work well.
>
> Best,
> Aljoscha
>

Re: RocksDB

[Aljoscha Krettek <al...@apache.org>]

On 10.03.20 11:36, Timothy Victor wrote:
> Can the RocksDB state backend used by Flink be queries from outside, e.g.
> via SQL?

That's not possible, but you might be interested in queryable state: 
https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/queryable_state.html

> Or maybe a better question, is there a RocksDB SinkFunction that exists?n

I'm afraid that doesn't exist, and it probably never will because 
usually RocksDB is not a distributed system so writing to it from a 
parallel stream processing application would most likely not work well.

Best,
Aljoscha