Does Flink support long term storage and complex queries

[Basanth Gowda <ba...@gmail.com>]

We are looking at Druid for long term storage and roll up metrics and adhoc
queries on time series data.

If Flink can support dynamic queries and rollups like Druid does, we would
like to just use Flink and leave Druid aside. Does Flink support such kind
of queries and what are the guidelines for data retention ?

thank you

Re: Does Flink support long term storage and complex queries

[Fabian Hueske <fh...@gmail.com>]

Hi Basanth,

Flink is not a storage system (neither for stream nor for batch data).
Flink applications can be stateful and maintain very large state (in the
order of several TBs) but state is always associated with an application.
State can be queryable, so outside applications can run key-lookup (point)
queries against state but bulk reading of active state is not supported.

So it depends on your use case if Flink is a good fit.
Flink excels at processing streams and can emit the results to a variety of
different storage systems.

Best, Fabian

2017-08-01 2:17 GMT+02:00 Basanth Gowda <ba...@gmail.com>:

> We are looking at Druid for long term storage and roll up metrics and
> adhoc queries on time series data.
>
> If Flink can support dynamic queries and rollups like Druid does, we would
> like to just use Flink and leave Druid aside. Does Flink support such kind
> of queries and what are the guidelines for data retention ?
>
> thank you
>