You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by Marco Villalobos <mv...@kineteque.com> on 2020/12/07 16:14:59 UTC

How does Flink cache values that also do not exist in the database?

How does Flink cache values that also do not exist in the database?

I would like to cache hits forever, but I would like to check items that do not exist in the database only every 15 minutes? Is there a way to set that up in the SQL / Table api?  Also, is there a way to set that up in Keyed State?

Re: How does Flink cache values that also do not exist in the database?

Posted by Timo Walther <tw...@apache.org>.

Hi Marco,

when you say "database" are you refering to the JDBC connector or would 
you like to perform a JDBC query within some UDF? In the latter case, I 
would recommend to use Flink's ProcessFunction because you can store the 
cache hits in state (and thus keep them forever). SQL/Table API does not 
expose state functionality. Aggregate functions are the only type of 
functions that are stateful but not useful in this case I guess.

Regards,
Timo

On 07.12.20 17:14, Marco Villalobos wrote:
> How does Flink cache values that also do not exist in the database?
> 
> I would like to cache hits forever, but I would like to check items that do not exist in the database only every 15 minutes? Is there a way to set that up in the SQL / Table api?  Also, is there a way to set that up in Keyed State?
>