You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by Pete Carlson <pg...@gmail.com> on 2018/03/25 16:12:05 UTC

How to handle large lookup tables that update rarely in Apache Flink - Stack Overflow

I found this issue on Stack overflow:

https://stackoverflow.com/questions/37448847/how-to-handle-large-lookup-tables-that-update-rarely-in-apache-flink

Can someone point me to an example?  I have a similar case where I need to
load a csv as a lookup table (or map state if that is still the best way),
and then enrich another csv stream using data from those lookups.

Thanks,

Pete

Re: How to handle large lookup tables that update rarely in Apache Flink - Stack Overflow

Posted by Timo Walther <tw...@apache.org>.

Hi Pete,

you can find some basic examples about stream enrichment here [1].

I hope this helps a bit.

Regards,
Timo

[1] http://training.data-artisans.com/exercises/rideEnrichment-flatmap.html
[2] 
http://training.data-artisans.com/exercises/rideEnrichment-processfunction.html


Am 25.03.18 um 18:12 schrieb Pete Carlson:
> I found this issue on Stack overflow:
>
> https://stackoverflow.com/questions/37448847/how-to-handle-large-lookup-tables-that-update-rarely-in-apache-flink 
>
>
> Can someone point me to an example?  I have a similar case where I 
> need to load a csv as a lookup table (or map state if that is still 
> the best way), and then enrich another csv stream using data from 
> those lookups.
>
> Thanks,
>
> Pete