[streaming] mappers(reducers) read database again to get the changed config

[Sendoh <un...@gmail.com>]

Hi Flink users,

We have an usecase that streaming Flink jobs reading small configuration(no
more than 100 rows/10 columns) to transform data source according to the
config. When there is change, database should be read again and the config
object is changed. 

Did someone has similar usage or have an in idea what would be the best
practice?
(Unrelated to Flink part: use Notify/Listen rather than to read DB every X
minute)

So we either 1.let workers(mappers/reducers) keep connection and read DB
again when there is change, or 2.let master read DB and transfer this small
data to workers(mappers/reducers). But how would the later look like? would
it be to put the data in ExecutionConfig and let worker read ExecutionConfig
repeatedly? Would there be a critical factor that determine which one is
better than the other?

Best,

Sendoh



--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/streaming-mappers-reducers-read-database-again-to-get-the-changed-config-tp14793.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.