You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by Chirag Dewan <ch...@yahoo.in> on 2018/05/31 04:56:21 UTC

Gluster as file system for state backend

Hi,
I am evaluating some File Systems as state backend. I can see that Flink currently supports S3, MAPRFS and HDFS as file systems. 
However, I was hoping I can use Gluster as my state backend, since its already a part of existing eco system. Since I have stateful operators in my job and I am expecting each operator to accumulate large amount of state(in RocksDB) in 10seconds(checkpointing interval), I am looking for a write optimized  file system.  

Does anyone has any prior experience with Gluster as file state backend? Are there any caveats with using a replicated/striped replicated volume for Flink state data? 
Thanks,
Chirag