RocksDB and local file system

[Jaqie Chan <ja...@gmail.com>]

Hello

I am using Flink rocksDb state backend, the documentation seems to imply i
can use a regular file system such as: file:///data/flink/checkpoints, but
the code javadoc only mentions hdfs or s3 option here.

I am wondering if it's possible to use local file system with flink rocksdb
backend.


Thanks

嘉琪

Re: RocksDB and local file system

[Congxian Qiu <qc...@gmail.com>]

Hi

The path will store the checkpoints, and Flink will fetch the checkpoint
files to restore the state if any failure occurred.  If you specify the
local file system, when restoring from checkpoint, Flink may can't find the
checkpoint files, and can't restore from last checkpoints.

Best,
Congxian


vino yang <ya...@gmail.com> 于2019年11月7日周四 上午10:57写道：

> Hi Jaqie,
>
> For testing, you can use the local file system pattern (e.g. "file:///").
> Technically speaking, it's OK to specify the string path provided by you.
>
> However, in the production environment, we do not recommend using the
> local file system. Because it does not provide high availability.
>
> Best,
> Vino
>
> Jaqie Chan <ja...@gmail.com> 于2019年11月6日周三 下午11:29写道：
>
>> Hello
>>
>> I am using Flink rocksDb state backend, the documentation seems to imply
>> i can use a regular file system such as: file:///data/flink/checkpoints,
>> but the code javadoc only mentions hdfs or s3 option here.
>>
>> I am wondering if it's possible to use local file system with flink
>> rocksdb backend.
>>
>>
>> Thanks
>>
>> 嘉琪
>>
>

Re: RocksDB and local file system

[vino yang <ya...@gmail.com>]

Hi Jaqie,

For testing, you can use the local file system pattern (e.g. "file:///").
Technically speaking, it's OK to specify the string path provided by you.

However, in the production environment, we do not recommend using the local
file system. Because it does not provide high availability.

Best,
Vino

Jaqie Chan <ja...@gmail.com> 于2019年11月6日周三 下午11:29写道：

> Hello
>
> I am using Flink rocksDb state backend, the documentation seems to imply i
> can use a regular file system such as: file:///data/flink/checkpoints,
> but the code javadoc only mentions hdfs or s3 option here.
>
> I am wondering if it's possible to use local file system with flink
> rocksdb backend.
>
>
> Thanks
>
> 嘉琪
>