You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@kafka.apache.org by "Parthasarathy, Mohan" <mp...@hpe.com> on 2019/05/15 20:55:53 UTC

Multiple state store files

Hi,

I am seeing multiple state store files (.sst) created under each <subtopology_partition> directory. Is this normal ? If an application crashes and comes back, would it cause this ?

Thanks
Mohan

Re: Multiple state store files

Posted by "Matthias J. Sax" <ma...@confluent.io>.

You might want to tune RocksDB as an alternative:

https://github.com/facebook/rocksdb/wiki/RocksDB-Tuning-Guide

You can pass in configs via `StreamsConfig` using `RocksDBConfigSetter`
object.

-Matthias

On 5/16/19 6:24 PM, Parthasarathy, Mohan wrote:
> Thanks. Unfortunately, this leads to lot of open file descriptors in the application. Anyone else faced that situation ? I will increase the limit for now.
> 
> Thanks
> Mohan
> 
> On 5/16/19, 5:18 AM, "Matthias J. Sax" <ma...@confluent.io> wrote:
> 
>     This is how RocksDB works, and it's normal.
>     
>     -Matthias
>     
>     On 5/15/19 10:55 PM, Parthasarathy, Mohan wrote:
>     > Hi,
>     > 
>     > I am seeing multiple state store files (.sst) created under each <subtopology_partition> directory. Is this normal ? If an application crashes and comes back, would it cause this ?
>     > 
>     > Thanks
>     > Mohan
>     > 
>     
>     
>

Re: Multiple state store files

Posted by "Parthasarathy, Mohan" <mp...@hpe.com>.

Thanks. Unfortunately, this leads to lot of open file descriptors in the application. Anyone else faced that situation ? I will increase the limit for now.

Thanks
Mohan

On 5/16/19, 5:18 AM, "Matthias J. Sax" <ma...@confluent.io> wrote:

    This is how RocksDB works, and it's normal.
    
    -Matthias
    
    On 5/15/19 10:55 PM, Parthasarathy, Mohan wrote:
    > Hi,
    > 
    > I am seeing multiple state store files (.sst) created under each <subtopology_partition> directory. Is this normal ? If an application crashes and comes back, would it cause this ?
    > 
    > Thanks
    > Mohan
    >

Re: Multiple state store files

Posted by "Matthias J. Sax" <ma...@confluent.io>.

This is how RocksDB works, and it's normal.

-Matthias

On 5/15/19 10:55 PM, Parthasarathy, Mohan wrote:
> Hi,
> 
> I am seeing multiple state store files (.sst) created under each <subtopology_partition> directory. Is this normal ? If an application crashes and comes back, would it cause this ?
> 
> Thanks
> Mohan
>