Re: Understanding configurations for IGFS

["ilya.kasnacheev" <il...@gmail.com>]

Hello!

Sorry that I didn't answer earlier. Took some courage.

> int getPerNodeBatchSize()
Internally, IGFS uses DataStreamer. This is DataStreamer's
perNodeBufferSize() that is called when it is created.

> int getPerNodeParallelBatchCount()
Ditto. It is perNodeParallelOperations().

So, remote node in this context is affinity node (not excluding local I
assume) that DataStreamer sends a batch to. I don't see how it would affect
backups, looks orthogonal.

> int getPrefetchBlocks()
When you request a block N of file, it will also prefetch block N + 1, N +
2, ..., N + prefetchBlocks.

> is there any reason to use TRANSACTIONAL for dataCacheConfiguration
I assume that IGFS uses transactions internally to guarantee metadata
consistency.

> Similarly, does the setExpiryPolicyFactory in dataCacheConfiguration and
> metaCacheConfiguration have any effect?
I think it would have effect, but I'm afraid it's not the effect you are
expecting. I couldn't find any tests on this, so I would not recommend it.

> Similarly, does the eviction policy configured for dataCacheConfiguration
> and metaCacheConfiguration have any effect?
> Does IGFS sync the eviction of entries in the data and the metadata cache?
> Even if I use 2 different data regions for the 2 caches? A metadata entry
> with no data entries can be useful, but not the other way around

I don't think you should expire metaCache, but it seems that you can expire
dataCache, and there's even IgfsPerBlockLruEvictionPolicy with configurable
filter for that.

metaCache should be much smaller that dataCache so you should probably just
expire latter and leave former as is.

> The readThrough, writeThrough,writeBehind fields for the
> CacheConfiguration dataCacheConfiguration and metaCacheConfiguration have
> any effect?

Frankly speaking, I can't recommend doing anything funny to caches created
for IGFS. Better stick to recommended defaults.

> Is there any recommended ratio between the page size used for the
> DataStorageConfiguration for a DataRegionConfiguration used for the IGFS
> dataCacheConfiguration, and the block size configured for IGFS? 

This is a very interesting question right here. I suspect you should
benchmark several combinations, see if there's any recommendations. I think
that IGFS was mainly created when Ignite was mostly on-heap and not in
durable memory, so maybe nobody ever figured this one out.

Regards,



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/