You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Kobe <rk...@mailcity.com> on 2015/12/04 18:26:47 UTC

IGFS with local filesystem: Lifecycle of content

Hello..

I have an IGFS instance managing content with a local file system backing
store.
I would like to understand the lifecycle of content in IGFS:
  1: Application A generates content and writes to IGFS
  2: IGFS writes to backing store (does this happen right away? can it be
batched?)
  3: Application B reads the content (the file) off IGFS. (I assume the
content will be delivered off the data cache and no reads from backing store
will be performed?)
  4: Application B deletes the content (is the backing store updated right
away?)

Any help gratefully appreciated,

thanx,

/Kobe

[CODE]

        IgfsSecondaryFileSystem igfsSec = new
IgniteHadoopIgfsSecondaryFileSystem("file:///", null, "data");
//
      dataCacheCfg.setName("partitioned");
        dataCacheCfg.setCacheMode(CacheMode.PARTITIONED);
        dataCacheCfg.setNearConfiguration(null);
       
dataCacheCfg.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC);
        dataCacheCfg.setAffinityMapper(new
IgfsGroupDataBlocksKeyMapper(128));
        dataCacheCfg.setBackups(0);
        dataCacheCfg.setAtomicityMode(CacheAtomicityMode.TRANSACTIONAL);
//...
        metaCacheCfg.setName("replicated");
        metaCacheCfg.setCacheMode(CacheMode.REPLICATED);
       
metaCacheCfg.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC);
  ///...
      try (Ignite ig = Ignition.start(igCfg)) {
            IgniteFileSystem igfs = ig.fileSystem("myigfs");

        // Create, manage data   
   
[/CODE]



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/IGFS-with-local-filesystem-Lifecycle-of-content-tp2141.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: IGFS with local filesystem: Lifecycle of content

Posted by Kobe <rk...@mailcity.com>.
Thank you Vladmir for your clarifications!
I understand.

Regards,

/Kobe



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/IGFS-with-local-filesystem-Lifecycle-of-content-tp2141p2195.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: IGFS with local filesystem: Lifecycle of content

Posted by Vladimir Ozerov <vo...@gridgain.com>.
Hi Kobe,

Ansewring your questions:
2)  Writes are batched based on "bufSize" parameter in methods
IgniteFileSystem.create() and IgniteFileSystem.append().
3) Do not understand the question. If you read files content through IGFS
and file is in cache, IGFS will not touch underlying file system. If the
file is not in cache, it will be cached during the first read. Essnetially
IGFS is a caching layer with read-through/write-through semantics.
4) Yes, other participants should observe file delete immediately.

On Fri, Dec 4, 2015 at 8:26 PM, Kobe <rk...@mailcity.com> wrote:

>
> Hello..
>
> I have an IGFS instance managing content with a local file system backing
> store.
> I would like to understand the lifecycle of content in IGFS:
>   1: Application A generates content and writes to IGFS
>   2: IGFS writes to backing store (does this happen right away? can it be
> batched?)
>   3: Application B reads the content (the file) off IGFS. (I assume the
> content will be delivered off the data cache and no reads from backing
> store
> will be performed?)
>   4: Application B deletes the content (is the backing store updated right
> away?)
>
> Any help gratefully appreciated,
>
> thanx,
>
> /Kobe
>
> [CODE]
>
>         IgfsSecondaryFileSystem igfsSec = new
> IgniteHadoopIgfsSecondaryFileSystem("file:///", null, "data");
> //
>       dataCacheCfg.setName("partitioned");
>         dataCacheCfg.setCacheMode(CacheMode.PARTITIONED);
>         dataCacheCfg.setNearConfiguration(null);
>
>
> dataCacheCfg.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC);
>         dataCacheCfg.setAffinityMapper(new
> IgfsGroupDataBlocksKeyMapper(128));
>         dataCacheCfg.setBackups(0);
>         dataCacheCfg.setAtomicityMode(CacheAtomicityMode.TRANSACTIONAL);
> //...
>         metaCacheCfg.setName("replicated");
>         metaCacheCfg.setCacheMode(CacheMode.REPLICATED);
>
>
> metaCacheCfg.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC);
>   ///...
>       try (Ignite ig = Ignition.start(igCfg)) {
>             IgniteFileSystem igfs = ig.fileSystem("myigfs");
>
>         // Create, manage data
>
> [/CODE]
>
>
>
> --
> View this message in context:
> http://apache-ignite-users.70518.x6.nabble.com/IGFS-with-local-filesystem-Lifecycle-of-content-tp2141.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>