You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@geode.apache.org by "Jason Huynh (JIRA)" <ji...@apache.org> on 2017/03/20 20:13:42 UTC
[jira] [Resolved] (GEODE-2679) Lucene asynchronous disk writes for
aeq can lead to data mismatch after compacting
[ https://issues.apache.org/jira/browse/GEODE-2679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Huynh resolved GEODE-2679.
--------------------------------
Resolution: Fixed
Fix Version/s: 1.1.0
> Lucene asynchronous disk writes for aeq can lead to data mismatch after compacting
> ----------------------------------------------------------------------------------
>
> Key: GEODE-2679
> URL: https://issues.apache.org/jira/browse/GEODE-2679
> Project: Geode
> Issue Type: Bug
> Components: lucene
> Reporter: Jason Huynh
> Assignee: Jason Huynh
> Fix For: 1.1.0
>
>
> Due to the way krf and drf oplogs are created/updated, if the aeq has disk synchronous as false, there is a window of time where a krf is rolled/created with the current region entries, but events that had been removed have not yet been written to the drf.
> We should probably change the aeq to be disk sync = true. This will alleviate some of the data mismatch where we have potential extra keys after compacting the oplogs.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
Re: [jira] [Resolved] (GEODE-2679) Lucene asynchronous disk writes
for aeq can lead to data mismatch after compacting
Posted by Dan Smith <ds...@pivotal.io>.
Hi Mike,
disk-synchronous=true is the default for geode regions.
disk-synchronous=false means that geode will buffer up writes in a queue
and flush them to disk asynchronously.
Neither setting actually flushes the os buffers. Thats gemfire.syncWrites.
-Dan
On Mon, Mar 20, 2017 at 2:41 PM, Michael Stolz <ms...@pivotal.io> wrote:
> Does this disk-sync=true really change anything?
> My understanding is all that does is cause us to flush the linux disk
> buffers after every write.
> I'm not aware of it changing behavior in terms of buffering inside Geode.
>
> --
> Mike Stolz
> Principal Engineer, GemFire Product Manager
> Mobile: +1-631-835-4771
>
> On Mon, Mar 20, 2017 at 4:13 PM, Jason Huynh (JIRA) <ji...@apache.org>
> wrote:
>
> >
> > [ https://issues.apache.org/jira/browse/GEODE-2679?page=
> > com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
> >
> > Jason Huynh resolved GEODE-2679.
> > --------------------------------
> > Resolution: Fixed
> > Fix Version/s: 1.1.0
> >
> > > Lucene asynchronous disk writes for aeq can lead to data mismatch after
> > compacting
> > > ------------------------------------------------------------
> > ----------------------
> > >
> > > Key: GEODE-2679
> > > URL: https://issues.apache.org/jira/browse/GEODE-2679
> > > Project: Geode
> > > Issue Type: Bug
> > > Components: lucene
> > > Reporter: Jason Huynh
> > > Assignee: Jason Huynh
> > > Fix For: 1.1.0
> > >
> > >
> > > Due to the way krf and drf oplogs are created/updated, if the aeq has
> > disk synchronous as false, there is a window of time where a krf is
> > rolled/created with the current region entries, but events that had been
> > removed have not yet been written to the drf.
> > > We should probably change the aeq to be disk sync = true. This will
> > alleviate some of the data mismatch where we have potential extra keys
> > after compacting the oplogs.
> >
> >
> >
> > --
> > This message was sent by Atlassian JIRA
> > (v6.3.15#6346)
> >
>
Re: [jira] [Resolved] (GEODE-2679) Lucene asynchronous disk writes
for aeq can lead to data mismatch after compacting
Posted by Michael Stolz <ms...@pivotal.io>.
Does this disk-sync=true really change anything?
My understanding is all that does is cause us to flush the linux disk
buffers after every write.
I'm not aware of it changing behavior in terms of buffering inside Geode.
--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: +1-631-835-4771
On Mon, Mar 20, 2017 at 4:13 PM, Jason Huynh (JIRA) <ji...@apache.org> wrote:
>
> [ https://issues.apache.org/jira/browse/GEODE-2679?page=
> com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
>
> Jason Huynh resolved GEODE-2679.
> --------------------------------
> Resolution: Fixed
> Fix Version/s: 1.1.0
>
> > Lucene asynchronous disk writes for aeq can lead to data mismatch after
> compacting
> > ------------------------------------------------------------
> ----------------------
> >
> > Key: GEODE-2679
> > URL: https://issues.apache.org/jira/browse/GEODE-2679
> > Project: Geode
> > Issue Type: Bug
> > Components: lucene
> > Reporter: Jason Huynh
> > Assignee: Jason Huynh
> > Fix For: 1.1.0
> >
> >
> > Due to the way krf and drf oplogs are created/updated, if the aeq has
> disk synchronous as false, there is a window of time where a krf is
> rolled/created with the current region entries, but events that had been
> removed have not yet been written to the drf.
> > We should probably change the aeq to be disk sync = true. This will
> alleviate some of the data mismatch where we have potential extra keys
> after compacting the oplogs.
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.3.15#6346)
>