You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "Megan Carey (Jira)" <ji...@apache.org> on 2020/08/25 21:09:00 UTC
[jira] [Updated] (SOLR-14778) Disabling UpdateLog leads to silently
lost updates
[ https://issues.apache.org/jira/browse/SOLR-14778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Megan Carey updated SOLR-14778:
-------------------------------
Description:
Solr currently "supports" disabling the UpdateLog, though it is "required" for NRT replicas (per the [docs|#transaction-log]]). However, when the update log is disabled and a replica is in BUFFERING state (e.g. during MigrateCmd or SplitShardCmd), updates are *lost silently*. While most users will likely never consider disabling the updateLog, it seems pertinent to provide a better support option.
Options as discussed in [ASF Slack|[https://the-asf.slack.com/archives/CEKUCUNE9/p1598373062262300]]:
# No longer support disabling the updateLog as it is considered a pertinent feature in SolrCloud. This might be undesirable for use cases where some data loss is acceptable and the updateLog takes up too much space.
# Improve Solr documentation to explicitly outline the risks of disabling the updateLog.
# Add logging to indicate when an update is swallowed in this state.
# _My preferred option:_ Support disabling the updateLog by providing additional replica states besides BUFFERING, so that there is no data loss when updateLog is disabled and replica goes offline for an operation like split. Some ideas:
## REJECTING: Fail updates so that the client can retry again once the operation is complete.
## BLOCKING: Stall update until operation is complete, and then execute update.
Feedback is welcome; once we establish a path forward I'd be happy to pick it up. If others are interested I can document my findings as well.
was:
Solr currently "supports" disabling the UpdateLog, though it is "required" for NRT replicas (per the [docs|[https://lucene.apache.org/solr/guide/8_6/updatehandlers-in-solrconfig.html#transaction-log]]). However, when the update log is disabled and a replica is in BUFFERING state (e.g. during MigrateCmd or SplitShardCmd), updates are *lost silently*. While most users will likely never consider disabling the updateLog, it seems pertinent to provide a better support option.
Options as discussed in [ASF Slack|[https://the-asf.slack.com/archives/CEKUCUNE9/p1598373062262300]:]
# No longer support disabling the updateLog as it is considered a pertinent feature in SolrCloud. This might be undesirable for use cases where some data loss is acceptable and the updateLog takes up too much space.
# Improve Solr documentation to explicitly outline the risks of disabling the updateLog.
# Add logging to indicate when an update is swallowed in this state.
# _My preferred option:_ Support disabling the updateLog by providing additional replica states so that there is no data loss when updateLog is disabled and replica goes offline for an operation like split.
## REJECTING: Fail updates so that the client can retry again once the operation is complete.
## BLOCKING: Stall update until operation is complete, and then execute update.
Feedback is welcome; once we establish a path forward I'll be happy to pick it up. If others are interested I can document my findings as well.
> Disabling UpdateLog leads to silently lost updates
> --------------------------------------------------
>
> Key: SOLR-14778
> URL: https://issues.apache.org/jira/browse/SOLR-14778
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrCloud, update
> Affects Versions: 8.6.1
> Reporter: Megan Carey
> Priority: Minor
>
> Solr currently "supports" disabling the UpdateLog, though it is "required" for NRT replicas (per the [docs|#transaction-log]]). However, when the update log is disabled and a replica is in BUFFERING state (e.g. during MigrateCmd or SplitShardCmd), updates are *lost silently*. While most users will likely never consider disabling the updateLog, it seems pertinent to provide a better support option.
> Options as discussed in [ASF Slack|[https://the-asf.slack.com/archives/CEKUCUNE9/p1598373062262300]]:
> # No longer support disabling the updateLog as it is considered a pertinent feature in SolrCloud. This might be undesirable for use cases where some data loss is acceptable and the updateLog takes up too much space.
> # Improve Solr documentation to explicitly outline the risks of disabling the updateLog.
> # Add logging to indicate when an update is swallowed in this state.
> # _My preferred option:_ Support disabling the updateLog by providing additional replica states besides BUFFERING, so that there is no data loss when updateLog is disabled and replica goes offline for an operation like split. Some ideas:
> ## REJECTING: Fail updates so that the client can retry again once the operation is complete.
> ## BLOCKING: Stall update until operation is complete, and then execute update.
> Feedback is welcome; once we establish a path forward I'd be happy to pick it up. If others are interested I can document my findings as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org