[jira] [Commented] (SOLR-5944) Support updates of numeric DocValues

["Hoss Man (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-5944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15806457#comment-15806457 ] 

Hoss Man commented on SOLR-5944:
--------------------------------

while reviewing some of ishan's recent commits to the jira/solr-5944 branch I was confused by how exactly {{TestRecovery.testLogReplayWithInPlaceUpdatesAndDeletes}} works and what it was doing, and asked ishan about it offline -- while he walked me through the test, we realized that at one point while writting it he had assumed some weird behavior he had seen relating to the update log replay was an artifact of the test harness, and he included this line to work around it...

{code}
      // Clearing the updatelog as it would happen after a fresh node restart
      h.getCore().getUpdateHandler().getUpdateLog().deleteAll();
{code}

..but the more we talked about it the more it seemed like a legitimate bug -- either in the new code, or in the existing log replay code.

I've been investigating and it seems like this is intentional, but weird, code on master, see SOLR-9941.

The net result being that the test as original written (w/o that call to {{getUpdateLog().deleteAll();}} really was finding a problematic situation with log recovery on the branch, because of how the existing DUH2 code tries to pre-emptively apply DBQs during log recovery.  We're either going to need to change things as part of SOLR-9941 first, or deal with out of order DBQs differently as part of this issue, because the current approach of "re-fetch whole doc from leader" won't work if the leader (or a single node install) is itself recovering from it's tlog.

----

Here's some simple steps to demonstrate the problem as it stands on the jira/solr-5944 branch (as of 5db04fd)...

{noformat}
bin/solr -e techproducts

curl -X POST http://localhost:8983/solr/techproducts/config -H 'Content-Type: application/json' --data-binary '{"set-property":{"updateHandler.autoCommit.maxTime":"-1"}}'

curl -X POST -H 'Content-type:application/json' --data-binary '{
  "add-field":{
     "name":"foo_dvo",
     "type":"int",
     "stored":false,
     "indexed":false,
     "docValues":true }
}' http://localhost:8983/solr/techproducts/schema

curl -X POST http://localhost:8983/solr/techproducts/update -H 'Content-Type: application/json' --data-binary '{
  "add":{
    "doc": {
      "id": "DOCX",
      "foo_dvo": 41
    }
   },
  "delete": { "query":"foo_dvo:42" },
  "delete": { "query":"foo_dvo:43" },
  "add":{
     "doc": {
      "id": "DOCX",
      "foo_dvo": { "inc" : "1" }
    }
  },
  "delete": { "query":"foo_dvo:41" },
  "delete": { "query":"foo_dvo:43" },
  "add":{
    "doc": {
      "id": "DOCX",
      "foo_dvo": { "inc" : "1" }
    }
  },
  "delete": { "query":"foo_dvo:41" },
  "delete": { "query":"foo_dvo:42" }
}'

# verify the in-place atomic updates were applied correctly...
curl 'http://localhost:8983/solr/techproducts/get?wt=json&id=DOCX'
{
  "doc":
  {
    "id":"DOCX",
    "_version_":1555823554278195200,
    "foo_dvo":43}}

# crash the node...
kill -9 PID # use whatever PID you get from "ps -ef | grep start.jar | grep techproducts"

# restart and let recovery from log replay happen...
bin/solr start -s example/techproducts/solr/

# because of how the DUH2/UpdateLog code interacts, the doc is deleted during one of the DBQs and the inplace update code can't recover it from anywhere else...
curl 'http://localhost:8983/solr/techproducts/get?wt=json&id=DOCX'
{
  "doc":null}

{noformat}



> Support updates of numeric DocValues
> ------------------------------------
>
>                 Key: SOLR-5944
>                 URL: https://issues.apache.org/jira/browse/SOLR-5944
>             Project: Solr
>          Issue Type: New Feature
>            Reporter: Ishan Chattopadhyaya
>            Assignee: Shalin Shekhar Mangar
>         Attachments: DUP.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, TestStressInPlaceUpdates.eb044ac71.beast-167-failure.stdout.txt, TestStressInPlaceUpdates.eb044ac71.beast-587-failure.stdout.txt, TestStressInPlaceUpdates.eb044ac71.failures.tar.gz, defensive-checks.log.gz, demo-why-dynamic-fields-cannot-be-inplace-updated-first-time.patch, hoss.62D328FA1DEA57FD.fail.txt, hoss.62D328FA1DEA57FD.fail2.txt, hoss.62D328FA1DEA57FD.fail3.txt, hoss.D768DD9443A98DC.fail.txt, hoss.D768DD9443A98DC.pass.txt
>
>
> LUCENE-5189 introduced support for updates to numeric docvalues. It would be really nice to have Solr support this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org