You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@couchdb.apache.org by GitBox <gi...@apache.org> on 2021/09/09 21:22:58 UTC
[GitHub] [couchdb] nickva opened a new issue #3738: Shard splitting job failed
nickva opened a new issue #3738:
URL: https://github.com/apache/couchdb/issues/3738
### Discussed in https://github.com/apache/couchdb/discussions/3737
<div type='discussions-op-text'>
<sup>Originally posted by **csemio** September 9, 2021</sup>
I have a database that contains around 100,000 docs, with 2 shards
I want to split one shards in two.
I do that :
````
range="80000000-ffffffff"
curl -u$credentials -s -H "Content-type: application/json" -XPOST \
$rootUrl"_reshard/jobs" \
-d '{"type": "split", "db":"'$dbName'", "range":"'$range'"}'
````
But the job failed with this error:
```
{
"total_rows": 1,
"offset": 0,
"jobs": [
{
"id": "001-c4c4d0cdde77c061d6c3fbb4183a5894a0675059a1acf9da34e7b965be8fdc34",
"type": "split",
"source": "shards/80000000-ffffffff/timezones.1631172775",
"target": [
"shards/80000000-bfffffff/timezones.1631172775",
"shards/c0000000-ffffffff/timezones.1631172775"
],
"job_state": "running",
"split_state": "initial_copy",
"state_info": {
"error": "{{invalid_start_purge_seq,0},\n [{couch_bt_engine,fold_purge_infos,5,\n [{file,\"src/couch_bt_engine.erl\"},{line,630}]},\n {couch_db_split,copy_purge_info,1,\n [{file,\"src/couch_db_split.erl\"},{line,304}]},\n {couch_db_split,split,6,[{file,\"src/couch_db_split.erl\"},{line,164}]},\n {couch_db_split,split,3,[{file,\"src/couch_db_split.erl\"},{line,65}]},\n {mem3_reshard_job,initial_copy_impl,1,\n [{file,\"src/mem3_reshard_job.erl\"},{line,400}]}]}"
},
"node": "nonode@nohost",
"start_time": "2021-09-09T07:38:29Z",
"update_time": "2021-09-09T07:38:59Z",
"history": [
{
"timestamp": "2021-09-09T07:38:29Z",
"type": "new",
"detail": null
},
{
"timestamp": "2021-09-09T07:38:29Z",
"type": "running",
"detail": "initial_copy"
}
]
}
]
}
```
</div>
I think this is a bug in how purge sequences are iterated over during shard splitting. It would happen in a database where purges were used extensively, as it seems to be this case, where the number of purges is higher than the max purge limit (1000) per shard. In that case, iterating over the purge sequence cannot happen from 0 and instead the minimal purge sequence has to be used, or it would crash.
It seems the fix would be to start from `couch_db_engine:get_oldest_purge_seq(Db) - 1` in [here](https://github.com/apache/couchdb/blob/3.x/src/couch/src/couch_db_split.erl#L304) instead of `0`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb] nickva closed issue #3738: Shard splitting job failed
Posted by GitBox <gi...@apache.org>.
nickva closed issue #3738:
URL: https://github.com/apache/couchdb/issues/3738
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb] nickva commented on issue #3738: Shard splitting job failed
Posted by GitBox <gi...@apache.org>.
nickva commented on issue #3738:
URL: https://github.com/apache/couchdb/issues/3738#issuecomment-918324335
This should be fixed. Closing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org