You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@couchdb.apache.org by va...@apache.org on 2017/04/06 19:55:39 UTC
[couchdb-documentation] 01/01: Documention for the scheduling
replicator
This is an automated email from the ASF dual-hosted git repository.
vatamane pushed a commit to branch 63012-scheduler
in repository https://gitbox.apache.org/repos/asf/couchdb-documentation.git
commit 86b251f9156ea1da9d6bfa6ac726382b7dc53f81
Author: Nick Vatamaniuc <va...@apache.org>
AuthorDate: Thu Apr 6 11:54:16 2017 -0400
Documention for the scheduling replicator
"What's New" update includes feature improvements and also an implicit to 3.0
"Replicator" section describing some new behavior and includes a state transition diagram.
Jira: COUCHDB-3324
---
images/replication-state-diagram.svg | 419 ++++++++++++++++++++++++++++++++
src/replication/replicator.rst | 450 +++++++++++++++++++++++------------
src/whatsnew/3.0.rst | 105 ++++++++
src/whatsnew/index.rst | 1 +
4 files changed, 821 insertions(+), 154 deletions(-)
diff --git a/images/replication-state-diagram.svg b/images/replication-state-diagram.svg
new file mode 100644
index 0000000..f6fb826
--- /dev/null
+++ b/images/replication-state-diagram.svg
@@ -0,0 +1,419 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<svg version="1.2" width="215.9mm" height="279.4mm" viewBox="0 0 21590 27940" preserveAspectRatio="xMidYMid" fill-rule="evenodd" stroke-width="28.222" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" xmlns:ooo="http://xml.openoffice.org/svg/export" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:presentation="http://sun.com/xmlns/staroffice/presentation" xmlns:smil="http://www.w3.org/2001/SMIL20/" xmlns:anim="urn:oasis:names:tc:opendocument:xmlns:animation:1.0" xml:space="preserve">
+ <defs class="ClipPathGroup">
+ <clipPath id="presentation_clip_path" clipPathUnits="userSpaceOnUse">
+ <rect x="0" y="0" width="21590" height="27940"/>
+ </clipPath>
+ <clipPath id="presentation_clip_path_shrink" clipPathUnits="userSpaceOnUse">
+ <rect x="21" y="27" width="21547" height="27885"/>
+ </clipPath>
+ </defs>
+ <defs>
+ <font id="EmbeddedFont_1" horiz-adv-x="2048">
+ <font-face font-family="Liberation Sans embedded" units-per-em="2048" font-weight="normal" font-style="normal" ascent="1839" descent="421"/>
+ <missing-glyph horiz-adv-x="2048" d="M 0,0 L 2047,0 2047,2047 0,2047 0,0 Z"/>
+ <glyph unicode="z" horiz-adv-x="842" d="M 83,0 L 83,137 688,943 117,943 117,1082 901,1082 901,945 295,139 922,139 922,0 Z"/>
+ <glyph unicode="y" horiz-adv-x="1014" d="M 191,-425 C 142,-425 100,-421 67,-414 L 67,-279 C 92,-283 120,-285 151,-285 263,-285 352,-203 417,-38 L 434,5 5,1082 197,1082 425,484 C 428,475 432,464 437,451 442,438 457,394 482,320 507,246 521,205 523,196 L 593,393 830,1082 1020,1082 604,0 C 559,-115 518,-201 479,-258 440,-314 398,-356 351,-384 304,-411 250,-425 191,-425 Z"/>
+ <glyph unicode="x" horiz-adv-x="982" d="M 801,0 L 510,444 217,0 23,0 408,556 41,1082 240,1082 510,661 778,1082 979,1082 612,558 1002,0 Z"/>
+ <glyph unicode="w" horiz-adv-x="1481" d="M 1174,0 L 965,0 776,765 740,934 C 734,904 725,861 712,805 699,748 631,480 508,0 L 300,0 -3,1082 175,1082 358,347 C 363,331 377,265 401,149 L 418,223 644,1082 837,1082 1026,339 1072,149 1103,288 1308,1082 1484,1082 Z"/>
+ <glyph unicode="u" horiz-adv-x="858" d="M 314,1082 L 314,396 C 314,325 321,269 335,230 349,191 371,162 402,145 433,128 478,119 537,119 624,119 692,149 742,208 792,267 817,350 817,455 L 817,1082 997,1082 997,231 C 997,105 999,28 1003,0 L 833,0 C 832,3 832,12 831,27 830,42 830,59 829,78 828,97 826,132 825,185 L 822,185 C 781,110 733,58 679,27 624,-5 557,-20 476,-20 357,-20 271,10 216,69 161,128 133,225 133,361 L 133,1082 Z"/>
+ <glyph unicode="t" horiz-adv-x="531" d="M 554,8 C 495,-8 434,-16 372,-16 228,-16 156,66 156,229 L 156,951 31,951 31,1082 163,1082 216,1324 336,1324 336,1082 536,1082 536,951 336,951 336,268 C 336,216 345,180 362,159 379,138 408,127 450,127 474,127 509,132 554,141 Z"/>
+ <glyph unicode="s" horiz-adv-x="890" d="M 950,299 C 950,197 912,118 835,63 758,8 650,-20 511,-20 376,-20 273,2 200,47 127,91 79,160 57,254 L 216,285 C 231,227 263,185 311,158 359,131 426,117 511,117 602,117 669,131 712,159 754,187 775,229 775,285 775,328 760,362 731,389 702,416 654,438 589,455 L 460,489 C 357,516 283,542 240,568 196,593 162,624 137,661 112,698 100,743 100,796 100,895 135,970 206,1022 276,1073 378,1099 513,1099 632,1099 727,1078 798,1036 868,994 912,927 931,834 L 769,8 [...]
+ <glyph unicode="r" horiz-adv-x="515" d="M 142,0 L 142,830 C 142,906 140,990 136,1082 L 306,1082 C 311,959 314,886 314,861 L 318,861 C 347,954 380,1017 417,1051 454,1085 507,1102 575,1102 599,1102 623,1099 648,1092 L 648,927 C 624,934 592,937 552,937 477,937 420,905 381,841 342,776 322,684 322,564 L 322,0 Z"/>
+ <glyph unicode="p" horiz-adv-x="936" d="M 1053,546 C 1053,169 920,-20 655,-20 488,-20 376,43 319,168 L 314,168 C 317,163 318,106 318,-2 L 318,-425 138,-425 138,861 C 138,972 136,1046 132,1082 L 306,1082 C 307,1079 308,1070 309,1054 310,1037 312,1012 314,978 315,944 316,921 316,908 L 320,908 C 352,975 394,1024 447,1055 500,1086 569,1101 655,1101 788,1101 888,1056 954,967 1020,878 1053,737 1053,546 Z M 864,542 C 864,693 844,800 803,865 762,930 698,962 609,962 538,962 482,947 442,917 401 [...]
+ <glyph unicode="o" horiz-adv-x="968" d="M 1053,542 C 1053,353 1011,212 928,119 845,26 724,-20 565,-20 407,-20 288,28 207,125 126,221 86,360 86,542 86,915 248,1102 571,1102 736,1102 858,1057 936,966 1014,875 1053,733 1053,542 Z M 864,542 C 864,691 842,800 798,868 753,935 679,969 574,969 469,969 393,935 346,866 299,797 275,689 275,542 275,399 298,292 345,221 391,149 464,113 563,113 671,113 748,148 795,217 841,286 864,395 864,542 Z"/>
+ <glyph unicode="n" horiz-adv-x="874" d="M 825,0 L 825,686 C 825,757 818,813 804,852 790,891 768,920 737,937 706,954 661,963 602,963 515,963 447,933 397,874 347,815 322,732 322,627 L 322,0 142,0 142,851 C 142,977 140,1054 136,1082 L 306,1082 C 307,1079 307,1070 308,1055 309,1040 310,1024 311,1005 312,986 313,950 314,897 L 317,897 C 358,972 406,1025 461,1056 515,1087 582,1102 663,1102 782,1102 869,1073 924,1014 979,955 1006,857 1006,721 L 1006,0 Z"/>
+ <glyph unicode="m" horiz-adv-x="1435" d="M 768,0 L 768,686 C 768,791 754,863 725,903 696,943 645,963 570,963 493,963 433,934 388,875 343,816 321,734 321,627 L 321,0 142,0 142,851 C 142,977 140,1054 136,1082 L 306,1082 C 307,1079 307,1070 308,1055 309,1040 310,1024 311,1005 312,986 313,950 314,897 L 317,897 C 356,974 400,1027 450,1057 500,1087 561,1102 633,1102 715,1102 780,1086 828,1053 875,1020 908,968 927,897 L 930,897 C 967,970 1013,1022 1066,1054 1119,1086 1183,1102 1258,1102 1367 [...]
+ <glyph unicode="l" horiz-adv-x="173" d="M 138,0 L 138,1484 318,1484 318,0 Z"/>
+ <glyph unicode="j" horiz-adv-x="360" d="M 137,1312 L 137,1484 317,1484 317,1312 Z M 317,-134 C 317,-236 297,-310 257,-356 217,-402 157,-425 77,-425 26,-425 -17,-422 -50,-416 L -50,-277 12,-283 C 58,-283 90,-271 109,-247 128,-223 137,-176 137,-107 L 137,1082 317,1082 Z"/>
+ <glyph unicode="i" horiz-adv-x="173" d="M 137,1312 L 137,1484 317,1484 317,1312 Z M 137,0 L 137,1082 317,1082 317,0 Z"/>
+ <glyph unicode="h" horiz-adv-x="874" d="M 317,897 C 356,968 402,1020 457,1053 511,1086 580,1102 663,1102 780,1102 867,1073 923,1015 978,956 1006,858 1006,721 L 1006,0 825,0 825,686 C 825,762 818,819 804,856 790,893 767,920 735,937 703,954 659,963 602,963 517,963 450,934 399,875 348,816 322,737 322,638 L 322,0 142,0 142,1484 322,1484 322,1098 C 322,1057 321,1015 319,972 316,929 315,904 314,897 Z"/>
+ <glyph unicode="g" horiz-adv-x="921" d="M 548,-425 C 430,-425 336,-402 266,-356 196,-309 151,-243 131,-158 L 312,-132 C 324,-182 351,-221 392,-248 433,-275 486,-288 553,-288 732,-288 822,-183 822,27 L 822,201 820,201 C 786,132 739,80 680,45 621,10 551,-8 472,-8 339,-8 242,36 180,124 117,212 86,350 86,539 86,730 120,872 187,963 254,1054 355,1099 492,1099 569,1099 635,1082 692,1047 748,1012 791,962 822,897 L 824,897 C 824,917 825,952 828,1001 831,1050 833,1077 836,1082 L 1007,1082 C 100 [...]
+ <glyph unicode="f" horiz-adv-x="547" d="M 361,951 L 361,0 181,0 181,951 29,951 29,1082 181,1082 181,1204 C 181,1303 203,1374 246,1417 289,1460 356,1482 445,1482 495,1482 537,1478 572,1470 L 572,1333 C 542,1338 515,1341 492,1341 446,1341 413,1329 392,1306 371,1283 361,1240 361,1179 L 361,1082 572,1082 572,951 Z"/>
+ <glyph unicode="e" horiz-adv-x="952" d="M 276,503 C 276,379 302,283 353,216 404,149 479,115 578,115 656,115 719,131 766,162 813,193 844,233 861,281 L 1019,236 C 954,65 807,-20 578,-20 418,-20 296,28 213,123 129,218 87,360 87,548 87,727 129,864 213,959 296,1054 416,1102 571,1102 889,1102 1048,910 1048,527 L 1048,503 Z M 862,641 C 852,755 823,838 775,891 727,943 658,969 568,969 481,969 412,940 361,882 310,823 282,743 278,641 Z"/>
+ <glyph unicode="d" horiz-adv-x="921" d="M 821,174 C 788,105 744,55 689,25 634,-5 565,-20 484,-20 347,-20 247,26 183,118 118,210 86,349 86,536 86,913 219,1102 484,1102 566,1102 634,1087 689,1057 744,1027 788,979 821,914 L 823,914 821,1035 821,1484 1001,1484 1001,223 C 1001,110 1003,36 1007,0 L 835,0 C 833,11 831,35 829,74 826,113 825,146 825,174 Z M 275,542 C 275,391 295,282 335,217 375,152 440,119 530,119 632,119 706,154 752,225 798,296 821,405 821,554 821,697 798,802 752,869 706,936 [...]
+ <glyph unicode="c" horiz-adv-x="874" d="M 275,546 C 275,402 298,295 343,226 388,157 457,122 548,122 612,122 666,139 709,174 752,209 778,262 788,334 L 970,322 C 956,218 912,135 837,73 762,11 668,-20 553,-20 402,-20 286,28 207,124 127,219 87,359 87,542 87,724 127,863 207,959 287,1054 402,1102 551,1102 662,1102 754,1073 827,1016 900,959 945,880 964,779 L 779,765 C 770,825 746,873 708,908 670,943 616,961 546,961 451,961 382,929 339,866 296,803 275,696 275,546 Z"/>
+ <glyph unicode="b" horiz-adv-x="936" d="M 1053,546 C 1053,169 920,-20 655,-20 573,-20 505,-5 451,25 396,54 352,102 318,168 L 316,168 C 316,147 315,116 312,74 309,31 307,7 306,0 L 132,0 C 136,36 138,110 138,223 L 138,1484 318,1484 318,1061 C 318,1018 317,967 314,908 L 318,908 C 351,977 396,1027 451,1057 506,1087 574,1102 655,1102 792,1102 892,1056 957,964 1021,872 1053,733 1053,546 Z M 864,540 C 864,691 844,800 804,865 764,930 699,963 609,963 508,963 434,928 388,859 341,790 318,680 318 [...]
+ <glyph unicode="a" horiz-adv-x="1046" d="M 414,-20 C 305,-20 224,9 169,66 114,123 87,202 87,302 87,414 124,500 198,560 271,620 390,652 554,656 L 797,660 797,719 C 797,807 778,870 741,908 704,946 645,965 565,965 484,965 426,951 389,924 352,897 330,853 323,793 L 135,810 C 166,1005 310,1102 569,1102 705,1102 807,1071 876,1009 945,946 979,856 979,738 L 979,272 C 979,219 986,179 1000,152 1014,125 1041,111 1080,111 1097,111 1117,113 1139,118 L 1139,6 C 1094,-5 1047,-10 1000,-10 933,-10 885, [...]
+ <glyph unicode="_" horiz-adv-x="1201" d="M -31,-407 L -31,-277 1162,-277 1162,-407 Z"/>
+ <glyph unicode="U" horiz-adv-x="1170" d="M 731,-20 C 616,-20 515,1 429,43 343,85 276,146 229,226 182,306 158,401 158,512 L 158,1409 349,1409 349,528 C 349,399 382,302 447,235 512,168 607,135 730,135 857,135 955,170 1026,239 1096,308 1131,408 1131,541 L 1131,1409 1321,1409 1321,530 C 1321,416 1297,318 1249,235 1200,152 1132,89 1044,46 955,2 851,-20 731,-20 Z"/>
+ <glyph unicode="T" horiz-adv-x="1154" d="M 720,1253 L 720,0 530,0 530,1253 46,1253 46,1409 1204,1409 1204,1253 Z"/>
+ <glyph unicode="S" horiz-adv-x="1186" d="M 1272,389 C 1272,259 1221,158 1120,87 1018,16 875,-20 690,-20 347,-20 148,99 93,338 L 278,375 C 299,290 345,228 414,189 483,149 578,129 697,129 820,129 916,150 983,193 1050,235 1083,297 1083,379 1083,425 1073,462 1052,491 1031,520 1001,543 963,562 925,581 880,596 827,609 774,622 716,635 652,650 541,675 456,699 399,724 341,749 295,776 262,807 229,837 203,872 186,913 168,954 159,1000 159,1053 159,1174 205,1267 298,1332 390,1397 522,1430 694,1430 [...]
+ <glyph unicode="R" horiz-adv-x="1217" d="M 1164,0 L 798,585 359,585 359,0 168,0 168,1409 831,1409 C 990,1409 1112,1374 1199,1303 1285,1232 1328,1133 1328,1006 1328,901 1298,813 1237,742 1176,671 1091,626 984,607 L 1384,0 Z M 1136,1004 C 1136,1086 1108,1149 1053,1192 997,1235 917,1256 812,1256 L 359,1256 359,736 820,736 C 921,736 999,760 1054,807 1109,854 1136,919 1136,1004 Z"/>
+ <glyph unicode="P" horiz-adv-x="1092" d="M 1258,985 C 1258,852 1215,746 1128,667 1041,588 922,549 773,549 L 359,549 359,0 168,0 168,1409 761,1409 C 919,1409 1041,1372 1128,1298 1215,1224 1258,1120 1258,985 Z M 1066,983 C 1066,1165 957,1256 738,1256 L 359,1256 359,700 746,700 C 959,700 1066,794 1066,983 Z"/>
+ <glyph unicode="O" horiz-adv-x="1404" d="M 1495,711 C 1495,564 1467,435 1411,324 1354,213 1273,128 1168,69 1063,10 938,-20 795,-20 650,-20 526,9 421,68 316,127 235,212 180,323 125,434 97,563 97,711 97,936 159,1113 282,1240 405,1367 577,1430 797,1430 940,1430 1065,1402 1170,1345 1275,1288 1356,1205 1412,1096 1467,987 1495,859 1495,711 Z M 1300,711 C 1300,886 1256,1024 1169,1124 1081,1224 957,1274 797,1274 636,1274 511,1225 423,1126 335,1027 291,889 291,711 291,534 336,394 425,291 514,1 [...]
+ <glyph unicode="N" horiz-adv-x="1139" d="M 1082,0 L 328,1200 333,1103 338,936 338,0 168,0 168,1409 390,1409 1152,201 C 1144,332 1140,426 1140,485 L 1140,1409 1312,1409 1312,0 Z"/>
+ <glyph unicode="I" horiz-adv-x="188" d="M 189,0 L 189,1409 380,1409 380,0 Z"/>
+ <glyph unicode="H" horiz-adv-x="1139" d="M 1121,0 L 1121,653 359,653 359,0 168,0 168,1409 359,1409 359,813 1121,813 1121,1409 1312,1409 1312,0 Z"/>
+ <glyph unicode="F" horiz-adv-x="999" d="M 359,1253 L 359,729 1145,729 1145,571 359,571 359,0 168,0 168,1409 1169,1409 1169,1253 Z"/>
+ <glyph unicode="E" horiz-adv-x="1108" d="M 168,0 L 168,1409 1237,1409 1237,1253 359,1253 359,801 1177,801 1177,647 359,647 359,156 1278,156 1278,0 Z"/>
+ <glyph unicode="C" horiz-adv-x="1294" d="M 792,1274 C 636,1274 515,1224 428,1124 341,1023 298,886 298,711 298,538 343,400 434,295 524,190 646,137 800,137 997,137 1146,235 1245,430 L 1401,352 C 1343,231 1262,138 1157,75 1052,12 930,-20 791,-20 649,-20 526,10 423,69 319,128 240,212 186,322 131,431 104,561 104,711 104,936 165,1112 286,1239 407,1366 575,1430 790,1430 940,1430 1065,1401 1166,1342 1267,1283 1341,1196 1388,1081 L 1207,1021 C 1174,1103 1122,1166 1050,1209 977,1252 891,1274 79 [...]
+ <glyph unicode="A" horiz-adv-x="1357" d="M 1167,0 L 1006,412 364,412 202,0 4,0 579,1409 796,1409 1362,0 Z M 685,1265 L 676,1237 C 659,1182 635,1111 602,1024 L 422,561 949,561 768,1026 C 749,1072 731,1124 712,1182 Z"/>
+ <glyph unicode=">" horiz-adv-x="999" d="M 101,154 L 101,307 959,674 101,1040 101,1194 1096,776 1096,571 Z"/>
+ <glyph unicode="<" horiz-adv-x="999" d="M 101,571 L 101,776 1096,1194 1096,1040 238,674 1096,307 1096,154 Z"/>
+ <glyph unicode="/" horiz-adv-x="578" d="M 0,-20 L 411,1484 569,1484 162,-20 Z"/>
+ <glyph unicode="-" horiz-adv-x="500" d="M 91,464 L 91,624 591,624 591,464 Z"/>
+ <glyph unicode=")" horiz-adv-x="546" d="M 555,528 C 555,335 525,162 465,9 404,-144 311,-289 186,-424 L 12,-424 C 137,-284 229,-137 287,19 345,174 374,344 374,530 374,716 345,887 287,1042 228,1197 137,1345 12,1484 L 186,1484 C 312,1348 405,1203 465,1050 525,896 555,723 555,532 Z"/>
+ <glyph unicode="(" horiz-adv-x="546" d="M 127,532 C 127,725 157,898 218,1051 278,1204 371,1349 496,1484 L 670,1484 C 545,1345 454,1198 396,1042 337,886 308,715 308,530 308,345 337,175 395,20 452,-135 544,-283 670,-424 L 496,-424 C 370,-288 277,-143 217,11 157,164 127,337 127,528 Z"/>
+ <glyph unicode=" " horiz-adv-x="561"/>
+ </font>
+ </defs>
+ <defs class="TextShapeIndex">
+ <g ooo:slide="id1" ooo:id-list="id3 id4 id5 id6 id7 id8 id9 id10 id11 id12 id13 id14 id15 id16 id17 id18 id19 id20 id21 id22 id23 id24 id25 id26 id27 id28 id29 id30 id31 id32 id33 id34 id35 id36 id37 id38 id39 id40"/>
+ </defs>
+ <defs class="EmbeddedBulletChars">
+ <g id="bullet-char-template(57356)" transform="scale(0.00048828125,-0.00048828125)">
+ <path d="M 580,1141 L 1163,571 580,0 -4,571 580,1141 Z"/>
+ </g>
+ <g id="bullet-char-template(57354)" transform="scale(0.00048828125,-0.00048828125)">
+ <path d="M 8,1128 L 1137,1128 1137,0 8,0 8,1128 Z"/>
+ </g>
+ <g id="bullet-char-template(10146)" transform="scale(0.00048828125,-0.00048828125)">
+ <path d="M 174,0 L 602,739 174,1481 1456,739 174,0 Z M 1358,739 L 309,1346 659,739 1358,739 Z"/>
+ </g>
+ <g id="bullet-char-template(10132)" transform="scale(0.00048828125,-0.00048828125)">
+ <path d="M 2015,739 L 1276,0 717,0 1260,543 174,543 174,936 1260,936 717,1481 1274,1481 2015,739 Z"/>
+ </g>
+ <g id="bullet-char-template(10007)" transform="scale(0.00048828125,-0.00048828125)">
+ <path d="M 0,-2 C -7,14 -16,27 -25,37 L 356,567 C 262,823 215,952 215,954 215,979 228,992 255,992 264,992 276,990 289,987 310,991 331,999 354,1012 L 381,999 492,748 772,1049 836,1024 860,1049 C 881,1039 901,1025 922,1006 886,937 835,863 770,784 769,783 710,716 594,584 L 774,223 C 774,196 753,168 711,139 L 727,119 C 717,90 699,76 672,76 641,76 570,178 457,381 L 164,-76 C 142,-110 111,-127 72,-127 30,-127 9,-110 8,-76 1,-67 -2,-52 -2,-32 -2,-23 -1,-13 0,-2 Z"/>
+ </g>
+ <g id="bullet-char-template(10004)" transform="scale(0.00048828125,-0.00048828125)">
+ <path d="M 285,-33 C 182,-33 111,30 74,156 52,228 41,333 41,471 41,549 55,616 82,672 116,743 169,778 240,778 293,778 328,747 346,684 L 369,508 C 377,444 397,411 428,410 L 1163,1116 C 1174,1127 1196,1133 1229,1133 1271,1133 1292,1118 1292,1087 L 1292,965 C 1292,929 1282,901 1262,881 L 442,47 C 390,-6 338,-33 285,-33 Z"/>
+ </g>
+ <g id="bullet-char-template(9679)" transform="scale(0.00048828125,-0.00048828125)">
+ <path d="M 813,0 C 632,0 489,54 383,161 276,268 223,411 223,592 223,773 276,916 383,1023 489,1130 632,1184 813,1184 992,1184 1136,1130 1245,1023 1353,916 1407,772 1407,592 1407,412 1353,268 1245,161 1136,54 992,0 813,0 Z"/>
+ </g>
+ <g id="bullet-char-template(8226)" transform="scale(0.00048828125,-0.00048828125)">
+ <path d="M 346,457 C 273,457 209,483 155,535 101,586 74,649 74,723 74,796 101,859 155,911 209,963 273,989 346,989 419,989 480,963 531,910 582,859 608,796 608,723 608,648 583,586 532,535 482,483 420,457 346,457 Z"/>
+ </g>
+ <g id="bullet-char-template(8211)" transform="scale(0.00048828125,-0.00048828125)">
+ <path d="M -4,459 L 1135,459 1135,606 -4,606 -4,459 Z"/>
+ </g>
+ <g id="bullet-char-template(61548)" transform="scale(0.00048828125,-0.00048828125)">
+ <path d="M 173,740 C 173,903 231,1043 346,1159 462,1274 601,1332 765,1332 928,1332 1067,1274 1183,1159 1299,1043 1357,903 1357,740 1357,577 1299,437 1183,322 1067,206 928,148 765,148 601,148 462,206 346,322 231,437 173,577 173,740 Z"/>
+ </g>
+ </defs>
+ <defs class="TextEmbeddedBitmaps"/>
+ <g>
+ <g id="id2" class="Master_Slide">
+ <g id="bg-id2" class="Background"/>
+ <g id="bo-id2" class="BackgroundObjects"/>
+ </g>
+ </g>
+ <g class="SlideGroup">
+ <g>
+ <g id="container-id1">
+ <g id="id1" class="Slide" clip-path="url(#presentation_clip_path)">
+ <g class="Page">
+ <g class="com.sun.star.drawing.CustomShape">
+ <g id="id3">
+ <rect class="BoundingBox" stroke="none" fill="none" x="4554" y="7857" width="2925" height="1274"/>
+ <path fill="rgb(255,204,204)" stroke="none" d="M 6015,7858 C 6843,7858 7476,8133 7476,8493 7476,8853 6843,9128 6015,9128 5187,9128 4555,8853 4555,8493 4555,8133 5187,7858 6015,7858 Z M 4555,7858 L 4555,7858 Z M 7477,9129 L 7477,9129 Z"/>
+ <path fill="none" stroke="rgb(52,101,164)" d="M 6015,7858 C 6843,7858 7476,8133 7476,8493 7476,8853 6843,9128 6015,9128 5187,9128 4555,8853 4555,8493 4555,8133 5187,7858 6015,7858 Z"/>
+ <path fill="none" stroke="rgb(52,101,164)" d="M 4555,7858 L 4555,7858 Z"/>
+ <path fill="none" stroke="rgb(52,101,164)" d="M 7477,9129 L 7477,9129 Z"/>
+ <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="635px" font-weight="400"><tspan class="TextPosition" x="5311" y="8714"><tspan fill="rgb(0,0,0)" stroke="none">Error</tspan></tspan></tspan></text>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.CustomShape">
+ <g id="id4">
+ <rect class="BoundingBox" stroke="none" fill="none" x="1887" y="15857" width="3433" height="1655"/>
+ <path fill="rgb(204,255,204)" stroke="none" d="M 3602,15858 C 4574,15858 5317,16215 5317,16683 5317,17151 4574,17509 3602,17509 2630,17509 1888,17151 1888,16683 1888,16215 2630,15858 3602,15858 Z M 1888,15858 L 1888,15858 Z M 5318,17510 L 5318,17510 Z"/>
+ <path fill="none" stroke="rgb(52,101,164)" d="M 3602,15858 C 4574,15858 5317,16215 5317,16683 5317,17151 4574,17509 3602,17509 2630,17509 1888,17151 1888,16683 1888,16215 2630,15858 3602,15858 Z"/>
+ <path fill="none" stroke="rgb(52,101,164)" d="M 1888,15858 L 1888,15858 Z"/>
+ <path fill="none" stroke="rgb(52,101,164)" d="M 5318,17510 L 5318,17510 Z"/>
+ <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="635px" font-weight="400"><tspan class="TextPosition" x="2442" y="16904"><tspan fill="rgb(0,0,0)" stroke="none">Pending</tspan></tspan></tspan></text>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.CustomShape">
+ <g id="id5">
+ <rect class="BoundingBox" stroke="none" fill="none" x="8999" y="15857" width="3433" height="1655"/>
+ <path fill="rgb(204,255,204)" stroke="none" d="M 10714,15858 C 11686,15858 12429,16215 12429,16683 12429,17151 11686,17509 10714,17509 9742,17509 9000,17151 9000,16683 9000,16215 9742,15858 10714,15858 Z M 9000,15858 L 9000,15858 Z M 12430,17510 L 12430,17510 Z"/>
+ <path fill="none" stroke="rgb(52,101,164)" d="M 10714,15858 C 11686,15858 12429,16215 12429,16683 12429,17151 11686,17509 10714,17509 9742,17509 9000,17151 9000,16683 9000,16215 9742,15858 10714,15858 Z"/>
+ <path fill="none" stroke="rgb(52,101,164)" d="M 9000,15858 L 9000,15858 Z"/>
+ <path fill="none" stroke="rgb(52,101,164)" d="M 12430,17510 L 12430,17510 Z"/>
+ <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="635px" font-weight="400"><tspan class="TextPosition" x="9537" y="16904"><tspan fill="rgb(0,0,0)" stroke="none">Running</tspan></tspan></tspan></text>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.CustomShape">
+ <g id="id6">
+ <rect class="BoundingBox" stroke="none" fill="none" x="15984" y="15985" width="3433" height="1528"/>
+ <path fill="rgb(255,204,204)" stroke="none" d="M 17699,15986 C 18671,15986 19414,16315 19414,16747 19414,17179 18671,17509 17699,17509 16727,17509 15985,17179 15985,16747 15985,16315 16727,15986 17699,15986 Z M 15985,15986 L 15985,15986 Z M 19415,17511 L 19415,17511 Z"/>
+ <path fill="none" stroke="rgb(52,101,164)" d="M 17699,15986 C 18671,15986 19414,16315 19414,16747 19414,17179 18671,17509 17699,17509 16727,17509 15985,17179 15985,16747 15985,16315 16727,15986 17699,15986 Z"/>
+ <path fill="none" stroke="rgb(52,101,164)" d="M 15985,15986 L 15985,15986 Z"/>
+ <path fill="none" stroke="rgb(52,101,164)" d="M 19415,17511 L 19415,17511 Z"/>
+ <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="635px" font-weight="400"><tspan class="TextPosition" x="16433" y="16969"><tspan fill="rgb(0,0,0)" stroke="none">Crashing</tspan></tspan></tspan></text>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.CustomShape">
+ <g id="id7">
+ <rect class="BoundingBox" stroke="none" fill="none" x="13318" y="7857" width="2797" height="1273"/>
+ <path fill="rgb(255,204,204)" stroke="none" d="M 14716,9128 L 13319,9128 13319,7858 16113,7858 16113,9128 14716,9128 Z"/>
+ <path fill="none" stroke="rgb(52,101,164)" d="M 14716,9128 L 13319,9128 13319,7858 16113,7858 16113,9128 14716,9128 Z"/>
+ <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="635px" font-weight="400"><tspan class="TextPosition" x="13855" y="8714"><tspan fill="rgb(0,0,0)" stroke="none">Failed</tspan></tspan></tspan></text>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.CustomShape">
+ <g id="id8">
+ <rect class="BoundingBox" stroke="none" fill="none" x="8872" y="21191" width="3686" height="1654"/>
+ <path fill="rgb(204,255,204)" stroke="none" d="M 10715,22843 L 8873,22843 8873,21192 12556,21192 12556,22843 10715,22843 Z"/>
+ <path fill="none" stroke="rgb(52,101,164)" d="M 10715,22843 L 8873,22843 8873,21192 12556,21192 12556,22843 10715,22843 Z"/>
+ <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="635px" font-weight="400"><tspan class="TextPosition" x="9184" y="22238"><tspan fill="rgb(0,0,0)" stroke="none">Completed</tspan></tspan></tspan></text>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.CustomShape">
+ <g id="id9">
+ <rect class="BoundingBox" stroke="none" fill="none" x="7603" y="5189" width="6100" height="639"/>
+ <path fill="rgb(255,255,255)" fill-opacity="0.988" stroke="rgb(255,255,255)" stroke-opacity="0.988" d="M 8831,5190 L 12473,5190 13701,5508 12473,5826 8831,5826 7604,5508 8831,5190 8831,5190 Z M 7604,5190 L 7604,5190 Z M 13701,5826 L 13701,5826 Z"/>
+ <path fill="none" stroke="rgb(52,101,164)" d="M 8831,5190 L 12473,5190 13701,5508 12473,5826 8831,5826 7604,5508 8831,5190 8831,5190 Z"/>
+ <path fill="none" stroke="rgb(52,101,164)" d="M 7604,5190 L 7604,5190 Z"/>
+ <path fill="none" stroke="rgb(52,101,164)" d="M 13701,5826 L 13701,5826 Z"/>
+ <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="353px" font-weight="400"><tspan class="TextPosition" x="8613" y="5654"><tspan fill="rgb(0,0,0)" stroke="none">Create job from documen</tspan><tspan font-size="423px" fill="rgb(0,0,0)" stroke="none">t</tspan></tspan></tspan></text>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.CustomShape">
+ <g id="id10">
+ <rect class="BoundingBox" stroke="none" fill="none" x="8110" y="12556" width="5211" height="512"/>
+ <path fill="rgb(255,255,255)" stroke="none" d="M 9159,12557 L 12270,12557 13319,12811 12270,13066 9159,13066 8111,12811 9159,12557 9159,12557 Z M 8111,12557 L 8111,12557 Z M 13319,13066 L 13319,13066 Z"/>
+ <path fill="none" stroke="rgb(52,101,164)" d="M 9159,12557 L 12270,12557 13319,12811 12270,13066 9159,13066 8111,12811 9159,12557 9159,12557 Z"/>
+ <path fill="none" stroke="rgb(52,101,164)" d="M 8111,12557 L 8111,12557 Z"/>
+ <path fill="none" stroke="rgb(52,101,164)" d="M 13319,13066 L 13319,13066 Z"/>
+ <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="353px" font-weight="400"><tspan class="TextPosition" x="9332" y="12935"><tspan fill="rgb(0,0,0)" stroke="none">Schedule new job</tspan></tspan></tspan></text>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.ConnectorShape">
+ <g id="id11">
+ <rect class="BoundingBox" stroke="none" fill="none" x="10502" y="2396" width="301" height="764"/>
+ <path fill="none" stroke="rgb(0,0,0)" d="M 10652,2397 L 10652,2729"/>
+ <path fill="rgb(0,0,0)" stroke="none" d="M 10652,3159 L 10802,2709 10502,2709 10652,3159 Z"/>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.ConnectorShape">
+ <g id="id12">
+ <rect class="BoundingBox" stroke="none" fill="none" x="10651" y="5825" width="4075" height="2034"/>
+ <path fill="none" stroke="rgb(0,0,0)" d="M 10652,5826 C 10652,7350 13936,6513 14600,7469"/>
+ <path fill="rgb(0,0,0)" stroke="none" d="M 14716,7858 L 14724,7384 14438,7474 14716,7858 Z"/>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.ConnectorShape">
+ <g id="id13">
+ <rect class="BoundingBox" stroke="none" fill="none" x="6017" y="5825" width="4637" height="2034"/>
+ <path fill="none" stroke="rgb(0,0,0)" d="M 10652,5826 C 10652,7350 6890,6510 6145,7475"/>
+ <path fill="rgb(0,0,0)" stroke="none" d="M 6017,7858 L 6307,7483 6024,7384 6017,7858 Z"/>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.ConnectorShape">
+ <g id="id14">
+ <rect class="BoundingBox" stroke="none" fill="none" x="10655" y="9127" width="8135" height="3431"/>
+ <path fill="none" stroke="rgb(0,0,0)" d="M 18788,9128 C 18788,11700 11591,10164 10788,12166"/>
+ <path fill="rgb(0,0,0)" stroke="none" d="M 10716,12557 L 10949,12144 10655,12087 10716,12557 Z"/>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.ConnectorShape">
+ <g id="id15">
+ <rect class="BoundingBox" stroke="none" fill="none" x="10565" y="5825" width="302" height="6733"/>
+ <path fill="none" stroke="rgb(0,0,0)" d="M 10652,5826 C 10652,10875 10712,7727 10716,12076"/>
+ <path fill="rgb(0,0,0)" stroke="none" d="M 10716,12557 L 10866,12107 10566,12107 10716,12557 Z"/>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.ConnectorShape">
+ <g id="id16">
+ <rect class="BoundingBox" stroke="none" fill="none" x="10566" y="13064" width="301" height="2796"/>
+ <path fill="none" stroke="rgb(0,0,0)" d="M 10716,13065 L 10716,15429"/>
+ <path fill="rgb(0,0,0)" stroke="none" d="M 10716,15859 L 10866,15409 10566,15409 10716,15859 Z"/>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.ConnectorShape">
+ <g id="id17">
+ <rect class="BoundingBox" stroke="none" fill="none" x="10565" y="17509" width="302" height="3685"/>
+ <path fill="none" stroke="rgb(0,0,0)" d="M 10716,17510 C 10716,20271 10715,18639 10715,20734"/>
+ <path fill="rgb(0,0,0)" stroke="none" d="M 10715,21193 L 10865,20743 10565,20743 10715,21193 Z"/>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.ConnectorShape">
+ <g id="id18">
+ <rect class="BoundingBox" stroke="none" fill="none" x="4816" y="15497" width="4689" height="605"/>
+ <path fill="none" stroke="rgb(0,0,0)" d="M 9503,16100 C 9503,15343 6051,15358 5072,15824"/>
+ <path fill="rgb(0,0,0)" stroke="none" d="M 4816,16100 L 5236,15880 5020,15672 4816,16100 Z"/>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.ConnectorShape">
+ <g id="id19">
+ <rect class="BoundingBox" stroke="none" fill="none" x="4815" y="17268" width="4689" height="670"/>
+ <path fill="none" stroke="rgb(0,0,0)" d="M 4816,17269 C 4816,18109 8314,18093 9266,17565"/>
+ <path fill="rgb(0,0,0)" stroke="none" d="M 9503,17269 L 9100,17519 9331,17711 9503,17269 Z"/>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.ConnectorShape">
+ <g id="id20">
+ <rect class="BoundingBox" stroke="none" fill="none" x="11927" y="15486" width="4562" height="724"/>
+ <path fill="none" stroke="rgb(0,0,0)" d="M 11928,16100 C 11928,15337 15364,15309 16271,15893"/>
+ <path fill="rgb(0,0,0)" stroke="none" d="M 16488,16209 L 16348,15756 16104,15930 16488,16209 Z"/>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.ConnectorShape">
+ <g id="id21">
+ <rect class="BoundingBox" stroke="none" fill="none" x="11928" y="17269" width="4562" height="571"/>
+ <path fill="none" stroke="rgb(0,0,0)" d="M 16488,17287 C 16488,17978 13165,17970 12192,17538"/>
+ <path fill="rgb(0,0,0)" stroke="none" d="M 11928,17269 L 12145,17691 12355,17477 11928,17269 Z"/>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.CustomShape">
+ <g id="id22">
+ <rect class="BoundingBox" stroke="none" fill="none" x="8492" y="3158" width="4322" height="1274"/>
+ <path fill="rgb(204,255,204)" stroke="none" d="M 10652,3159 C 11876,3159 12811,3434 12811,3794 12811,4154 11876,4429 10652,4429 9428,4429 8493,4154 8493,3794 8493,3434 9428,3159 10652,3159 Z M 8493,3159 L 8493,3159 Z M 12812,4430 L 12812,4430 Z"/>
+ <path fill="none" stroke="rgb(52,101,164)" d="M 10652,3159 C 11876,3159 12811,3434 12811,3794 12811,4154 11876,4429 10652,4429 9428,4429 8493,4154 8493,3794 8493,3434 9428,3159 10652,3159 Z"/>
+ <path fill="none" stroke="rgb(52,101,164)" d="M 8493,3159 L 8493,3159 Z"/>
+ <path fill="none" stroke="rgb(52,101,164)" d="M 12812,4430 L 12812,4430 Z"/>
+ <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="635px" font-weight="400"><tspan class="TextPosition" x="9263" y="4015"><tspan fill="rgb(0,0,0)" stroke="none">Initializing</tspan></tspan></tspan></text>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.ConnectorShape">
+ <g id="id23">
+ <rect class="BoundingBox" stroke="none" fill="none" x="3573" y="13064" width="7145" height="2796"/>
+ <path fill="none" stroke="rgb(0,0,0)" d="M 10716,13065 C 10716,15160 4548,13939 3700,15474"/>
+ <path fill="rgb(0,0,0)" stroke="none" d="M 3604,15859 L 3864,15462 3574,15386 3604,15859 Z"/>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.ConnectorShape">
+ <g id="id24">
+ <rect class="BoundingBox" stroke="none" fill="none" x="10502" y="4428" width="301" height="764"/>
+ <path fill="none" stroke="rgb(0,0,0)" d="M 10652,4429 L 10652,4761"/>
+ <path fill="rgb(0,0,0)" stroke="none" d="M 10652,5191 L 10802,4741 10502,4741 10652,5191 Z"/>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.CustomShape">
+ <g id="id25">
+ <rect class="BoundingBox" stroke="none" fill="none" x="998" y="24873" width="638" height="352"/>
+ <path fill="rgb(255,255,255)" stroke="none" d="M 1316,24874 L 1634,24874 1634,25223 999,25223 999,24874 1316,24874 Z"/>
+ <path fill="none" stroke="rgb(0,0,0)" d="M 1316,24874 L 1634,24874 1634,25223 999,25223 999,24874 1316,24874 Z"/>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.CustomShape">
+ <g id="id26">
+ <rect class="BoundingBox" stroke="none" fill="none" x="998" y="25382" width="639" height="385"/>
+ <path fill="rgb(255,255,255)" stroke="none" d="M 1316,25383 C 1496,25383 1634,25465 1634,25573 1634,25681 1496,25764 1316,25764 1136,25764 999,25681 999,25573 999,25465 1136,25383 1316,25383 Z M 999,25383 L 999,25383 Z M 1635,25765 L 1635,25765 Z"/>
+ <path fill="none" stroke="rgb(0,0,0)" d="M 1316,25383 C 1496,25383 1634,25465 1634,25573 1634,25681 1496,25764 1316,25764 1136,25764 999,25681 999,25573 999,25465 1136,25383 1316,25383 Z"/>
+ <path fill="none" stroke="rgb(0,0,0)" d="M 999,25383 L 999,25383 Z"/>
+ <path fill="none" stroke="rgb(0,0,0)" d="M 1635,25765 L 1635,25765 Z"/>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.CustomShape">
+ <g id="id27">
+ <rect class="BoundingBox" stroke="none" fill="none" x="998" y="24239" width="638" height="384"/>
+ <path fill="rgb(255,204,204)" stroke="none" d="M 1317,24621 L 999,24621 999,24240 1634,24240 1634,24621 1317,24621 Z"/>
+ <path fill="none" stroke="rgb(255,255,255)" d="M 1317,24621 L 999,24621 999,24240 1634,24240 1634,24621 1317,24621 Z"/>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.CustomShape">
+ <g id="id28">
+ <rect class="BoundingBox" stroke="none" fill="none" x="998" y="23731" width="638" height="384"/>
+ <path fill="rgb(204,255,204)" stroke="none" d="M 1317,24113 L 999,24113 999,23732 1634,23732 1634,24113 1317,24113 Z"/>
+ <path fill="none" stroke="rgb(255,255,255)" d="M 1317,24113 L 999,24113 999,23732 1634,23732 1634,24113 1317,24113 Z"/>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.TableShape">
+ <g>
+ <rect class="BoundingBox" stroke="none" fill="none" x="22055" y="25834" width="4448" height="619"/>
+ <path fill="rgb(255,255,255)" stroke="none" d="M 22056,25835 L 26501,25835 26501,26451 22056,26451 22056,25835 Z"/>
+ <path fill="none" stroke="rgb(255,255,255)" d="M 22056,25835 L 22056,26451"/>
+ <path fill="none" stroke="rgb(255,255,255)" d="M 22056,26451 L 26501,26451"/>
+ <path fill="none" stroke="rgb(255,255,255)" d="M 26501,25835 L 26501,26451"/>
+ <path fill="none" stroke="rgb(255,255,255)" d="M 22056,25835 L 26501,25835"/>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.TextShape">
+ <g id="id29">
+ <rect class="BoundingBox" stroke="none" fill="none" x="1635" y="24749" width="2510" height="751"/>
+ <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="282px" font-weight="400"><tspan class="TextPosition" x="1885" y="25128"><tspan fill="rgb(0,0,0)" stroke="none">Terminal state</tspan></tspan></tspan></text>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.TextShape">
+ <g id="id30">
+ <rect class="BoundingBox" stroke="none" fill="none" x="1683" y="25257" width="4148" height="608"/>
+ <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="282px" font-weight="400"><tspan class="TextPosition" x="1933" y="25666"><tspan fill="rgb(0,0,0)" stroke="none">Non-terminal (retryin</tspan><tspan font-size="318px" fill="rgb(0,0,0)" stroke="none">g) state</tspan></tspan></tspan></text>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.TextShape">
+ <g id="id31">
+ <rect class="BoundingBox" stroke="none" fill="none" x="1635" y="23606" width="2541" height="636"/>
+ <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="282px" font-weight="400"><tspan class="TextPosition" x="1885" y="23985"><tspan fill="rgb(0,0,0)" stroke="none">Healthy state</tspan></tspan></tspan></text>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.TextShape">
+ <g id="id32">
+ <rect class="BoundingBox" stroke="none" fill="none" x="1635" y="24185" width="2476" height="565"/>
+ <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="282px" font-weight="400"><tspan class="TextPosition" x="1885" y="24564"><tspan fill="rgb(0,0,0)" stroke="none">Unhealthy state</tspan></tspan></tspan></text>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.ConnectorShape">
+ <g id="id33">
+ <rect class="BoundingBox" stroke="none" fill="none" x="6016" y="9127" width="4794" height="3431"/>
+ <path fill="none" stroke="rgb(0,0,0)" d="M 6017,9128 C 6017,11700 10163,10178 10665,12135"/>
+ <path fill="rgb(0,0,0)" stroke="none" d="M 10715,12557 L 10809,12092 10511,12129 10715,12557 Z"/>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.ConnectorShape">
+ <g id="id34">
+ <rect class="BoundingBox" stroke="none" fill="none" x="4053" y="7390" width="994" height="2170"/>
+ <path fill="none" stroke="rgb(0,0,0)" d="M 4983,8942 C 4983,9974 4054,9631 4054,8494 4054,7358 4686,7159 4907,7626"/>
+ <path fill="rgb(0,0,0)" stroke="none" d="M 4983,8044 L 5045,7574 4751,7631 4983,8044 Z"/>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.CustomShape">
+ <g id="id35">
+ <rect class="BoundingBox" stroke="none" fill="none" x="998" y="25890" width="639" height="385"/>
+ <path fill="rgb(255,255,255)" stroke="none" d="M 1127,25891 L 1506,25891 1635,26082 1506,26273 1127,26273 999,26082 1127,25891 1127,25891 Z M 999,25891 L 999,25891 Z M 1635,26273 L 1635,26273 Z"/>
+ <path fill="none" stroke="rgb(0,0,0)" d="M 1127,25891 L 1506,25891 1635,26082 1506,26273 1127,26273 999,26082 1127,25891 1127,25891 Z"/>
+ <path fill="none" stroke="rgb(0,0,0)" d="M 999,25891 L 999,25891 Z"/>
+ <path fill="none" stroke="rgb(0,0,0)" d="M 1635,26273 L 1635,26273 Z"/>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.TextShape">
+ <g id="id36">
+ <rect class="BoundingBox" stroke="none" fill="none" x="1649" y="25836" width="4178" height="565"/>
+ <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="282px" font-weight="400"><tspan class="TextPosition" x="1899" y="26215"><tspan fill="rgb(0,0,0)" stroke="none">Internal API (not a state)</tspan></tspan></tspan></text>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.TextShape">
+ <g id="id37">
+ <rect class="BoundingBox" stroke="none" fill="none" x="1683" y="26273" width="3665" height="565"/>
+ <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="282px" font-weight="400"><tspan class="TextPosition" x="1933" y="26652"><tspan fill="rgb(0,0,0)" stroke="none">External API (not a state)</tspan></tspan></tspan></text>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.CustomShape">
+ <g id="id38">
+ <rect class="BoundingBox" stroke="none" fill="none" x="17001" y="7857" width="3576" height="1274"/>
+ <path fill="rgb(255,255,204)" stroke="none" d="M 17002,7858 L 20575,7858 19940,9129 17636,9129 17002,7858 Z M 17002,7858 L 17002,7858 Z M 20575,9129 L 20575,9129 Z"/>
+ <path fill="none" stroke="rgb(52,101,164)" d="M 17002,7858 L 20575,7858 19940,9129 17636,9129 17002,7858 Z"/>
+ <path fill="none" stroke="rgb(52,101,164)" d="M 17002,7858 L 17002,7858 Z"/>
+ <path fill="none" stroke="rgb(52,101,164)" d="M 20575,9129 L 20575,9129 Z"/>
+ <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="353px" font-weight="400"><tspan class="TextPosition" x="18119" y="8421"><tspan fill="rgb(0,0,0)" stroke="none">POST to</tspan></tspan></tspan><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="353px" font-weight="400"><tspan class="TextPosition" x="17980" y="8811"><tspan fill="rgb(0,0,0)" stroke="none">/_replicate</tspan></tspan></tspan></text>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.CustomShape">
+ <g id="id39">
+ <rect class="BoundingBox" stroke="none" fill="none" x="7984" y="1126" width="5338" height="1274"/>
+ <path fill="rgb(255,255,204)" stroke="none" d="M 7985,1127 L 13320,1127 12392,2398 8913,2398 7985,1127 Z M 7985,1127 L 7985,1127 Z M 13320,2398 L 13320,2398 Z"/>
+ <path fill="none" stroke="rgb(52,101,164)" d="M 7985,1127 L 13320,1127 12392,2398 8913,2398 7985,1127 Z"/>
+ <path fill="none" stroke="rgb(52,101,164)" d="M 7985,1127 L 7985,1127 Z"/>
+ <path fill="none" stroke="rgb(52,101,164)" d="M 13320,2398 L 13320,2398 Z"/>
+ <text class="TextShape"><tspan class="TextParagraph" font-family="Liberation Sans, sans-serif" font-size="353px" font-weight="400"><tspan class="TextPosition" x="9298" y="1885"><tspan fill="rgb(0,0,0)" stroke="none">_replicator/<doc></tspan></tspan></tspan></text>
+ </g>
+ </g>
+ <g class="com.sun.star.drawing.CustomShape">
+ <g id="id40">
+ <rect class="BoundingBox" stroke="none" fill="none" x="998" y="26494" width="639" height="290"/>
+ <path fill="rgb(255,255,255)" stroke="none" d="M 999,26495 L 1635,26495 1476,26782 1158,26782 999,26495 Z M 999,26495 L 999,26495 Z M 1635,26782 L 1635,26782 Z"/>
+ <path fill="none" stroke="rgb(0,0,0)" d="M 999,26495 L 1635,26495 1476,26782 1158,26782 999,26495 Z"/>
+ <path fill="none" stroke="rgb(0,0,0)" d="M 999,26495 L 999,26495 Z"/>
+ <path fill="none" stroke="rgb(0,0,0)" d="M 1635,26782 L 1635,26782 Z"/>
+ </g>
+ </g>
+ </g>
+ </g>
+ </g>
+ </g>
+ </g>
+</svg>
\ No newline at end of file
diff --git a/src/replication/replicator.rst b/src/replication/replicator.rst
index 1db5c8f..414cbe2 100644
--- a/src/replication/replicator.rst
+++ b/src/replication/replicator.rst
@@ -16,6 +16,11 @@
Replicator Database
===================
+.. versionchanged:: 3.0.0 Scheduling replicator was
+ introduced. Replication states, by default are not written back to
+ documents anymore. There are new replication job states and new API
+ endpoints ``_scheduler/jobs`` and ``_scheduler/docs``.
+
The ``_replicator`` database works like any other in CouchDB, but documents
added to it will trigger replications. Create (``PUT`` or ``POST``) a document
to start replication. ``DELETE`` a replication document to cancel an ongoing
@@ -30,7 +35,7 @@ Replication documents can have a user defined ``_id`` (handy for finding a
specific replication request later). Design Documents (and ``_local`` documents)
added to the replicator database are ignored.
-The default name of this database is ``_replicator``. Additional replicator
+The default replicator database is ``_replicator``. Additional replicator
databases can be created. To be recognized as such by the system, their database
names should end with ``/_replicator``.
@@ -43,108 +48,151 @@ Let's say you POST the following document into ``_replicator``:
{
"_id": "my_rep",
- "source": "http://myserver.com:5984/foo",
+ "source": "http://user:pass@localhost:5984/foo",
"target": "http://user:pass@localhost:5984/bar",
- "create_target": true
+ "create_target": true,
+ "continuous": true
}
In the couch log you'll see 2 entries like these:
.. code-block:: text
- [Thu, 17 Feb 2011 19:43:59 GMT] [info] [<0.291.0>] Document `my_rep` triggered replication `c0ebe9256695ff083347cbf95f93e280+create_target`
- [Thu, 17 Feb 2011 19:44:37 GMT] [info] [<0.124.0>] Replication `c0ebe9256695ff083347cbf95f93e280+create_target` finished (triggered by document `my_rep`)
-
-As soon as the replication is triggered, the document will be updated by
-CouchDB with 3 new fields:
+ [notice] 2017-04-05T17:16:19.646716Z node1@127.0.0.1 <0.29432.0> -------- Replication `"a81a78e822837e66df423d54279c15fe+continuous+create_target"` is using:
+ 4 worker processes
+ a worker batch size of 500
+ 20 HTTP connections
+ a connection timeout of 30000 milliseconds
+ 10 retries per request
+ socket options are: [{keepalive,true},{nodelay,false}]
+ [notice] 2017-04-05T17:16:19.646759Z node1@127.0.0.1 <0.29432.0> -------- Document `my_rep` triggered replication `a81a78e822837e66df423d54279c15fe+continuous+create_target`
+
+Replication state of this document can then be queried from
+``http://adm:pass@localhost:5984/_scheduler/docs/my_rep``
+
+.. code-block:: json
+
+ {
+ "database": "_replicator",
+ "doc_id": "my_rep",
+ "error_count": 0,
+ "id": "a81a78e822837e66df423d54279c15fe+continuous+create_target",
+ "info": null,
+ "last_updated": "2017-04-05T19:18:15Z",
+ "node": "node1@127.0.0.1",
+ "proxy": null,
+ "source": "http://adm:*****@localhost:5984/foo/",
+ "start_time": "2017-04-05T19:18:15Z",
+ "state": "running",
+ "target": "http://adm:*****@localhost:5984/bar/"
+ }
+
+The state is `running` that means replicator has scheduled this
+replication job to run. The replication job will also appear in
+
+``http://adm:pass@localhost:5984/_scheduler/jobs``
+
+.. code-block:: json
+
+ {
+ "jobs": [
+ {
+ "database": "_replicator",
+ "doc_id": "my_rep",
+ "history": [
+ {
+ "timestamp": "2017-04-05T19:18:15Z",
+ "type": "started"
+ },
+ {
+ "timestamp": "2017-04-05T19:18:15Z",
+ "type": "added"
+ }
+ ],
+ "id": "a81a78e822837e66df423d54279c15fe+continuous+create_target",
+ "node": "node1@127.0.0.1",
+ "pid": "<0.1174.0>",
+ "source": "http://adm:*****@localhost:5984/foo/",
+ "start_time": "2017-04-05T19:18:15Z",
+ "target": "http://adm:*****@localhost:5984/bar/",
+ "user": null
+ }
+ ],
+ "offset": 0,
+ "total": 1
+ }
+
+``_scheduler/jobs`` shows more information such as a detailed history of
+state changes. However if replication has completed or has failed to
+start it would not appear here, only in ``_scheduler/docs``.
+
+If there is an error, for example the source database is missing, the
+replication job will crash and retry after a wait period. Each
+sucesssive crash will result in a longer waiting period.
+
+For example, POST-ing this document
.. code-block:: javascript
{
- "_id": "my_rep",
- "source": "http://myserver.com:5984/foo",
+ "_id": "my_rep_crashing",
+ "source": "http://user:pass@localhost:5984/missing",
"target": "http://user:pass@localhost:5984/bar",
"create_target": true,
- "_replication_id": "c0ebe9256695ff083347cbf95f93e280",
- "_replication_state": "triggered",
- "_replication_state_time": 1297974122
+ "continuous": true
}
-Special fields set by the replicator start with the prefix
-``_replication_``.
-
-- ``_replication_id``
-
- The ID internally assigned to the replication. This is also the ID
- exposed by ``/_active_tasks``.
-
-- ``_replication_state``
-
- The current state of the replication.
-
-- ``_replication_state_time``
-
- A Unix timestamp (number of seconds since 1 Jan 1970) that tells us
- when the current replication state (marked in ``_replication_state``)
- was set.
-
-- ``_replication_state_reason``
-
- If ``replication_state`` is ``error``, this field contains the reason.
-
-.. code-block:: javascript
+when source database is missing, will result in periodic starts and
+crashes with an increasingly larger interval. The ``history`` list from
+``_scheduler/jobs`` for this replication would look something like
+this:
+
+.. code-block:: json
+
+ [
+ {
+ "reason": "db_not_found: could not open http://adm:*****@localhost:5984/missing/",
+ "timestamp": "2017-04-05T20:55:10Z",
+ "type": "crashed"
+ },
+ {
+ "timestamp": "2017-04-05T20:55:10Z",
+ "type": "started"
+ },
+ {
+ "reason": "db_not_found: could not open http://adm:*****@localhost:5984/missing/",
+ "timestamp": "2017-04-05T20:47:10Z",
+ "type": "crashed"
+ },
+ {
+ "timestamp": "2017-04-05T20:47:10Z",
+ "type": "started"
+ }
+ ]
+
+``_scheduler/docs`` shows a shorter summary:
+
+.. code-block:: json
{
- "_id": "my_rep",
- "_rev": "2-9f2c0d9372f4ee4dc75652ab8f8e7c70",
- "source": "http://myserver.com:5984/foo",
- "target": "http://user:pass@localhost:5984/bar",
- "_replication_state": "error",
- "_replication_state_time": "2013-12-13T18:48:00+01:00",
- "_replication_state_reason": "db_not_found: could not open foodb",
- "_replication_id": "c0ebe9256695ff083347cbf95f93e280"
+ "database": "_replicator",
+ "doc_id": "my_rep_crashing",
+ "error_count": 6,
+ "id": "cb78391640ed34e9578e638d9bb00e44+create_target",
+ "info": "db_not_found: could not open http://adm:*****@localhost:5984/missing/",
+ "last_updated": "2017-04-05T20:55:10Z",
+ "node": "node1@127.0.0.1",
+ "proxy": null,
+ "source": "http://adm:*****@localhost:5984/missing/",
+ "start_time": "2017-04-05T20:38:34Z",
+ "state": "crashing",
+ "target": "http://adm:*****@localhost:5984/bar/"
}
-When the replication finishes, it will update the ``_replication_state``
-field (and ``_replication_state_time``) with the value ``completed``, so
-the document will look like:
-
-.. code-block:: javascript
-
- {
- "_id": "my_rep",
- "source": "http://myserver.com:5984/foo",
- "target": "http://user:pass@localhost:5984/bar",
- "create_target": true,
- "_replication_id": "c0ebe9256695ff083347cbf95f93e280",
- "_replication_state": "completed",
- "_replication_state_time": 1297974122
- }
-
-When an error happens during replication, the ``_replication_state``
-field is set to ``error`` (and ``_replication_state_reason`` and
-``_replication_state_time`` are updated).
-
-When you PUT/POST a document to the ``_replicator`` database, CouchDB
-will attempt to start the replication up to 10 times (configurable under
-``[replicator]``, parameter ``max_replication_retry_count``). If it
-fails on the first attempt, it waits 5 seconds before doing a second
-attempt. If the second attempt fails, it waits 10 seconds before doing a
-third attempt. If the third attempt fails, it waits 20 seconds before
-doing a fourth attempt (each attempt doubles the previous wait period).
-When an attempt fails, the Couch log will show you something like:
-
-.. code-block:: text
-
- [error] [<0.149.0>] Error starting replication `67c1bb92010e7abe35d7d629635f18b6+create_target` (document `my_rep_2`): {db_not_found,<<"could not open http://myserver:5986/foo/">>
-
-.. note::
- The ``_replication_state`` field is only set to ``error`` when all the
- attempts were unsuccessful.
-
-There are only 3 possible values for the ``_replication_state`` field:
-``triggered``, ``completed`` and ``error``. Continuous replications
-never get their state set to ``completed``.
+Repeated crashes are described as a ``crashing`` state. ``-ing`` suffix
+implies this is a temporary state. User at any moment could create the
+missing database and then replication job could return back to the
+normal.
Documents describing the same replication
=========================================
@@ -155,9 +203,11 @@ the following order:
.. code-block:: javascript
{
- "_id": "doc_A",
- "source": "http://myserver.com:5984/foo",
- "target": "http://user:pass@localhost:5984/bar"
+ "_id": "my_rep",
+ "source": "http://user:pass@localhost:5984/foo",
+ "target": "http://user:pass@localhost:5984/bar",
+ "create_target": true,
+ "continuous": true
}
and
@@ -165,88 +215,180 @@ and
.. code-block:: javascript
{
- "_id": "doc_B",
- "source": "http://myserver.com:5984/foo",
- "target": "http://user:pass@localhost:5984/bar"
- }
-
-Both describe exactly the same replication (only their ``_ids`` differ). In this
-case document ``doc_A`` triggers the replication, getting updated by CouchDB
-with the fields ``_replication_state``, ``_replication_state_time`` and
-``_replication_id``, just like it was described before. Document ``doc_B``
-however, is only updated with one field, the ``_replication_id`` so it will
-look like this:
-
-.. code-block:: javascript
-
- {
- "_id": "doc_B",
- "source": "http://myserver.com:5984/foo",
+ "_id": "my_rep_dup",
+ "source": "http://user:pass@localhost:5984/foo",
"target": "http://user:pass@localhost:5984/bar",
- "_replication_id": "c0ebe9256695ff083347cbf95f93e280"
+ "create_target": true,
+ "continuous": true
}
-While document ``doc_A`` will look like this:
-
-.. code-block:: javascript
-
- {
- "_id": "doc_A",
- "source": "http://myserver.com:5984/foo",
- "target": "http://user:pass@localhost:5984/bar",
- "_replication_id": "c0ebe9256695ff083347cbf95f93e280",
- "_replication_state": "triggered",
- "_replication_state_time": 1297974122
- }
+Both describe exactly the same replication (only their ``_ids``
+differ). In this case document ``my_rep`` triggers the
+replication. While ``my_rep_dup``` will fail. Inspecting
+``_scheduler/docs`` explains exactly why it failed:
+
+.. code-block:: json
+
+ {
+ "database": "_replicator",
+ "doc_id": "my_rep_dup",
+ "error_count": 1,
+ "id": null,
+ "info": "Replication `a81a78e822837e66df423d54279c15fe+continuous+create_target` specified by document `my_rep_dup` already started, triggered by document `my_rep` from db `_replicator`",
+ "last_updated": "2017-04-05T21:41:51Z",
+ "source": "http://adm:*****@localhost:5984/foo/",
+ "start_time": "2017-04-05T21:41:51Z",
+ "state": "failed",
+ "target": "http://adm:*****@localhost:5984/bar/"
+ }
-Note that both document get exactly the same value for the ``_replication_id``
-field. This way you can identify which documents refer to the same replication -
-you can for example define a view which maps replication IDs to document IDs.
+Notice the state for this replication is ``failed``. Unlike
+``crashing``, ``failed`` state is terminal. As long as both documents
+are present replicator will not retry to run my_rep_dub
+replication. Another reason could be malformed documents. For
+example if worker process count is specified as a string
+(``"worker_processes": "a few"``) instead of an integer.
+
+Replication states
+==================
+
+Replication jobs during their life-cycle pass through various
+states. This is a diagram of all the states and transitions
+between them:
+
+.. figure:: ../../images/replication-state-diagram.svg
+ :align: center
+ :alt: Replication state diagram
+
+ Replication state diagram
+
+Green and red shapes represent replication job states.
+
+Yellow shapes represent external APIs, that's how users interact with
+the replicator. Writing documents to ``_replicator`` is the preferred
+way of creating replications, but posting to the ``_replicate`` HTTP
+endpoint is also supported.
+
+.. note:: Replications created through the ``_replicate`` endpoint
+ will not survive cluster node restart where they are running, and
+ once completed they will be removed from the system, so their
+ completion state is preserved in ``_scheduler/jobs`` endpoint.
+
+White shapes indicate internal API boundaries and point to how the
+replicator is structured internally. There are two stages in the
+processing: the first is where replication documents are parsed and
+become replication jobs, and the second is the scheduler. The
+scheduler runs replication jobs, periodically stopping and starting
+some. Jobs posted via the ``_replicate`` endpoint bypass the first
+component and go straight to the scheduler.
+
+States descriptions
+-------------------
+
+Before explaining the details of each state, it is worth noticing that
+color and shape of each state in the diagram:
+
+`Green` vs `red` partitions states into "healthy" and "unhealthy",
+respectively. Unhealthy states indicate something has gone wrong and
+it might need user's attention.
+
+`Rectangle` vs `oval` separates "terminal" states from "non-terminal"
+ones. Terminal states are those which will not transition to other
+states any more. Informally, jobs in a terminal state will not be
+retried and don't consume memory or CPU resources.
+
+ * ``Initializing``: Indicates replicator has noticed the change
+ from the replication document. Jobs should transition quickly
+ through this state. Being stuck here for a while could mean there
+ is an internal error.
+
+ * ``Failed``: Replication document could not be processed and turned
+ into a valid replication job for the scheduler. This state is
+ terminal and requires user intervention to fix the problem. Typical
+ reasons for ending up in this state is a malformed document. For
+ example specifying an integer for a parameter which accepts a
+ boolean. Another reason could be specifying a duplicate
+ replication. A duplicate replication is a replication with
+ identical parameters but a different document ID.
+
+ * ``Error``: Replication document update could not be turned into a
+ replication job. Unlike the ``Failed`` state, this one is
+ temporary, and replicator will keep retrying periodically. There is
+ an exponential backoff applied in case of consecutive failures.
+ The main reason this state exists is to handle filtered
+ replications with custom user functions. Filter function content is
+ needed in order to calculate the replication ID. A replication job
+ could not be created until the function code is retrieved. Because
+ retrieval happens over the network, temporary failures have to be
+ handled.
+
+ * ``Running``: Replication job is running normally. This means,
+ there might be a change feed open, and if changes are noticed, they
+ would be processed and posted to the target. Job is still
+ considered ``Running`` even if its workers are currently not
+ streaming changes from source to target and are just waiting on the
+ change feed.
+
+ * ``Pending``: Replication job is not running and is waiting its
+ turn. This state is reached when the number of replication jobs
+ added to the scheduler exceeds ``replicator.max_jobs``. In that
+ case scheduler will periodically stop and start subsets of jobs
+ trying to give each one a fair chance at making progress.
+
+ * ``Crashing``: Replication job has been successfully added to the
+ replication scheduler. However an error was encountered during the
+ last run. Error could be a network failure, a missing source
+ database, a permissions error, etc. Repeated consecutive crashes
+ result in an exponential backoff. This state is considered temporary
+ (non-terminal) and replication jobs will be periodically
+ retried. Maximum backoff interval is around a day or so.
+
+ * ``Completed``: This is a terminal, successful state for
+ non-continuous replications. Once in this state the replication is
+ "forgotten" by the scheduler and it doesn't consume any more CPU or
+ memory resorces. Continuous replication jobs will never reach this
+ state.
+
+Compatibility Mode
+==================
+
+Previous version of CouchDB replicator wrote state updates back to
+replication documents. In case were user code programmatically read
+those states, there is compatibility mode enabled via
+``replicator.update_docs = true`` configuration setting. In this mode
+replicator will continue to write state updates to the
+documents.
Canceling replications
======================
-To cancel a replication simply ``DELETE`` the document which triggered the
-replication. The Couch log will show you an entry like the following:
-
-.. code-block:: text
-
- [Thu, 17 Feb 2011 20:16:29 GMT] [info] [<0.125.0>] Stopped replication `c0ebe9256695ff083347cbf95f93e280+continuous+create_target` because replication document `doc_A` was deleted
-
-.. note::
- You need to ``DELETE`` the document that triggered the replication.
- ``DELETE``-ing another document that describes the same replication
- but did not trigger it, will not cancel the replication.
+To cancel a replication simply ``DELETE`` the document which triggered
+the replication. To update a replication, for example, change the
+number of worker or the source, simply update the document with new
+data. If there is extra application-specific data in the replication
+documents, that data is ignored by the replicator.
Server restart
==============
-When CouchDB is restarted, it checks its ``_replicator`` database and
-restarts any replication that is described by a document that either has
-its ``_replication_state`` field set to ``triggered`` or it doesn't have
-yet the ``_replication_state`` field set.
-
-.. note::
- Continuous replications always have a ``_replication_state`` field
- with the value ``triggered``, therefore they're always restarted
- when CouchDB is restarted.
-
-Updating Documents in the Replicator Database
-=============================================
+When CouchDB is restarted, it checks its ``_replicator`` databases and
+restarts replications described by documents if they are not already in
+in a ``completed`` or ``failed`` state. If they are, they are ignored.
-Once the replicator has started work on a job defined in the ``_replicator``
-database, modifying the replication document is no longer allowed. Attempting
-to do this will result in the following response
+Clustering
+==========
-.. code-block:: javascript
-
- {
- "error": "forbidden",
- "reason": "Only the replicator can edit replication documents that are in the triggered state."
- }
+In a cluster, replication jobs are balanced evenly among all the nodes
+nodes such that a replication job runs on only one node at a time.
-The way to accomplish this is to first delete the old version and then insert
-the new one.
+Every time there is a cluster membership change, that is when nodes
+are added or removed, as it happens in a rolling reboot, replicator
+application will notice the change, rescan all the document and
+running replication, and re-evaluate their cluster placement in light
+of the new set of live nodes. This mechanism also provides
+replication fail-over in case on node fails. Replication jobs started
+form replication documents (but not those started from ``_replicate``
+HTTP endpoint) will automatically migrate one of the live nodes.
Additional Replicator Databases
===============================
diff --git a/src/whatsnew/3.0.rst b/src/whatsnew/3.0.rst
new file mode 100644
index 0000000..cea8ab0
--- /dev/null
+++ b/src/whatsnew/3.0.rst
@@ -0,0 +1,105 @@
+.. Licensed under the Apache License, Version 2.0 (the "License"); you may not
+.. use this file except in compliance with the License. You may obtain a copy of
+.. the License at
+..
+.. http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing, software
+.. distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+.. WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+.. License for the specific language governing permissions and limitations under
+.. the License.
+
+.. _release/3.0.x:
+
+============
+3.0.x Branch
+============
+
+.. contents::
+ :depth: 1
+ :local:
+
+.. _release/3.0.0:
+
+Version 3.0.0
+=============
+
+* New scheduling replicator. The core of the new replicator is a
+ scheduler which allows running a large number of replication
+ jobs by switching between them, stopping some and starting others
+ periodically. Jobs which fail are backed off exponentially. There is
+ also an improved inspection and querying API: ``_scheduler/jobs`` and
+ ``_scheduler/docs``:
+
+ * ``_scheduler/jobs`` : This endpoint shows active replication
+ jobs. These are jobs managed by the scheduler. Some of them might
+ be running, some might be waiting to run, or backed off
+ (penalized) because they crashed too many times. Semantically this
+ is somewhat equivalent to ``_active_tasks`` but focuses only on
+ replications. Jobs which have completed or which were never
+ created because of malformed replication documents will not be
+ shown here as they are not managed by the scheduler.
+ ``_replicate`` replications, started form _replicate endpoint not
+ from a document in a ``_replicator`` db, will also show up here.
+
+ * ``_scheduler/docs`` : This endpoint is an improvement on having to go
+ back and read replication documents to query their state. It
+ represents the state of all the replications started from
+ documents in _replicator db. Unlike ``_scheduler/jobs`` it will also
+ show jobs which have failed or have completed.
+
+ By default, scheduling replicator will not update documents with
+ transient states like ``triggered`` or ``error`` anymore, instead
+ ``_scheduler/docs`` API should be used to query replication document
+ states.
+
+Other scheduling replicator improvements
+-----------------------------------------
+
+ * Network resource usage and performance was improved by
+ implementing a shared connection pool. This should help in cases
+ of a large number of connections to the same sources or
+ target. Previously connection pools were shared only withing a
+ single replication job.
+
+ * Improved request rate limit handling. Replicator requests will
+ auto-discover rate limit capacity on targets and sources based on
+ a proven Additive Increase / Multiplicative Decrease feedback
+ control algorithm.
+
+ * Improved performance by having exponential backoff for all
+ replication jobs failures. Previously there were some scenarios
+ were failure led to continuous repeated retries, consuming CPU and
+ disk resources in the process.
+
+ * Improved recovery from long but temporary network
+ failure. Currently if replications jobs fail to start 10 times in
+ a row, they will not be retried anymore. This is sometimes
+ desirable, but in some cases, for example, after a sustained DNS
+ failure which eventually recovers, replications reach their retry
+ limit, stop retrying and never recover. Previously it required
+ user intervention to continue. Scheduling replicator will never
+ give up retrying a valid scheduled replication job and so it
+ should recover automatically.
+
+ * Better handling of filtered replications. Failing user filter code
+ fetches from the source will not block replicator manager and
+ stall other replications. Failing filter fetches will also be
+ backed off exponentially. Another improvement is when filter code
+ changes on the source, a running replication will detect that and
+ restart itself with a new replication ID automatically.
+
+.. _release/3.0.x/upgrade:
+
+Upgrade Notes
+=============
+
+* If user code reads or manipulates replicator document states,
+ consider using ``replicator.update_docs = true`` compatibility
+ parameter. In that case replicator will continue updating documents
+ with transient replication states. However, that will incure a
+ performance cost. Consider instead switching using
+ ``_scheduler/docs`` HTTP endpoint.
+
+.. _CouchDB JIRA instance: https://issues.apache.org/jira/browse/COUCHDB
diff --git a/src/whatsnew/index.rst b/src/whatsnew/index.rst
index 5448fba..66ea131 100644
--- a/src/whatsnew/index.rst
+++ b/src/whatsnew/index.rst
@@ -19,6 +19,7 @@ Release History
.. toctree::
:glob:
+ 3.0
2.0
1.6
1.5
--
To stop receiving notification emails like this one, please contact
"commits@couchdb.apache.org" <co...@couchdb.apache.org>.