You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2017/10/29 19:47:18 UTC
[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build #
1496 - Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1496/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC
1 tests failed.
FAILED: org.apache.solr.search.function.TestFunctionQuery.testLongComparisons
Error Message:
Exception during query
Stack Trace:
java.lang.RuntimeException: Exception during query
at __randomizedtesting.SeedInfo.seed([C51A3ABEB4C1B3BD:C682A5DC7AA7E44F]:0)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:885)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:845)
at org.apache.solr.search.function.TestFunctionQuery.singleTest(TestFunctionQuery.java:131)
at org.apache.solr.search.function.TestFunctionQuery.singleTest(TestFunctionQuery.java:137)
at org.apache.solr.search.function.TestFunctionQuery.testLongComparisons(TestFunctionQuery.java:943)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//doc[./str[@name='id']='2' and ./float[@name='score']='2.0']
xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">0</int>
</lst>
<result name="response" numFound="144" start="0" maxScore="5.0">
<doc>
<str name="id">1</str>
<int name="id_i">1</int>
<long name="number_of_atoms_in_universe_l">9223372036854775807</long>
<float name="score">5.0</float></doc>
<doc>
<str name="id">0</str>
<int name="id_i">0</int>
<float name="default_amount_f">17307.0</float>
<str name="amounts_dpf">1|1.28 2|2.20 3|3.29 4|4.83 5|5.72 6|6.15 7|7.79 8|8.59 9|9.62 10|10.44 11|11.44 12|12.64 13|13.93 14|14.87 15|15.50 16|16.33 17|17.52 18|18.82 19|19.55 20|20.84 21|21.41 22|22.6 23|23.74 24|24.2 25|25.16 26|26.58 27|27.3 28|28.51 29|29.68 30|30.7 31|31.22 32|32.81 33|33.93 34|34.58 35|35.43 36|36.77 37|37.15 38|38.20 39|39.98 40|40.32 41|41.23 42|42.6 43|43.43 44|44.1 45|45.53 46|46.63 47|47.2 48|48.21 49|49.3 50|50.79 51|51.34 52|52.29 53|53.80 54|54.63 55|55.45 56|56.47 57|57.63 58|58.55 59|59.6 60|60.0 61|61.47 62|62.71 63|63.55 64|64.40 65|65.75 66|66.72 67|67.34 68|68.43 69|69.32 70|70.36 71|71.17 72|72.24 73|73.82 74|74.24 75|75.52 76|76.26 77|77.80 78|78.90 79|79.25 80|80.80 81|81.5 82|82.69 83|83.48 84|84.41 85|85.25 86|86.40 87|87.2 88|88.68 89|89.37 90|90.26 91|91.37 92|92.66 93|93.18 94|94.48 95|95.62 96|96.18 97|97.66 98|98.78 99|99.73 100|100.5 101|101.35 102|102.96 103|103.43 104|104.37 105|105.57 106|106.62 107|107.85 108|108.3 109|109.17 110|110.19 111|111.20 112|112.95 113|113.87 114|114.94 115|115.99 116|116.51 117|117.96 118|118.61 119|119.38 120|120.24 121|121.94 122|122.42 123|123.95 124|124.69 125|125.65 126|126.28 127|127.85 128|128.83 129|129.6 130|130.85 131|131.16 132|132.41 133|133.9 134|134.22 135|135.76 136|136.45 137|137.61 138|138.93 139|139.6 140|140.92 141|141.4 142|142.2 143|143.21 144|144.91 145|145.26 146|146.68 147|147.3 148|148.20 149|149.49 150|150.32 151|151.27 152|152.25 153|153.56 154|154.50 155|155.57 156|156.99 157|157.41 158|158.32 159|159.52 160|160.89 161|161.80 162|162.36 163|163.77 164|164.19 165|165.31 166|166.31 167|167.99 168|168.59 169|169.89 170|170.78 171|171.53 172|172.60 173|173.87 174|174.12 175|175.29 176|176.28 177|177.56 178|178.33 179|179.95 180|180.23 181|181.38 182|182.73 183|183.69 184|184.42 185|185.36 186|186.30 187|187.94 188|188.25 189|189.27 190|190.61 191|191.36 192|192.30 193|193.14 194|194.33 195|195.42 196|196.45 197|197.16 198|198.84 199|199.34 200|200.37 201|201.12 202|202.91 203|203.59 204|204.76 205|205.91 206|206.60 207|207.71 208|208.74 209|209.46 210|210.85 211|211.24 212|212.63 213|213.73 214|214.74 215|215.13 216|216.89 217|217.34 218|218.61 219|219.10 220|220.10 221|221.21 222|222.76 223|223.31 224|224.7 225|225.33 226|226.63 227|227.70 228|228.67 229|229.1 230|230.87 231|231.40 232|232.33 233|233.68 234|234.98 235|235.78 236|236.97 237|237.54 238|238.88 239|239.57 240|240.20 241|241.67 242|242.97 243|243.23 244|244.89 245|245.86 246|246.33 247|247.54 248|248.32 249|249.49 250|250.96 251|251.56 252|252.69 253|253.41 254|254.34 255|255.52 256|256.18 257|257.7 258|258.45 259|259.98 260|260.50 261|261.90 262|262.30 263|263.39 264|264.43 265|265.53 266|266.31 267|267.87 268|268.28 269|269.50 270|270.15 271|271.42 272|272.12 273|273.47 274|274.45 275|275.58 276|276.23 277|277.0 278|278.49 279|279.68 280|280.6 281|281.42 282|282.30 283|283.83 284|284.77 285|285.56 286|286.16 287|287.6 288|288.9 289|289.39 290|290.90 291|291.66 292|292.70 293|293.84 294|294.26 295|295.1 296|296.90 297|297.75 298|298.81 299|299.80 300|300.16 301|301.52 302|302.3 303|303.20 304|304.88 305|305.65 306|306.23 307|307.80 308|308.58 309|309.4 310|310.61 311|311.46 312|312.99 313|313.3 314|314.31 315|315.22 316|316.26 317|317.30 318|318.71 319|319.69 320|320.76 321|321.21 322|322.96 323|323.60 324|324.99 325|325.96 326|326.97 327|327.21 328|328.61 329|329.77 330|330.5 331|331.26 332|332.58 333|333.75 334|334.87 335|335.27 336|336.64 337|337.87 338|338.89 339|339.37 340|340.21 341|341.86 342|342.77 343|343.64 344|344.76 345|345.50 346|346.87 347|347.42 348|348.75 349|349.51 350|350.26 351|351.44 352|352.68 353|353.29 354|354.75 355|355.46 356|356.23 357|357.95 358|358.64 359|359.2 360|360.48 361|361.68 362|362.40 363|363.19 364|364.12 365|365.8 366|366.8 367|367.96 368|368.45 369|369.36 370|370.60 371|371.47 372|372.29 373|373.33 374|374.31 375|375.77 376|376.90 377|377.82 378|378.79 379|379.37 380|380.88 381|381.60 382|382.39 383|383.83 384|384.73 385|385.73 386|386.9 387|387.83 388|388.84 389|389.75 390|390.29 391|391.90 392|392.4 393|393.2 394|394.61 395|395.62 396|396.80 397|397.97 398|398.28 399|399.83 400|400.83 401|401.87 402|402.95 403|403.19 404|404.86 405|405.50 406|406.49 407|407.39 408|408.76 409|409.44 410|410.1 411|411.66 412|412.79 413|413.6 414|414.55 415|415.8 416|416.6 417|417.94 418|418.98 419|419.92 420|420.95 421|421.34 422|422.11 423|423.50 424|424.10 425|425.48 426|426.88 427|427.67 428|428.81 429|429.62 430|430.29 431|431.91 432|432.76 433|433.45 434|434.45 435|435.54 436|436.95 437|437.83 438|438.91 439|439.5 440|440.19 441|441.97 442|442.55 443|443.54 444|444.90 445|445.63 446|446.49 447|447.67 448|448.13 449|449.75 450|450.55 451|451.26 452|452.40 453|453.14 454|454.32 455|455.23 456|456.21 457|457.73 458|458.38 459|459.55 460|460.60 461|461.23 462|462.17 463|463.70 464|464.23 465|465.79 466|466.16 467|467.50 468|468.29 469|469.16 470|470.23 471|471.87 472|472.82 473|473.35 474|474.16 475|475.91 476|476.35 477|477.36 478|478.32 479|479.45 480|480.41 481|481.10 482|482.30 483|483.9 484|484.72 485|485.26 486|486.73 487|487.57 488|488.60 489|489.73 490|490.66 491|491.92 492|492.57 493|493.67 494|494.33 495|495.27 496|496.23 497|497.62 498|498.51 499|499.59 500|500.91 501|501.48 502|502.37 503|503.14 504|504.2 505|505.32 506|506.29 507|507.8 508|508.0 509|509.55 510|510.3 511|511.46 512|512.42 513|513.0 514|514.38 515|515.45 516|516.90 517|517.71 518|518.14 519|519.88 520|520.42 521|521.21 522|522.99 523|523.26 524|524.21 525|525.34 526|526.33 527|527.53 528|528.41 529|529.67 530|530.35 531|531.33 532|532.64 533|533.45 534|534.42 535|535.68 536|536.95 537|537.51 538|538.5 539|539.11 540|540.92 541|541.28 542|542.76 543|543.29 544|544.97 545|545.79 546|546.80 547|547.43 548|548.66 549|549.37 550|550.15 551|551.33 552|552.40 553|553.1 554|554.31 555|555.20 556|556.98 557|557.24 558|558.18 559|559.30 560|560.97 561|561.68 562|562.62 563|563.48 564|564.4 565|565.6 566|566.49 567|567.78 568|568.37 569|569.11 570|570.45 571|571.99 572|572.92 573|573.34 574|574.2 575|575.45 576|576.29 577|577.3 578|578.78 579|579.31 580|580.87 581|581.92 582|582.87 583|583.72 584|584.29 585|585.79 586|586.73 587|587.93 588|588.48 589|589.84 590|590.95 591|591.44 592|592.33 593|593.59 594|594.33 595|595.67 596|596.44 597|597.45 598|598.61 599|599.17 600|600.80 601|601.60 602|602.42 603|603.41 604|604.50 605|605.79 606|606.58 607|607.95 608|608.86 609|609.59 610|610.72 611|611.37 612|612.76 613|613.73 614|614.49 615|615.81 616|616.64 617|617.34 618|618.0 619|619.77 620|620.19 621|621.39 622|622.56 623|623.52 624|624.25 625|625.52 626|626.78 627|627.13 628|628.10 629|629.49 630|630.83 631|631.11 632|632.89 633|633.48 634|634.81 635|635.27 636|636.17 637|637.74 638|638.30 639|639.36 640|640.62 641|641.98 642|642.36 643|643.34 644|644.16 645|645.71 646|646.25 647|647.44 648|648.55 649|649.96 650|650.72 651|651.15 652|652.74 653|653.22 654|654.39 655|655.42 656|656.90 657|657.79 658|658.1 659|659.99 660|660.1 661|661.11 662|662.78 663|663.5 664|664.95 665|665.99 666|666.47 667|667.42 668|668.29 669|669.38 670|670.78 671|671.23 672|672.19 673|673.12 674|674.30 675|675.69 676|676.24 677|677.51 678|678.93 679|679.92 680|680.40 681|681.15 682|682.3 683|683.6 684|684.67 685|685.80 686|686.1 687|687.19 688|688.53 689|689.51 690|690.21 691|691.79 692|692.9 693|693.81 694|694.32 695|695.46 696|696.25 697|697.33 698|698.16 699|699.94 700|700.24 701|701.25 702|702.94 703|703.60 704|704.85 705|705.50 706|706.82 707|707.26 708|708.61 709|709.26 710|710.69 711|711.73 712|712.31 713|713.16 714|714.41 715|715.65 716|716.11 717|717.59 718|718.7 719|719.15 720|720.58 721|721.84 722|722.88 723|723.94 724|724.6 725|725.32 726|726.39 727|727.32 728|728.11 729|729.48 730|730.2 731|731.90 732|732.22 733|733.68 734|734.43 735|735.95 736|736.71 737|737.15 738|738.15 739|739.61 740|740.59 741|741.32 742|742.73 743|743.72 744|744.64 745|745.78 746|746.31 747|747.26 748|748.48 749|749.44 750|750.83 751|751.22 752|752.28 753|753.76 754|754.82 755|755.88 756|756.84 757|757.11 758|758.99 759|759.51 760|760.13 761|761.96 762|762.18 763|763.81 764|764.38 765|765.88 766|766.64 767|767.79 768|768.60 769|769.35 770|770.7 771|771.21 772|772.90 773|773.39 774|774.7 775|775.1 776|776.5 777|777.45 778|778.47 779|779.10 780|780.10 781|781.42 782|782.43 783|783.77 784|784.59 785|785.61 786|786.30 787|787.78 788|788.42 789|789.97 790|790.42 791|791.80 792|792.42 793|793.71 794|794.26 795|795.7 796|796.45 797|797.75 798|798.2 799|799.26 800|800.70 801|801.65 802|802.75 803|803.4 804|804.82 805|805.10 806|806.71 807|807.36 808|808.44 809|809.32 810|810.47 811|811.59 812|812.14 813|813.1 814|814.24 815|815.17 816|816.94 817|817.3 818|818.88 819|819.39 820|820.42 821|821.61 822|822.26 823|823.39 824|824.93 825|825.67 826|826.59 827|827.7 828|828.54 829|829.96 830|830.90 831|831.64 832|832.82 833|833.36 834|834.66 835|835.79 836|836.99 837|837.82 838|838.22 839|839.19 840|840.26 841|841.11 842|842.4 843|843.56 844|844.72 845|845.0 846|846.90 847|847.14 848|848.52 849|849.9 850|850.72 851|851.91 852|852.24 853|853.50 854|854.25 855|855.34 856|856.78 857|857.61 858|858.79 859|859.37 860|860.15 861|861.34 862|862.80 863|863.78 864|864.56 865|865.60 866|866.32 867|867.9 868|868.79 869|869.11 870|870.27 871|871.30 872|872.71 873|873.74 874|874.22 875|875.87 876|876.81 877|877.4 878|878.64 879|879.64 880|880.1 881|881.74 882|882.91 883|883.23 884|884.16 885|885.69 886|886.72 887|887.34 888|888.28 889|889.44 890|890.98 891|891.11 892|892.55 893|893.87 894|894.20 895|895.76 896|896.48 897|897.65 898|898.33 899|899.51 900|900.72 901|901.46 902|902.40 903|903.47 904|904.67 905|905.51 906|906.44 907|907.25 908|908.67 909|909.49 910|910.65 911|911.55 912|912.87 913|913.18 914|914.5 915|915.33 916|916.25 917|917.51 918|918.32 919|919.25 920|920.30 921|921.43 922|922.7 923|923.59 924|924.53 925|925.72 926|926.85 927|927.19 928|928.22 929|929.13 930|930.4 931|931.9 932|932.65 933|933.81 934|934.93 935|935.48 936|936.99 937|937.35 938|938.55 939|939.9 940|940.55 941|941.82 942|942.77 943|943.17 944|944.45 945|945.1 946|946.13 947|947.32 948|948.44 949|949.98 950|950.3 951|951.56 952|952.3 953|953.87 954|954.83 955|955.9 956|956.73 957|957.90 958|958.29 959|959.32 960|960.22 961|961.23 962|962.80 963|963.40 964|964.54 965|965.95 966|966.87 967|967.7 968|968.84 969|969.45 970|970.47 971|971.34 972|972.50 973|973.48 974|974.95 975|975.31 976|976.81 977|977.83 978|978.42 979|979.3 980|980.31 981|981.30 982|982.51 983|983.15 984|984.96 985|985.62 986|986.59 987|987.37 988|988.81 989|989.90 990|990.93 991|991.81 992|992.90 993|993.88 994|994.99 995|995.39 996|996.32 997|997.6 998|998.29 999|999.85 1000|1000.66 1001|1001.22 1002|1002.8 1003|1003.89 1004|1004.54 1005|1005.64 1006|1006.30 1007|1007.41 1008|1008.16 1009|1009.90 1010|1010.59 1011|1011.14 1012|1012.54 1013|1013.9 1014|1014.82 1015|1015.68 1016|1016.43 1017|1017.54 1018|1018.55 1019|1019.35 1020|1020.56 1021|1021.53 1022|1022.91 1023|1023.7 1024|1024.20 1025|1025.24 1026|1026.78 1027|1027.15 1028|1028.32 1029|1029.58 1030|1030.8 1031|1031.0 1032|1032.60 1033|1033.22 1034|1034.98 1035|1035.86 1036|1036.53 1037|1037.43 1038|1038.75 1039|1039.71 1040|1040.52 1041|1041.58 1042|1042.84 1043|1043.56 1044|1044.8 1045|1045.64 1046|1046.11 1047|1047.8 1048|1048.33 1049|1049.1 1050|1050.71 1051|1051.0 1052|1052.94 1053|1053.28 1054|1054.19 1055|1055.34 1056|1056.62 1057|1057.59 1058|1058.58 1059|1059.85 1060|1060.3 1061|1061.95 1062|1062.89 1063|1063.86 1064|1064.34 1065|1065.37 1066|1066.76 1067|1067.29 1068|1068.51 1069|1069.93 1070|1070.89 1071|1071.15 1072|1072.26 1073|1073.20 1074|1074.42 1075|1075.95 </str>
<float name="score">2.0</float></doc>
<doc>
<str name="id">3</str>
<int name="id_i">3</int>
<float name="default_amount_f">13688.0</float>
<str name="amounts_dpf">1|1.0 2|2.0 3|3.52 4|4.86 5|5.10 6|6.4 7|7.87 8|8.97 9|9.24 10|10.83 11|11.88 12|12.51 13|13.20 14|14.46 15|15.4 16|16.1 17|17.36 18|18.59 19|19.0 20|20.73 21|21.17 22|22.12 23|23.16 24|24.75 25|25.58 26|26.80 27|27.96 28|28.32 29|29.6 30|30.89 31|31.12 32|32.16 33|33.72 34|34.68 35|35.6 36|36.14 37|37.9 38|38.19 39|39.15 40|40.24 41|41.19 42|42.32 43|43.8 44|44.46 45|45.55 46|46.0 47|47.62 48|48.41 49|49.23 50|50.31 51|51.39 52|52.73 53|53.65 54|54.65 55|55.71 56|56.94 57|57.8 58|58.88 59|59.34 60|60.49 61|61.0 62|62.19 63|63.20 64|64.21 65|65.40 66|66.66 67|67.67 68|68.4 69|69.89 70|70.7 71|71.19 72|72.98 73|73.34 74|74.76 75|75.4 76|76.14 77|77.86 78|78.84 79|79.98 80|80.38 81|81.64 82|82.38 83|83.43 84|84.71 85|85.88 86|86.91 87|87.74 88|88.13 89|89.16 90|90.35 91|91.88 92|92.42 93|93.59 94|94.61 95|95.11 96|96.5 97|97.59 98|98.1 99|99.58 100|100.90 101|101.56 102|102.63 103|103.38 104|104.11 105|105.91 106|106.41 107|107.62 108|108.36 109|109.67 110|110.33 111|111.93 112|112.54 113|113.55 114|114.5 115|115.2 116|116.3 117|117.90 118|118.95 119|119.9 120|120.80 121|121.57 122|122.90 123|123.89 124|124.25 125|125.16 126|126.48 127|127.69 128|128.79 129|129.2 130|130.83 131|131.34 132|132.85 133|133.23 134|134.80 135|135.24 136|136.5 137|137.27 138|138.91 139|139.0 140|140.0 141|141.2 142|142.78 143|143.16 144|144.32 145|145.34 146|146.58 147|147.13 148|148.42 149|149.79 150|150.65 151|151.80 152|152.42 153|153.76 154|154.37 155|155.19 156|156.28 157|157.76 158|158.96 159|159.4 160|160.96 161|161.0 162|162.24 163|163.90 164|164.30 165|165.57 166|166.18 167|167.56 168|168.87 169|169.0 170|170.98 171|171.6 172|172.59 173|173.67 174|174.35 175|175.74 176|176.48 177|177.10 178|178.5 179|179.26 180|180.42 181|181.20 182|182.39 183|183.85 184|184.32 185|185.97 186|186.41 187|187.79 188|188.72 189|189.98 190|190.56 191|191.22 192|192.66 193|193.2 194|194.27 195|195.61 196|196.29 197|197.13 198|198.89 199|199.23 200|200.15 201|201.42 202|202.85 203|203.25 204|204.72 205|205.51 206|206.93 207|207.36 208|208.31 209|209.57 210|210.7 211|211.92 212|212.77 213|213.68 214|214.44 215|215.22 216|216.91 217|217.21 218|218.58 219|219.96 220|220.58 221|221.35 222|222.6 223|223.84 224|224.57 225|225.59 226|226.79 227|227.53 228|228.96 229|229.71 230|230.21 231|231.48 232|232.36 233|233.84 234|234.53 235|235.76 236|236.72 237|237.80 238|238.31 239|239.16 240|240.21 241|241.72 242|242.40 243|243.95 244|244.23 245|245.22 246|246.63 247|247.19 248|248.30 249|249.35 250|250.10 251|251.25 252|252.94 253|253.56 254|254.10 255|255.1 256|256.47 257|257.89 258|258.34 259|259.36 260|260.34 261|261.89 262|262.11 263|263.0 264|264.52 265|265.49 266|266.5 267|267.80 268|268.60 269|269.3 270|270.63 271|271.47 272|272.45 273|273.93 274|274.56 275|275.98 276|276.4 277|277.10 278|278.7 279|279.71 280|280.41 281|281.39 282|282.26 283|283.97 284|284.94 285|285.95 286|286.1 287|287.58 288|288.15 289|289.46 290|290.88 291|291.71 292|292.48 293|293.10 294|294.7 295|295.21 296|296.75 297|297.92 298|298.31 299|299.19 300|300.28 301|301.79 302|302.38 303|303.57 304|304.69 305|305.30 306|306.48 307|307.69 308|308.81 309|309.93 310|310.82 311|311.44 312|312.82 313|313.91 314|314.95 315|315.99 316|316.53 317|317.59 318|318.63 319|319.63 320|320.34 321|321.62 322|322.92 323|323.33 324|324.28 325|325.41 326|326.29 327|327.97 328|328.82 329|329.35 330|330.81 331|331.66 332|332.4 333|333.59 334|334.45 335|335.18 336|336.49 337|337.80 338|338.33 339|339.11 340|340.44 341|341.52 342|342.43 343|343.48 344|344.30 345|345.92 346|346.93 347|347.67 348|348.41 349|349.85 350|350.44 351|351.23 352|352.14 353|353.11 354|354.50 355|355.91 356|356.25 357|357.93 358|358.57 359|359.77 360|360.70 361|361.10 362|362.23 363|363.98 364|364.77 365|365.1 366|366.18 367|367.83 368|368.89 369|369.50 370|370.33 371|371.9 372|372.1 373|373.19 374|374.47 375|375.53 376|376.65 377|377.62 378|378.92 379|379.51 380|380.75 381|381.26 382|382.89 383|383.12 384|384.64 385|385.33 386|386.81 387|387.97 388|388.15 389|389.40 390|390.81 391|391.56 392|392.34 393|393.5 394|394.78 395|395.21 396|396.31 397|397.9 398|398.86 399|399.42 400|400.28 401|401.59 402|402.29 403|403.51 404|404.49 405|405.33 406|406.25 407|407.38 408|408.13 409|409.29 410|410.26 411|411.54 412|412.52 413|413.60 414|414.11 415|415.34 416|416.43 417|417.36 418|418.27 419|419.66 420|420.7 421|421.43 422|422.17 423|423.81 424|424.72 425|425.95 426|426.4 427|427.74 428|428.70 429|429.55 430|430.26 431|431.42 432|432.43 433|433.3 434|434.51 435|435.41 436|436.24 437|437.45 438|438.76 439|439.32 440|440.33 441|441.35 442|442.58 443|443.62 444|444.98 445|445.18 446|446.83 447|447.41 448|448.12 449|449.17 450|450.3 451|451.15 452|452.41 453|453.88 454|454.23 455|455.88 456|456.14 457|457.87 458|458.22 459|459.6 460|460.86 461|461.39 462|462.47 463|463.7 464|464.65 465|465.20 466|466.72 467|467.68 468|468.58 469|469.49 470|470.48 471|471.68 472|472.2 473|473.65 474|474.99 475|475.5 476|476.50 477|477.72 478|478.69 479|479.3 480|480.88 481|481.6 482|482.60 483|483.33 484|484.12 485|485.67 486|486.2 487|487.57 488|488.98 489|489.93 490|490.98 491|491.58 492|492.60 493|493.53 494|494.85 495|495.61 496|496.75 497|497.94 498|498.59 499|499.51 500|500.2 501|501.95 502|502.82 503|503.67 504|504.68 505|505.11 506|506.58 507|507.72 508|508.10 509|509.95 510|510.15 511|511.20 512|512.35 513|513.29 514|514.11 515|515.42 516|516.50 517|517.54 518|518.49 519|519.0 520|520.72 521|521.27 522|522.33 523|523.2 524|524.34 525|525.71 526|526.18 527|527.41 528|528.22 529|529.85 530|530.37 531|531.83 532|532.12 533|533.87 534|534.78 535|535.75 536|536.68 537|537.23 538|538.1 539|539.3 540|540.40 541|541.78 542|542.53 543|543.58 544|544.9 545|545.47 546|546.27 547|547.18 548|548.61 549|549.15 550|550.12 551|551.64 552|552.97 553|553.23 554|554.17 555|555.23 556|556.83 557|557.46 558|558.92 559|559.27 560|560.84 561|561.74 562|562.88 563|563.40 564|564.31 565|565.19 566|566.53 567|567.45 568|568.33 569|569.12 570|570.14 571|571.61 572|572.76 573|573.71 574|574.86 575|575.9 576|576.40 577|577.60 578|578.79 579|579.2 580|580.84 581|581.60 582|582.23 583|583.80 584|584.78 585|585.98 586|586.46 587|587.70 588|588.39 589|589.26 590|590.63 591|591.59 592|592.93 593|593.84 594|594.23 595|595.16 596|596.16 597|597.84 598|598.46 599|599.35 600|600.30 601|601.97 602|602.58 603|603.73 604|604.12 605|605.8 606|606.57 607|607.3 608|608.13 609|609.88 610|610.85 611|611.99 612|612.43 613|613.23 614|614.49 615|615.77 616|616.82 617|617.74 618|618.17 619|619.68 620|620.32 621|621.3 622|622.60 623|623.28 624|624.89 625|625.94 626|626.41 627|627.14 628|628.35 629|629.33 630|630.25 631|631.9 632|632.47 633|633.87 634|634.22 635|635.30 636|636.11 637|637.83 638|638.30 639|639.16 640|640.34 641|641.52 642|642.75 643|643.6 644|644.51 645|645.2 646|646.51 647|647.33 648|648.63 649|649.81 650|650.97 651|651.34 652|652.13 653|653.85 654|654.56 655|655.65 656|656.38 657|657.15 658|658.54 659|659.44 660|660.6 661|661.89 662|662.35 663|663.48 664|664.15 665|665.84 666|666.26 667|667.48 668|668.56 669|669.49 670|670.96 671|671.30 672|672.32 673|673.99 674|674.69 675|675.99 676|676.68 677|677.88 678|678.82 679|679.16 680|680.71 681|681.96 682|682.34 683|683.56 684|684.41 685|685.38 686|686.43 687|687.84 688|688.51 689|689.31 690|690.24 691|691.76 692|692.54 693|693.45 694|694.26 695|695.28 696|696.88 697|697.0 698|698.74 699|699.19 700|700.96 701|701.24 702|702.78 703|703.67 704|704.72 705|705.51 706|706.43 707|707.48 708|708.81 709|709.23 710|710.32 711|711.90 712|712.29 713|713.97 714|714.54 715|715.92 716|716.87 717|717.68 718|718.23 719|719.78 720|720.18 721|721.3 722|722.51 723|723.55 724|724.37 725|725.97 726|726.99 727|727.7 728|728.33 729|729.45 730|730.72 731|731.32 732|732.54 733|733.33 734|734.78 735|735.12 736|736.99 737|737.42 738|738.86 739|739.30 740|740.87 741|741.59 742|742.22 743|743.8 744|744.81 745|745.86 746|746.83 747|747.49 748|748.28 749|749.11 750|750.96 751|751.10 752|752.3 753|753.47 754|754.9 755|755.50 756|756.4 757|757.85 758|758.70 759|759.84 760|760.16 761|761.36 762|762.0 763|763.57 764|764.77 765|765.30 766|766.5 767|767.41 768|768.28 769|769.74 770|770.29 771|771.45 772|772.90 773|773.3 774|774.68 775|775.84 776|776.7 777|777.89 778|778.28 779|779.33 780|780.82 781|781.35 782|782.86 783|783.49 784|784.40 785|785.12 786|786.70 787|787.34 788|788.54 789|789.25 790|790.65 791|791.28 792|792.57 793|793.22 794|794.30 795|795.80 796|796.65 797|797.32 798|798.29 799|799.0 800|800.27 801|801.28 802|802.54 803|803.96 804|804.15 805|805.36 806|806.52 807|807.45 808|808.38 809|809.2 810|810.15 811|811.58 812|812.29 813|813.15 814|814.77 815|815.98 816|816.90 817|817.77 818|818.94 819|819.26 820|820.39 821|821.96 822|822.36 823|823.58 824|824.80 825|825.95 826|826.37 827|827.17 828|828.98 829|829.36 830|830.54 831|831.20 832|832.11 833|833.55 834|834.87 835|835.44 836|836.21 837|837.48 838|838.7 839|839.46 840|840.53 841|841.27 842|842.51 843|843.75 844|844.69 845|845.41 846|846.42 847|847.45 848|848.54 849|849.59 850|850.8 851|851.59 852|852.6 853|853.16 854|854.91 855|855.57 856|856.45 857|857.86 858|858.78 859|859.43 860|860.77 861|861.92 862|862.25 863|863.49 864|864.15 865|865.65 866|866.76 867|867.2 868|868.96 869|869.69 870|870.45 871|871.81 872|872.88 873|873.50 874|874.86 875|875.36 876|876.40 877|877.95 878|878.59 879|879.55 880|880.55 881|881.71 882|882.3 883|883.75 884|884.56 885|885.48 886|886.12 887|887.54 888|888.40 889|889.17 890|890.7 891|891.18 892|892.41 893|893.99 894|894.30 895|895.75 896|896.66 897|897.21 898|898.59 899|899.30 900|900.75 901|901.95 902|902.35 903|903.3 904|904.85 905|905.91 906|906.35 907|907.89 908|908.59 909|909.33 910|910.61 911|911.39 912|912.46 913|913.49 914|914.8 915|915.10 916|916.77 917|917.3 918|918.43 919|919.32 920|920.13 921|921.18 922|922.76 923|923.52 924|924.83 925|925.59 926|926.22 927|927.63 928|928.40 929|929.29 930|930.65 931|931.32 932|932.20 933|933.97 934|934.57 935|935.96 936|936.7 937|937.12 938|938.73 939|939.59 940|940.18 941|941.8 942|942.79 943|943.76 944|944.67 945|945.43 946|946.83 947|947.1 948|948.19 949|949.34 950|950.14 951|951.37 952|952.29 953|953.3 954|954.94 955|955.21 956|956.93 957|957.51 958|958.3 959|959.89 960|960.27 961|961.76 962|962.71 963|963.73 964|964.48 965|965.5 966|966.63 967|967.29 968|968.58 969|969.65 970|970.42 971|971.88 972|972.73 973|973.57 974|974.23 975|975.5 976|976.14 977|977.2 978|978.34 979|979.83 980|980.71 981|981.97 982|982.95 983|983.91 984|984.48 985|985.98 986|986.0 987|987.25 988|988.88 989|989.34 990|990.8 991|991.72 992|992.30 993|993.33 994|994.55 995|995.57 996|996.91 997|997.54 998|998.5 999|999.65 1000|1000.11 1001|1001.50 1002|1002.39 1003|1003.67 1004|1004.9 1005|1005.31 1006|1006.51 1007|1007.82 1008|1008.25 1009|1009.20 1010|1010.25 1011|1011.57 1012|1012.68 1013|1013.50 1014|1014.79 1015|1015.62 1016|1016.1 1017|1017.43 1018|1018.21 1019|1019.59 1020|1020.36 1021|1021.36 1022|1022.59 1023|1023.96 1024|1024.87 1025|1025.10 1026|1026.35 1027|1027.51 1028|1028.81 1029|1029.83 1030|1030.95 1031|1031.10 1032|1032.72 1033|1033.36 1034|1034.87 1035|1035.74 1036|1036.39 1037|1037.39 1038|1038.43 1039|1039.16 1040|1040.47 1041|1041.61 1042|1042.1 1043|1043.13 1044|1044.76 1045|1045.43 1046|1046.0 1047|1047.33 1048|1048.26 1049|1049.16 1050|1050.12 1051|1051.81 1052|1052.5 1053|1053.87 1054|1054.84 1055|1055.52 1056|1056.55 1057|1057.25 1058|1058.96 1059|1059.81 1060|1060.87 1061|1061.50 1062|1062.30 1063|1063.55 1064|1064.87 1065|1065.60 1066|1066.84 1067|1067.43 1068|1068.89 1069|1069.7 1070|1070.7 1071|1071.42 1072|1072.23 1073|1073.81 1074|1074.74 1075|1075.78 </str>
<float name="score">2.0</float></doc>
<doc>
<str name="id">4</str>
<int name="id_i">4</int>
<float name="default_amount_f">11326.0</float>
<str name="amounts_dpf">1|1.21 2|2.16 3|3.6 4|4.93 5|5.34 6|6.11 7|7.20 8|8.35 9|9.39 10|10.47 11|11.1 12|12.2 13|13.76 14|14.28 15|15.86 16|16.73 17|17.61 18|18.99 19|19.88 20|20.75 21|21.4 22|22.70 23|23.13 24|24.4 25|25.51 26|26.82 27|27.12 28|28.46 29|29.53 30|30.99 31|31.71 32|32.68 33|33.77 34|34.10 35|35.80 36|36.44 37|37.45 38|38.70 39|39.19 40|40.17 41|41.28 42|42.56 43|43.28 44|44.46 45|45.68 46|46.99 47|47.27 48|48.19 49|49.96 50|50.56 51|51.5 52|52.64 53|53.50 54|54.72 55|55.97 56|56.83 57|57.39 58|58.91 59|59.26 60|60.25 61|61.60 62|62.56 63|63.57 64|64.40 65|65.73 66|66.13 67|67.97 68|68.69 69|69.83 70|70.48 71|71.49 72|72.6 73|73.21 74|74.70 75|75.31 76|76.79 77|77.94 78|78.9 79|79.8 80|80.93 81|81.40 82|82.90 83|83.90 84|84.59 85|85.16 86|86.23 87|87.78 88|88.70 89|89.97 90|90.85 91|91.20 92|92.93 93|93.66 94|94.29 95|95.42 96|96.39 97|97.98 98|98.60 99|99.1 100|100.36 101|101.30 102|102.23 103|103.39 104|104.78 105|105.30 106|106.36 107|107.3 108|108.14 109|109.83 110|110.26 111|111.47 112|112.62 113|113.45 114|114.40 115|115.28 116|116.9 117|117.0 118|118.50 119|119.22 120|120.27 121|121.66 122|122.70 123|123.57 124|124.27 125|125.30 126|126.16 127|127.21 128|128.59 129|129.6 130|130.99 131|131.83 132|132.20 133|133.3 134|134.15 135|135.58 136|136.82 137|137.7 138|138.72 139|139.34 140|140.88 141|141.59 142|142.41 143|143.61 144|144.74 145|145.99 146|146.27 147|147.61 148|148.31 149|149.17 150|150.19 151|151.33 152|152.44 153|153.26 154|154.7 155|155.7 156|156.0 157|157.44 158|158.55 159|159.54 160|160.51 161|161.86 162|162.15 163|163.80 164|164.16 165|165.35 166|166.47 167|167.18 168|168.85 169|169.59 170|170.42 171|171.86 172|172.48 173|173.54 174|174.75 175|175.43 176|176.38 177|177.98 178|178.17 179|179.97 180|180.76 181|181.22 182|182.31 183|183.13 184|184.84 185|185.21 186|186.59 187|187.54 188|188.87 189|189.57 190|190.25 191|191.68 192|192.89 193|193.65 194|194.56 195|195.80 196|196.87 197|197.18 198|198.55 199|199.19 200|200.88 201|201.78 202|202.86 203|203.1 204|204.35 205|205.81 206|206.1 207|207.35 208|208.98 209|209.57 210|210.57 211|211.14 212|212.96 213|213.89 214|214.4 215|215.57 216|216.6 217|217.72 218|218.52 219|219.91 220|220.80 221|221.91 222|222.89 223|223.71 224|224.27 225|225.3 226|226.22 227|227.45 228|228.96 229|229.29 230|230.32 231|231.58 232|232.20 233|233.95 234|234.19 235|235.42 236|236.80 237|237.85 238|238.36 239|239.83 240|240.42 241|241.92 242|242.84 243|243.32 244|244.23 245|245.32 246|246.83 247|247.72 248|248.63 249|249.61 250|250.53 251|251.11 252|252.45 253|253.95 254|254.45 255|255.14 256|256.50 257|257.51 258|258.60 259|259.10 260|260.41 261|261.34 262|262.86 263|263.63 264|264.44 265|265.95 266|266.73 267|267.46 268|268.98 269|269.34 270|270.0 271|271.86 272|272.23 273|273.58 274|274.53 275|275.62 276|276.81 277|277.78 278|278.11 279|279.64 280|280.24 281|281.55 282|282.2 283|283.46 284|284.62 285|285.81 286|286.74 287|287.16 288|288.8 289|289.31 290|290.40 291|291.86 292|292.79 293|293.44 294|294.70 295|295.45 296|296.2 297|297.84 298|298.1 299|299.1 300|300.49 301|301.47 302|302.93 303|303.62 304|304.55 305|305.51 306|306.60 307|307.56 308|308.98 309|309.3 310|310.43 311|311.19 312|312.73 313|313.69 314|314.1 315|315.13 316|316.51 317|317.51 318|318.27 319|319.29 320|320.34 321|321.89 322|322.21 323|323.44 324|324.81 325|325.6 326|326.52 327|327.12 328|328.2 329|329.68 330|330.79 331|331.70 332|332.7 333|333.71 334|334.47 335|335.77 336|336.71 337|337.95 338|338.18 339|339.74 340|340.42 341|341.32 342|342.63 343|343.31 344|344.75 345|345.85 346|346.81 347|347.41 348|348.68 349|349.20 350|350.32 351|351.80 352|352.62 353|353.12 354|354.62 355|355.44 356|356.10 357|357.76 358|358.65 359|359.58 360|360.39 361|361.95 362|362.72 363|363.91 364|364.56 365|365.79 366|366.99 367|367.13 368|368.12 369|369.97 370|370.70 371|371.37 372|372.39 373|373.80 374|374.46 375|375.45 376|376.0 377|377.21 378|378.52 379|379.58 380|380.73 381|381.43 382|382.63 383|383.43 384|384.80 385|385.83 386|386.55 387|387.33 388|388.66 389|389.90 390|390.71 391|391.7 392|392.3 393|393.79 394|394.72 395|395.75 396|396.44 397|397.1 398|398.53 399|399.25 400|400.46 401|401.79 402|402.44 403|403.45 404|404.22 405|405.60 406|406.79 407|407.22 408|408.11 409|409.10 410|410.27 411|411.93 412|412.11 413|413.24 414|414.54 415|415.88 416|416.75 417|417.53 418|418.79 419|419.54 420|420.11 421|421.44 422|422.82 423|423.3 424|424.39 425|425.17 426|426.20 427|427.29 428|428.11 429|429.56 430|430.58 431|431.91 432|432.73 433|433.25 434|434.20 435|435.18 436|436.99 437|437.64 438|438.70 439|439.29 440|440.31 441|441.70 442|442.8 443|443.72 444|444.64 445|445.67 446|446.26 447|447.35 448|448.24 449|449.5 450|450.37 451|451.59 452|452.18 453|453.79 454|454.62 455|455.28 456|456.9 457|457.42 458|458.69 459|459.65 460|460.5 461|461.83 462|462.23 463|463.59 464|464.50 465|465.90 466|466.60 467|467.51 468|468.72 469|469.49 470|470.42 471|471.8 472|472.42 473|473.70 474|474.5 475|475.2 476|476.21 477|477.59 478|478.56 479|479.56 480|480.73 481|481.3 482|482.89 483|483.78 484|484.75 485|485.15 486|486.27 487|487.34 488|488.24 489|489.60 490|490.49 491|491.65 492|492.34 493|493.35 494|494.35 495|495.59 496|496.33 497|497.37 498|498.3 499|499.97 500|500.5 501|501.25 502|502.22 503|503.95 504|504.87 505|505.32 506|506.74 507|507.9 508|508.64 509|509.98 510|510.42 511|511.67 512|512.66 513|513.44 514|514.97 515|515.30 516|516.61 517|517.11 518|518.66 519|519.55 520|520.79 521|521.64 522|522.57 523|523.93 524|524.79 525|525.42 526|526.82 527|527.86 528|528.63 529|529.23 530|530.62 531|531.4 532|532.79 533|533.46 534|534.26 535|535.64 536|536.16 537|537.26 538|538.72 539|539.36 540|540.60 541|541.86 542|542.2 543|543.56 544|544.98 545|545.58 546|546.57 547|547.96 548|548.72 549|549.86 550|550.4 551|551.16 552|552.33 553|553.51 554|554.20 555|555.52 556|556.27 557|557.12 558|558.60 559|559.51 560|560.12 561|561.42 562|562.66 563|563.68 564|564.43 565|565.42 566|566.34 567|567.95 568|568.27 569|569.24 570|570.25 571|571.27 572|572.55 573|573.50 574|574.74 575|575.50 576|576.9 577|577.67 578|578.58 579|579.78 580|580.84 581|581.41 582|582.63 583|583.62 584|584.90 585|585.26 586|586.73 587|587.93 588|588.18 589|589.20 590|590.82 591|591.73 592|592.9 593|593.61 594|594.17 595|595.45 596|596.49 597|597.2 598|598.28 599|599.66 600|600.41 601|601.68 602|602.78 603|603.5 604|604.82 605|605.60 606|606.48 607|607.82 608|608.38 609|609.45 610|610.38 611|611.56 612|612.0 613|613.0 614|614.1 615|615.78 616|616.58 617|617.7 618|618.29 619|619.28 620|620.48 621|621.89 622|622.42 623|623.53 624|624.30 625|625.47 626|626.53 627|627.42 628|628.36 629|629.30 630|630.52 631|631.61 632|632.91 633|633.5 634|634.56 635|635.30 636|636.1 637|637.64 638|638.99 639|639.33 640|640.21 641|641.9 642|642.23 643|643.32 644|644.22 645|645.50 646|646.44 647|647.68 648|648.1 649|649.97 650|650.30 651|651.5 652|652.8 653|653.15 654|654.84 655|655.32 656|656.66 657|657.33 658|658.87 659|659.30 660|660.86 661|661.15 662|662.61 663|663.45 664|664.74 665|665.96 666|666.6 667|667.16 668|668.2 669|669.20 670|670.19 671|671.97 672|672.66 673|673.15 674|674.68 675|675.69 676|676.24 677|677.50 678|678.95 679|679.84 680|680.11 681|681.35 682|682.38 683|683.54 684|684.81 685|685.46 686|686.54 687|687.16 688|688.79 689|689.52 690|690.22 691|691.75 692|692.6 693|693.94 694|694.43 695|695.95 696|696.68 697|697.77 698|698.52 699|699.36 700|700.87 701|701.93 702|702.88 703|703.48 704|704.96 705|705.94 706|706.58 707|707.91 708|708.16 709|709.32 710|710.82 711|711.39 712|712.99 713|713.99 714|714.30 715|715.0 716|716.15 717|717.26 718|718.28 719|719.91 720|720.52 721|721.46 722|722.86 723|723.51 724|724.19 725|725.25 726|726.90 727|727.8 728|728.84 729|729.66 730|730.21 731|731.21 732|732.46 733|733.72 734|734.3 735|735.38 736|736.83 737|737.79 738|738.81 739|739.21 740|740.15 741|741.14 742|742.19 743|743.47 744|744.95 745|745.88 746|746.17 747|747.20 748|748.45 749|749.93 750|750.96 751|751.98 752|752.20 753|753.43 754|754.37 755|755.70 756|756.11 757|757.12 758|758.94 759|759.25 760|760.51 761|761.47 762|762.24 763|763.85 764|764.65 765|765.70 766|766.70 767|767.70 768|768.14 769|769.27 770|770.67 771|771.22 772|772.32 773|773.4 774|774.65 775|775.75 776|776.90 777|777.90 778|778.59 779|779.58 780|780.8 781|781.21 782|782.52 783|783.68 784|784.39 785|785.22 786|786.86 787|787.33 788|788.94 789|789.60 790|790.88 791|791.2 792|792.74 793|793.50 794|794.95 795|795.36 796|796.31 797|797.12 798|798.12 799|799.20 800|800.5 801|801.56 802|802.23 803|803.76 804|804.66 805|805.81 806|806.31 807|807.77 808|808.20 809|809.37 810|810.71 811|811.63 812|812.87 813|813.48 814|814.32 815|815.71 816|816.43 817|817.64 818|818.64 819|819.43 820|820.35 821|821.40 822|822.35 823|823.97 824|824.40 825|825.83 826|826.90 827|827.76 828|828.25 829|829.96 830|830.77 831|831.78 832|832.17 833|833.46 834|834.69 835|835.12 836|836.4 837|837.33 838|838.9 839|839.81 840|840.53 841|841.95 842|842.14 843|843.88 844|844.78 845|845.46 846|846.39 847|847.6 848|848.31 849|849.58 850|850.12 851|851.89 852|852.5 853|853.74 854|854.59 855|855.72 856|856.92 857|857.14 858|858.73 859|859.89 860|860.11 861|861.42 862|862.98 863|863.45 864|864.54 865|865.98 866|866.45 867|867.8 868|868.89 869|869.90 870|870.88 871|871.59 872|872.68 873|873.76 874|874.11 875|875.36 876|876.76 877|877.96 878|878.53 879|879.34 880|880.42 881|881.80 882|882.61 883|883.2 884|884.80 885|885.33 886|886.22 887|887.41 888|888.2 889|889.85 890|890.84 891|891.17 892|892.15 893|893.34 894|894.75 895|895.55 896|896.53 897|897.35 898|898.47 899|899.35 900|900.91 901|901.67 902|902.59 903|903.50 904|904.36 905|905.29 906|906.22 907|907.60 908|908.17 909|909.98 910|910.63 911|911.35 912|912.28 913|913.97 914|914.60 915|915.77 916|916.92 917|917.73 918|918.63 919|919.26 920|920.45 921|921.85 922|922.95 923|923.13 924|924.60 925|925.84 926|926.37 927|927.56 928|928.84 929|929.16 930|930.9 931|931.64 932|932.53 933|933.99 934|934.27 935|935.89 936|936.81 937|937.77 938|938.38 939|939.5 940|940.3 941|941.78 942|942.6 943|943.32 944|944.24 945|945.48 946|946.45 947|947.14 948|948.41 949|949.2 950|950.44 951|951.59 952|952.24 953|953.97 954|954.9 955|955.21 956|956.83 957|957.40 958|958.72 959|959.50 960|960.99 961|961.38 962|962.16 963|963.64 964|964.98 965|965.91 966|966.5 967|967.43 968|968.78 969|969.8 970|970.16 971|971.86 972|972.31 973|973.12 974|974.72 975|975.35 976|976.78 977|977.0 978|978.90 979|979.40 980|980.46 981|981.60 982|982.9 983|983.21 984|984.67 985|985.12 986|986.80 987|987.39 988|988.10 989|989.35 990|990.40 991|991.72 992|992.61 993|993.22 994|994.3 995|995.52 996|996.11 997|997.58 998|998.68 999|999.18 1000|1000.13 1001|1001.50 1002|1002.45 1003|1003.76 1004|1004.70 1005|1005.18 1006|1006.10 1007|1007.91 1008|1008.72 1009|1009.81 1010|1010.55 1011|1011.53 1012|1012.47 1013|1013.48 1014|1014.97 1015|1015.42 1016|1016.11 1017|1017.45 1018|1018.42 1019|1019.8 1020|1020.42 1021|1021.60 1022|1022.61 1023|1023.88 1024|1024.71 1025|1025.83 1026|1026.69 1027|1027.62 1028|1028.28 1029|1029.68 1030|1030.28 1031|1031.19 1032|1032.14 1033|1033.20 1034|1034.75 1035|1035.25 1036|1036.48 1037|1037.10 1038|1038.22 1039|1039.96 1040|1040.47 1041|1041.59 1042|1042.87 1043|1043.47 1044|1044.68 1045|1045.38 1046|1046.15 1047|1047.97 1048|1048.67 1049|1049.47 1050|1050.55 1051|1051.40 1052|1052.76 1053|1053.59 1054|1054.11 1055|1055.57 1056|1056.14 1057|1057.23 1058|1058.81 1059|1059.18 1060|1060.29 1061|1061.63 1062|1062.7 1063|1063.10 1064|1064.9 1065|1065.65 1066|1066.42 1067|1067.63 1068|1068.7 1069|1069.29 1070|1070.10 1071|1071.2 1072|1072.35 1073|1073.72 1074|1074.32 1075|1075.71 </str>
<float name="score">2.0</float></doc>
<doc>
<str name="id">5</str>
<int name="id_i">5</int>
<float name="default_amount_f">11617.0</float>
<str name="amounts_dpf">1|1.48 2|2.32 3|3.90 4|4.96 5|5.28 6|6.66 7|7.78 8|8.48 9|9.69 10|10.72 11|11.51 12|12.96 13|13.49 14|14.48 15|15.75 16|16.54 17|17.24 18|18.32 19|19.58 20|20.35 21|21.64 22|22.93 23|23.81 24|24.20 25|25.52 26|26.95 27|27.23 28|28.29 29|29.76 30|30.46 31|31.6 32|32.20 33|33.55 34|34.56 35|35.24 36|36.91 37|37.17 38|38.28 39|39.31 40|40.67 41|41.28 42|42.77 43|43.75 44|44.54 45|45.32 46|46.41 47|47.43 48|48.88 49|49.27 50|50.48 51|51.36 52|52.17 53|53.70 54|54.52 55|55.98 56|56.92 57|57.11 58|58.97 59|59.85 60|60.90 61|61.11 62|62.66 63|63.88 64|64.0 65|65.53 66|66.3 67|67.90 68|68.66 69|69.22 70|70.12 71|71.14 72|72.42 73|73.42 74|74.64 75|75.7 76|76.30 77|77.27 78|78.97 79|79.56 80|80.34 81|81.19 82|82.75 83|83.72 84|84.27 85|85.11 86|86.63 87|87.6 88|88.66 89|89.40 90|90.98 91|91.28 92|92.83 93|93.69 94|94.59 95|95.42 96|96.68 97|97.25 98|98.71 99|99.66 100|100.27 101|101.48 102|102.19 103|103.39 104|104.90 105|105.82 106|106.91 107|107.13 108|108.42 109|109.44 110|110.22 111|111.76 112|112.2 113|113.16 114|114.69 115|115.61 116|116.96 117|117.63 118|118.93 119|119.84 120|120.83 121|121.89 122|122.22 123|123.11 124|124.78 125|125.75 126|126.23 127|127.84 128|128.85 129|129.70 130|130.25 131|131.95 132|132.20 133|133.15 134|134.83 135|135.35 136|136.84 137|137.7 138|138.38 139|139.26 140|140.28 141|141.60 142|142.0 143|143.6 144|144.45 145|145.18 146|146.43 147|147.39 148|148.22 149|149.43 150|150.79 151|151.72 152|152.60 153|153.34 154|154.88 155|155.24 156|156.70 157|157.22 158|158.27 159|159.93 160|160.61 161|161.70 162|162.97 163|163.60 164|164.64 165|165.69 166|166.80 167|167.52 168|168.88 169|169.84 170|170.78 171|171.60 172|172.35 173|173.98 174|174.70 175|175.87 176|176.16 177|177.47 178|178.40 179|179.30 180|180.37 181|181.81 182|182.14 183|183.12 184|184.44 185|185.46 186|186.8 187|187.39 188|188.6 189|189.99 190|190.89 191|191.14 192|192.32 193|193.56 194|194.83 195|195.58 196|196.78 197|197.5 198|198.56 199|199.11 200|200.1 201|201.84 202|202.74 203|203.54 204|204.99 205|205.43 206|206.15 207|207.38 208|208.42 209|209.92 210|210.34 211|211.88 212|212.69 213|213.0 214|214.61 215|215.0 216|216.44 217|217.50 218|218.38 219|219.16 220|220.47 221|221.23 222|222.8 223|223.36 224|224.70 225|225.71 226|226.97 227|227.74 228|228.77 229|229.42 230|230.3 231|231.80 232|232.15 233|233.79 234|234.88 235|235.32 236|236.26 237|237.1 238|238.81 239|239.77 240|240.20 241|241.31 242|242.65 243|243.71 244|244.62 245|245.45 246|246.34 247|247.27 248|248.22 249|249.68 250|250.91 251|251.46 252|252.46 253|253.87 254|254.12 255|255.31 256|256.57 257|257.99 258|258.1 259|259.14 260|260.51 261|261.49 262|262.78 263|263.57 264|264.54 265|265.97 266|266.9 267|267.29 268|268.22 269|269.9 270|270.38 271|271.22 272|272.0 273|273.58 274|274.48 275|275.3 276|276.54 277|277.71 278|278.80 279|279.75 280|280.6 281|281.56 282|282.67 283|283.12 284|284.63 285|285.4 286|286.98 287|287.21 288|288.62 289|289.55 290|290.84 291|291.32 292|292.29 293|293.6 294|294.91 295|295.81 296|296.74 297|297.61 298|298.57 299|299.31 300|300.25 301|301.85 302|302.68 303|303.55 304|304.94 305|305.81 306|306.9 307|307.43 308|308.67 309|309.38 310|310.50 311|311.33 312|312.82 313|313.12 314|314.82 315|315.19 316|316.56 317|317.47 318|318.48 319|319.97 320|320.45 321|321.13 322|322.2 323|323.38 324|324.95 325|325.90 326|326.78 327|327.83 328|328.29 329|329.61 330|330.12 331|331.6 332|332.23 333|333.22 334|334.48 335|335.8 336|336.98 337|337.10 338|338.54 339|339.88 340|340.28 341|341.5 342|342.0 343|343.51 344|344.11 345|345.51 346|346.6 347|347.0 348|348.99 349|349.90 350|350.3 351|351.75 352|352.1 353|353.53 354|354.76 355|355.94 356|356.65 357|357.79 358|358.55 359|359.75 360|360.26 361|361.68 362|362.76 363|363.35 364|364.16 365|365.5 366|366.47 367|367.59 368|368.33 369|369.95 370|370.97 371|371.16 372|372.15 373|373.89 374|374.57 375|375.0 376|376.54 377|377.98 378|378.75 379|379.47 380|380.72 381|381.65 382|382.4 383|383.72 384|384.36 385|385.85 386|386.5 387|387.34 388|388.46 389|389.80 390|390.76 391|391.99 392|392.33 393|393.21 394|394.64 395|395.97 396|396.96 397|397.98 398|398.74 399|399.99 400|400.42 401|401.49 402|402.41 403|403.55 404|404.84 405|405.0 406|406.50 407|407.59 408|408.81 409|409.69 410|410.46 411|411.93 412|412.73 413|413.2 414|414.29 415|415.59 416|416.5 417|417.7 418|418.32 419|419.1 420|420.12 421|421.42 422|422.6 423|423.64 424|424.71 425|425.95 426|426.48 427|427.57 428|428.21 429|429.37 430|430.51 431|431.61 432|432.39 433|433.0 434|434.80 435|435.15 436|436.3 437|437.70 438|438.58 439|439.26 440|440.93 441|441.13 442|442.29 443|443.49 444|444.57 445|445.38 446|446.47 447|447.43 448|448.8 449|449.13 450|450.61 451|451.78 452|452.67 453|453.54 454|454.81 455|455.12 456|456.68 457|457.42 458|458.73 459|459.64 460|460.5 461|461.12 462|462.9 463|463.60 464|464.97 465|465.66 466|466.82 467|467.46 468|468.39 469|469.35 470|470.61 471|471.33 472|472.10 473|473.84 474|474.30 475|475.70 476|476.83 477|477.28 478|478.26 479|479.71 480|480.54 481|481.69 482|482.36 483|483.21 484|484.44 485|485.19 486|486.13 487|487.99 488|488.13 489|489.12 490|490.89 491|491.94 492|492.53 493|493.12 494|494.63 495|495.10 496|496.80 497|497.69 498|498.19 499|499.62 500|500.3 501|501.84 502|502.32 503|503.96 504|504.22 505|505.82 506|506.49 507|507.99 508|508.97 509|509.19 510|510.46 511|511.5 512|512.5 513|513.58 514|514.21 515|515.54 516|516.70 517|517.11 518|518.58 519|519.8 520|520.36 521|521.91 522|522.11 523|523.65 524|524.43 525|525.28 526|526.45 527|527.6 528|528.98 529|529.42 530|530.78 531|531.49 532|532.70 533|533.32 534|534.55 535|535.80 536|536.55 537|537.24 538|538.26 539|539.55 540|540.30 541|541.85 542|542.6 543|543.59 544|544.67 545|545.38 546|546.4 547|547.6 548|548.29 549|549.98 550|550.70 551|551.31 552|552.82 553|553.51 554|554.18 555|555.76 556|556.54 557|557.52 558|558.75 559|559.33 560|560.68 561|561.43 562|562.22 563|563.70 564|564.94 565|565.77 566|566.46 567|567.87 568|568.12 569|569.82 570|570.24 571|571.23 572|572.19 573|573.8 574|574.91 575|575.53 576|576.2 577|577.29 578|578.50 579|579.46 580|580.84 581|581.27 582|582.51 583|583.11 584|584.45 585|585.83 586|586.62 587|587.66 588|588.12 589|589.67 590|590.70 591|591.9 592|592.14 593|593.40 594|594.73 595|595.17 596|596.72 597|597.16 598|598.99 599|599.28 600|600.52 601|601.68 602|602.51 603|603.2 604|604.51 605|605.18 606|606.89 607|607.67 608|608.18 609|609.23 610|610.88 611|611.10 612|612.89 613|613.53 614|614.83 615|615.22 616|616.68 617|617.11 618|618.58 619|619.82 620|620.39 621|621.76 622|622.3 623|623.83 624|624.27 625|625.37 626|626.11 627|627.43 628|628.10 629|629.54 630|630.31 631|631.13 632|632.89 633|633.80 634|634.88 635|635.52 636|636.51 637|637.24 638|638.64 639|639.62 640|640.58 641|641.5 642|642.33 643|643.19 644|644.47 645|645.55 646|646.36 647|647.5 648|648.32 649|649.81 650|650.79 651|651.95 652|652.54 653|653.68 654|654.15 655|655.10 656|656.50 657|657.94 658|658.62 659|659.74 660|660.99 661|661.82 662|662.69 663|663.10 664|664.4 665|665.28 666|666.36 667|667.90 668|668.16 669|669.5 670|670.54 671|671.41 672|672.18 673|673.92 674|674.99 675|675.63 676|676.90 677|677.30 678|678.94 679|679.91 680|680.22 681|681.19 682|682.41 683|683.94 684|684.51 685|685.62 686|686.2 687|687.72 688|688.68 689|689.64 690|690.24 691|691.61 692|692.41 693|693.11 694|694.10 695|695.86 696|696.44 697|697.60 698|698.69 699|699.0 700|700.22 701|701.21 702|702.40 703|703.44 704|704.99 705|705.42 706|706.86 707|707.16 708|708.41 709|709.58 710|710.35 711|711.44 712|712.60 713|713.4 714|714.11 715|715.80 716|716.58 717|717.43 718|718.96 719|719.93 720|720.17 721|721.16 722|722.88 723|723.57 724|724.22 725|725.8 726|726.53 727|727.48 728|728.50 729|729.48 730|730.70 731|731.74 732|732.72 733|733.3 734|734.6 735|735.89 736|736.71 737|737.57 738|738.37 739|739.65 740|740.4 741|741.25 742|742.86 743|743.37 744|744.28 745|745.58 746|746.43 747|747.88 748|748.88 749|749.22 750|750.18 751|751.96 752|752.83 753|753.19 754|754.76 755|755.21 756|756.9 757|757.6 758|758.49 759|759.66 760|760.22 761|761.78 762|762.21 763|763.89 764|764.97 765|765.14 766|766.82 767|767.28 768|768.71 769|769.14 770|770.69 771|771.18 772|772.93 773|773.54 774|774.83 775|775.29 776|776.6 777|777.21 778|778.89 779|779.94 780|780.36 781|781.29 782|782.13 783|783.79 784|784.6 785|785.80 786|786.79 787|787.7 788|788.67 789|789.21 790|790.97 791|791.28 792|792.95 793|793.80 794|794.6 795|795.88 796|796.44 797|797.94 798|798.16 799|799.52 800|800.59 801|801.92 802|802.95 803|803.97 804|804.0 805|805.59 806|806.53 807|807.49 808|808.28 809|809.93 810|810.88 811|811.43 812|812.4 813|813.59 814|814.12 815|815.60 816|816.30 817|817.28 818|818.30 819|819.30 820|820.89 821|821.17 822|822.78 823|823.48 824|824.79 825|825.96 826|826.44 827|827.36 828|828.56 829|829.93 830|830.95 831|831.67 832|832.94 833|833.66 834|834.10 835|835.52 836|836.54 837|837.97 838|838.29 839|839.65 840|840.63 841|841.36 842|842.55 843|843.36 844|844.87 845|845.77 846|846.65 847|847.53 848|848.15 849|849.94 850|850.32 851|851.24 852|852.12 853|853.25 854|854.3 855|855.85 856|856.84 857|857.89 858|858.72 859|859.64 860|860.7 861|861.73 862|862.44 863|863.31 864|864.46 865|865.2 866|866.2 867|867.28 868|868.81 869|869.23 870|870.79 871|871.14 872|872.45 873|873.69 874|874.32 875|875.86 876|876.22 877|877.69 878|878.22 879|879.56 880|880.90 881|881.76 882|882.21 883|883.63 884|884.38 885|885.79 886|886.14 887|887.49 888|888.34 889|889.10 890|890.47 891|891.82 892|892.48 893|893.74 894|894.39 895|895.1 896|896.87 897|897.55 898|898.32 899|899.62 900|900.48 901|901.43 902|902.43 903|903.17 904|904.58 905|905.98 906|906.41 907|907.7 908|908.41 909|909.19 910|910.63 911|911.67 912|912.14 913|913.45 914|914.69 915|915.5 916|916.27 917|917.85 918|918.68 919|919.28 920|920.28 921|921.24 922|922.89 923|923.43 924|924.79 925|925.32 926|926.42 927|927.65 928|928.97 929|929.39 930|930.25 931|931.65 932|932.43 933|933.59 934|934.81 935|935.86 936|936.54 937|937.55 938|938.33 939|939.42 940|940.40 941|941.55 942|942.28 943|943.93 944|944.72 945|945.50 946|946.63 947|947.10 948|948.3 949|949.36 950|950.16 951|951.1 952|952.9 953|953.90 954|954.37 955|955.67 956|956.32 957|957.97 958|958.1 959|959.65 960|960.10 961|961.84 962|962.11 963|963.55 964|964.83 965|965.51 966|966.75 967|967.25 968|968.91 969|969.26 970|970.25 971|971.89 972|972.55 973|973.85 974|974.49 975|975.24 976|976.99 977|977.71 978|978.45 979|979.38 980|980.54 981|981.50 982|982.47 983|983.23 984|984.92 985|985.88 986|986.14 987|987.86 988|988.97 989|989.62 990|990.81 991|991.15 992|992.25 993|993.76 994|994.19 995|995.13 996|996.61 997|997.4 998|998.67 999|999.14 1000|1000.62 1001|1001.52 1002|1002.2 1003|1003.62 1004|1004.82 1005|1005.92 1006|1006.28 1007|1007.33 1008|1008.91 1009|1009.86 1010|1010.85 1011|1011.76 1012|1012.54 1013|1013.35 1014|1014.98 1015|1015.36 1016|1016.7 1017|1017.53 1018|1018.97 1019|1019.62 1020|1020.48 1021|1021.14 1022|1022.81 1023|1023.54 1024|1024.12 1025|1025.43 1026|1026.28 1027|1027.95 1028|1028.29 1029|1029.11 1030|1030.28 1031|1031.60 1032|1032.34 1033|1033.37 1034|1034.78 1035|1035.36 1036|1036.71 1037|1037.32 1038|1038.83 1039|1039.96 1040|1040.4 1041|1041.15 1042|1042.26 1043|1043.41 1044|1044.73 1045|1045.59 1046|1046.24 1047|1047.30 1048|1048.26 1049|1049.4 1050|1050.84 1051|1051.63 1052|1052.29 1053|1053.86 1054|1054.33 1055|1055.15 1056|1056.20 1057|1057.11 1058|1058.56 1059|1059.11 1060|1060.11 1061|1061.2 1062|1062.43 1063|1063.84 1064|1064.71 1065|1065.54 1066|1066.89 1067|1067.71 1068|1068.31 1069|1069.17 1070|1070.50 1071|1071.56 1072|1072.62 1073|1073.81 1074|1074.5 1075|1075.94 </str>
<float name="score">2.0</float></doc>
<doc>
<str name="id">6</str>
<int name="id_i">6</int>
<float name="default_amount_f">16554.0</float>
<str name="amounts_dpf">1|1.17 2|2.77 3|3.45 4|4.19 5|5.98 6|6.87 7|7.36 8|8.23 9|9.49 10|10.30 11|11.72 12|12.98 13|13.3 14|14.93 15|15.52 16|16.84 17|17.77 18|18.36 19|19.39 20|20.71 21|21.6 22|22.52 23|23.47 24|24.56 25|25.87 26|26.48 27|27.7 28|28.83 29|29.67 30|30.49 31|31.0 32|32.77 33|33.8 34|34.60 35|35.34 36|36.21 37|37.87 38|38.29 39|39.86 40|40.81 41|41.4 42|42.60 43|43.15 44|44.4 45|45.6 46|46.1 47|47.40 48|48.98 49|49.48 50|50.1 51|51.42 52|52.51 53|53.21 54|54.85 55|55.18 56|56.48 57|57.60 58|58.92 59|59.82 60|60.39 61|61.63 62|62.97 63|63.82 64|64.59 65|65.53 66|66.62 67|67.27 68|68.28 69|69.15 70|70.15 71|71.73 72|72.92 73|73.31 74|74.38 75|75.53 76|76.19 77|77.68 78|78.45 79|79.40 80|80.72 81|81.3 82|82.81 83|83.86 84|84.70 85|85.85 86|86.64 87|87.17 88|88.94 89|89.19 90|90.86 91|91.94 92|92.71 93|93.25 94|94.33 95|95.18 96|96.94 97|97.47 98|98.37 99|99.7 100|100.38 101|101.44 102|102.45 103|103.18 104|104.48 105|105.3 106|106.49 107|107.82 108|108.93 109|109.4 110|110.4 111|111.48 112|112.6 113|113.10 114|114.57 115|115.69 116|116.18 117|117.26 118|118.63 119|119.15 120|120.11 121|121.31 122|122.74 123|123.77 124|124.11 125|125.66 126|126.84 127|127.32 128|128.50 129|129.89 130|130.34 131|131.49 132|132.54 133|133.50 134|134.31 135|135.35 136|136.63 137|137.17 138|138.72 139|139.91 140|140.70 141|141.13 142|142.37 143|143.60 144|144.71 145|145.81 146|146.29 147|147.67 148|148.18 149|149.61 150|150.74 151|151.19 152|152.94 153|153.77 154|154.24 155|155.52 156|156.67 157|157.47 158|158.30 159|159.21 160|160.43 161|161.76 162|162.61 163|163.66 164|164.96 165|165.24 166|166.9 167|167.93 168|168.37 169|169.83 170|170.32 171|171.25 172|172.58 173|173.52 174|174.18 175|175.96 176|176.17 177|177.39 178|178.56 179|179.41 180|180.43 181|181.86 182|182.97 183|183.20 184|184.73 185|185.40 186|186.39 187|187.4 188|188.51 189|189.76 190|190.75 191|191.57 192|192.92 193|193.74 194|194.92 195|195.30 196|196.2 197|197.65 198|198.87 199|199.6 200|200.99 201|201.99 202|202.64 203|203.84 204|204.70 205|205.39 206|206.45 207|207.44 208|208.81 209|209.31 210|210.61 211|211.34 212|212.56 213|213.66 214|214.78 215|215.96 216|216.50 217|217.85 218|218.42 219|219.22 220|220.2 221|221.81 222|222.1 223|223.68 224|224.12 225|225.62 226|226.71 227|227.90 228|228.57 229|229.12 230|230.50 231|231.70 232|232.6 233|233.97 234|234.90 235|235.4 236|236.82 237|237.14 238|238.54 239|239.70 240|240.77 241|241.0 242|242.36 243|243.60 244|244.17 245|245.69 246|246.69 247|247.79 248|248.40 249|249.0 250|250.91 251|251.65 252|252.37 253|253.35 254|254.22 255|255.54 256|256.72 257|257.16 258|258.71 259|259.61 260|260.20 261|261.42 262|262.31 263|263.95 264|264.84 265|265.72 266|266.20 267|267.94 268|268.67 269|269.30 270|270.4 271|271.32 272|272.39 273|273.35 274|274.84 275|275.90 276|276.18 277|277.94 278|278.58 279|279.76 280|280.41 281|281.63 282|282.81 283|283.51 284|284.54 285|285.70 286|286.85 287|287.37 288|288.93 289|289.1 290|290.56 291|291.82 292|292.49 293|293.41 294|294.58 295|295.41 296|296.22 297|297.45 298|298.45 299|299.48 300|300.87 301|301.48 302|302.22 303|303.65 304|304.2 305|305.55 306|306.54 307|307.55 308|308.97 309|309.83 310|310.58 311|311.63 312|312.4 313|313.66 314|314.4 315|315.65 316|316.72 317|317.61 318|318.32 319|319.55 320|320.1 321|321.12 322|322.9 323|323.42 324|324.36 325|325.69 326|326.78 327|327.50 328|328.19 329|329.26 330|330.51 331|331.74 332|332.82 333|333.75 334|334.6 335|335.2 336|336.69 337|337.45 338|338.71 339|339.29 340|340.71 341|341.62 342|342.39 343|343.25 344|344.5 345|345.6 346|346.22 347|347.85 348|348.30 349|349.90 350|350.52 351|351.41 352|352.92 353|353.91 354|354.6 355|355.76 356|356.49 357|357.77 358|358.27 359|359.66 360|360.21 361|361.98 362|362.13 363|363.56 364|364.46 365|365.52 366|366.12 367|367.15 368|368.60 369|369.0 370|370.62 371|371.14 372|372.5 373|373.11 374|374.67 375|375.57 376|376.2 377|377.86 378|378.99 379|379.73 380|380.72 381|381.0 382|382.95 383|383.81 384|384.29 385|385.48 386|386.57 387|387.18 388|388.65 389|389.41 390|390.64 391|391.47 392|392.57 393|393.74 394|394.23 395|395.49 396|396.40 397|397.98 398|398.82 399|399.50 400|400.83 401|401.50 402|402.51 403|403.81 404|404.42 405|405.92 406|406.27 407|407.12 408|408.93 409|409.4 410|410.28 411|411.66 412|412.70 413|413.95 414|414.98 415|415.43 416|416.51 417|417.64 418|418.13 419|419.51 420|420.16 421|421.42 422|422.42 423|423.17 424|424.95 425|425.52 426|426.39 427|427.29 428|428.94 429|429.8 430|430.88 431|431.37 432|432.72 433|433.46 434|434.90 435|435.10 436|436.27 437|437.8 438|438.64 439|439.89 440|440.3 441|441.13 442|442.36 443|443.90 444|444.71 445|445.13 446|446.94 447|447.17 448|448.24 449|449.9 450|450.86 451|451.18 452|452.37 453|453.33 454|454.48 455|455.29 456|456.47 457|457.37 458|458.86 459|459.30 460|460.98 461|461.12 462|462.60 463|463.91 464|464.10 465|465.89 466|466.16 467|467.14 468|468.78 469|469.2 470|470.48 471|471.97 472|472.75 473|473.46 474|474.44 475|475.6 476|476.96 477|477.20 478|478.24 479|479.11 480|480.45 481|481.25 482|482.70 483|483.31 484|484.59 485|485.56 486|486.40 487|487.61 488|488.41 489|489.92 490|490.22 491|491.55 492|492.19 493|493.7 494|494.49 495|495.4 496|496.97 497|497.42 498|498.95 499|499.89 500|500.2 501|501.75 502|502.14 503|503.73 504|504.55 505|505.63 506|506.90 507|507.17 508|508.13 509|509.47 510|510.51 511|511.36 512|512.33 513|513.71 514|514.26 515|515.50 516|516.62 517|517.69 518|518.42 519|519.52 520|520.98 521|521.29 522|522.49 523|523.98 524|524.48 525|525.4 526|526.94 527|527.44 528|528.90 529|529.2 530|530.38 531|531.4 532|532.37 533|533.85 534|534.18 535|535.7 536|536.17 537|537.32 538|538.71 539|539.39 540|540.91 541|541.5 542|542.68 543|543.1 544|544.34 545|545.54 546|546.57 547|547.10 548|548.80 549|549.24 550|550.42 551|551.52 552|552.3 553|553.32 554|554.21 555|555.2 556|556.29 557|557.12 558|558.8 559|559.92 560|560.24 561|561.24 562|562.18 563|563.58 564|564.19 565|565.13 566|566.36 567|567.68 568|568.37 569|569.12 570|570.61 571|571.35 572|572.96 573|573.57 574|574.35 575|575.44 576|576.80 577|577.14 578|578.1 579|579.63 580|580.78 581|581.58 582|582.52 583|583.27 584|584.41 585|585.14 586|586.87 587|587.16 588|588.10 589|589.50 590|590.61 591|591.96 592|592.46 593|593.53 594|594.51 595|595.60 596|596.51 597|597.37 598|598.33 599|599.25 600|600.51 601|601.52 602|602.61 603|603.50 604|604.97 605|605.97 606|606.87 607|607.48 608|608.60 609|609.0 610|610.23 611|611.75 612|612.71 613|613.84 614|614.21 615|615.20 616|616.44 617|617.94 618|618.93 619|619.38 620|620.41 621|621.1 622|622.92 623|623.50 624|624.29 625|625.82 626|626.53 627|627.11 628|628.53 629|629.14 630|630.97 631|631.93 632|632.67 633|633.78 634|634.69 635|635.72 636|636.9 637|637.89 638|638.75 639|639.58 640|640.28 641|641.39 642|642.26 643|643.82 644|644.39 645|645.47 646|646.35 647|647.20 648|648.38 649|649.23 650|650.81 651|651.57 652|652.89 653|653.22 654|654.3 655|655.84 656|656.53 657|657.47 658|658.51 659|659.62 660|660.34 661|661.33 662|662.61 663|663.16 664|664.88 665|665.34 666|666.38 667|667.2 668|668.88 669|669.68 670|670.81 671|671.64 672|672.22 673|673.97 674|674.19 675|675.82 676|676.7 677|677.29 678|678.58 679|679.56 680|680.53 681|681.11 682|682.46 683|683.73 684|684.29 685|685.22 686|686.46 687|687.6 688|688.23 689|689.67 690|690.86 691|691.50 692|692.86 693|693.70 694|694.18 695|695.98 696|696.11 697|697.10 698|698.55 699|699.31 700|700.69 701|701.21 702|702.21 703|703.49 704|704.78 705|705.6 706|706.31 707|707.55 708|708.20 709|709.75 710|710.2 711|711.10 712|712.97 713|713.82 714|714.59 715|715.96 716|716.26 717|717.11 718|718.48 719|719.62 720|720.84 721|721.57 722|722.33 723|723.28 724|724.97 725|725.40 726|726.26 727|727.48 728|728.48 729|729.72 730|730.35 731|731.6 732|732.72 733|733.96 734|734.76 735|735.37 736|736.75 737|737.12 738|738.15 739|739.83 740|740.65 741|741.49 742|742.15 743|743.24 744|744.52 745|745.21 746|746.51 747|747.48 748|748.36 749|749.46 750|750.4 751|751.78 752|752.68 753|753.84 754|754.28 755|755.97 756|756.99 757|757.0 758|758.3 759|759.34 760|760.89 761|761.40 762|762.82 763|763.66 764|764.56 765|765.56 766|766.91 767|767.69 768|768.22 769|769.7 770|770.29 771|771.80 772|772.89 773|773.20 774|774.81 775|775.52 776|776.80 777|777.79 778|778.76 779|779.63 780|780.71 781|781.44 782|782.34 783|783.97 784|784.4 785|785.98 786|786.48 787|787.46 788|788.95 789|789.87 790|790.27 791|791.46 792|792.84 793|793.5 794|794.25 795|795.21 796|796.47 797|797.93 798|798.77 799|799.47 800|800.17 801|801.84 802|802.84 803|803.7 804|804.32 805|805.82 806|806.52 807|807.83 808|808.48 809|809.36 810|810.28 811|811.38 812|812.97 813|813.46 814|814.42 815|815.57 816|816.15 817|817.42 818|818.65 819|819.24 820|820.60 821|821.0 822|822.71 823|823.39 824|824.80 825|825.0 826|826.6 827|827.21 828|828.2 829|829.43 830|830.5 831|831.73 832|832.37 833|833.6 834|834.70 835|835.87 836|836.44 837|837.44 838|838.36 839|839.97 840|840.92 841|841.67 842|842.9 843|843.42 844|844.20 845|845.48 846|846.39 847|847.1 848|848.38 849|849.24 850|850.24 851|851.55 852|852.52 853|853.75 854|854.41 855|855.18 856|856.32 857|857.11 858|858.70 859|859.22 860|860.3 861|861.79 862|862.86 863|863.20 864|864.69 865|865.15 866|866.10 867|867.81 868|868.35 869|869.83 870|870.40 871|871.95 872|872.16 873|873.79 874|874.16 875|875.56 876|876.43 877|877.61 878|878.64 879|879.45 880|880.27 881|881.50 882|882.69 883|883.58 884|884.44 885|885.22 886|886.74 887|887.74 888|888.65 889|889.43 890|890.19 891|891.60 892|892.40 893|893.81 894|894.21 895|895.88 896|896.3 897|897.80 898|898.46 899|899.46 900|900.67 901|901.80 902|902.59 903|903.23 904|904.62 905|905.67 906|906.79 907|907.61 908|908.36 909|909.84 910|910.43 911|911.88 912|912.33 913|913.85 914|914.12 915|915.2 916|916.11 917|917.12 918|918.65 919|919.71 920|920.35 921|921.86 922|922.48 923|923.2 924|924.20 925|925.54 926|926.68 927|927.40 928|928.29 929|929.11 930|930.41 931|931.61 932|932.11 933|933.60 934|934.84 935|935.89 936|936.95 937|937.64 938|938.49 939|939.2 940|940.97 941|941.72 942|942.76 943|943.76 944|944.33 945|945.68 946|946.67 947|947.20 948|948.25 949|949.89 950|950.96 951|951.58 952|952.75 953|953.75 954|954.85 955|955.34 956|956.22 957|957.52 958|958.11 959|959.22 960|960.94 961|961.14 962|962.13 963|963.27 964|964.42 965|965.58 966|966.57 967|967.38 968|968.55 969|969.35 970|970.91 971|971.16 972|972.23 973|973.54 974|974.72 975|975.86 976|976.98 977|977.63 978|978.60 979|979.97 980|980.58 981|981.44 982|982.64 983|983.74 984|984.31 985|985.73 986|986.38 987|987.94 988|988.70 989|989.39 990|990.37 991|991.63 992|992.50 993|993.14 994|994.95 995|995.0 996|996.57 997|997.69 998|998.90 999|999.91 1000|1000.93 1001|1001.78 1002|1002.46 1003|1003.79 1004|1004.76 1005|1005.82 1006|1006.0 1007|1007.41 1008|1008.91 1009|1009.70 1010|1010.58 1011|1011.13 1012|1012.15 1013|1013.24 1014|1014.61 1015|1015.14 1016|1016.51 1017|1017.74 1018|1018.73 1019|1019.28 1020|1020.62 1021|1021.19 1022|1022.26 1023|1023.53 1024|1024.80 1025|1025.59 1026|1026.0 1027|1027.25 1028|1028.42 1029|1029.85 1030|1030.59 1031|1031.37 1032|1032.94 1033|1033.62 1034|1034.8 1035|1035.35 1036|1036.26 1037|1037.67 1038|1038.0 1039|1039.20 1040|1040.6 1041|1041.94 1042|1042.28 1043|1043.77 1044|1044.26 1045|1045.56 1046|1046.2 1047|1047.5 1048|1048.28 1049|1049.81 1050|1050.49 1051|1051.57 1052|1052.6 1053|1053.12 1054|1054.69 1055|1055.16 1056|1056.69 1057|1057.38 1058|1058.25 1059|1059.31 1060|1060.38 1061|1061.86 1062|1062.59 1063|1063.4 1064|1064.51 1065|1065.43 1066|1066.4 1067|1067.88 1068|1068.62 1069|1069.25 1070|1070.45 1071|1071.59 1072|1072.47 1073|1073.51 1074|1074.24 1075|1075.81 </str>
<float name="score">2.0</float></doc>
<doc>
<str name="id">7</str>
<int name="id_i">7</int>
<float name="default_amount_f">19049.0</float>
<str name="amounts_dpf">1|1.65 2|2.61 3|3.81 4|4.49 5|5.8 6|6.49 7|7.48 8|8.44 9|9.48 10|10.98 11|11.82 12|12.83 13|13.3 14|14.57 15|15.33 16|16.78 17|17.67 18|18.14 19|19.19 20|20.39 21|21.76 22|22.29 23|23.62 24|24.86 25|25.83 26|26.45 27|27.12 28|28.73 29|29.15 30|30.68 31|31.81 32|32.19 33|33.76 34|34.74 35|35.90 36|36.78 37|37.3 38|38.82 39|39.40 40|40.29 41|41.54 42|42.7 43|43.12 44|44.29 45|45.5 46|46.70 47|47.2 48|48.22 49|49.88 50|50.11 51|51.90 52|52.87 53|53.69 54|54.27 55|55.52 56|56.42 57|57.44 58|58.70 59|59.81 60|60.95 61|61.50 62|62.66 63|63.40 64|64.47 65|65.83 66|66.59 67|67.45 68|68.7 69|69.48 70|70.63 71|71.82 72|72.18 73|73.73 74|74.58 75|75.91 76|76.48 77|77.41 78|78.63 79|79.95 80|80.50 81|81.33 82|82.86 83|83.76 84|84.39 85|85.55 86|86.80 87|87.91 88|88.41 89|89.10 90|90.49 91|91.6 92|92.38 93|93.97 94|94.82 95|95.88 96|96.64 97|97.50 98|98.24 99|99.23 100|100.85 101|101.91 102|102.54 103|103.99 104|104.15 105|105.87 106|106.33 107|107.90 108|108.63 109|109.16 110|110.75 111|111.43 112|112.47 113|113.99 114|114.76 115|115.42 116|116.72 117|117.71 118|118.32 119|119.41 120|120.40 121|121.24 122|122.60 123|123.48 124|124.68 125|125.89 126|126.31 127|127.10 128|128.0 129|129.44 130|130.43 131|131.70 132|132.39 133|133.48 134|134.41 135|135.64 136|136.87 137|137.23 138|138.9 139|139.58 140|140.27 141|141.60 142|142.9 143|143.18 144|144.24 145|145.66 146|146.56 147|147.66 148|148.33 149|149.79 150|150.10 151|151.92 152|152.82 153|153.95 154|154.95 155|155.67 156|156.57 157|157.90 158|158.74 159|159.17 160|160.84 161|161.26 162|162.51 163|163.7 164|164.23 165|165.86 166|166.29 167|167.89 168|168.85 169|169.62 170|170.86 171|171.34 172|172.78 173|173.19 174|174.72 175|175.24 176|176.97 177|177.71 178|178.4 179|179.95 180|180.89 181|181.48 182|182.71 183|183.80 184|184.52 185|185.1 186|186.19 187|187.24 188|188.29 189|189.52 190|190.26 191|191.61 192|192.89 193|193.54 194|194.4 195|195.74 196|196.2 197|197.94 198|198.87 199|199.93 200|200.44 201|201.79 202|202.8 203|203.11 204|204.41 205|205.82 206|206.53 207|207.31 208|208.67 209|209.40 210|210.73 211|211.71 212|212.70 213|213.83 214|214.95 215|215.80 216|216.67 217|217.11 218|218.11 219|219.56 220|220.68 221|221.29 222|222.71 223|223.55 224|224.78 225|225.43 226|226.33 227|227.46 228|228.25 229|229.60 230|230.84 231|231.97 232|232.69 233|233.42 234|234.66 235|235.72 236|236.76 237|237.13 238|238.63 239|239.98 240|240.56 241|241.34 242|242.11 243|243.98 244|244.75 245|245.55 246|246.77 247|247.16 248|248.70 249|249.44 250|250.42 251|251.1 252|252.87 253|253.65 254|254.26 255|255.21 256|256.84 257|257.68 258|258.85 259|259.98 260|260.61 261|261.41 262|262.38 263|263.61 264|264.60 265|265.95 266|266.46 267|267.73 268|268.50 269|269.44 270|270.51 271|271.51 272|272.38 273|273.6 274|274.24 275|275.25 276|276.17 277|277.67 278|278.17 279|279.88 280|280.14 281|281.30 282|282.5 283|283.5 284|284.35 285|285.1 286|286.90 287|287.28 288|288.80 289|289.45 290|290.59 291|291.75 292|292.66 293|293.22 294|294.13 295|295.31 296|296.83 297|297.75 298|298.70 299|299.56 300|300.50 301|301.62 302|302.99 303|303.25 304|304.82 305|305.25 306|306.48 307|307.48 308|308.3 309|309.48 310|310.39 311|311.21 312|312.19 313|313.1 314|314.79 315|315.48 316|316.11 317|317.20 318|318.63 319|319.48 320|320.78 321|321.6 322|322.38 323|323.79 324|324.89 325|325.20 326|326.22 327|327.66 328|328.14 329|329.69 330|330.21 331|331.47 332|332.95 333|333.70 334|334.64 335|335.55 336|336.53 337|337.59 338|338.43 339|339.1 340|340.44 341|341.11 342|342.88 343|343.35 344|344.6 345|345.0 346|346.87 347|347.92 348|348.46 349|349.97 350|350.48 351|351.22 352|352.65 353|353.96 354|354.20 355|355.72 356|356.79 357|357.52 358|358.39 359|359.44 360|360.12 361|361.43 362|362.98 363|363.50 364|364.21 365|365.70 366|366.49 367|367.17 368|368.18 369|369.51 370|370.34 371|371.45 372|372.5 373|373.44 374|374.21 375|375.59 376|376.14 377|377.82 378|378.82 379|379.35 380|380.79 381|381.27 382|382.75 383|383.54 384|384.44 385|385.31 386|386.65 387|387.46 388|388.93 389|389.18 390|390.97 391|391.24 392|392.47 393|393.94 394|394.22 395|395.36 396|396.30 397|397.22 398|398.31 399|399.20 400|400.71 401|401.54 402|402.15 403|403.82 404|404.79 405|405.93 406|406.40 407|407.47 408|408.39 409|409.99 410|410.17 411|411.69 412|412.73 413|413.65 414|414.93 415|415.24 416|416.82 417|417.92 418|418.55 419|419.83 420|420.80 421|421.54 422|422.83 423|423.6 424|424.77 425|425.58 426|426.26 427|427.40 428|428.14 429|429.2 430|430.64 431|431.73 432|432.68 433|433.60 434|434.20 435|435.53 436|436.19 437|437.30 438|438.18 439|439.77 440|440.88 441|441.51 442|442.34 443|443.53 444|444.44 445|445.33 446|446.50 447|447.61 448|448.61 449|449.77 450|450.71 451|451.44 452|452.27 453|453.58 454|454.10 455|455.25 456|456.29 457|457.28 458|458.56 459|459.8 460|460.47 461|461.20 462|462.71 463|463.84 464|464.60 465|465.62 466|466.65 467|467.88 468|468.31 469|469.23 470|470.75 471|471.53 472|472.19 473|473.39 474|474.47 475|475.99 476|476.61 477|477.35 478|478.76 479|479.32 480|480.81 481|481.30 482|482.47 483|483.65 484|484.93 485|485.85 486|486.89 487|487.92 488|488.77 489|489.57 490|490.43 491|491.81 492|492.89 493|493.84 494|494.92 495|495.1 496|496.31 497|497.85 498|498.14 499|499.63 500|500.2 501|501.7 502|502.31 503|503.15 504|504.91 505|505.23 506|506.40 507|507.44 508|508.72 509|509.37 510|510.41 511|511.71 512|512.43 513|513.19 514|514.45 515|515.81 516|516.52 517|517.28 518|518.36 519|519.3 520|520.70 521|521.54 522|522.24 523|523.78 524|524.53 525|525.36 526|526.61 527|527.44 528|528.23 529|529.30 530|530.68 531|531.8 532|532.45 533|533.93 534|534.99 535|535.58 536|536.98 537|537.54 538|538.11 539|539.99 540|540.81 541|541.46 542|542.33 543|543.39 544|544.17 545|545.8 546|546.58 547|547.90 548|548.14 549|549.63 550|550.87 551|551.6 552|552.57 553|553.67 554|554.59 555|555.0 556|556.80 557|557.32 558|558.21 559|559.14 560|560.36 561|561.12 562|562.3 563|563.64 564|564.85 565|565.91 566|566.92 567|567.94 568|568.89 569|569.74 570|570.40 571|571.14 572|572.48 573|573.71 574|574.67 575|575.75 576|576.61 577|577.91 578|578.23 579|579.59 580|580.90 581|581.15 582|582.40 583|583.35 584|584.76 585|585.81 586|586.38 587|587.58 588|588.14 589|589.96 590|590.98 591|591.60 592|592.50 593|593.45 594|594.15 595|595.42 596|596.74 597|597.36 598|598.23 599|599.22 600|600.82 601|601.45 602|602.11 603|603.79 604|604.55 605|605.39 606|606.25 607|607.54 608|608.81 609|609.15 610|610.30 611|611.28 612|612.95 613|613.29 614|614.27 615|615.45 616|616.48 617|617.1 618|618.19 619|619.29 620|620.53 621|621.91 622|622.62 623|623.35 624|624.97 625|625.56 626|626.7 627|627.42 628|628.88 629|629.63 630|630.2 631|631.23 632|632.20 633|633.40 634|634.49 635|635.42 636|636.74 637|637.39 638|638.24 639|639.31 640|640.55 641|641.32 642|642.67 643|643.0 644|644.6 645|645.67 646|646.34 647|647.46 648|648.89 649|649.7 650|650.33 651|651.99 652|652.14 653|653.42 654|654.12 655|655.44 656|656.4 657|657.55 658|658.12 659|659.67 660|660.70 661|661.99 662|662.25 663|663.1 664|664.58 665|665.48 666|666.65 667|667.0 668|668.96 669|669.67 670|670.70 671|671.68 672|672.50 673|673.81 674|674.25 675|675.6 676|676.86 677|677.15 678|678.97 679|679.54 680|680.3 681|681.57 682|682.11 683|683.93 684|684.76 685|685.22 686|686.71 687|687.46 688|688.23 689|689.79 690|690.95 691|691.12 692|692.87 693|693.70 694|694.65 695|695.65 696|696.39 697|697.45 698|698.53 699|699.51 700|700.9 701|701.69 702|702.73 703|703.49 704|704.28 705|705.17 706|706.36 707|707.80 708|708.51 709|709.99 710|710.16 711|711.48 712|712.6 713|713.23 714|714.26 715|715.58 716|716.24 717|717.95 718|718.34 719|719.14 720|720.30 721|721.35 722|722.79 723|723.44 724|724.22 725|725.7 726|726.66 727|727.11 728|728.59 729|729.47 730|730.35 731|731.42 732|732.88 733|733.45 734|734.24 735|735.36 736|736.23 737|737.72 738|738.9 739|739.77 740|740.66 741|741.48 742|742.9 743|743.50 744|744.84 745|745.30 746|746.61 747|747.84 748|748.42 749|749.91 750|750.44 751|751.0 752|752.18 753|753.15 754|754.99 755|755.69 756|756.81 757|757.17 758|758.47 759|759.36 760|760.11 761|761.78 762|762.15 763|763.97 764|764.25 765|765.17 766|766.94 767|767.63 768|768.91 769|769.19 770|770.31 771|771.79 772|772.81 773|773.36 774|774.91 775|775.1 776|776.10 777|777.24 778|778.81 779|779.28 780|780.20 781|781.86 782|782.87 783|783.57 784|784.81 785|785.89 786|786.38 787|787.25 788|788.23 789|789.14 790|790.46 791|791.19 792|792.63 793|793.55 794|794.88 795|795.69 796|796.19 797|797.0 798|798.38 799|799.23 800|800.52 801|801.80 802|802.33 803|803.29 804|804.42 805|805.1 806|806.44 807|807.76 808|808.91 809|809.22 810|810.38 811|811.71 812|812.0 813|813.36 814|814.11 815|815.58 816|816.18 817|817.10 818|818.77 819|819.51 820|820.36 821|821.79 822|822.63 823|823.66 824|824.12 825|825.53 826|826.39 827|827.32 828|828.96 829|829.7 830|830.17 831|831.36 832|832.72 833|833.23 834|834.88 835|835.59 836|836.36 837|837.40 838|838.71 839|839.26 840|840.69 841|841.50 842|842.65 843|843.8 844|844.22 845|845.59 846|846.70 847|847.17 848|848.12 849|849.44 850|850.88 851|851.73 852|852.55 853|853.98 854|854.10 855|855.82 856|856.10 857|857.65 858|858.35 859|859.5 860|860.66 861|861.67 862|862.53 863|863.72 864|864.78 865|865.43 866|866.25 867|867.81 868|868.19 869|869.53 870|870.59 871|871.8 872|872.58 873|873.15 874|874.53 875|875.35 876|876.38 877|877.90 878|878.7 879|879.38 880|880.28 881|881.23 882|882.14 883|883.85 884|884.21 885|885.67 886|886.27 887|887.22 888|888.16 889|889.16 890|890.15 891|891.95 892|892.63 893|893.74 894|894.72 895|895.95 896|896.20 897|897.71 898|898.61 899|899.61 900|900.58 901|901.89 902|902.76 903|903.6 904|904.65 905|905.72 906|906.86 907|907.3 908|908.72 909|909.50 910|910.18 911|911.41 912|912.56 913|913.7 914|914.52 915|915.85 916|916.46 917|917.15 918|918.21 919|919.10 920|920.45 921|921.25 922|922.17 923|923.33 924|924.61 925|925.98 926|926.91 927|927.45 928|928.82 929|929.74 930|930.44 931|931.65 932|932.23 933|933.22 934|934.5 935|935.46 936|936.81 937|937.49 938|938.45 939|939.90 940|940.15 941|941.24 942|942.82 943|943.61 944|944.90 945|945.71 946|946.30 947|947.98 948|948.1 949|949.86 950|950.38 951|951.50 952|952.99 953|953.14 954|954.17 955|955.97 956|956.90 957|957.70 958|958.3 959|959.19 960|960.68 961|961.26 962|962.4 963|963.3 964|964.15 965|965.59 966|966.75 967|967.66 968|968.5 969|969.57 970|970.42 971|971.5 972|972.92 973|973.1 974|974.53 975|975.30 976|976.65 977|977.3 978|978.78 979|979.61 980|980.25 981|981.82 982|982.18 983|983.71 984|984.17 985|985.99 986|986.90 987|987.94 988|988.40 989|989.82 990|990.50 991|991.77 992|992.99 993|993.5 994|994.87 995|995.10 996|996.82 997|997.39 998|998.74 999|999.88 1000|1000.93 1001|1001.12 1002|1002.31 1003|1003.57 1004|1004.65 1005|1005.77 1006|1006.18 1007|1007.13 1008|1008.37 1009|1009.56 1010|1010.43 1011|1011.79 1012|1012.86 1013|1013.35 1014|1014.75 1015|1015.63 1016|1016.80 1017|1017.48 1018|1018.97 1019|1019.71 1020|1020.31 1021|1021.56 1022|1022.88 1023|1023.82 1024|1024.30 1025|1025.20 1026|1026.1 1027|1027.23 1028|1028.80 1029|1029.54 1030|1030.27 1031|1031.13 1032|1032.24 1033|1033.1 1034|1034.51 1035|1035.73 1036|1036.17 1037|1037.90 1038|1038.62 1039|1039.34 1040|1040.23 1041|1041.32 1042|1042.51 1043|1043.0 1044|1044.7 1045|1045.6 1046|1046.20 1047|1047.41 1048|1048.14 1049|1049.0 1050|1050.82 1051|1051.20 1052|1052.66 1053|1053.71 1054|1054.48 1055|1055.73 1056|1056.36 1057|1057.98 1058|1058.88 1059|1059.46 1060|1060.29 1061|1061.7 1062|1062.4 1063|1063.3 1064|1064.91 1065|1065.77 1066|1066.80 1067|1067.13 1068|1068.45 1069|1069.83 1070|1070.77 1071|1071.89 1072|1072.66 1073|1073.9 1074|1074.83 1075|1075.6 </str>
<float name="score">2.0</float></doc>
<doc>
<str name="id">8</str>
<int name="id_i">8</int>
<float name="default_amount_f">13191.0</float>
<str name="amounts_dpf">1|1.20 2|2.15 3|3.83 4|4.55 5|5.38 6|6.89 7|7.31 8|8.58 9|9.45 10|10.17 11|11.20 12|12.93 13|13.58 14|14.66 15|15.63 16|16.56 17|17.46 18|18.12 19|19.85 20|20.36 21|21.84 22|22.52 23|23.89 24|24.50 25|25.32 26|26.65 27|27.46 28|28.19 29|29.44 30|30.55 31|31.54 32|32.60 33|33.39 34|34.27 35|35.13 36|36.45 37|37.93 38|38.63 39|39.98 40|40.65 41|41.57 42|42.68 43|43.73 44|44.39 45|45.67 46|46.17 47|47.33 48|48.11 49|49.40 50|50.51 51|51.12 52|52.38 53|53.89 54|54.4 55|55.1 56|56.68 57|57.85 58|58.15 59|59.58 60|60.67 61|61.85 62|62.93 63|63.76 64|64.52 65|65.89 66|66.77 67|67.90 68|68.92 69|69.68 70|70.64 71|71.51 72|72.9 73|73.81 74|74.39 75|75.40 76|76.55 77|77.81 78|78.8 79|79.13 80|80.68 81|81.11 82|82.78 83|83.21 84|84.51 85|85.57 86|86.18 87|87.7 88|88.48 89|89.91 90|90.72 91|91.32 92|92.2 93|93.76 94|94.82 95|95.29 96|96.65 97|97.2 98|98.54 99|99.65 100|100.25 101|101.84 102|102.25 103|103.84 104|104.48 105|105.32 106|106.43 107|107.96 108|108.40 109|109.15 110|110.84 111|111.74 112|112.35 113|113.76 114|114.85 115|115.20 116|116.85 117|117.66 118|118.28 119|119.20 120|120.49 121|121.73 122|122.30 123|123.58 124|124.53 125|125.21 126|126.33 127|127.18 128|128.90 129|129.70 130|130.51 131|131.50 132|132.87 133|133.43 134|134.87 135|135.56 136|136.60 137|137.51 138|138.99 139|139.87 140|140.55 141|141.74 142|142.14 143|143.98 144|144.96 145|145.96 146|146.48 147|147.16 148|148.73 149|149.61 150|150.20 151|151.12 152|152.31 153|153.19 154|154.4 155|155.50 156|156.96 157|157.55 158|158.98 159|159.30 160|160.26 161|161.14 162|162.67 163|163.27 164|164.4 165|165.85 166|166.77 167|167.79 168|168.68 169|169.2 170|170.51 171|171.78 172|172.61 173|173.35 174|174.68 175|175.2 176|176.1 177|177.63 178|178.78 179|179.37 180|180.30 181|181.64 182|182.16 183|183.35 184|184.5 185|185.92 186|186.84 187|187.14 188|188.87 189|189.47 190|190.95 191|191.28 192|192.65 193|193.79 194|194.63 195|195.86 196|196.11 197|197.56 198|198.66 199|199.94 200|200.95 201|201.30 202|202.8 203|203.78 204|204.52 205|205.43 206|206.47 207|207.11 208|208.29 209|209.65 210|210.85 211|211.61 212|212.38 213|213.31 214|214.95 215|215.78 216|216.10 217|217.34 218|218.38 219|219.71 220|220.75 221|221.13 222|222.75 223|223.56 224|224.47 225|225.94 226|226.93 227|227.2 228|228.67 229|229.58 230|230.66 231|231.35 232|232.88 233|233.92 234|234.41 235|235.58 236|236.73 237|237.37 238|238.36 239|239.45 240|240.63 241|241.9 242|242.85 243|243.25 244|244.86 245|245.86 246|246.31 247|247.21 248|248.81 249|249.22 250|250.54 251|251.54 252|252.39 253|253.30 254|254.30 255|255.34 256|256.15 257|257.91 258|258.52 259|259.32 260|260.80 261|261.87 262|262.93 263|263.61 264|264.73 265|265.47 266|266.0 267|267.56 268|268.76 269|269.23 270|270.74 271|271.43 272|272.20 273|273.88 274|274.39 275|275.15 276|276.75 277|277.69 278|278.41 279|279.87 280|280.50 281|281.86 282|282.49 283|283.43 284|284.91 285|285.26 286|286.16 287|287.72 288|288.55 289|289.55 290|290.37 291|291.19 292|292.98 293|293.42 294|294.60 295|295.49 296|296.78 297|297.30 298|298.79 299|299.51 300|300.83 301|301.53 302|302.77 303|303.21 304|304.61 305|305.18 306|306.27 307|307.67 308|308.45 309|309.40 310|310.16 311|311.39 312|312.6 313|313.81 314|314.95 315|315.9 316|316.23 317|317.15 318|318.96 319|319.90 320|320.2 321|321.52 322|322.52 323|323.71 324|324.10 325|325.88 326|326.60 327|327.73 328|328.69 329|329.46 330|330.40 331|331.71 332|332.72 333|333.94 334|334.18 335|335.62 336|336.81 337|337.47 338|338.92 339|339.36 340|340.91 341|341.29 342|342.31 343|343.39 344|344.56 345|345.58 346|346.99 347|347.65 348|348.20 349|349.4 350|350.55 351|351.14 352|352.80 353|353.81 354|354.77 355|355.11 356|356.8 357|357.83 358|358.73 359|359.61 360|360.53 361|361.51 362|362.92 363|363.41 364|364.66 365|365.61 366|366.48 367|367.93 368|368.3 369|369.58 370|370.86 371|371.70 372|372.73 373|373.30 374|374.5 375|375.55 376|376.89 377|377.17 378|378.72 379|379.30 380|380.10 381|381.81 382|382.77 383|383.36 384|384.52 385|385.66 386|386.59 387|387.23 388|388.68 389|389.33 390|390.11 391|391.82 392|392.8 393|393.26 394|394.1 395|395.54 396|396.40 397|397.37 398|398.72 399|399.28 400|400.20 401|401.90 402|402.31 403|403.33 404|404.39 405|405.49 406|406.65 407|407.69 408|408.23 409|409.69 410|410.26 411|411.49 412|412.15 413|413.53 414|414.35 415|415.31 416|416.13 417|417.81 418|418.53 419|419.81 420|420.90 421|421.94 422|422.52 423|423.17 424|424.46 425|425.3 426|426.38 427|427.92 428|428.27 429|429.66 430|430.73 431|431.81 432|432.87 433|433.31 434|434.58 435|435.2 436|436.51 437|437.59 438|438.78 439|439.56 440|440.69 441|441.80 442|442.98 443|443.27 444|444.51 445|445.80 446|446.2 447|447.39 448|448.5 449|449.24 450|450.62 451|451.29 452|452.54 453|453.94 454|454.38 455|455.6 456|456.33 457|457.48 458|458.23 459|459.82 460|460.18 461|461.75 462|462.7 463|463.22 464|464.27 465|465.7 466|466.69 467|467.89 468|468.21 469|469.8 470|470.78 471|471.86 472|472.6 473|473.66 474|474.75 475|475.43 476|476.13 477|477.69 478|478.46 479|479.80 480|480.67 481|481.65 482|482.88 483|483.21 484|484.85 485|485.95 486|486.77 487|487.58 488|488.48 489|489.99 490|490.17 491|491.76 492|492.25 493|493.69 494|494.22 495|495.76 496|496.4 497|497.56 498|498.92 499|499.88 500|500.60 501|501.81 502|502.63 503|503.68 504|504.21 505|505.24 506|506.66 507|507.60 508|508.8 509|509.31 510|510.80 511|511.30 512|512.1 513|513.76 514|514.2 515|515.91 516|516.98 517|517.15 518|518.84 519|519.33 520|520.10 521|521.14 522|522.47 523|523.44 524|524.45 525|525.65 526|526.90 527|527.65 528|528.78 529|529.29 530|530.46 531|531.68 532|532.49 533|533.23 534|534.87 535|535.91 536|536.86 537|537.66 538|538.36 539|539.86 540|540.19 541|541.33 542|542.35 543|543.33 544|544.45 545|545.34 546|546.9 547|547.63 548|548.91 549|549.58 550|550.49 551|551.30 552|552.16 553|553.28 554|554.5 555|555.33 556|556.58 557|557.84 558|558.22 559|559.93 560|560.85 561|561.24 562|562.77 563|563.58 564|564.94 565|565.96 566|566.83 567|567.18 568|568.77 569|569.30 570|570.20 571|571.92 572|572.33 573|573.26 574|574.0 575|575.38 576|576.59 577|577.2 578|578.42 579|579.63 580|580.68 581|581.86 582|582.10 583|583.14 584|584.1 585|585.20 586|586.17 587|587.70 588|588.87 589|589.4 590|590.56 591|591.59 592|592.72 593|593.6 594|594.0 595|595.68 596|596.94 597|597.8 598|598.5 599|599.4 600|600.52 601|601.87 602|602.75 603|603.37 604|604.37 605|605.32 606|606.30 607|607.28 608|608.44 609|609.27 610|610.2 611|611.57 612|612.15 613|613.30 614|614.11 615|615.83 616|616.50 617|617.72 618|618.42 619|619.64 620|620.4 621|621.56 622|622.7 623|623.41 624|624.42 625|625.13 626|626.65 627|627.35 628|628.4 629|629.36 630|630.47 631|631.18 632|632.23 633|633.74 634|634.9 635|635.96 636|636.25 637|637.84 638|638.56 639|639.40 640|640.36 641|641.77 642|642.42 643|643.4 644|644.43 645|645.22 646|646.98 647|647.74 648|648.11 649|649.29 650|650.67 651|651.76 652|652.43 653|653.27 654|654.3 655|655.81 656|656.54 657|657.63 658|658.40 659|659.76 660|660.67 661|661.13 662|662.38 663|663.80 664|664.80 665|665.91 666|666.15 667|667.45 668|668.7 669|669.47 670|670.59 671|671.19 672|672.14 673|673.14 674|674.99 675|675.3 676|676.9 677|677.5 678|678.61 679|679.26 680|680.52 681|681.79 682|682.11 683|683.32 684|684.42 685|685.75 686|686.60 687|687.16 688|688.16 689|689.39 690|690.75 691|691.98 692|692.46 693|693.55 694|694.59 695|695.26 696|696.32 697|697.95 698|698.67 699|699.59 700|700.46 701|701.17 702|702.54 703|703.23 704|704.51 705|705.49 706|706.39 707|707.28 708|708.14 709|709.50 710|710.67 711|711.49 712|712.32 713|713.32 714|714.82 715|715.49 716|716.38 717|717.60 718|718.28 719|719.23 720|720.50 721|721.78 722|722.4 723|723.52 724|724.36 725|725.7 726|726.96 727|727.0 728|728.6 729|729.25 730|730.41 731|731.18 732|732.94 733|733.32 734|734.13 735|735.90 736|736.55 737|737.34 738|738.92 739|739.69 740|740.73 741|741.45 742|742.28 743|743.14 744|744.71 745|745.84 746|746.39 747|747.16 748|748.79 749|749.69 750|750.29 751|751.80 752|752.94 753|753.62 754|754.14 755|755.10 756|756.94 757|757.52 758|758.52 759|759.98 760|760.68 761|761.89 762|762.4 763|763.3 764|764.3 765|765.13 766|766.70 767|767.1 768|768.94 769|769.79 770|770.5 771|771.69 772|772.19 773|773.75 774|774.29 775|775.78 776|776.83 777|777.87 778|778.26 779|779.77 780|780.86 781|781.11 782|782.96 783|783.40 784|784.10 785|785.1 786|786.31 787|787.72 788|788.54 789|789.99 790|790.49 791|791.50 792|792.9 793|793.35 794|794.12 795|795.31 796|796.19 797|797.12 798|798.21 799|799.52 800|800.88 801|801.55 802|802.39 803|803.31 804|804.58 805|805.39 806|806.73 807|807.52 808|808.3 809|809.24 810|810.52 811|811.92 812|812.2 813|813.4 814|814.16 815|815.8 816|816.30 817|817.12 818|818.67 819|819.29 820|820.41 821|821.7 822|822.10 823|823.68 824|824.80 825|825.42 826|826.2 827|827.68 828|828.40 829|829.12 830|830.47 831|831.54 832|832.30 833|833.54 834|834.90 835|835.18 836|836.59 837|837.31 838|838.22 839|839.9 840|840.96 841|841.42 842|842.51 843|843.47 844|844.5 845|845.13 846|846.35 847|847.42 848|848.19 849|849.40 850|850.25 851|851.50 852|852.92 853|853.22 854|854.31 855|855.47 856|856.25 857|857.38 858|858.45 859|859.37 860|860.99 861|861.54 862|862.94 863|863.48 864|864.50 865|865.50 866|866.53 867|867.19 868|868.25 869|869.4 870|870.55 871|871.53 872|872.52 873|873.56 874|874.63 875|875.48 876|876.24 877|877.87 878|878.76 879|879.35 880|880.96 881|881.93 882|882.71 883|883.1 884|884.22 885|885.51 886|886.32 887|887.3 888|888.35 889|889.13 890|890.19 891|891.16 892|892.77 893|893.2 894|894.4 895|895.45 896|896.63 897|897.15 898|898.61 899|899.40 900|900.52 901|901.72 902|902.93 903|903.78 904|904.50 905|905.79 906|906.42 907|907.26 908|908.60 909|909.30 910|910.71 911|911.19 912|912.82 913|913.18 914|914.34 915|915.62 916|916.81 917|917.77 918|918.20 919|919.98 920|920.65 921|921.89 922|922.86 923|923.5 924|924.3 925|925.14 926|926.12 927|927.21 928|928.3 929|929.9 930|930.45 931|931.15 932|932.8 933|933.78 934|934.15 935|935.5 936|936.37 937|937.63 938|938.45 939|939.91 940|940.21 941|941.62 942|942.64 943|943.21 944|944.62 945|945.40 946|946.21 947|947.51 948|948.76 949|949.73 950|950.88 951|951.97 952|952.23 953|953.42 954|954.10 955|955.7 956|956.99 957|957.6 958|958.54 959|959.23 960|960.88 961|961.53 962|962.51 963|963.31 964|964.54 965|965.71 966|966.76 967|967.46 968|968.92 969|969.56 970|970.61 971|971.9 972|972.87 973|973.34 974|974.48 975|975.38 976|976.0 977|977.48 978|978.98 979|979.21 980|980.11 981|981.35 982|982.74 983|983.47 984|984.67 985|985.5 986|986.43 987|987.37 988|988.73 989|989.14 990|990.84 991|991.56 992|992.90 993|993.7 994|994.37 995|995.85 996|996.18 997|997.79 998|998.61 999|999.93 1000|1000.48 1001|1001.92 1002|1002.32 1003|1003.99 1004|1004.66 1005|1005.47 1006|1006.29 1007|1007.21 1008|1008.24 1009|1009.8 1010|1010.77 1011|1011.54 1012|1012.80 1013|1013.48 1014|1014.97 1015|1015.68 1016|1016.36 1017|1017.30 1018|1018.84 1019|1019.20 1020|1020.24 1021|1021.38 1022|1022.82 1023|1023.5 1024|1024.19 1025|1025.88 1026|1026.60 1027|1027.9 1028|1028.72 1029|1029.97 1030|1030.60 1031|1031.10 1032|1032.34 1033|1033.36 1034|1034.69 1035|1035.10 1036|1036.76 1037|1037.39 1038|1038.0 1039|1039.62 1040|1040.13 1041|1041.66 1042|1042.59 1043|1043.24 1044|1044.11 1045|1045.54 1046|1046.67 1047|1047.0 1048|1048.5 1049|1049.95 1050|1050.35 1051|1051.3 1052|1052.55 1053|1053.88 1054|1054.34 1055|1055.5 1056|1056.14 1057|1057.51 1058|1058.32 1059|1059.60 1060|1060.15 1061|1061.59 1062|1062.73 1063|1063.43 1064|1064.12 1065|1065.30 1066|1066.11 1067|1067.97 1068|1068.20 1069|1069.57 1070|1070.87 1071|1071.2 1072|1072.60 1073|1073.49 1074|1074.1 1075|1075.73 </str>
<float name="score">2.0</float></doc>
<doc>
<str name="id">9</str>
<int name="id_i">9</int>
<float name="default_amount_f">13780.0</float>
<str name="amounts_dpf">1|1.93 2|2.99 3|3.89 4|4.40 5|5.44 6|6.78 7|7.24 8|8.70 9|9.85 10|10.79 11|11.87 12|12.42 13|13.77 14|14.98 15|15.78 16|16.58 17|17.45 18|18.82 19|19.94 20|20.62 21|21.65 22|22.11 23|23.44 24|24.26 25|25.11 26|26.11 27|27.34 28|28.67 29|29.71 30|30.14 31|31.72 32|32.98 33|33.41 34|34.67 35|35.23 36|36.35 37|37.16 38|38.29 39|39.43 40|40.89 41|41.40 42|42.54 43|43.13 44|44.10 45|45.38 46|46.51 47|47.0 48|48.4 49|49.11 50|50.91 51|51.40 52|52.72 53|53.35 54|54.55 55|55.1 56|56.51 57|57.18 58|58.75 59|59.56 60|60.28 61|61.88 62|62.32 63|63.66 64|64.41 65|65.55 66|66.40 67|67.79 68|68.98 69|69.9 70|70.34 71|71.41 72|72.0 73|73.56 74|74.23 75|75.64 76|76.53 77|77.37 78|78.79 79|79.92 80|80.95 81|81.13 82|82.15 83|83.34 84|84.88 85|85.62 86|86.19 87|87.58 88|88.11 89|89.34 90|90.22 91|91.39 92|92.96 93|93.80 94|94.68 95|95.57 96|96.21 97|97.88 98|98.90 99|99.10 100|100.9 101|101.73 102|102.52 103|103.42 104|104.84 105|105.30 106|106.95 107|107.43 108|108.71 109|109.37 110|110.59 111|111.74 112|112.81 113|113.10 114|114.6 115|115.48 116|116.23 117|117.94 118|118.13 119|119.9 120|120.99 121|121.29 122|122.92 123|123.60 124|124.13 125|125.5 126|126.68 127|127.65 128|128.9 129|129.32 130|130.32 131|131.7 132|132.25 133|133.39 134|134.52 135|135.70 136|136.57 137|137.51 138|138.99 139|139.30 140|140.83 141|141.98 142|142.33 143|143.90 144|144.79 145|145.80 146|146.58 147|147.20 148|148.22 149|149.78 150|150.30 151|151.98 152|152.29 153|153.8 154|154.42 155|155.13 156|156.66 157|157.90 158|158.12 159|159.69 160|160.65 161|161.62 162|162.82 163|163.4 164|164.9 165|165.81 166|166.75 167|167.16 168|168.23 169|169.43 170|170.48 171|171.53 172|172.36 173|173.20 174|174.31 175|175.90 176|176.9 177|177.66 178|178.29 179|179.77 180|180.28 181|181.66 182|182
[...truncated too long message...]
actory [collection1] webapp=null path=null params={}{commit=} 0 2
[junit4] 2> 2865867 INFO (TEST-TestFunctionQuery.testStrDistance-seed#[C51A3ABEB4C1B3BD]) [ ] o.a.s.c.S.Request [collection1] webapp=null path=null params={q={!func}strdist(x_s,+'foit',+edit)&fl=*,score&fq=id:1&wt=xml} hits=1 status=0 QTime=1
[junit4] 2> 2865867 INFO (TEST-TestFunctionQuery.testStrDistance-seed#[C51A3ABEB4C1B3BD]) [ ] o.a.s.c.S.Request [collection1] webapp=null path=null params={q={!func}strdist(x_s,+'foit',+jw)&fl=*,score&fq=id:1&wt=xml} hits=1 status=0 QTime=0
[junit4] 2> 2865869 INFO (TEST-TestFunctionQuery.testStrDistance-seed#[C51A3ABEB4C1B3BD]) [ ] o.a.s.c.S.Request [collection1] webapp=null path=null params={q={!func}strdist(x_s,+'foit',+ngram,+2)&fl=*,score&fq=id:1&wt=xml} hits=1 status=0 QTime=1
[junit4] 2> 2865870 INFO (TEST-TestFunctionQuery.testStrDistance-seed#[C51A3ABEB4C1B3BD]) [ ] o.a.s.c.S.Request [collection1] webapp=null path=null params={q=id:1&fl=good:strdist(x_s,+'toil',+edit)&fl=bad1:strdist(missing1_s,+missing2_s,+edit)&fl=bad2:strdist(missing1_s,+'something',+edit)&fl=bad3:strdist(missing1_s,+x_s,+edit)&wt=xml} hits=1 status=0 QTime=0
[junit4] 2> 2865870 INFO (TEST-TestFunctionQuery.testStrDistance-seed#[C51A3ABEB4C1B3BD]) [ ] o.a.s.c.S.Request [collection1] webapp=null path=null params={q={!func}strdist(missing1_s,+missing2_s,+edit)&fl=score&fq=id:1&wt=xml} hits=1 status=0 QTime=0
[junit4] 2> 2865874 INFO (TEST-TestFunctionQuery.testStrDistance-seed#[C51A3ABEB4C1B3BD]) [ ] o.a.s.c.S.Request [collection1] webapp=null path=null params={q={!func}strdist(missing1_s,+x_s,+edit)&fl=score&fq=id:1&wt=xml} hits=1 status=0 QTime=0
[junit4] 2> 2865874 INFO (TEST-TestFunctionQuery.testStrDistance-seed#[C51A3ABEB4C1B3BD]) [ ] o.a.s.c.S.Request [collection1] webapp=null path=null params={q={!func}strdist(missing1_s,+'const',+edit)&fl=score&fq=id:1&wt=xml} hits=1 status=0 QTime=0
[junit4] 2> 2865875 INFO (TEST-TestFunctionQuery.testStrDistance-seed#[C51A3ABEB4C1B3BD]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testStrDistance
[junit4] 2> 2865875 INFO (SUITE-TestFunctionQuery-seed#[C51A3ABEB4C1B3BD]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> 2865875 INFO (SUITE-TestFunctionQuery-seed#[C51A3ABEB4C1B3BD]-worker) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1288446886
[junit4] 2> 2865875 INFO (SUITE-TestFunctionQuery-seed#[C51A3ABEB4C1B3BD]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 2865875 INFO (SUITE-TestFunctionQuery-seed#[C51A3ABEB4C1B3BD]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@597175b2: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@2679fcc1
[junit4] 2> 2865878 INFO (SUITE-TestFunctionQuery-seed#[C51A3ABEB4C1B3BD]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 2865878 INFO (SUITE-TestFunctionQuery-seed#[C51A3ABEB4C1B3BD]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@19ba1a44: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@5af5925e
[junit4] 2> 2865880 INFO (SUITE-TestFunctionQuery-seed#[C51A3ABEB4C1B3BD]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 2865880 INFO (SUITE-TestFunctionQuery-seed#[C51A3ABEB4C1B3BD]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4e9981e1: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@4607fa5
[junit4] 2> 2865900 INFO (coreCloseExecutor-7733-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@650e90e9
[junit4] 2> 2865900 INFO (coreCloseExecutor-7733-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=1695453417
[junit4] 2> 2865900 INFO (coreCloseExecutor-7733-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2480d1fb: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@9d0de2c
[junit4] 2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.search.function.TestFunctionQuery_C51A3ABEB4C1B3BD-001
[junit4] 2> NOTE: test params are: codec=Lucene70, sim=RandomSimilarity(queryNorm=false): {}, locale=ro-RO, timezone=America/Paramaribo
[junit4] 2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=3,threads=1,free=251862200,total=464519168
[junit4] 2> NOTE: All tests run in this JVM: [ConfigSetsAPITest, TestClassicSimilarityFactory, DeleteShardTest, TestSolr4Spatial, TestExportWriter, RegexBoostProcessorTest, TestNumericRangeQuery64, TestMultiValuedNumericRangeQuery, TestReplicationHandlerBackup, TestBinaryField, TestSolrDeletionPolicy2, TestHashQParserPlugin, TestFiltering, SolrInfoBeanTest, FastVectorHighlighterTest, InfoHandlerTest, TestJmxIntegration, OverseerModifyCollectionTest, IndexBasedSpellCheckerTest, TestIBSimilarityFactory, TestHalfAndHalfDocValues, UUIDFieldTest, TestDeleteCollectionOnDownNodes, TestSQLHandlerNonCloud, BlockJoinFacetRandomTest, TestUpdate, TestLeaderElectionZkExpiry, TestRandomDVFaceting, TestStressRecovery, TestObjectReleaseTracker, MoveReplicaHDFSFailoverTest, CursorPagingTest, ShardRoutingTest, TestRawTransformer, ConnectionReuseTest, ResponseLogComponentTest, CollectionReloadTest, TestPseudoReturnFields, XsltUpdateRequestHandlerTest, CdcrUpdateLogTest, V2ApiIntegrationTest, TestDocBasedVersionConstraints, TestFieldCache, UUIDUpdateProcessorFallbackTest, TestPayloadCheckQParserPlugin, TestSolrCloudWithSecureImpersonation, HighlighterMaxOffsetTest, TestSmileRequest, TestConfigSetsAPIZkFailure, SharedFSAutoReplicaFailoverTest, SpellCheckComponentTest, CdcrReplicationHandlerTest, FieldAnalysisRequestHandlerTest, TestManagedSchemaAPI, TestSegmentSorting, TestReversedWildcardFilterFactory, ClassificationUpdateProcessorTest, MultiTermTest, DistribJoinFromCollectionTest, SolrCLIZkUtilsTest, DeleteInactiveReplicaTest, TestTrieFacet, SegmentsInfoRequestHandlerTest, TestDistributedStatsComponentCardinality, TestCodecSupport, TestSolrCLIRunExample, AsyncCallRequestStatusResponseTest, TestFieldCacheVsDocValues, GraphQueryTest, TestZkAclsWithHadoopAuth, CSVRequestHandlerTest, TestAtomicUpdateErrorCases, HdfsUnloadDistributedZkTest, JSONWriterTest, BigEndianAscendingWordDeserializerTest, TestBulkSchemaAPI, HdfsCollectionsAPIDistributedZkTest, SuggesterWFSTTest, CloudMLTQParserTest, DocValuesMissingTest, CoreAdminRequestStatusTest, RulesTest, TestCloudDeleteByQuery, SuggestComponentContextFilterQueryTest, AnalyticsQueryTest, TestSolrXml, DirectUpdateHandlerOptimizeTest, PropertiesRequestHandlerTest, TestCollectionAPI, ImplicitSnitchTest, TestSchemaSimilarityResource, SolrMetricManagerTest, TestBinaryResponseWriter, TestSolrCoreProperties, HdfsDirectoryFactoryTest, TestNamedUpdateProcessors, AddReplicaTest, TestRealTimeGet, TestPKIAuthenticationPlugin, TestPhraseSuggestions, TestSlowCompositeReaderWrapper, TestSolrConfigHandlerConcurrent, BinaryUpdateRequestHandlerTest, MetricUtilsTest, TestFilteredDocIdSet, SolrMetricsIntegrationTest, ShufflingReplicaListTransformerTest, ShowFileRequestHandlerTest, TestDFRSimilarityFactory, RemoteQueryErrorTest, TestBackupRepositoryFactory, TestManagedResourceStorage, TestSweetSpotSimilarityFactory, SolrCmdDistributorTest, TestHighlightDedupGrouping, SearchRateTriggerTest, TestSolrCloudWithDelegationTokens, TestCloudJSONFacetJoinDomain, SyncSliceTest, DistributedVersionInfoTest, NodeMutatorTest, TestReloadAndDeleteDocs, DistribCursorPagingTest, NumberUtilsTest, ScriptEngineTest, NodeAddedTriggerTest, TestDistribIDF, SliceStateTest, SuggesterTest, SearchHandlerTest, TestTolerantSearch, ZkFailoverTest, HdfsRecoverLeaseTest, UniqFieldsUpdateProcessorFactoryTest, TestOmitPositions, ClassificationUpdateProcessorFactoryTest, ZkControllerTest, BadIndexSchemaTest, AutoscalingHistoryHandlerTest, StressHdfsTest, DistributedFacetExistsSmallTest, TestQuerySenderListener, TestPullReplicaErrorHandling, TestManagedStopFilterFactory, TestMissingGroups, DistributedFacetPivotSmallTest, TestUtils, TestWriterPerf, TestTlogReplica, AutoAddReplicasPlanActionTest, ClusterStateUpdateTest, HighlighterTest, HighlighterConfigTest, CdcrVersionReplicationTest, ResourceLoaderTest, LeaderInitiatedRecoveryOnCommitTest, MetricsConfigTest, TestConfigsApi, TestReplicaProperties, BitVectorTest, DistanceFunctionTest, TestFuzzyAnalyzedSuggestions, TestExactStatsCache, CdcrBootstrapTest, TestMacroExpander, TestDynamicFieldCollectionResource, DirectSolrConnectionTest, SignatureUpdateProcessorFactoryTest, TestCloudPseudoReturnFields, BasicDistributedZkTest, DirectoryFactoryTest, ComputePlanActionTest, DirectUpdateHandlerTest, TestCustomStream, TestLegacyNumericUtils, HdfsAutoAddReplicasIntegrationTest, RAMDirectoryFactoryTest, SuggesterFSTTest, CollectionsAPIDistributedZkTest, TestReplicationHandler, LeaderElectionTest, ZkCLITest, TestDistributedGrouping, DistributedSpellCheckComponentTest, TestStressReorder, TestJoin, DistributedTermsComponentTest, TestRangeQuery, SimpleFacetsTest, PeerSyncTest, ConvertedLegacyTest, TestFunctionQuery]
[junit4] Completed [403/746 (1!)] on J0 in 2.16s, 22 tests, 1 error <<< FAILURES!
[...truncated 48346 lines...]
[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build #
1502 - Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1502/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC
2 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation
Error Message:
2 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation: 1) Thread[id=26185, name=jetty-launcher-4215-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037) at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328) at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277) at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105) at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41) at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244) at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44) at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61) at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505) 2) Thread[id=26189, name=jetty-launcher-4215-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037) at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328) at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277) at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105) at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41) at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244) at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44) at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61) at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
1) Thread[id=26185, name=jetty-launcher-4215-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
2) Thread[id=26189, name=jetty-launcher-4215-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
at __randomizedtesting.SeedInfo.seed([A40E31BF3AE97F14]:0)
FAILED: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI
Error Message:
Error from server at http://127.0.0.1:42522/solr/awhollynewcollection_0_shard2_replica_n2: ClusterState says we are the leader (http://127.0.0.1:42522/solr/awhollynewcollection_0_shard2_replica_n2), but locally we don't think so. Request came from null
Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from server at http://127.0.0.1:42522/solr/awhollynewcollection_0_shard2_replica_n2: ClusterState says we are the leader (http://127.0.0.1:42522/solr/awhollynewcollection_0_shard2_replica_n2), but locally we don't think so. Request came from null
at __randomizedtesting.SeedInfo.seed([A40E31BF3AE97F14:EC7B450B3CDA5081]:0)
at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:541)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1008)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:875)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:937)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:937)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:937)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:937)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:937)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:808)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:459)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:42522/solr/awhollynewcollection_0_shard2_replica_n2: ClusterState says we are the leader (http://127.0.0.1:42522/solr/awhollynewcollection_0_shard2_replica_n2), but locally we don't think so. Request came from null
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:658)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:518)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
Build Log:
[...truncated 12681 lines...]
[junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
[junit4] 2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A40E31BF3AE97F14-001/init-core-data-001
[junit4] 2> 1854465 WARN (SUITE-CollectionsAPIDistributedZkTest-seed#[A40E31BF3AE97F14]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=25 numCloses=25
[junit4] 2> 1854465 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[A40E31BF3AE97F14]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 1854467 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[A40E31BF3AE97F14]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 1854467 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[A40E31BF3AE97F14]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 1854467 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[A40E31BF3AE97F14]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A40E31BF3AE97F14-001/tempDir-001
[junit4] 2> 1854468 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[A40E31BF3AE97F14]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1854468 INFO (Thread-4693) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1854468 INFO (Thread-4693) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1854470 ERROR (Thread-4693) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1854568 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[A40E31BF3AE97F14]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:65097
[junit4] 2> 1854575 INFO (jetty-launcher-2913-thread-1) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 1854575 INFO (jetty-launcher-2913-thread-2) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 1854575 INFO (jetty-launcher-2913-thread-3) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 1854576 INFO (jetty-launcher-2913-thread-4) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 1854579 INFO (jetty-launcher-2913-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@19f7121a{/solr,null,AVAILABLE}
[junit4] 2> 1854579 INFO (jetty-launcher-2913-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@16439443{/solr,null,AVAILABLE}
[junit4] 2> 1854579 INFO (jetty-launcher-2913-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@707d2a12{HTTP/1.1,[http/1.1]}{127.0.0.1:61293}
[junit4] 2> 1854579 INFO (jetty-launcher-2913-thread-3) [ ] o.e.j.s.Server Started @1856413ms
[junit4] 2> 1854579 INFO (jetty-launcher-2913-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=61293}
[junit4] 2> 1854579 INFO (jetty-launcher-2913-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@6baa882a{HTTP/1.1,[http/1.1]}{127.0.0.1:59792}
[junit4] 2> 1854579 ERROR (jetty-launcher-2913-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1854579 INFO (jetty-launcher-2913-thread-1) [ ] o.e.j.s.Server Started @1856413ms
[junit4] 2> 1854579 INFO (jetty-launcher-2913-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@ced1c28{/solr,null,AVAILABLE}
[junit4] 2> 1854580 INFO (jetty-launcher-2913-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=59792}
[junit4] 2> 1854580 ERROR (jetty-launcher-2913-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1854580 INFO (jetty-launcher-2913-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@30e22d3c{/solr,null,AVAILABLE}
[junit4] 2> 1854580 INFO (jetty-launcher-2913-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 1854580 INFO (jetty-launcher-2913-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1854580 INFO (jetty-launcher-2913-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1854580 INFO (jetty-launcher-2913-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@635fe25b{HTTP/1.1,[http/1.1]}{127.0.0.1:42173}
[junit4] 2> 1854580 INFO (jetty-launcher-2913-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-11-01T05:37:23.553Z
[junit4] 2> 1854580 INFO (jetty-launcher-2913-thread-4) [ ] o.e.j.s.Server Started @1856414ms
[junit4] 2> 1854580 INFO (jetty-launcher-2913-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42173}
[junit4] 2> 1854580 ERROR (jetty-launcher-2913-thread-4) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1854580 INFO (jetty-launcher-2913-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 1854580 INFO (jetty-launcher-2913-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1854580 INFO (jetty-launcher-2913-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1854580 INFO (jetty-launcher-2913-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-11-01T05:37:23.553Z
[junit4] 2> 1854580 INFO (jetty-launcher-2913-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 1854581 INFO (jetty-launcher-2913-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1854581 INFO (jetty-launcher-2913-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1854581 INFO (jetty-launcher-2913-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-11-01T05:37:23.554Z
[junit4] 2> 1854580 INFO (jetty-launcher-2913-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@34033258{HTTP/1.1,[http/1.1]}{127.0.0.1:42522}
[junit4] 2> 1854581 INFO (jetty-launcher-2913-thread-2) [ ] o.e.j.s.Server Started @1856415ms
[junit4] 2> 1854581 INFO (jetty-launcher-2913-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42522}
[junit4] 2> 1854582 ERROR (jetty-launcher-2913-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1854582 INFO (jetty-launcher-2913-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 1854582 INFO (jetty-launcher-2913-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1854582 INFO (jetty-launcher-2913-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1854582 INFO (jetty-launcher-2913-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-11-01T05:37:23.555Z
[junit4] 2> 1854584 INFO (jetty-launcher-2913-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1854584 INFO (jetty-launcher-2913-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1854585 INFO (jetty-launcher-2913-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1854585 INFO (jetty-launcher-2913-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1854588 INFO (jetty-launcher-2913-thread-4) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 1854588 INFO (jetty-launcher-2913-thread-4) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 1854588 INFO (jetty-launcher-2913-thread-1) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 1854589 INFO (jetty-launcher-2913-thread-1) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 1854588 INFO (jetty-launcher-2913-thread-3) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 1854589 INFO (jetty-launcher-2913-thread-3) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 1854591 INFO (jetty-launcher-2913-thread-2) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 1854591 INFO (jetty-launcher-2913-thread-2) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 1854592 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15f761587f20002, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1854595 INFO (jetty-launcher-2913-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:65097/solr
[junit4] 2> 1854597 INFO (jetty-launcher-2913-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:65097/solr
[junit4] 2> 1854597 INFO (jetty-launcher-2913-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:65097/solr
[junit4] 2> 1854600 INFO (jetty-launcher-2913-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:65097/solr
[junit4] 2> 1854781 INFO (jetty-launcher-2913-thread-2) [n:127.0.0.1:42522_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1854781 INFO (jetty-launcher-2913-thread-2) [n:127.0.0.1:42522_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42522_solr
[junit4] 2> 1854783 INFO (jetty-launcher-2913-thread-2) [n:127.0.0.1:42522_solr ] o.a.s.c.Overseer Overseer (id=98927551672614923-127.0.0.1:42522_solr-n_0000000000) starting
[junit4] 2> 1854797 INFO (jetty-launcher-2913-thread-2) [n:127.0.0.1:42522_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42522_solr
[junit4] 2> 1854803 INFO (zkCallback-2932-thread-1-processing-n:127.0.0.1:59792_solr) [n:127.0.0.1:59792_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1854804 INFO (jetty-launcher-2913-thread-1) [n:127.0.0.1:59792_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1854804 INFO (OverseerStateUpdate-98927551672614923-127.0.0.1:42522_solr-n_0000000000) [n:127.0.0.1:42522_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1854808 INFO (jetty-launcher-2913-thread-4) [n:127.0.0.1:42173_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1854809 INFO (jetty-launcher-2913-thread-1) [n:127.0.0.1:59792_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59792_solr
[junit4] 2> 1854811 INFO (zkCallback-2934-thread-1-processing-n:127.0.0.1:42522_solr) [n:127.0.0.1:42522_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1854812 INFO (zkCallback-2932-thread-1-processing-n:127.0.0.1:59792_solr) [n:127.0.0.1:59792_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1854813 INFO (zkCallback-2933-thread-1-processing-n:127.0.0.1:42173_solr) [n:127.0.0.1:42173_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1854813 INFO (jetty-launcher-2913-thread-4) [n:127.0.0.1:42173_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1854816 INFO (jetty-launcher-2913-thread-4) [n:127.0.0.1:42173_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42173_solr
[junit4] 2> 1854819 INFO (zkCallback-2933-thread-1-processing-n:127.0.0.1:42173_solr) [n:127.0.0.1:42173_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1854819 INFO (zkCallback-2932-thread-1-processing-n:127.0.0.1:59792_solr) [n:127.0.0.1:59792_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1854820 INFO (zkCallback-2934-thread-1-processing-n:127.0.0.1:42522_solr) [n:127.0.0.1:42522_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1854877 INFO (jetty-launcher-2913-thread-1) [n:127.0.0.1:59792_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4ffccd9c
[junit4] 2> 1854885 INFO (jetty-launcher-2913-thread-1) [n:127.0.0.1:59792_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4ffccd9c
[junit4] 2> 1854885 INFO (jetty-launcher-2913-thread-1) [n:127.0.0.1:59792_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4ffccd9c
[junit4] 2> 1854887 INFO (jetty-launcher-2913-thread-1) [n:127.0.0.1:59792_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A40E31BF3AE97F14-001/tempDir-001/node1
[junit4] 2> 1854893 INFO (jetty-launcher-2913-thread-2) [n:127.0.0.1:42522_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4ffccd9c
[junit4] 2> 1854902 INFO (jetty-launcher-2913-thread-2) [n:127.0.0.1:42522_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4ffccd9c
[junit4] 2> 1854903 INFO (jetty-launcher-2913-thread-2) [n:127.0.0.1:42522_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4ffccd9c
[junit4] 2> 1854904 INFO (jetty-launcher-2913-thread-2) [n:127.0.0.1:42522_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A40E31BF3AE97F14-001/tempDir-001/node2
[junit4] 2> 1854929 INFO (jetty-launcher-2913-thread-4) [n:127.0.0.1:42173_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4ffccd9c
[junit4] 2> 1854934 WARN (jetty-launcher-2913-thread-2) [n:127.0.0.1:42522_solr ] o.a.s.m.r.j.JmxMetricsReporter Unable to register gauge
[junit4] 2> javax.management.InstanceNotFoundException: solr:dom1=jvm,name0=memory,name1=pools,name2=Par-Survivor-Space,name=usage
[junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1095)
[junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:427)
[junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
[junit4] 2> at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
[junit4] 2> at org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter$JmxListener.registerMBean(JmxMetricsReporter.java:530)
[junit4] 2> at org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter$JmxListener.onGaugeAdded(JmxMetricsReporter.java:565)
[junit4] 2> at com.codahale.metrics.MetricRegistry.notifyListenerOfAddedMetric(MetricRegistry.java:454)
[junit4] 2> at com.codahale.metrics.MetricRegistry.onMetricAdded(MetricRegistry.java:448)
[junit4] 2> at com.codahale.metrics.MetricRegistry.register(MetricRegistry.java:89)
[junit4] 2> at org.apache.solr.metrics.SolrMetricManager.registerAll(SolrMetricManager.java:525)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.setupJvmMetrics(SolrDispatchFilter.java:214)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:186)
[junit4] 2> at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1596)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1659)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1316)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1145)
[junit4] 2> at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
[junit4] 2> at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
[junit4] 2> at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
[junit4] 2> at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:384)
[junit4] 2> at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:247)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1854936 INFO (jetty-launcher-2913-thread-4) [n:127.0.0.1:42173_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4ffccd9c
[junit4] 2> 1854937 INFO (jetty-launcher-2913-thread-4) [n:127.0.0.1:42173_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4ffccd9c
[junit4] 2> 1854938 INFO (jetty-launcher-2913-thread-4) [n:127.0.0.1:42173_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A40E31BF3AE97F14-001/tempDir-001/node4
[junit4] 2> 1854941 INFO (jetty-launcher-2913-thread-3) [n:127.0.0.1:61293_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 1854943 INFO (jetty-launcher-2913-thread-3) [n:127.0.0.1:61293_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1854945 INFO (jetty-launcher-2913-thread-3) [n:127.0.0.1:61293_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:61293_solr
[junit4] 2> 1854947 INFO (zkCallback-2933-thread-1-processing-n:127.0.0.1:42173_solr) [n:127.0.0.1:42173_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1854947 INFO (zkCallback-2932-thread-1-processing-n:127.0.0.1:59792_solr) [n:127.0.0.1:59792_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1854964 INFO (zkCallback-2934-thread-1-processing-n:127.0.0.1:42522_solr) [n:127.0.0.1:42522_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1854966 INFO (zkCallback-2935-thread-1-processing-n:127.0.0.1:61293_solr) [n:127.0.0.1:61293_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1855118 INFO (jetty-launcher-2913-thread-3) [n:127.0.0.1:61293_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4ffccd9c
[junit4] 2> 1855123 INFO (jetty-launcher-2913-thread-3) [n:127.0.0.1:61293_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4ffccd9c
[junit4] 2> 1855124 INFO (jetty-launcher-2913-thread-3) [n:127.0.0.1:61293_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4ffccd9c
[junit4] 2> 1855124 INFO (jetty-launcher-2913-thread-3) [n:127.0.0.1:61293_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A40E31BF3AE97F14-001/tempDir-001/node3
[junit4] 2> 1855150 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15f761587f2000d, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1855153 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[A40E31BF3AE97F14]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 1855155 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[A40E31BF3AE97F14]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:65097/solr ready
[junit4] 2> 1855182 INFO (TEST-CollectionsAPIDistributedZkTest.testCollectionsAPI-seed#[A40E31BF3AE97F14]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCollectionsAPI
[junit4] 2> 1855184 INFO (TEST-CollectionsAPIDistributedZkTest.testCollectionsAPI-seed#[A40E31BF3AE97F14]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 1855187 INFO (qtp820274126-17667) [n:127.0.0.1:61293_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params async=fb041d48-216d-41c9-9f39-799afedccb1f&replicationFactor=1&collection.configName=conf2&maxShardsPerNode=2&name=awhollynewcollection_0&nrtReplicas=1&action=CREATE&numShards=5&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1855189 INFO (qtp820274126-17667) [n:127.0.0.1:61293_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=fb041d48-216d-41c9-9f39-799afedccb1f&replicationFactor=1&collection.configName=conf2&maxShardsPerNode=2&name=awhollynewcollection_0&nrtReplicas=1&action=CREATE&numShards=5&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 1855192 INFO (OverseerThreadFactory-7116-thread-1-processing-n:127.0.0.1:42522_solr) [n:127.0.0.1:42522_solr ] o.a.s.c.CreateCollectionCmd Create collection awhollynewcollection_0
[junit4] 2> 1855200 INFO (zkCallback-2944-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeCreated path:/collections/awhollynewcollection_0/state.json] for collection [awhollynewcollection_0] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1855304 INFO (OverseerStateUpdate-98927551672614923-127.0.0.1:42522_solr-n_0000000000) [n:127.0.0.1:42522_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"awhollynewcollection_0",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"awhollynewcollection_0_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:59792/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1855307 INFO (OverseerStateUpdate-98927551672614923-127.0.0.1:42522_solr-n_0000000000) [n:127.0.0.1:42522_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"awhollynewcollection_0",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"awhollynewcollection_0_shard2_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:42522/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1855310 INFO (OverseerStateUpdate-98927551672614923-127.0.0.1:42522_solr-n_0000000000) [n:127.0.0.1:42522_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"awhollynewcollection_0",
[junit4] 2> "shard":"shard3",
[junit4] 2> "core":"awhollynewcollection_0_shard3_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:61293/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1855312 INFO (OverseerStateUpdate-98927551672614923-127.0.0.1:42522_solr-n_0000000000) [n:127.0.0.1:42522_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"awhollynewcollection_0",
[junit4] 2> "shard":"shard4",
[junit4] 2> "core":"awhollynewcollection_0_shard4_replica_n6",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:42173/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1855315 INFO (OverseerStateUpdate-98927551672614923-127.0.0.1:42522_solr-n_0000000000) [n:127.0.0.1:42522_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"awhollynewcollection_0",
[junit4] 2> "shard":"shard5",
[junit4] 2> "core":"awhollynewcollection_0_shard5_replica_n8",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:59792/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1855417 INFO (zkCallback-2944-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/awhollynewcollection_0/state.json] for collection [awhollynewcollection_0] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1855525 INFO (qtp417198643-17690) [n:127.0.0.1:42522_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf2&newCollection=true&collection=awhollynewcollection_0&version=2&replicaType=NRT&async=fb041d48-216d-41c9-9f39-799afedccb1f844344460217368&coreNodeName=core_node5&name=awhollynewcollection_0_shard2_replica_n2&action=CREATE&numShards=5&shard=shard2&wt=javabin} status=0 QTime=0
[junit4] 2> 1855526 INFO (qtp805569476-17663) [n:127.0.0.1:59792_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf2&newCollection=true&collection=awhollynewcollection_0&version=2&replicaType=NRT&async=fb041d48-216d-41c9-9f39-799afedccb1f844344457431382&coreNodeName=core_node3&name=awhollynewcollection_0_shard1_replica_n1&action=CREATE&numShards=5&shard=shard1&wt=javabin} status=0 QTime=0
[junit4] 2> 1855526 INFO (parallelCoreAdminExecutor-7118-thread-1-processing-n:127.0.0.1:42522_solr fb041d48-216d-41c9-9f39-799afedccb1f844344460217368 CREATE) [n:127.0.0.1:42522_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf2&newCollection=true&collection=awhollynewcollection_0&version=2&replicaType=NRT&async=fb041d48-216d-41c9-9f39-799afedccb1f844344460217368&coreNodeName=core_node5&name=awhollynewcollection_0_shard2_replica_n2&action=CREATE&numShards=5&shard=shard2&wt=javabin
[junit4] 2> 1855529 INFO (parallelCoreAdminExecutor-7117-thread-1-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344457431382 CREATE) [n:127.0.0.1:59792_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf2&newCollection=true&collection=awhollynewcollection_0&version=2&replicaType=NRT&async=fb041d48-216d-41c9-9f39-799afedccb1f844344457431382&coreNodeName=core_node3&name=awhollynewcollection_0_shard1_replica_n1&action=CREATE&numShards=5&shard=shard1&wt=javabin
[junit4] 2> 1855529 INFO (parallelCoreAdminExecutor-7118-thread-1-processing-n:127.0.0.1:42522_solr fb041d48-216d-41c9-9f39-799afedccb1f844344460217368 CREATE) [n:127.0.0.1:42522_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1855529 INFO (qtp820274126-17680) [n:127.0.0.1:61293_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf2&newCollection=true&collection=awhollynewcollection_0&version=2&replicaType=NRT&async=fb041d48-216d-41c9-9f39-799afedccb1f844344463297441&coreNodeName=core_node7&name=awhollynewcollection_0_shard3_replica_n4&action=CREATE&numShards=5&shard=shard3&wt=javabin} status=0 QTime=0
[junit4] 2> 1855530 INFO (parallelCoreAdminExecutor-7117-thread-1-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344457431382 CREATE) [n:127.0.0.1:59792_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1855531 INFO (parallelCoreAdminExecutor-7123-thread-1-processing-n:127.0.0.1:61293_solr fb041d48-216d-41c9-9f39-799afedccb1f844344463297441 CREATE) [n:127.0.0.1:61293_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf2&newCollection=true&collection=awhollynewcollection_0&version=2&replicaType=NRT&async=fb041d48-216d-41c9-9f39-799afedccb1f844344463297441&coreNodeName=core_node7&name=awhollynewcollection_0_shard3_replica_n4&action=CREATE&numShards=5&shard=shard3&wt=javabin
[junit4] 2> 1855533 INFO (parallelCoreAdminExecutor-7123-thread-1-processing-n:127.0.0.1:61293_solr fb041d48-216d-41c9-9f39-799afedccb1f844344463297441 CREATE) [n:127.0.0.1:61293_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1855533 INFO (qtp805569476-17665) [n:127.0.0.1:59792_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf2&newCollection=true&collection=awhollynewcollection_0&version=2&replicaType=NRT&async=fb041d48-216d-41c9-9f39-799afedccb1f844344468697888&coreNodeName=core_node10&name=awhollynewcollection_0_shard5_replica_n8&action=CREATE&numShards=5&shard=shard5&wt=javabin} status=0 QTime=7
[junit4] 2> 1855534 INFO (qtp890055569-17673) [n:127.0.0.1:42173_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf2&newCollection=true&collection=awhollynewcollection_0&version=2&replicaType=NRT&async=fb041d48-216d-41c9-9f39-799afedccb1f844344465948674&coreNodeName=core_node9&name=awhollynewcollection_0_shard4_replica_n6&action=CREATE&numShards=5&shard=shard4&wt=javabin} status=0 QTime=1
[junit4] 2> 1855535 INFO (parallelCoreAdminExecutor-7121-thread-1-processing-n:127.0.0.1:42173_solr fb041d48-216d-41c9-9f39-799afedccb1f844344465948674 CREATE) [n:127.0.0.1:42173_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf2&newCollection=true&collection=awhollynewcollection_0&version=2&replicaType=NRT&async=fb041d48-216d-41c9-9f39-799afedccb1f844344465948674&coreNodeName=core_node9&name=awhollynewcollection_0_shard4_replica_n6&action=CREATE&numShards=5&shard=shard4&wt=javabin
[junit4] 2> 1855535 INFO (parallelCoreAdminExecutor-7117-thread-2-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344468697888 CREATE) [n:127.0.0.1:59792_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf2&newCollection=true&collection=awhollynewcollection_0&version=2&replicaType=NRT&async=fb041d48-216d-41c9-9f39-799afedccb1f844344468697888&coreNodeName=core_node10&name=awhollynewcollection_0_shard5_replica_n8&action=CREATE&numShards=5&shard=shard5&wt=javabin
[junit4] 2> 1855536 INFO (parallelCoreAdminExecutor-7121-thread-1-processing-n:127.0.0.1:42173_solr fb041d48-216d-41c9-9f39-799afedccb1f844344465948674 CREATE) [n:127.0.0.1:42173_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1855538 INFO (qtp890055569-17675) [n:127.0.0.1:42173_solr ] o.a.s.h.a.CoreAdminOperation Checking request status for : fb041d48-216d-41c9-9f39-799afedccb1f844344465948674
[junit4] 2> 1855541 INFO (qtp890055569-17675) [n:127.0.0.1:42173_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=fb041d48-216d-41c9-9f39-799afedccb1f844344465948674&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=4
[junit4] 2> 1855661 INFO (zkCallback-2932-thread-1-processing-n:127.0.0.1:59792_solr) [n:127.0.0.1:59792_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/awhollynewcollection_0/state.json] for collection [awhollynewcollection_0] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1855661 INFO (zkCallback-2934-thread-1-processing-n:127.0.0.1:42522_solr) [n:127.0.0.1:42522_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/awhollynewcollection_0/state.json] for collection [awhollynewcollection_0] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1855662 INFO (zkCallback-2944-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/awhollynewcollection_0/state.json] for collection [awhollynewcollection_0] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1855662 INFO (zkCallback-2933-thread-1-processing-n:127.0.0.1:42173_solr) [n:127.0.0.1:42173_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/awhollynewcollection_0/state.json] for collection [awhollynewcollection_0] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1855662 INFO (zkCallback-2932-thread-2-processing-n:127.0.0.1:59792_solr) [n:127.0.0.1:59792_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/awhollynewcollection_0/state.json] for collection [awhollynewcollection_0] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1855662 INFO (zkCallback-2934-thread-2-processing-n:127.0.0.1:42522_solr) [n:127.0.0.1:42522_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/awhollynewcollection_0/state.json] for collection [awhollynewcollection_0] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1855663 INFO (zkCallback-2933-thread-2-processing-n:127.0.0.1:42173_solr) [n:127.0.0.1:42173_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/awhollynewcollection_0/state.json] for collection [awhollynewcollection_0] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1855663 INFO (zkCallback-2935-thread-1-processing-n:127.0.0.1:61293_solr) [n:127.0.0.1:61293_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/awhollynewcollection_0/state.json] for collection [awhollynewcollection_0] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1855663 INFO (zkCallback-2935-thread-2-processing-n:127.0.0.1:61293_solr) [n:127.0.0.1:61293_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/awhollynewcollection_0/state.json] for collection [awhollynewcollection_0] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1856545 INFO (qtp890055569-17674) [n:127.0.0.1:42173_solr ] o.a.s.h.a.CoreAdminOperation Checking request status for : fb041d48-216d-41c9-9f39-799afedccb1f844344465948674
[junit4] 2> 1856545 INFO (qtp890055569-17674) [n:127.0.0.1:42173_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=fb041d48-216d-41c9-9f39-799afedccb1f844344465948674&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1856561 INFO (parallelCoreAdminExecutor-7117-thread-1-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344457431382 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard1 r:core_node3 x:awhollynewcollection_0_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 1856565 INFO (parallelCoreAdminExecutor-7123-thread-1-processing-n:127.0.0.1:61293_solr fb041d48-216d-41c9-9f39-799afedccb1f844344463297441 CREATE) [n:127.0.0.1:61293_solr c:awhollynewcollection_0 s:shard3 r:core_node7 x:awhollynewcollection_0_shard3_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 1856580 INFO (parallelCoreAdminExecutor-7121-thread-1-processing-n:127.0.0.1:42173_solr fb041d48-216d-41c9-9f39-799afedccb1f844344465948674 CREATE) [n:127.0.0.1:42173_solr c:awhollynewcollection_0 s:shard4 r:core_node9 x:awhollynewcollection_0_shard4_replica_n6] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 1856582 INFO (parallelCoreAdminExecutor-7123-thread-1-processing-n:127.0.0.1:61293_solr fb041d48-216d-41c9-9f39-799afedccb1f844344463297441 CREATE) [n:127.0.0.1:61293_solr c:awhollynewcollection_0 s:shard3 r:core_node7 x:awhollynewcollection_0_shard3_replica_n4] o.a.s.s.IndexSchema [awhollynewcollection_0_shard3_replica_n4] Schema name=minimal
[junit4] 2> 1856585 INFO (parallelCoreAdminExecutor-7123-thread-1-processing-n:127.0.0.1:61293_solr fb041d48-216d-41c9-9f39-799afedccb1f844344463297441 CREATE) [n:127.0.0.1:61293_solr c:awhollynewcollection_0 s:shard3 r:core_node7 x:awhollynewcollection_0_shard3_replica_n4] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1856585 INFO (parallelCoreAdminExecutor-7123-thread-1-processing-n:127.0.0.1:61293_solr fb041d48-216d-41c9-9f39-799afedccb1f844344463297441 CREATE) [n:127.0.0.1:61293_solr c:awhollynewcollection_0 s:shard3 r:core_node7 x:awhollynewcollection_0_shard3_replica_n4] o.a.s.c.CoreContainer Creating SolrCore 'awhollynewcollection_0_shard3_replica_n4' using configuration from collection awhollynewcollection_0, trusted=true
[junit4] 2> 1856585 INFO (parallelCoreAdminExecutor-7123-thread-1-processing-n:127.0.0.1:61293_solr fb041d48-216d-41c9-9f39-799afedccb1f844344463297441 CREATE) [n:127.0.0.1:61293_solr c:awhollynewcollection_0 s:shard3 r:core_node7 x:awhollynewcollection_0_shard3_replica_n4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.awhollynewcollection_0.shard3.replica_n4' (registry 'solr.core.awhollynewcollection_0.shard3.replica_n4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4ffccd9c
[junit4] 2> 1856585 INFO (parallelCoreAdminExecutor-7123-thread-1-processing-n:127.0.0.1:61293_solr fb041d48-216d-41c9-9f39-799afedccb1f844344463297441 CREATE) [n:127.0.0.1:61293_solr c:awhollynewcollection_0 s:shard3 r:core_node7 x:awhollynewcollection_0_shard3_replica_n4] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1856585 INFO (parallelCoreAdminExecutor-7123-thread-1-processing-n:127.0.0.1:61293_solr fb041d48-216d-41c9-9f39-799afedccb1f844344463297441 CREATE) [n:127.0.0.1:61293_solr c:awhollynewcollection_0 s:shard3 r:core_node7 x:awhollynewcollection_0_shard3_replica_n4] o.a.s.c.SolrCore [[awhollynewcollection_0_shard3_replica_n4] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A40E31BF3AE97F14-001/tempDir-001/node3/awhollynewcollection_0_shard3_replica_n4], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A40E31BF3AE97F14-001/tempDir-001/node3/awhollynewcollection_0_shard3_replica_n4/data/]
[junit4] 2> 1856587 INFO (parallelCoreAdminExecutor-7117-thread-1-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344457431382 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard1 r:core_node3 x:awhollynewcollection_0_shard1_replica_n1] o.a.s.s.IndexSchema [awhollynewcollection_0_shard1_replica_n1] Schema name=minimal
[junit4] 2> 1856589 INFO (parallelCoreAdminExecutor-7117-thread-1-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344457431382 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard1 r:core_node3 x:awhollynewcollection_0_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1856589 INFO (parallelCoreAdminExecutor-7117-thread-1-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344457431382 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard1 r:core_node3 x:awhollynewcollection_0_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'awhollynewcollection_0_shard1_replica_n1' using configuration from collection awhollynewcollection_0, trusted=true
[junit4] 2> 1856589 INFO (parallelCoreAdminExecutor-7117-thread-2-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344468697888 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard5 r:core_node10 x:awhollynewcollection_0_shard5_replica_n8] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 1856589 INFO (parallelCoreAdminExecutor-7117-thread-1-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344457431382 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard1 r:core_node3 x:awhollynewcollection_0_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.awhollynewcollection_0.shard1.replica_n1' (registry 'solr.core.awhollynewcollection_0.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4ffccd9c
[junit4] 2> 1856589 INFO (parallelCoreAdminExecutor-7117-thread-1-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344457431382 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard1 r:core_node3 x:awhollynewcollection_0_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1856589 INFO (parallelCoreAdminExecutor-7117-thread-1-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344457431382 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard1 r:core_node3 x:awhollynewcollection_0_shard1_replica_n1] o.a.s.c.SolrCore [[awhollynewcollection_0_shard1_replica_n1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A40E31BF3AE97F14-001/tempDir-001/node1/awhollynewcollection_0_shard1_replica_n1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A40E31BF3AE97F14-001/tempDir-001/node1/awhollynewcollection_0_shard1_replica_n1/data/]
[junit4] 2> 1856592 INFO (parallelCoreAdminExecutor-7121-thread-1-processing-n:127.0.0.1:42173_solr fb041d48-216d-41c9-9f39-799afedccb1f844344465948674 CREATE) [n:127.0.0.1:42173_solr c:awhollynewcollection_0 s:shard4 r:core_node9 x:awhollynewcollection_0_shard4_replica_n6] o.a.s.s.IndexSchema [awhollynewcollection_0_shard4_replica_n6] Schema name=minimal
[junit4] 2> 1856594 INFO (parallelCoreAdminExecutor-7121-thread-1-processing-n:127.0.0.1:42173_solr fb041d48-216d-41c9-9f39-799afedccb1f844344465948674 CREATE) [n:127.0.0.1:42173_solr c:awhollynewcollection_0 s:shard4 r:core_node9 x:awhollynewcollection_0_shard4_replica_n6] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1856594 INFO (parallelCoreAdminExecutor-7121-thread-1-processing-n:127.0.0.1:42173_solr fb041d48-216d-41c9-9f39-799afedccb1f844344465948674 CREATE) [n:127.0.0.1:42173_solr c:awhollynewcollection_0 s:shard4 r:core_node9 x:awhollynewcollection_0_shard4_replica_n6] o.a.s.c.CoreContainer Creating SolrCore 'awhollynewcollection_0_shard4_replica_n6' using configuration from collection awhollynewcollection_0, trusted=true
[junit4] 2> 1856594 INFO (parallelCoreAdminExecutor-7121-thread-1-processing-n:127.0.0.1:42173_solr fb041d48-216d-41c9-9f39-799afedccb1f844344465948674 CREATE) [n:127.0.0.1:42173_solr c:awhollynewcollection_0 s:shard4 r:core_node9 x:awhollynewcollection_0_shard4_replica_n6] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.awhollynewcollection_0.shard4.replica_n6' (registry 'solr.core.awhollynewcollection_0.shard4.replica_n6') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4ffccd9c
[junit4] 2> 1856595 INFO (parallelCoreAdminExecutor-7121-thread-1-processing-n:127.0.0.1:42173_solr fb041d48-216d-41c9-9f39-799afedccb1f844344465948674 CREATE) [n:127.0.0.1:42173_solr c:awhollynewcollection_0 s:shard4 r:core_node9 x:awhollynewcollection_0_shard4_replica_n6] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1856595 INFO (parallelCoreAdminExecutor-7121-thread-1-processing-n:127.0.0.1:42173_solr fb041d48-216d-41c9-9f39-799afedccb1f844344465948674 CREATE) [n:127.0.0.1:42173_solr c:awhollynewcollection_0 s:shard4 r:core_node9 x:awhollynewcollection_0_shard4_replica_n6] o.a.s.c.SolrCore [[awhollynewcollection_0_shard4_replica_n6] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A40E31BF3AE97F14-001/tempDir-001/node4/awhollynewcollection_0_shard4_replica_n6], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A40E31BF3AE97F14-001/tempDir-001/node4/awhollynewcollection_0_shard4_replica_n6/data/]
[junit4] 2> 1856611 INFO (parallelCoreAdminExecutor-7118-thread-1-processing-n:127.0.0.1:42522_solr fb041d48-216d-41c9-9f39-799afedccb1f844344460217368 CREATE) [n:127.0.0.1:42522_solr c:awhollynewcollection_0 s:shard2 r:core_node5 x:awhollynewcollection_0_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 1856618 INFO (parallelCoreAdminExecutor-7117-thread-2-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344468697888 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard5 r:core_node10 x:awhollynewcollection_0_shard5_replica_n8] o.a.s.s.IndexSchema [awhollynewcollection_0_shard5_replica_n8] Schema name=minimal
[junit4] 2> 1856619 INFO (parallelCoreAdminExecutor-7117-thread-2-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344468697888 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard5 r:core_node10 x:awhollynewcollection_0_shard5_replica_n8] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1856619 INFO (parallelCoreAdminExecutor-7117-thread-2-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344468697888 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard5 r:core_node10 x:awhollynewcollection_0_shard5_replica_n8] o.a.s.c.CoreContainer Creating SolrCore 'awhollynewcollection_0_shard5_replica_n8' using configuration from collection awhollynewcollection_0, trusted=true
[junit4] 2> 1856619 INFO (parallelCoreAdminExecutor-7117-thread-2-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344468697888 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard5 r:core_node10 x:awhollynewcollection_0_shard5_replica_n8] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.awhollynewcollection_0.shard5.replica_n8' (registry 'solr.core.awhollynewcollection_0.shard5.replica_n8') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4ffccd9c
[junit4] 2> 1856620 INFO (parallelCoreAdminExecutor-7117-thread-2-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344468697888 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard5 r:core_node10 x:awhollynewcollection_0_shard5_replica_n8] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1856620 INFO (parallelCoreAdminExecutor-7117-thread-2-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344468697888 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard5 r:core_node10 x:awhollynewcollection_0_shard5_replica_n8] o.a.s.c.SolrCore [[awhollynewcollection_0_shard5_replica_n8] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A40E31BF3AE97F14-001/tempDir-001/node1/awhollynewcollection_0_shard5_replica_n8], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A40E31BF3AE97F14-001/tempDir-001/node1/awhollynewcollection_0_shard5_replica_n8/data/]
[junit4] 2> 1856626 INFO (parallelCoreAdminExecutor-7118-thread-1-processing-n:127.0.0.1:42522_solr fb041d48-216d-41c9-9f39-799afedccb1f844344460217368 CREATE) [n:127.0.0.1:42522_solr c:awhollynewcollection_0 s:shard2 r:core_node5 x:awhollynewcollection_0_shard2_replica_n2] o.a.s.s.IndexSchema [awhollynewcollection_0_shard2_replica_n2] Schema name=minimal
[junit4] 2> 1856628 INFO (parallelCoreAdminExecutor-7118-thread-1-processing-n:127.0.0.1:42522_solr fb041d48-216d-41c9-9f39-799afedccb1f844344460217368 CREATE) [n:127.0.0.1:42522_solr c:awhollynewcollection_0 s:shard2 r:core_node5 x:awhollynewcollection_0_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1856629 INFO (parallelCoreAdminExecutor-7118-thread-1-processing-n:127.0.0.1:42522_solr fb041d48-216d-41c9-9f39-799afedccb1f844344460217368 CREATE) [n:127.0.0.1:42522_solr c:awhollynewcollection_0 s:shard2 r:core_node5 x:awhollynewcollection_0_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'awhollynewcollection_0_shard2_replica_n2' using configuration from collection awhollynewcollection_0, trusted=true
[junit4] 2> 1856629 INFO (parallelCoreAdminExecutor-7118-thread-1-processing-n:127.0.0.1:42522_solr fb041d48-216d-41c9-9f39-799afedccb1f844344460217368 CREATE) [n:127.0.0.1:42522_solr c:awhollynewcollection_0 s:shard2 r:core_node5 x:awhollynewcollection_0_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.awhollynewcollection_0.shard2.replica_n2' (registry 'solr.core.awhollynewcollection_0.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4ffccd9c
[junit4] 2> 1856629 INFO (parallelCoreAdminExecutor-7118-thread-1-processing-n:127.0.0.1:42522_solr fb041d48-216d-41c9-9f39-799afedccb1f844344460217368 CREATE) [n:127.0.0.1:42522_solr c:awhollynewcollection_0 s:shard2 r:core_node5 x:awhollynewcollection_0_shard2_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1856629 INFO (parallelCoreAdminExecutor-7118-thread-1-processing-n:127.0.0.1:42522_solr fb041d48-216d-41c9-9f39-799afedccb1f844344460217368 CREATE) [n:127.0.0.1:42522_solr c:awhollynewcollection_0 s:shard2 r:core_node5 x:awhollynewcollection_0_shard2_replica_n2] o.a.s.c.SolrCore [[awhollynewcollection_0_shard2_replica_n2] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A40E31BF3AE97F14-001/tempDir-001/node2/awhollynewcollection_0_shard2_replica_n2], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A40E31BF3AE97F14-001/tempDir-001/node2/awhollynewcollection_0_shard2_replica_n2/data/]
[junit4] 2> 1856687 INFO (parallelCoreAdminExecutor-7121-thread-1-processing-n:127.0.0.1:42173_solr fb041d48-216d-41c9-9f39-799afedccb1f844344465948674 CREATE) [n:127.0.0.1:42173_solr c:awhollynewcollection_0 s:shard4 r:core_node9 x:awhollynewcollection_0_shard4_replica_n6] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1856687 INFO (parallelCoreAdminExecutor-7121-thread-1-processing-n:127.0.0.1:42173_solr fb041d48-216d-41c9-9f39-799afedccb1f844344465948674 CREATE) [n:127.0.0.1:42173_solr c:awhollynewcollection_0 s:shard4 r:core_node9 x:awhollynewcollection_0_shard4_replica_n6] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1856688 INFO (parallelCoreAdminExecutor-7121-thread-1-processing-n:127.0.0.1:42173_solr fb041d48-216d-41c9-9f39-799afedccb1f844344465948674 CREATE) [n:127.0.0.1:42173_solr c:awhollynewcollection_0 s:shard4 r:core_node9 x:awhollynewcollection_0_shard4_replica_n6] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1856688 INFO (parallelCoreAdminExecutor-7121-thread-1-processing-n:127.0.0.1:42173_solr fb041d48-216d-41c9-9f39-799afedccb1f844344465948674 CREATE) [n:127.0.0.1:42173_solr c:awhollynewcollection_0 s:shard4 r:core_node9 x:awhollynewcollection_0_shard4_replica_n6] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1856689 INFO (parallelCoreAdminExecutor-7121-thread-1-processing-n:127.0.0.1:42173_solr fb041d48-216d-41c9-9f39-799afedccb1f844344465948674 CREATE) [n:127.0.0.1:42173_solr c:awhollynewcollection_0 s:shard4 r:core_node9 x:awhollynewcollection_0_shard4_replica_n6] o.a.s.s.SolrIndexSearcher Opening [Searcher@6dc10144[awhollynewcollection_0_shard4_replica_n6] main]
[junit4] 2> 1856690 INFO (parallelCoreAdminExecutor-7121-thread-1-processing-n:127.0.0.1:42173_solr fb041d48-216d-41c9-9f39-799afedccb1f844344465948674 CREATE) [n:127.0.0.1:42173_solr c:awhollynewcollection_0 s:shard4 r:core_node9 x:awhollynewcollection_0_shard4_replica_n6] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf2
[junit4] 2> 1856690 INFO (parallelCoreAdminExecutor-7121-thread-1-processing-n:127.0.0.1:42173_solr fb041d48-216d-41c9-9f39-799afedccb1f844344465948674 CREATE) [n:127.0.0.1:42173_solr c:awhollynewcollection_0 s:shard4 r:core_node9 x:awhollynewcollection_0_shard4_replica_n6] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf2
[junit4] 2> 1856691 INFO (parallelCoreAdminExecutor-7121-thread-1-processing-n:127.0.0.1:42173_solr fb041d48-216d-41c9-9f39-799afedccb1f844344465948674 CREATE) [n:127.0.0.1:42173_solr c:awhollynewcollection_0 s:shard4 r:core_node9 x:awhollynewcollection_0_shard4_replica_n6] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1856691 INFO (parallelCoreAdminExecutor-7121-thread-1-processing-n:127.0.0.1:42173_solr fb041d48-216d-41c9-9f39-799afedccb1f844344465948674 CREATE) [n:127.0.0.1:42173_solr c:awhollynewcollection_0 s:shard4 r:core_node9 x:awhollynewcollection_0_shard4_replica_n6] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1582840829091774464
[junit4] 2> 1856691 INFO (parallelCoreAdminExecutor-7117-thread-1-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344457431382 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard1 r:core_node3 x:awhollynewcollection_0_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1856691 INFO (parallelCoreAdminExecutor-7117-thread-1-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344457431382 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard1 r:core_node3 x:awhollynewcollection_0_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1856692 INFO (parallelCoreAdminExecutor-7117-thread-1-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344457431382 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard1 r:core_node3 x:awhollynewcollection_0_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1856692 INFO (parallelCoreAdminExecutor-7117-thread-1-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344457431382 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard1 r:core_node3 x:awhollynewcollection_0_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1856693 INFO (parallelCoreAdminExecutor-7117-thread-1-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344457431382 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard1 r:core_node3 x:awhollynewcollection_0_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4057429[awhollynewcollection_0_shard1_replica_n1] main]
[junit4] 2> 1856695 INFO (parallelCoreAdminExecutor-7117-thread-1-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344457431382 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard1 r:core_node3 x:awhollynewcollection_0_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf2
[junit4] 2> 1856697 INFO (parallelCoreAdminExecutor-7121-thread-1-processing-n:127.0.0.1:42173_solr fb041d48-216d-41c9-9f39-799afedccb1f844344465948674 CREATE) [n:127.0.0.1:42173_solr c:awhollynewcollection_0 s:shard4 r:core_node9 x:awhollynewcollection_0_shard4_replica_n6] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1856697 INFO (parallelCoreAdminExecutor-7121-thread-1-processing-n:127.0.0.1:42173_solr fb041d48-216d-41c9-9f39-799afedccb1f844344465948674 CREATE) [n:127.0.0.1:42173_solr c:awhollynewcollection_0 s:shard4 r:core_node9 x:awhollynewcollection_0_shard4_replica_n6] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1856697 INFO (parallelCoreAdminExecutor-7121-thread-1-processing-n:127.0.0.1:42173_solr fb041d48-216d-41c9-9f39-799afedccb1f844344465948674 CREATE) [n:127.0.0.1:42173_solr c:awhollynewcollection_0 s:shard4 r:core_node9 x:awhollynewcollection_0_shard4_replica_n6] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:42173/solr/awhollynewcollection_0_shard4_replica_n6/
[junit4] 2> 1856697 INFO (parallelCoreAdminExecutor-7121-thread-1-processing-n:127.0.0.1:42173_solr fb041d48-216d-41c9-9f39-799afedccb1f844344465948674 CREATE) [n:127.0.0.1:42173_solr c:awhollynewcollection_0 s:shard4 r:core_node9 x:awhollynewcollection_0_shard4_replica_n6] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1856697 INFO (parallelCoreAdminExecutor-7121-thread-1-processing-n:127.0.0.1:42173_solr fb041d48-216d-41c9-9f39-799afedccb1f844344465948674 CREATE) [n:127.0.0.1:42173_solr c:awhollynewcollection_0 s:shard4 r:core_node9 x:awhollynewcollection_0_shard4_replica_n6] o.a.s.c.SyncStrategy http://127.0.0.1:42173/solr/awhollynewcollection_0_shard4_replica_n6/ has no replicas
[junit4] 2> 1856697 INFO (parallelCoreAdminExecutor-7121-thread-1-processing-n:127.0.0.1:42173_solr fb041d48-216d-41c9-9f39-799afedccb1f844344465948674 CREATE) [n:127.0.0.1:42173_solr c:awhollynewcollection_0 s:shard4 r:core_node9 x:awhollynewcollection_0_shard4_replica_n6] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1856702 INFO (parallelCoreAdminExecutor-7121-thread-1-processing-n:127.0.0.1:42173_solr fb041d48-216d-41c9-9f39-799afedccb1f844344465948674 CREATE) [n:127.0.0.1:42173_solr c:awhollynewcollection_0 s:shard4 r:core_node9 x:awhollynewcollection_0_shard4_replica_n6] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:42173/solr/awhollynewcollection_0_shard4_replica_n6/ shard4
[junit4] 2> 1856703 INFO (searcherExecutor-7127-thread-1-processing-x:awhollynewcollection_0_shard4_replica_n6 r:core_node9 fb041d48-216d-41c9-9f39-799afedccb1f844344465948674 CREATE n:127.0.0.1:42173_solr s:shard4 c:awhollynewcollection_0) [n:127.0.0.1:42173_solr c:awhollynewcollection_0 s:shard4 r:core_node9 x:awhollynewcollection_0_shard4_replica_n6] o.a.s.c.SolrCore [awhollynewcollection_0_shard4_replica_n6] Registered new searcher Searcher@6dc10144[awhollynewcollection_0_shard4_replica_n6] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1856704 INFO (parallelCoreAdminExecutor-7117-thread-1-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344457431382 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard1 r:core_node3 x:awhollynewcollection_0_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf2
[junit4] 2> 1856729 INFO (parallelCoreAdminExecutor-7117-thread-2-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344468697888 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard5 r:core_node10 x:awhollynewcollection_0_shard5_replica_n8] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1856729 INFO (parallelCoreAdminExecutor-7117-thread-2-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344468697888 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard5 r:core_node10 x:awhollynewcollection_0_shard5_replica_n8] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1856731 INFO (parallelCoreAdminExecutor-7117-thread-2-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344468697888 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard5 r:core_node10 x:awhollynewcollection_0_shard5_replica_n8] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1856731 INFO (parallelCoreAdminExecutor-7117-thread-2-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344468697888 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard5 r:core_node10 x:awhollynewcollection_0_shard5_replica_n8] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1856732 INFO (parallelCoreAdminExecutor-7117-thread-2-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344468697888 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard5 r:core_node10 x:awhollynewcollection_0_shard5_replica_n8] o.a.s.s.SolrIndexSearcher Opening [Searcher@77f8cf31[awhollynewcollection_0_shard5_replica_n8] main]
[junit4] 2> 1856732 INFO (parallelCoreAdminExecutor-7117-thread-1-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344457431382 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard1 r:core_node3 x:awhollynewcollection_0_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1856733 INFO (parallelCoreAdminExecutor-7117-thread-2-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344468697888 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard5 r:core_node10 x:awhollynewcollection_0_shard5_replica_n8] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf2
[junit4] 2> 1856733 INFO (parallelCoreAdminExecutor-7123-thread-1-processing-n:127.0.0.1:61293_solr fb041d48-216d-41c9-9f39-799afedccb1f844344463297441 CREATE) [n:127.0.0.1:61293_solr c:awhollynewcollection_0 s:shard3 r:core_node7 x:awhollynewcollection_0_shard3_replica_n4] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1856733 INFO (parallelCoreAdminExecutor-7123-thread-1-processing-n:127.0.0.1:61293_solr fb041d48-216d-41c9-9f39-799afedccb1f844344463297441 CREATE) [n:127.0.0.1:61293_solr c:awhollynewcollection_0 s:shard3 r:core_node7 x:awhollynewcollection_0_shard3_replica_n4] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1856733 INFO (searcherExecutor-7126-thread-1-processing-x:awhollynewcollection_0_shard1_replica_n1 r:core_node3 fb041d48-216d-41c9-9f39-799afedccb1f844344457431382 CREATE n:127.0.0.1:59792_solr s:shard1 c:awhollynewcollection_0) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard1 r:core_node3 x:awhollynewcollection_0_shard1_replica_n1] o.a.s.c.SolrCore [awhollynewcollection_0_shard1_replica_n1] Registered new searcher Searcher@4057429[awhollynewcollection_0_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1856734 INFO (parallelCoreAdminExecutor-7117-thread-2-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344468697888 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard5 r:core_node10 x:awhollynewcollection_0_shard5_replica_n8] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf2
[junit4] 2> 1856734 INFO (parallelCoreAdminExecutor-7117-thread-2-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344468697888 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard5 r:core_node10 x:awhollynewcollection_0_shard5_replica_n8] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1856734 INFO (parallelCoreAdminExecutor-7123-thread-1-processing-n:127.0.0.1:61293_solr fb041d48-216d-41c9-9f39-799afedccb1f844344463297441 CREATE) [n:127.0.0.1:61293_solr c:awhollynewcollection_0 s:shard3 r:core_node7 x:awhollynewcollection_0_shard3_replica_n4] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1856734 INFO (parallelCoreAdminExecutor-7117-thread-2-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344468697888 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard5 r:core_node10 x:awhollynewcollection_0_shard5_replica_n8] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1582840829136863232
[junit4] 2> 1856734 INFO (parallelCoreAdminExecutor-7123-thread-1-processing-n:127.0.0.1:61293_solr fb041d48-216d-41c9-9f39-799afedccb1f844344463297441 CREATE) [n:127.0.0.1:61293_solr c:awhollynewcollection_0 s:shard3 r:core_node7 x:awhollynewcollection_0_shard3_replica_n4] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1856735 INFO (parallelCoreAdminExecutor-7123-thread-1-processing-n:127.0.0.1:61293_solr fb041d48-216d-41c9-9f39-799afedccb1f844344463297441 CREATE) [n:127.0.0.1:61293_solr c:awhollynewcollection_0 s:shard3 r:core_node7 x:awhollynewcollection_0_shard3_replica_n4] o.a.s.s.SolrIndexSearcher Opening [Searcher@55f1e77f[awhollynewcollection_0_shard3_replica_n4] main]
[junit4] 2> 1856736 INFO (parallelCoreAdminExecutor-7123-thread-1-processing-n:127.0.0.1:61293_solr fb041d48-216d-41c9-9f39-799afedccb1f844344463297441 CREATE) [n:127.0.0.1:61293_solr c:awhollynewcollection_0 s:shard3 r:core_node7 x:awhollynewcollection_0_shard3_replica_n4] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf2
[junit4] 2> 1856736 INFO (parallelCoreAdminExecutor-7117-thread-1-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344457431382 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard1 r:core_node3 x:awhollynewcollection_0_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1582840829138960384
[junit4] 2> 1856737 INFO (parallelCoreAdminExecutor-7123-thread-1-processing-n:127.0.0.1:61293_solr fb041d48-216d-41c9-9f39-799afedccb1f844344463297441 CREATE) [n:127.0.0.1:61293_solr c:awhollynewcollection_0 s:shard3 r:core_node7 x:awhollynewcollection_0_shard3_replica_n4] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf2
[junit4] 2> 1856738 INFO (parallelCoreAdminExecutor-7123-thread-1-processing-n:127.0.0.1:61293_solr fb041d48-216d-41c9-9f39-799afedccb1f844344463297441 CREATE) [n:127.0.0.1:61293_solr c:awhollynewcollection_0 s:shard3 r:core_node7 x:awhollynewcollection_0_shard3_replica_n4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1856738 INFO (parallelCoreAdminExecutor-7123-thread-1-processing-n:127.0.0.1:61293_solr fb041d48-216d-41c9-9f39-799afedccb1f844344463297441 CREATE) [n:127.0.0.1:61293_solr c:awhollynewcollection_0 s:shard3 r:core_node7 x:awhollynewcollection_0_shard3_replica_n4] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1582840829141057536
[junit4] 2> 1856740 INFO (searcherExecutor-7125-thread-1-processing-x:awhollynewcollection_0_shard3_replica_n4 r:core_node7 fb041d48-216d-41c9-9f39-799afedccb1f844344463297441 CREATE n:127.0.0.1:61293_solr s:shard3 c:awhollynewcollection_0) [n:127.0.0.1:61293_solr c:awhollynewcollection_0 s:shard3 r:core_node7 x:awhollynewcollection_0_shard3_replica_n4] o.a.s.c.SolrCore [awhollynewcollection_0_shard3_replica_n4] Registered new searcher Searcher@55f1e77f[awhollynewcollection_0_shard3_replica_n4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1856740 INFO (parallelCoreAdminExecutor-7117-thread-2-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344468697888 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard5 r:core_node10 x:awhollynewcollection_0_shard5_replica_n8] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1856740 INFO (parallelCoreAdminExecutor-7117-thread-2-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344468697888 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard5 r:core_node10 x:awhollynewcollection_0_shard5_replica_n8] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1856740 INFO (parallelCoreAdminExecutor-7117-thread-2-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344468697888 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard5 r:core_node10 x:awhollynewcollection_0_shard5_replica_n8] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:59792/solr/awhollynewcollection_0_shard5_replica_n8/
[junit4] 2> 1856740 INFO (parallelCoreAdminExecutor-7117-thread-2-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344468697888 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard5 r:core_node10 x:awhollynewcollection_0_shard5_replica_n8] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1856740 INFO (parallelCoreAdminExecutor-7117-thread-2-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344468697888 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard5 r:core_node10 x:awhollynewcollection_0_shard5_replica_n8] o.a.s.c.SyncStrategy http://127.0.0.1:59792/solr/awhollynewcollection_0_shard5_replica_n8/ has no replicas
[junit4] 2> 1856741 INFO (parallelCoreAdminExecutor-7117-thread-2-processing-n:127.0.0.1:59792_solr fb041d48-216d-41c9-9f39-799afedccb1f844344468697888 CREATE) [n:127.0.0.1:59792_solr c:awhollynewcollection_0 s:shard5 r:core_node10 x:awhollynewcollection_0_shard5_replica_n8] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1856743 INFO (parallelCoreAdminExecutor-7123-thread-1-processing-n:127.0.0.1:61293_solr fb041d48-216d-41c9-9f39-799afedccb1f844344463297441 CREATE) [n:127.0.0.1:61293_solr c:awhollynewcollection_0 s:shard3 r:core_node7 x:awhollynewcollection_0_shard3_replica_n4] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1856744 INFO (parallelCoreAdminExecutor-7123-thread-1-processing-n:127.0.0.1:61293_solr fb041d48-216d-41c9-9f39-799afedccb1f844344463297441 CREATE) [n:127.0.0.1:61293_solr c:awhollynewcollection_0 s:shard3 r:core_node7 x:awhollynewcollection_0_shard3_replica_n4] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1856744 INFO (parallelCoreAdminExecutor-7123-thread-1-processing-n:127.0.0.1:61293_solr fb041d48-216d-41c9-9f39-799afedccb1f844344463297441 CREATE) [n:127.0.0.1:61293_solr c:awhollynewcollection_0 s:shard3 r:core_node7 x:awhollynewcollection_0_shard3_replica_n4] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:61293/solr/awhollynewcollection_0_shard3_replica_n4/
[junit4] 2> 1856744 INFO (parallelCoreAdminExecutor-712
[...truncated too long message...]
TestCoreAdminApis, TestLegacyNumericRangeQueryBuilder, CdcrUpdateLogTest, ClassificationUpdateProcessorFactoryTest, TestTolerantUpdateProcessorCloud, TestBulkSchemaAPI, UtilsToolTest, TestShardHandlerFactory, ChaosMonkeyNothingIsSafeWithPullReplicasTest, SecurityConfHandlerTest, TestExpandComponent, HdfsBasicDistributedZkTest, TestCollectionAPIs, TestCryptoKeys, CoreMergeIndexesAdminHandlerTest, AtomicUpdateProcessorFactoryTest, TestBlendedInfixSuggestions, LukeRequestHandlerTest, ExecutePlanActionTest, RestartWhileUpdatingTest, NodeLostTriggerTest, MetricUtilsTest, CacheHeaderTest, CurrencyFieldTypeTest, HdfsTlogReplayBufferedWhileIndexingTest, SoftAutoCommitTest, TestNestedDocsSort, TestRemoteStreaming, TestClusterStateMutator, TestDistribDocBasedVersion, BlockJoinFacetRandomTest, TestFieldCollectionResource, SolrCloudExampleTest, ManagedSchemaRoundRobinCloudTest, TestSolrCloudSnapshots, HLLUtilTest, TermVectorComponentTest, SolrShardReporterTest, ConcurrentDeleteAndCreateCollectionTest, TestQueryUtils, TestFilteredDocIdSet, CoreAdminHandlerTest, UpdateRequestProcessorFactoryTest, CloudMLTQParserTest, TestConfigSetProperties, TestStressReorder, TestPostingsSolrHighlighter, PolyFieldTest, SampleTest, ZkStateWriterTest, TestSolrConfigHandler, SSLMigrationTest, SimpleFacetsTest, TestSolrCoreParser, TestPerFieldSimilarity, SystemInfoHandlerTest, MoveReplicaHDFSTest, HdfsSyncSliceTest, TestPivotHelperCode, DocValuesTest, TestNoOpRegenerator, TestSkipOverseerOperations, TestCorePropertiesReload, MetricsHandlerTest, TestSolrJ, RecoveryAfterSoftCommitTest, DirectSolrSpellCheckerTest, TestNumericRangeQuery32, FullHLLTest, TestNumericTerms64, PKIAuthenticationIntegrationTest, TestFaceting, TestPartialUpdateDeduplication, RuleEngineTest, TestDistribIDF, TestConfig, TestIntervalFaceting, TestSolrIndexConfig, ZkNodePropsTest, VersionInfoTest, TestBM25SimilarityFactory, DistribDocExpirationUpdateProcessorTest, AnalyticsQueryTest, TestRandomFaceting, TestElisionMultitermQuery, TestCSVLoader, TestRawTransformer, TestSystemIdResolver, PingRequestHandlerTest, TestRebalanceLeaders, BasicFunctionalityTest, TestPKIAuthenticationPlugin, TestSubQueryTransformer, DistributedMLTComponentTest, TestStressLucene, SystemLogListenerTest, TestSchemaNameResource, TestBadConfig, HealthCheckHandlerTest, SynonymTokenizerTest, JsonLoaderTest, QueryElevationComponentTest, OpenExchangeRatesOrgProviderTest, DocExpirationUpdateProcessorFactoryTest, BigEndianAscendingWordDeserializerTest, TestSolrQueryResponse, DistributedFacetPivotLargeTest, TestReqParamsAPI, AnalysisAfterCoreReloadTest, TestBinaryResponseWriter, TestCollectionsAPIViaSolrCloudCluster, TestTrieFacet, TestDocSet, TestFreeTextSuggestions, SpatialHeatmapFacetsTest, TestShortCircuitedRequests, IndexSchemaRuntimeFieldTest, DocValuesNotIndexedTest, OverseerRolesTest, TestAuthenticationFramework, DebugComponentTest, TestExportWriter, PeerSyncTest, SOLR749Test, BasicAuthIntegrationTest, TestPushWriter, TestLMJelinekMercerSimilarityFactory, TestCloudPivotFacet, TestAddFieldRealTimeGet, FieldMutatingUpdateProcessorTest, TestManagedResourceStorage, JvmMetricsTest, TestStandardQParsers, RequestLoggingTest, TestTrie, TestAnalyzedSuggestions, TestFieldCache, TestUseDocValuesAsStored2, TestQueryTypes, TestManagedStopFilterFactory, HdfsNNFailoverTest, TestLegacyFieldReuse, TestSerializedLuceneMatchVersion, QueryResultKeyTest, CloudExitableDirectoryReaderTest, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest, SyncSliceTest, OpenCloseCoreStressTest, BasicZkTest, FullSolrCloudDistribCmdsTest, LeaderElectionTest, TestZkChroot, ShardRoutingCustomTest, TestDistributedSearch, TestRecovery, TestRealTimeGet, TestJoin, TestStressVersions, TestRangeQuery, TestGroupingSearch, SolrCmdDistributorTest, BadIndexSchemaTest, TestSort, TestFiltering, TestIndexSearcher, ShowFileRequestHandlerTest, SolrIndexSplitterTest, SolrRequestParserTest, SuggesterTest, SuggesterWFSTTest, SchemaVersionSpecificBehaviorTest, QueryEqualityTest, DistanceFunctionTest, TestSolrDeletionPolicy1, SolrInfoBeanTest, DisMaxRequestHandlerTest, DocumentAnalysisRequestHandlerTest, TestOmitPositions, TermsComponentTest, SolrPluginUtilsTest, TestCollationField, MBeansHandlerTest, BadComponentTest, TestBinaryField, TestSearchPerf, NumericFieldsTest, MinimalSchemaTest, ExternalFileFieldSortTest, SpellPossibilityIteratorTest, EchoParamsTest, TimeZoneUtilsTest, ScriptEngineTest, PluginInfoTest, ChaosMonkeyNothingIsSafeTest, AsyncCallRequestStatusResponseTest, BaseCdcrDistributedZkTest, CdcrReplicationDistributedZkTest, CdcrReplicationHandlerTest, CdcrRequestHandlerTest, CollectionReloadTest, CollectionStateFormat2Test, CollectionTooManyReplicasTest, CustomCollectionTest, DeleteStatusTest, DistributedQueueTest, LeaderFailureAfterFreshStartTest, LeaderInitiatedRecoveryOnShardRestartTest, MigrateRouteKeyTest, MultiThreadedOCPTest, OverseerCollectionConfigSetProcessorTest, RemoteQueryErrorTest, ReplaceNodeTest, SaslZkACLProviderTest, SolrCLIZkUtilsTest, TestCloudInspectUtil, TestCloudJSONFacetJoinDomain, TestCloudSearcherWarming, TestExclusionRuleCollectionAccess, TestHdfsCloudBackupRestore, TestLeaderInitiatedRecoveryThread, TestOnReconnectListenerSupport, TestPullReplica, TestRandomFlRTGCloud, TestReplicaProperties, TestSegmentSorting, TestSolrCloudWithSecureImpersonation]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestSolrCloudWithSecureImpersonation -Dtests.seed=A40E31BF3AE97F14 -Dtests.slow=true -Dtests.locale=zh-HK -Dtests.timezone=Africa/Lagos -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J1 | TestSolrCloudWithSecureImpersonation (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
[junit4] > 1) Thread[id=26185, name=jetty-launcher-4215-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
[junit4] > at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
[junit4] > at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
[junit4] > at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
[junit4] > at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
[junit4] > 2) Thread[id=26189, name=jetty-launcher-4215-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
[junit4] > at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
[junit4] > at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
[junit4] > at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
[junit4] > at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
[junit4] > at __randomizedtesting.SeedInfo.seed([A40E31BF3AE97F14]:0)
[junit4] Completed [564/747 (2!)] on J1 in 46.02s, 12 tests, 1 error <<< FAILURES!
[...truncated 47811 lines...]
[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build #
1501 - Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1501/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseSerialGC
2 tests failed.
FAILED: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI
Error Message:
Error from server at http://127.0.0.1:43905/solr/awhollynewcollection_0_shard3_replica_n4: ClusterState says we are the leader (http://127.0.0.1:43905/solr/awhollynewcollection_0_shard3_replica_n4), but locally we don't think so. Request came from null
Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from server at http://127.0.0.1:43905/solr/awhollynewcollection_0_shard3_replica_n4: ClusterState says we are the leader (http://127.0.0.1:43905/solr/awhollynewcollection_0_shard3_replica_n4), but locally we don't think so. Request came from null
at __randomizedtesting.SeedInfo.seed([39C95B6195AFBB9F:71BC2FD5939C940A]:0)
at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:541)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1008)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:875)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:937)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:937)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:937)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:937)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:937)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:808)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:459)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:43905/solr/awhollynewcollection_0_shard3_replica_n4: ClusterState says we are the leader (http://127.0.0.1:43905/solr/awhollynewcollection_0_shard3_replica_n4), but locally we don't think so. Request came from null
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:658)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:518)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
FAILED: org.apache.solr.client.solrj.io.stream.StreamExpressionTest.testParallelCommitStream
Error Message:
expected:<5> but was:<2>
Stack Trace:
java.lang.AssertionError: expected:<5> but was:<2>
at __randomizedtesting.SeedInfo.seed([5B7B0EE40B6887BC:7B916CE497296AF0]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.client.solrj.io.stream.StreamExpressionTest.testParallelCommitStream(StreamExpressionTest.java:4594)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12791 lines...]
[junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
[junit4] 2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_39C95B6195AFBB9F-001/init-core-data-001
[junit4] 2> 2294164 WARN (SUITE-CollectionsAPIDistributedZkTest-seed#[39C95B6195AFBB9F]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=53 numCloses=53
[junit4] 2> 2294164 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[39C95B6195AFBB9F]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 2294165 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[39C95B6195AFBB9F]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 2294165 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[39C95B6195AFBB9F]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 2294166 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[39C95B6195AFBB9F]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_39C95B6195AFBB9F-001/tempDir-001
[junit4] 2> 2294166 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[39C95B6195AFBB9F]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2294166 INFO (Thread-6745) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2294166 INFO (Thread-6745) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2294167 ERROR (Thread-6745) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2294266 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[39C95B6195AFBB9F]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:56086
[junit4] 2> 2294272 INFO (jetty-launcher-3106-thread-1) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 2294273 INFO (jetty-launcher-3106-thread-3) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 2294273 INFO (jetty-launcher-3106-thread-4) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 2294273 INFO (jetty-launcher-3106-thread-2) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 2294275 INFO (jetty-launcher-3106-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@72608f71{/solr,null,AVAILABLE}
[junit4] 2> 2294275 INFO (jetty-launcher-3106-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@41982076{/solr,null,AVAILABLE}
[junit4] 2> 2294277 INFO (jetty-launcher-3106-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5b74ea50{HTTP/1.1,[http/1.1]}{127.0.0.1:54423}
[junit4] 2> 2294277 INFO (jetty-launcher-3106-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@d1502f1{HTTP/1.1,[http/1.1]}{127.0.0.1:62610}
[junit4] 2> 2294277 INFO (jetty-launcher-3106-thread-3) [ ] o.e.j.s.Server Started @2296371ms
[junit4] 2> 2294277 INFO (jetty-launcher-3106-thread-1) [ ] o.e.j.s.Server Started @2296371ms
[junit4] 2> 2294277 INFO (jetty-launcher-3106-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=54423}
[junit4] 2> 2294277 INFO (jetty-launcher-3106-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=62610}
[junit4] 2> 2294277 INFO (jetty-launcher-3106-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@a8974a7{/solr,null,AVAILABLE}
[junit4] 2> 2294277 ERROR (jetty-launcher-3106-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2294277 INFO (jetty-launcher-3106-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 2294277 INFO (jetty-launcher-3106-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@65b22f9a{HTTP/1.1,[http/1.1]}{127.0.0.1:43905}
[junit4] 2> 2294277 INFO (jetty-launcher-3106-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2294277 INFO (jetty-launcher-3106-thread-4) [ ] o.e.j.s.Server Started @2296371ms
[junit4] 2> 2294277 INFO (jetty-launcher-3106-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2294277 INFO (jetty-launcher-3106-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43905}
[junit4] 2> 2294277 INFO (jetty-launcher-3106-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-10-31T19:59:25.671Z
[junit4] 2> 2294277 ERROR (jetty-launcher-3106-thread-4) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2294278 INFO (jetty-launcher-3106-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 2294278 INFO (jetty-launcher-3106-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2294277 ERROR (jetty-launcher-3106-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2294278 INFO (jetty-launcher-3106-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 2294278 INFO (jetty-launcher-3106-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2294278 INFO (jetty-launcher-3106-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2294278 INFO (jetty-launcher-3106-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-10-31T19:59:25.672Z
[junit4] 2> 2294278 INFO (jetty-launcher-3106-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2294278 INFO (jetty-launcher-3106-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-10-31T19:59:25.672Z
[junit4] 2> 2294279 INFO (jetty-launcher-3106-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6fad596a{/solr,null,AVAILABLE}
[junit4] 2> 2294279 INFO (jetty-launcher-3106-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4506312b{HTTP/1.1,[http/1.1]}{127.0.0.1:60697}
[junit4] 2> 2294279 INFO (jetty-launcher-3106-thread-2) [ ] o.e.j.s.Server Started @2296374ms
[junit4] 2> 2294279 INFO (jetty-launcher-3106-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=60697}
[junit4] 2> 2294280 ERROR (jetty-launcher-3106-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2294280 INFO (jetty-launcher-3106-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 2294280 INFO (jetty-launcher-3106-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2294280 INFO (jetty-launcher-3106-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2294280 INFO (jetty-launcher-3106-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-10-31T19:59:25.674Z
[junit4] 2> 2294281 INFO (jetty-launcher-3106-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2294281 INFO (jetty-launcher-3106-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2294281 INFO (jetty-launcher-3106-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2294283 INFO (jetty-launcher-3106-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2294284 INFO (jetty-launcher-3106-thread-3) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 2294284 INFO (jetty-launcher-3106-thread-3) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 2294285 INFO (jetty-launcher-3106-thread-2) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 2294285 INFO (jetty-launcher-3106-thread-2) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 2294285 INFO (jetty-launcher-3106-thread-1) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 2294285 INFO (jetty-launcher-3106-thread-1) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 2294287 INFO (jetty-launcher-3106-thread-4) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 2294287 INFO (jetty-launcher-3106-thread-4) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 2294289 INFO (jetty-launcher-3106-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56086/solr
[junit4] 2> 2294289 INFO (jetty-launcher-3106-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56086/solr
[junit4] 2> 2294289 INFO (jetty-launcher-3106-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56086/solr
[junit4] 2> 2294292 INFO (jetty-launcher-3106-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56086/solr
[junit4] 2> 2294532 INFO (jetty-launcher-3106-thread-2) [n:127.0.0.1:60697_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2294534 INFO (jetty-launcher-3106-thread-2) [n:127.0.0.1:60697_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:60697_solr
[junit4] 2> 2294534 INFO (jetty-launcher-3106-thread-3) [n:127.0.0.1:54423_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2294535 INFO (jetty-launcher-3106-thread-1) [n:127.0.0.1:62610_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2294537 INFO (jetty-launcher-3106-thread-2) [n:127.0.0.1:60697_solr ] o.a.s.c.Overseer Overseer (id=98925279023005705-127.0.0.1:60697_solr-n_0000000000) starting
[junit4] 2> 2294537 INFO (jetty-launcher-3106-thread-3) [n:127.0.0.1:54423_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54423_solr
[junit4] 2> 2294538 INFO (jetty-launcher-3106-thread-1) [n:127.0.0.1:62610_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:62610_solr
[junit4] 2> 2294539 INFO (zkCallback-3126-thread-1-processing-n:127.0.0.1:62610_solr) [n:127.0.0.1:62610_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2294540 INFO (zkCallback-3127-thread-1-processing-n:127.0.0.1:54423_solr) [n:127.0.0.1:54423_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2294541 INFO (zkCallback-3128-thread-1-processing-n:127.0.0.1:43905_solr) [n:127.0.0.1:43905_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2294542 INFO (zkCallback-3125-thread-1-processing-n:127.0.0.1:60697_solr) [n:127.0.0.1:60697_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2294543 INFO (zkCallback-3126-thread-1-processing-n:127.0.0.1:62610_solr) [n:127.0.0.1:62610_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2294543 INFO (jetty-launcher-3106-thread-4) [n:127.0.0.1:43905_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2294543 INFO (zkCallback-3127-thread-2-processing-n:127.0.0.1:54423_solr) [n:127.0.0.1:54423_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2294547 INFO (jetty-launcher-3106-thread-4) [n:127.0.0.1:43905_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43905_solr
[junit4] 2> 2294550 INFO (zkCallback-3128-thread-1-processing-n:127.0.0.1:43905_solr) [n:127.0.0.1:43905_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2294560 INFO (zkCallback-3127-thread-2-processing-n:127.0.0.1:54423_solr) [n:127.0.0.1:54423_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2294560 INFO (zkCallback-3125-thread-1-processing-n:127.0.0.1:60697_solr) [n:127.0.0.1:60697_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2294561 INFO (zkCallback-3126-thread-1-processing-n:127.0.0.1:62610_solr) [n:127.0.0.1:62610_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2294630 INFO (jetty-launcher-3106-thread-4) [n:127.0.0.1:43905_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@60102d0c
[junit4] 2> 2294632 INFO (jetty-launcher-3106-thread-2) [n:127.0.0.1:60697_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60697_solr
[junit4] 2> 2294634 INFO (zkCallback-3127-thread-2-processing-n:127.0.0.1:54423_solr) [n:127.0.0.1:54423_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2294634 INFO (zkCallback-3128-thread-1-processing-n:127.0.0.1:43905_solr) [n:127.0.0.1:43905_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2294637 INFO (jetty-launcher-3106-thread-4) [n:127.0.0.1:43905_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@60102d0c
[junit4] 2> 2294637 INFO (jetty-launcher-3106-thread-4) [n:127.0.0.1:43905_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@60102d0c
[junit4] 2> 2294638 INFO (jetty-launcher-3106-thread-4) [n:127.0.0.1:43905_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_39C95B6195AFBB9F-001/tempDir-001/node4
[junit4] 2> 2294643 INFO (jetty-launcher-3106-thread-3) [n:127.0.0.1:54423_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@60102d0c
[junit4] 2> 2294650 INFO (jetty-launcher-3106-thread-3) [n:127.0.0.1:54423_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@60102d0c
[junit4] 2> 2294651 INFO (jetty-launcher-3106-thread-3) [n:127.0.0.1:54423_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@60102d0c
[junit4] 2> 2294653 INFO (jetty-launcher-3106-thread-3) [n:127.0.0.1:54423_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_39C95B6195AFBB9F-001/tempDir-001/node3
[junit4] 2> 2294655 INFO (zkCallback-3125-thread-1-processing-n:127.0.0.1:60697_solr) [n:127.0.0.1:60697_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2294677 INFO (jetty-launcher-3106-thread-1) [n:127.0.0.1:62610_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@60102d0c
[junit4] 2> 2294683 INFO (zkCallback-3126-thread-1-processing-n:127.0.0.1:62610_solr) [n:127.0.0.1:62610_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2294685 INFO (jetty-launcher-3106-thread-1) [n:127.0.0.1:62610_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@60102d0c
[junit4] 2> 2294685 INFO (jetty-launcher-3106-thread-1) [n:127.0.0.1:62610_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@60102d0c
[junit4] 2> 2294686 INFO (jetty-launcher-3106-thread-1) [n:127.0.0.1:62610_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_39C95B6195AFBB9F-001/tempDir-001/node1
[junit4] 2> 2294778 INFO (jetty-launcher-3106-thread-2) [n:127.0.0.1:60697_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@60102d0c
[junit4] 2> 2294785 INFO (jetty-launcher-3106-thread-2) [n:127.0.0.1:60697_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@60102d0c
[junit4] 2> 2294785 INFO (jetty-launcher-3106-thread-2) [n:127.0.0.1:60697_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@60102d0c
[junit4] 2> 2294786 INFO (jetty-launcher-3106-thread-2) [n:127.0.0.1:60697_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_39C95B6195AFBB9F-001/tempDir-001/node2
[junit4] 2> 2294821 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[39C95B6195AFBB9F]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 2294823 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[39C95B6195AFBB9F]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:56086/solr ready
[junit4] 2> 2294864 INFO (TEST-CollectionsAPIDistributedZkTest.deleteCollectionOnlyInZk-seed#[39C95B6195AFBB9F]) [ ] o.a.s.SolrTestCaseJ4 ###Starting deleteCollectionOnlyInZk
[junit4] 2> 2294866 INFO (TEST-CollectionsAPIDistributedZkTest.deleteCollectionOnlyInZk-seed#[39C95B6195AFBB9F]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 2294873 INFO (qtp425568925-20948) [n:127.0.0.1:62610_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=onlyinzk&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2294882 INFO (qtp425568925-20948) [n:127.0.0.1:62610_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=onlyinzk&action=DELETE&wt=javabin&version=2} status=0 QTime=8
[junit4] 2> 2294882 INFO (qtp425568925-20949) [n:127.0.0.1:62610_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2294882 INFO (qtp425568925-20949) [n:127.0.0.1:62610_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2294883 INFO (qtp425568925-20947) [n:127.0.0.1:62610_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&name=onlyinzk&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2294886 INFO (OverseerThreadFactory-8392-thread-2-processing-n:127.0.0.1:60697_solr) [n:127.0.0.1:60697_solr ] o.a.s.c.CreateCollectionCmd Create collection onlyinzk
[junit4] 2> 2294887 INFO (OverseerCollectionConfigSetProcessor-98925279023005705-127.0.0.1:60697_solr-n_0000000000) [n:127.0.0.1:60697_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2295000 INFO (OverseerStateUpdate-98925279023005705-127.0.0.1:60697_solr-n_0000000000) [n:127.0.0.1:60697_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"onlyinzk",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"onlyinzk_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:60697/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2295004 INFO (OverseerStateUpdate-98925279023005705-127.0.0.1:60697_solr-n_0000000000) [n:127.0.0.1:60697_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"onlyinzk",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"onlyinzk_shard2_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:54423/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2295214 INFO (qtp270024657-20955) [n:127.0.0.1:60697_solr ] o.a.s.u.TestInjection Inject random core creation delay of 2s
[junit4] 2> 2295215 INFO (qtp1892567940-20957) [n:127.0.0.1:54423_solr ] o.a.s.u.TestInjection Inject random core creation delay of 2s
[junit4] 2> 2297215 INFO (qtp1892567940-20957) [n:127.0.0.1:54423_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=onlyinzk_shard2_replica_n2&action=CREATE&numShards=2&collection=onlyinzk&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2297215 INFO (qtp1892567940-20957) [n:127.0.0.1:54423_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2297220 INFO (qtp270024657-20955) [n:127.0.0.1:60697_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=onlyinzk_shard1_replica_n1&action=CREATE&numShards=2&collection=onlyinzk&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2297220 INFO (qtp270024657-20955) [n:127.0.0.1:60697_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2297223 INFO (zkCallback-3127-thread-2-processing-n:127.0.0.1:54423_solr) [n:127.0.0.1:54423_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2297223 INFO (zkCallback-3127-thread-1-processing-n:127.0.0.1:54423_solr) [n:127.0.0.1:54423_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2297329 INFO (zkCallback-3125-thread-1-processing-n:127.0.0.1:60697_solr) [n:127.0.0.1:60697_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2297329 INFO (zkCallback-3127-thread-1-processing-n:127.0.0.1:54423_solr) [n:127.0.0.1:54423_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2297329 INFO (zkCallback-3127-thread-2-processing-n:127.0.0.1:54423_solr) [n:127.0.0.1:54423_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2298235 INFO (qtp1892567940-20957) [n:127.0.0.1:54423_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2298237 INFO (qtp270024657-20955) [n:127.0.0.1:60697_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2298241 INFO (qtp1892567940-20957) [n:127.0.0.1:54423_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.s.IndexSchema [onlyinzk_shard2_replica_n2] Schema name=minimal
[junit4] 2> 2298243 INFO (qtp1892567940-20957) [n:127.0.0.1:54423_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2298243 INFO (qtp1892567940-20957) [n:127.0.0.1:54423_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'onlyinzk_shard2_replica_n2' using configuration from collection onlyinzk, trusted=true
[junit4] 2> 2298243 INFO (qtp270024657-20955) [n:127.0.0.1:60697_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.s.IndexSchema [onlyinzk_shard1_replica_n1] Schema name=minimal
[junit4] 2> 2298243 INFO (qtp1892567940-20957) [n:127.0.0.1:54423_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.onlyinzk.shard2.replica_n2' (registry 'solr.core.onlyinzk.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@60102d0c
[junit4] 2> 2298244 INFO (qtp1892567940-20957) [n:127.0.0.1:54423_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2298244 INFO (qtp1892567940-20957) [n:127.0.0.1:54423_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.c.SolrCore [[onlyinzk_shard2_replica_n2] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_39C95B6195AFBB9F-001/tempDir-001/node3/onlyinzk_shard2_replica_n2], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_39C95B6195AFBB9F-001/tempDir-001/node3/onlyinzk_shard2_replica_n2/data/]
[junit4] 2> 2298245 INFO (qtp270024657-20955) [n:127.0.0.1:60697_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2298245 INFO (qtp270024657-20955) [n:127.0.0.1:60697_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'onlyinzk_shard1_replica_n1' using configuration from collection onlyinzk, trusted=true
[junit4] 2> 2298245 INFO (qtp270024657-20955) [n:127.0.0.1:60697_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.onlyinzk.shard1.replica_n1' (registry 'solr.core.onlyinzk.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@60102d0c
[junit4] 2> 2298245 INFO (qtp270024657-20955) [n:127.0.0.1:60697_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2298246 INFO (qtp270024657-20955) [n:127.0.0.1:60697_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.c.SolrCore [[onlyinzk_shard1_replica_n1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_39C95B6195AFBB9F-001/tempDir-001/node2/onlyinzk_shard1_replica_n1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_39C95B6195AFBB9F-001/tempDir-001/node2/onlyinzk_shard1_replica_n1/data/]
[junit4] 2> 2298318 INFO (qtp270024657-20955) [n:127.0.0.1:60697_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2298318 INFO (qtp270024657-20955) [n:127.0.0.1:60697_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2298319 INFO (qtp1892567940-20957) [n:127.0.0.1:54423_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2298319 INFO (qtp1892567940-20957) [n:127.0.0.1:54423_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2298319 INFO (qtp270024657-20955) [n:127.0.0.1:60697_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2298319 INFO (qtp270024657-20955) [n:127.0.0.1:60697_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2298320 INFO (qtp1892567940-20957) [n:127.0.0.1:54423_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2298320 INFO (qtp1892567940-20957) [n:127.0.0.1:54423_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2298320 INFO (qtp270024657-20955) [n:127.0.0.1:60697_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@24b81759[onlyinzk_shard1_replica_n1] main]
[junit4] 2> 2298321 INFO (qtp1892567940-20957) [n:127.0.0.1:54423_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@5e10a005[onlyinzk_shard2_replica_n2] main]
[junit4] 2> 2298322 INFO (qtp270024657-20955) [n:127.0.0.1:60697_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 2298323 INFO (qtp1892567940-20957) [n:127.0.0.1:54423_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 2298323 INFO (qtp270024657-20955) [n:127.0.0.1:60697_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 2298323 INFO (qtp1892567940-20957) [n:127.0.0.1:54423_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 2298324 INFO (qtp270024657-20955) [n:127.0.0.1:60697_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2298324 INFO (qtp270024657-20955) [n:127.0.0.1:60697_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1582804468727021568
[junit4] 2> 2298324 INFO (qtp1892567940-20957) [n:127.0.0.1:54423_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2298324 INFO (qtp1892567940-20957) [n:127.0.0.1:54423_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1582804468727021568
[junit4] 2> 2298325 INFO (searcherExecutor-8400-thread-1-processing-n:127.0.0.1:60697_solr x:onlyinzk_shard1_replica_n1 s:shard1 c:onlyinzk r:core_node3) [n:127.0.0.1:60697_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.c.SolrCore [onlyinzk_shard1_replica_n1] Registered new searcher Searcher@24b81759[onlyinzk_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2298325 INFO (searcherExecutor-8399-thread-1-processing-n:127.0.0.1:54423_solr x:onlyinzk_shard2_replica_n2 s:shard2 c:onlyinzk r:core_node4) [n:127.0.0.1:54423_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.c.SolrCore [onlyinzk_shard2_replica_n2] Registered new searcher Searcher@5e10a005[onlyinzk_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2298332 INFO (qtp1892567940-20957) [n:127.0.0.1:54423_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2298332 INFO (qtp1892567940-20957) [n:127.0.0.1:54423_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2298332 INFO (qtp1892567940-20957) [n:127.0.0.1:54423_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:54423/solr/onlyinzk_shard2_replica_n2/
[junit4] 2> 2298332 INFO (qtp1892567940-20957) [n:127.0.0.1:54423_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 2298332 INFO (qtp1892567940-20957) [n:127.0.0.1:54423_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.c.SyncStrategy http://127.0.0.1:54423/solr/onlyinzk_shard2_replica_n2/ has no replicas
[junit4] 2> 2298332 INFO (qtp270024657-20955) [n:127.0.0.1:60697_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2298332 INFO (qtp270024657-20955) [n:127.0.0.1:60697_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2298332 INFO (qtp1892567940-20957) [n:127.0.0.1:54423_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 2298332 INFO (qtp270024657-20955) [n:127.0.0.1:60697_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:60697/solr/onlyinzk_shard1_replica_n1/
[junit4] 2> 2298332 INFO (qtp270024657-20955) [n:127.0.0.1:60697_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 2298332 INFO (qtp270024657-20955) [n:127.0.0.1:60697_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:60697/solr/onlyinzk_shard1_replica_n1/ has no replicas
[junit4] 2> 2298333 INFO (qtp270024657-20955) [n:127.0.0.1:60697_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 2298339 INFO (qtp1892567940-20957) [n:127.0.0.1:54423_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:54423/solr/onlyinzk_shard2_replica_n2/ shard2
[junit4] 2> 2298341 INFO (qtp270024657-20955) [n:127.0.0.1:60697_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:60697/solr/onlyinzk_shard1_replica_n1/ shard1
[junit4] 2> 2298443 INFO (zkCallback-3125-thread-1-processing-n:127.0.0.1:60697_solr) [n:127.0.0.1:60697_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2298443 INFO (zkCallback-3127-thread-1-processing-n:127.0.0.1:54423_solr) [n:127.0.0.1:54423_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2298443 INFO (zkCallback-3127-thread-2-processing-n:127.0.0.1:54423_solr) [n:127.0.0.1:54423_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2298446 INFO (qtp270024657-20955) [n:127.0.0.1:60697_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2298448 INFO (qtp270024657-20955) [n:127.0.0.1:60697_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=onlyinzk_shard1_replica_n1&action=CREATE&numShards=2&collection=onlyinzk&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=3234
[junit4] 2> 2298491 INFO (qtp1892567940-20957) [n:127.0.0.1:54423_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2298494 INFO (qtp1892567940-20957) [n:127.0.0.1:54423_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=onlyinzk_shard2_replica_n2&action=CREATE&numShards=2&collection=onlyinzk&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=3279
[junit4] 2> 2298496 INFO (qtp425568925-20947) [n:127.0.0.1:62610_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 2298595 INFO (zkCallback-3127-thread-2-processing-n:127.0.0.1:54423_solr) [n:127.0.0.1:54423_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2298595 INFO (zkCallback-3125-thread-1-processing-n:127.0.0.1:60697_solr) [n:127.0.0.1:60697_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2298595 INFO (zkCallback-3127-thread-1-processing-n:127.0.0.1:54423_solr) [n:127.0.0.1:54423_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2298892 INFO (OverseerCollectionConfigSetProcessor-98925279023005705-127.0.0.1:60697_solr-n_0000000000) [n:127.0.0.1:60697_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2299501 INFO (qtp425568925-20947) [n:127.0.0.1:62610_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf&name=onlyinzk&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=4618
[junit4] 2> 2299501 INFO (qtp425568925-20951) [n:127.0.0.1:62610_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2299502 INFO (qtp425568925-20951) [n:127.0.0.1:62610_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2299502 INFO (TEST-CollectionsAPIDistributedZkTest.deleteCollectionOnlyInZk-seed#[39C95B6195AFBB9F]) [ ] o.a.s.SolrTestCaseJ4 ###Ending deleteCollectionOnlyInZk
[junit4] 2> 2299537 INFO (TEST-CollectionsAPIDistributedZkTest.testCreateShouldFailOnExistingCore-seed#[39C95B6195AFBB9F]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCreateShouldFailOnExistingCore
[junit4] 2> 2299541 INFO (TEST-CollectionsAPIDistributedZkTest.testCreateShouldFailOnExistingCore-seed#[39C95B6195AFBB9F]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 2299543 INFO (qtp425568925-20952) [n:127.0.0.1:62610_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=onlyinzk&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2299555 INFO (OverseerThreadFactory-8392-thread-3-processing-n:127.0.0.1:60697_solr) [n:127.0.0.1:60697_solr ] o.a.s.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
[junit4] 2> 2299556 INFO (qtp270024657-20967) [n:127.0.0.1:60697_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.onlyinzk.shard1.replica_n1, tag=null
[junit4] 2> 2299556 INFO (qtp1892567940-20958) [n:127.0.0.1:54423_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.onlyinzk.shard2.replica_n2, tag=null
[junit4] 2> 2299556 INFO (qtp270024657-20967) [n:127.0.0.1:60697_solr ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@959ce22: rootName = null, domain = solr.core.onlyinzk.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.onlyinzk.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@55715ed
[junit4] 2> 2299556 INFO (qtp1892567940-20958) [n:127.0.0.1:54423_solr ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1d02609f: rootName = null, domain = solr.core.onlyinzk.shard2.replica_n2, service url = null, agent id = null] for registry solr.core.onlyinzk.shard2.replica_n2 / com.codahale.metrics.MetricRegistry@48a950e4
[junit4] 2> 2299570 INFO (qtp1892567940-20958) [n:127.0.0.1:54423_solr ] o.a.s.c.SolrCore [onlyinzk_shard2_replica_n2] CLOSING SolrCore org.apache.solr.core.SolrCore@6256c314
[junit4] 2> 2299570 INFO (qtp1892567940-20958) [n:127.0.0.1:54423_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.onlyinzk.shard2.replica_n2, tag=1649853204
[junit4] 2> 2299570 INFO (qtp1892567940-20958) [n:127.0.0.1:54423_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.onlyinzk.shard2.leader, tag=1649853204
[junit4] 2> 2299570 INFO (qtp270024657-20967) [n:127.0.0.1:60697_solr ] o.a.s.c.SolrCore [onlyinzk_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@5b0d1628
[junit4] 2> 2299570 INFO (qtp270024657-20967) [n:127.0.0.1:60697_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.onlyinzk.shard1.replica_n1, tag=1527584296
[junit4] 2> 2299570 INFO (qtp270024657-20967) [n:127.0.0.1:60697_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.onlyinzk.shard1.leader, tag=1527584296
[junit4] 2> 2299573 INFO (qtp270024657-20967) [n:127.0.0.1:60697_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=onlyinzk_shard1_replica_n1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=17
[junit4] 2> 2299573 INFO (qtp1892567940-20958) [n:127.0.0.1:54423_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=onlyinzk_shard2_replica_n2&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=17
[junit4] 2> 2300279 INFO (qtp425568925-20952) [n:127.0.0.1:62610_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=onlyinzk&action=DELETE&wt=javabin&version=2} status=0 QTime=736
[junit4] 2> 2300279 INFO (qtp425568925-20963) [n:127.0.0.1:62610_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&name=halfcollectionblocker&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2300283 INFO (OverseerThreadFactory-8392-thread-4-processing-n:127.0.0.1:60697_solr) [n:127.0.0.1:60697_solr ] o.a.s.c.CreateCollectionCmd Create collection halfcollectionblocker
[junit4] 2> 2300283 INFO (OverseerCollectionConfigSetProcessor-98925279023005705-127.0.0.1:60697_solr-n_0000000000) [n:127.0.0.1:60697_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2300283 WARN (OverseerThreadFactory-8392-thread-4-processing-n:127.0.0.1:60697_solr) [n:127.0.0.1:60697_solr ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (halfcollectionblocker) without cores.
[junit4] 2> 2300497 INFO (qtp425568925-20963) [n:127.0.0.1:62610_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 2300498 INFO (qtp425568925-20963) [n:127.0.0.1:62610_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf&name=halfcollectionblocker&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&wt=javabin&version=2} status=0 QTime=218
[junit4] 2> 2300498 INFO (qtp425568925-20945) [n:127.0.0.1:62610_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:43905_solr&name=halfcollection_shard1_replica_n1&action=ADDREPLICA&collection=halfcollectionblocker&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2300502 INFO (OverseerCollectionConfigSetProcessor-98925279023005705-127.0.0.1:60697_solr-n_0000000000) [n:127.0.0.1:60697_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2300503 INFO (OverseerThreadFactory-8392-thread-5-processing-n:127.0.0.1:60697_solr) [n:127.0.0.1:60697_solr ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:43905_solr for creating new replica
[junit4] 2> 2300506 INFO (OverseerStateUpdate-98925279023005705-127.0.0.1:60697_solr-n_0000000000) [n:127.0.0.1:60697_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"addreplica",
[junit4] 2> "collection":"halfcollectionblocker",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"halfcollection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:43905/solr",
[junit4] 2> "node_name":"127.0.0.1:43905_solr",
[junit4] 2> "type":"NRT"}
[junit4] 2> 2300716 INFO (qtp871521207-20972) [n:127.0.0.1:43905_solr ] o.a.s.u.TestInjection Inject random core creation delay of 2s
[junit4] 2> 2302718 INFO (qtp871521207-20972) [n:127.0.0.1:43905_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node21&collection.configName=conf&name=halfcollection_shard1_replica_n1&action=CREATE&collection=halfcollectionblocker&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2302719 INFO (qtp871521207-20972) [n:127.0.0.1:43905_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2302727 INFO (zkCallback-3128-thread-1-processing-n:127.0.0.1:43905_solr) [n:127.0.0.1:43905_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfcollectionblocker/state.json] for collection [halfcollectionblocker] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2302727 INFO (zkCallback-3128-thread-2-processing-n:127.0.0.1:43905_solr) [n:127.0.0.1:43905_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfcollectionblocker/state.json] for collection [halfcollectionblocker] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2302734 INFO (qtp871521207-20972) [n:127.0.0.1:43905_solr c:halfcollectionblocker s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2302738 INFO (qtp871521207-20972) [n:127.0.0.1:43905_solr c:halfcollectionblocker s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.s.IndexSchema [halfcollection_shard1_replica_n1] Schema name=minimal
[junit4] 2> 2302739 INFO (qtp871521207-20972) [n:127.0.0.1:43905_solr c:halfcollectionblocker s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2302739 INFO (qtp871521207-20972) [n:127.0.0.1:43905_solr c:halfcollectionblocker s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'halfcollection_shard1_replica_n1' using configuration from collection halfcollectionblocker, trusted=true
[junit4] 2> 2302740 INFO (qtp871521207-20972) [n:127.0.0.1:43905_solr c:halfcollectionblocker s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.halfcollectionblocker.shard1.core_node21' (registry 'solr.core.halfcollectionblocker.shard1.core_node21') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@60102d0c
[junit4] 2> 2302740 INFO (qtp871521207-20972) [n:127.0.0.1:43905_solr c:halfcollectionblocker s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2302740 INFO (qtp871521207-20972) [n:127.0.0.1:43905_solr c:halfcollectionblocker s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.c.SolrCore [[halfcollection_shard1_replica_n1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_39C95B6195AFBB9F-001/tempDir-001/node4/halfcollection_shard1_replica_n1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_39C95B6195AFBB9F-001/tempDir-001/node4/halfcollection_shard1_replica_n1/data/]
[junit4] 2> 2302781 INFO (qtp871521207-20972) [n:127.0.0.1:43905_solr c:halfcollectionblocker s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2302781 INFO (qtp871521207-20972) [n:127.0.0.1:43905_solr c:halfcollectionblocker s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2302782 INFO (qtp871521207-20972) [n:127.0.0.1:43905_solr c:halfcollectionblocker s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2302782 INFO (qtp871521207-20972) [n:127.0.0.1:43905_solr c:halfcollectionblocker s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2302783 INFO (qtp871521207-20972) [n:127.0.0.1:43905_solr c:halfcollectionblocker s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@417543f2[halfcollection_shard1_replica_n1] main]
[junit4] 2> 2302784 INFO (qtp871521207-20972) [n:127.0.0.1:43905_solr c:halfcollectionblocker s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 2302784 INFO (qtp871521207-20972) [n:127.0.0.1:43905_solr c:halfcollectionblocker s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 2302785 INFO (qtp871521207-20972) [n:127.0.0.1:43905_solr c:halfcollectionblocker s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2302785 INFO (searcherExecutor-8409-thread-1-processing-n:127.0.0.1:43905_solr x:halfcollection_shard1_replica_n1 s:shard1 c:halfcollectionblocker r:core_node21) [n:127.0.0.1:43905_solr c:halfcollectionblocker s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.c.SolrCore [halfcollection_shard1_replica_n1] Registered new searcher Searcher@417543f2[halfcollection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2302785 INFO (qtp871521207-20972) [n:127.0.0.1:43905_solr c:halfcollectionblocker s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1582804473404719104
[junit4] 2> 2302791 INFO (qtp871521207-20972) [n:127.0.0.1:43905_solr c:halfcollectionblocker s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2302791 INFO (qtp871521207-20972) [n:127.0.0.1:43905_solr c:halfcollectionblocker s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2302791 INFO (qtp871521207-20972) [n:127.0.0.1:43905_solr c:halfcollectionblocker s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:43905/solr/halfcollection_shard1_replica_n1/
[junit4] 2> 2302791 INFO (qtp871521207-20972) [n:127.0.0.1:43905_solr c:halfcollectionblocker s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 2302791 INFO (qtp871521207-20972) [n:127.0.0.1:43905_solr c:halfcollectionblocker s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:43905/solr/halfcollection_shard1_replica_n1/ has no replicas
[junit4] 2> 2302791 INFO (qtp871521207-20972) [n:127.0.0.1:43905_solr c:halfcollectionblocker s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 2302795 INFO (qtp871521207-20972) [n:127.0.0.1:43905_solr c:halfcollectionblocker s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:43905/solr/halfcollection_shard1_replica_n1/ shard1
[junit4] 2> 2302898 INFO (zkCallback-3128-thread-1-processing-n:127.0.0.1:43905_solr) [n:127.0.0.1:43905_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfcollectionblocker/state.json] for collection [halfcollectionblocker] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2302898 INFO (zkCallback-3128-thread-2-processing-n:127.0.0.1:43905_solr) [n:127.0.0.1:43905_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfcollectionblocker/state.json] for collection [halfcollectionblocker] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2302946 INFO (qtp871521207-20972) [n:127.0.0.1:43905_solr c:halfcollectionblocker s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2302949 INFO (qtp871521207-20972) [n:127.0.0.1:43905_solr c:halfcollectionblocker s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node21&collection.configName=conf&name=halfcollection_shard1_replica_n1&action=CREATE&collection=halfcollectionblocker&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2233
[junit4] 2> 2302955 INFO (qtp425568925-20945) [n:127.0.0.1:62610_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:43905_solr&name=halfcollection_shard1_replica_n1&action=ADDREPLICA&collection=halfcollectionblocker&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=2456
[junit4] 2> 2302955 INFO (qtp425568925-20948) [n:127.0.0.1:62610_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&name=halfcollectionblocker2&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2302960 INFO (OverseerThreadFactory-8392-thread-5-processing-n:127.0.0.1:60697_solr) [n:127.0.0.1:60697_solr ] o.a.s.c.CreateCollectionCmd Create collection halfcollectionblocker2
[junit4] 2> 2302961 INFO (OverseerCollectionConfigSetProcessor-98925279023005705-127.0.0.1:60697_solr-n_0000000000) [n:127.0.0.1:60697_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000008 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2302962 WARN (OverseerThreadFactory-8392-thread-5-processing-n:127.0.0.1:60697_solr) [n:127.0.0.1:60697_solr ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (halfcollectionblocker2) without cores.
[junit4] 2> 2303076 INFO (zkCallback-3128-thread-2-processing-n:127.0.0.1:43905_solr) [n:127.0.0.1:43905_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfcollectionblocker/state.json] for collection [halfcollectionblocker] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2303076 INFO (zkCallback-3128-thread-1-processing-n:127.0.0.1:43905_solr) [n:127.0.0.1:43905_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfcollectionblocker/state.json] for collection [halfcollectionblocker] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2303181 INFO (qtp425568925-20948) [n:127.0.0.1:62610_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 2303182 INFO (qtp425568925-20948) [n:127.0.0.1:62610_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf&name=halfcollectionblocker2&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&wt=javabin&version=2} status=0 QTime=226
[junit4] 2> 2303182 INFO (qtp425568925-20949) [n:127.0.0.1:62610_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:54423_solr&name=halfcollection_shard1_replica_n1&action=ADDREPLICA&collection=halfcollectionblocker2&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2303189 INFO (OverseerCollectionConfigSetProcessor-98925279023005705-127.0.0.1:60697_solr-n_0000000000) [n:127.0.0.1:60697_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000010 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2303192 INFO (OverseerThreadFactory-8392-thread-5-processing-n:127.0.0.1:60697_solr) [n:127.0.0.1:60697_solr ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:54423_solr for creating new replica
[junit4] 2> 2303196 INFO (OverseerStateUpdate-98925279023005705-127.0.0.1:60697_solr-n_0000000000) [n:127.0.0.1:60697_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"addreplica",
[junit4] 2> "collection":"halfcollectionblocker2",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"halfcollection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:54423/solr",
[junit4] 2> "node_name":"127.0.0.1:54423_solr",
[junit4] 2> "type":"NRT"}
[junit4] 2> 2303411 INFO (qtp1892567940-20959) [n:127.0.0.1:54423_solr ] o.a.s.u.TestInjection Inject random core creation delay of 2s
[junit4] 2> 2305412 INFO (qtp1892567940-20959) [n:127.0.0.1:54423_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node21&collection.configName=conf&name=halfcollection_shard1_replica_n1&action=CREATE&collection=halfcollectionblocker2&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2305421 INFO (zkCallback-3127-thread-2-processing-n:127.0.0.1:54423_solr) [n:127.0.0.1:54423_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfcollectionblocker2/state.json] for collection [halfcollectionblocker2] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2305421 INFO (zkCallback-3127-thread-1-processing-n:127.0.0.1:54423_solr) [n:127.0.0.1:54423_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfcollectionblocker2/state.json] for collection [halfcollectionblocker2] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2305462 INFO (qtp1892567940-20959) [n:127.0.0.1:54423_solr c:halfcollectionblocker2 s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2305467 INFO (qtp1892567940-20959) [n:127.0.0.1:54423_solr c:halfcollectionblocker2 s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.s.IndexSchema [halfcollection_shard1_replica_n1] Schema name=minimal
[junit4] 2> 2305468 INFO (qtp1892567940-20959) [n:127.0.0.1:54423_solr c:halfcollectionblocker2 s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2305468 INFO (qtp1892567940-20959) [n:127.0.0.1:54423_solr c:halfcollectionblocker2 s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'halfcollection_shard1_replica_n1' using configuration from collection halfcollectionblocker2, trusted=true
[junit4] 2> 2305468 INFO (qtp1892567940-20959) [n:127.0.0.1:54423_solr c:halfcollectionblocker2 s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.halfcollectionblocker2.shard1.core_node21' (registry 'solr.core.halfcollectionblocker2.shard1.core_node21') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@60102d0c
[junit4] 2> 2305468 INFO (qtp1892567940-20959) [n:127.0.0.1:54423_solr c:halfcollectionblocker2 s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2305469 INFO (qtp1892567940-20959) [n:127.0.0.1:54423_solr c:halfcollectionblocker2 s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.c.SolrCore [[halfcollection_shard1_replica_n1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_39C95B6195AFBB9F-001/tempDir-001/node3/halfcollection_shard1_replica_n1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest_39C95B6195AFBB9F-001/tempDir-001/node3/halfcollection_shard1_replica_n1/data/]
[junit4] 2> 2305503 INFO (qtp1892567940-20959) [n:127.0.0.1:54423_solr c:halfcollectionblocker2 s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2305503 INFO (qtp1892567940-20959) [n:127.0.0.1:54423_solr c:halfcollectionblocker2 s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2305504 INFO (qtp1892567940-20959) [n:127.0.0.1:54423_solr c:halfcollectionblocker2 s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2305504 INFO (qtp1892567940-20959) [n:127.0.0.1:54423_solr c:halfcollectionblocker2 s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2305505 INFO (qtp1892567940-20959) [n:127.0.0.1:54423_solr c:halfcollectionblocker2 s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@250924d5[halfcollection_shard1_replica_n1] main]
[junit4] 2> 2305506 INFO (qtp1892567940-20959) [n:127.0.0.1:54423_solr c:halfcollectionblocker2 s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 2305507 INFO (qtp1892567940-20959) [n:127.0.0.1:54423_solr c:halfcollectionblocker2 s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 2305507 INFO (qtp1892567940-20959) [n:127.0.0.1:54423_solr c:halfcollectionblocker2 s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2305508 INFO (searcherExecutor-8414-thread-1-processing-n:127.0.0.1:54423_solr x:halfcollection_shard1_replica_n1 s:shard1 c:halfcollectionblocker2 r:core_node21) [n:127.0.0.1:54423_solr c:halfcollectionblocker2 s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.c.SolrCore [halfcollection_shard1_replica_n1] Registered new searcher Searcher@250924d5[halfcollection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2305508 INFO (qtp1892567940-20959) [n:127.0.0.1:54423_solr c:halfcollectionblocker2 s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1582804476259991552
[junit4] 2> 2305513 INFO (qtp1892567940-20959) [n:127.0.0.1:54423_solr c:halfcollectionblocker2 s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2305513 INFO (qtp1892567940-20959) [n:127.0.0.1:54423_solr c:halfcollectionblocker2 s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2305513 INFO (qtp1892567940-20959) [n:127.0.0.1:54423_solr c:halfcollectionblocker2 s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:54423/solr/halfcollection_shard1_replica_n1/
[junit4] 2> 2305513 INFO (qtp1892567940-20959) [n:127.0.0.1:54423_solr c:halfcollectionblocker2 s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 2305513 INFO (qtp1892567940-20959) [n:127.0.0.1:54423_solr c:halfcollectionblocker2 s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:54423/solr/halfcollection_shard1_replica_n1/ has no replicas
[junit4] 2> 2305513 INFO (qtp1892567940-20959) [n:127.0.0.1:54423_solr c:halfcollectionblocker2 s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 2305517 INFO (qtp1892567940-20959) [n:127.0.0.1:54423_solr c:halfcollectionblocker2 s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:54423/solr/halfcollection_shard1_replica_n1/ shard1
[junit4] 2> 2305623 INFO (zkCallback-3127-thread-2-processing-n:127.0.0.1:54423_solr) [n:127.0.0.1:54423_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfcollectionblocker2/state.json] for collection [halfcollectionblocker2] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2305623 INFO (zkCallback-3127-thread-1-processing-n:127.0.0.1:54423_solr) [n:127.0.0.1:54423_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfcollectionblocker2/state.json] for collection [halfcollectionblocker2] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2305672 INFO (qtp1892567940-20959) [n:127.0.0.1:54423_solr c:halfcollectionblocker2 s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2305674 INFO (qtp1892567940-20959) [n:127.0.0.1:54423_solr c:halfcollectionblocker2 s:shard1 r:core_node21 x:halfcollection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node21&collection.configName=conf&name=halfcollection_shard1_replica_n1&action=CREATE&collection=halfcollectionblocker2&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2263
[junit4] 2> 2305677 INFO (qtp425568925-20949) [n:127.0.0.1:62610_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:54423_solr&name=halfcollection_shard1_replica_n1&action=ADDREPLICA&collection=halfcollectionblocker2&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=2494
[junit4] 2> 2305678 INFO (qtp425568925-20947) [n:127.0.0.1:62610_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&name=halfcollection&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=127.0.0.1:43905_solr,127.0.0.1:54423_solr&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2305680 INFO (OverseerThreadFactory-8392-thread-5-processing-n:127.0.0.1:60697_solr) [n:127.0.0.1:60697_solr ] o.a.s.c.CreateCollectionCmd Create collection halfcollection
[junit4] 2> 2305680 INFO (OverseerCollectionConfigSetProcessor-98925279023005705-127.0.0.1:60697_solr-n_0000000000) [n:127.0.0.1:60697_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000012 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2305788 INFO (zkCallback-3127-thread-1-processing-n:127.0.0.1:54423_solr) [n:127.0.0.1:54423_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfcollectionblocker2/state.json] for collection [halfcollectionblocker2] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2305788 INFO (zkCallback-3127-thread-2-processing-n:127.0.0.1:54423_solr) [n:127.0.0.1:54423_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfcollectionblocker2/state.json] for collection [halfcollectionblocker2] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2305897 INFO (OverseerStateUpdate-98925279023005705-127.0.0.1:60697_solr-n_0000000000) [n:127.0.0.1:60697_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"halfcollection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"halfcollection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:54423/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2305904 INFO (OverseerStateUpdate-98925279023005705-127.0.0.1:60697_solr-n_0000000000) [n:127.0.0.1:60697_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"halfcollection",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"halfcollection_shard2_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:43905/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2306105 INFO (qtp1892567940-20960) [n:127.0.0.1:54423_solr ] o.a.s.u.TestInjection Inject random core creation delay of 2s
[junit4] 2> 2306106 INFO (qtp871521207-20953) [n:127.0.0.1:43905_solr ] o.a.s.u.TestInjection Inject random core creation delay of 2s
[junit4] 2> 2308105 INFO (qtp1892567940-20960) [n:127.0.0.1:54423_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=halfcollection_shard1_replica_n1&action=CREATE&numShards=2&collection=halfcollection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2308105 WARN (qtp1892567940-20960) [n:127.0.0.1:54423_solr ] o.a.s.c.CoreContainer Creating a core with existing name is not allowed
[junit4] 2> 2308106 ERROR (qtp1892567940-20960) [n:127.0.0.1:54423_solr ] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Core with name 'halfcollection_shard1_replica_n1' already exists.
[junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:930)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:735)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:716)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:497)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.han
[...truncated too long message...]
oseExecutor-235-thread-1) [n:127.0.0.1:55443_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1.shard2.replica_n2, tag=690313736
[junit4] 2> 127017 INFO (coreCloseExecutor-235-thread-1) [n:127.0.0.1:55443_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@64b2ee0b: rootName = solr_55443, domain = solr.core.collection1.shard2.replica_n2, service url = null, agent id = null] for registry solr.core.collection1.shard2.replica_n2 / com.codahale.metrics.MetricRegistry@2dc99e01
[junit4] 2> 127018 INFO (jetty-closer-4-thread-1) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:54271_solr
[junit4] 2> 127019 INFO (zkCallback-24-thread-2-processing-n:127.0.0.1:36573_solr) [n:127.0.0.1:36573_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 127020 INFO (zkCallback-23-thread-3-processing-n:127.0.0.1:54271_solr) [n:127.0.0.1:54271_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 127021 INFO (zkCallback-1620-thread-1-processing-n:127.0.0.1:36573_solr x:collection1_shard1_replica_n1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:36573_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 127021 INFO (zkCallback-22-thread-2-processing-n:127.0.0.1:55443_solr) [n:127.0.0.1:55443_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 127021 INFO (jetty-closer-4-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 127024 INFO (coreCloseExecutor-234-thread-1) [n:127.0.0.1:36573_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=13456165
[junit4] 2> 127032 INFO (jetty-closer-4-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3f7a6f64{/solr,null,UNAVAILABLE}
[junit4] 2> 127036 INFO (coreCloseExecutor-235-thread-1) [n:127.0.0.1:55443_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard2.leader, tag=690313736
[junit4] 2> 127043 INFO (jetty-closer-4-thread-4) [ ] o.a.s.c.Overseer Overseer (id=98925431059316746-127.0.0.1:36573_solr-n_0000000000) closing
[junit4] 2> 127044 INFO (OverseerStateUpdate-98925431059316746-127.0.0.1:36573_solr-n_0000000000) [n:127.0.0.1:36573_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:36573_solr
[junit4] 2> 127047 WARN (OverseerAutoScalingTriggerThread-98925431059316746-127.0.0.1:36573_solr-n_0000000000) [n:127.0.0.1:36573_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 127054 INFO (jetty-closer-4-thread-4) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5d3dd2c4{/solr,null,UNAVAILABLE}
[junit4] 2> 128562 INFO (jetty-closer-4-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5bd15a6a{/solr,null,UNAVAILABLE}
[junit4] 2> 128565 ERROR (SUITE-StreamExpressionTest-seed#[5B7B0EE40B6887BC]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 128569 INFO (SUITE-StreamExpressionTest-seed#[5B7B0EE40B6887BC]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:62206 62206
[junit4] 2> 138689 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxnFactory Ignoring unexpected runtime exception
[junit4] 2> java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.readyOps(SelectionKeyImpl.java:87)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:182)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 138691 INFO (Thread-12) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:62206 62206
[junit4] 2> 138695 WARN (Thread-12) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 100 /solr/aliases.json
[junit4] 2> 100 /solr/clusterprops.json
[junit4] 2> 14 /solr/configs/conf
[junit4] 2> 7 /solr/configs/ml
[junit4] 2> 4 /solr/security.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 100 /solr/clusterstate.json
[junit4] 2> 38 /solr/collections/destinationCollection/state.json
[junit4] 2> 26 /solr/collections/parallelDestinationCollection1/state.json
[junit4] 2> 20 /solr/collections/parallelDestinationCollection/state.json
[junit4] 2> 10 /solr/collections/mainCorpus1/state.json
[junit4] 2> 10 /solr/collections/destination/state.json
[junit4] 2> 10 /solr/collections/destination1/state.json
[junit4] 2> 10 /solr/collections/collection1/state.json
[junit4] 2> 10 /solr/collections/workQueue1/state.json
[junit4] 2> 10 /solr/collections/workQueue/state.json
[junit4] 2> 10 /solr/collections/uknownCollection/state.json
[junit4] 2> 10 /solr/collections/modelCollection/state.json
[junit4] 2> 10 /solr/collections/checkpointCollection/state.json
[junit4] 2> 10 /solr/collections/mainCorpus/state.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 100 /solr/live_nodes
[junit4] 2> 100 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.io.stream.StreamExpressionTest_5B7B0EE40B6887BC-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {expr_s=Lucene50(blocksize=128), name_s=BlockTreeOrds(blocksize=128), terms_ss=PostingsFormat(name=LuceneVarGapDocFreqInterval), join2_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), field_s=Lucene50(blocksize=128), multiDefault=PostingsFormat(name=LuceneVarGapDocFreqInterval), subject=PostingsFormat(name=LuceneVarGapDocFreqInterval), a1_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), a2_s=BlockTreeOrds(blocksize=128), level1_s=Lucene50(blocksize=128), body_t=BlockTreeOrds(blocksize=128), s_multi=BlockTreeOrds(blocksize=128), col_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), a_ss=Lucene50(blocksize=128), level2_s=Lucene50(blocksize=128), a_s=BlockTreeOrds(blocksize=128), tv_text=Lucene50(blocksize=128), term_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), a_t=Lucene50(blocksize=128), text_s=Lucene50(blocksize=128), id=Lucene50(blocksize=128), text=BlockTreeOrds(blocksize=128), featureSet_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), checkpoint_ss=BlockTreeOrds(blocksize=128), test_t=BlockTreeOrds(blocksize=128), ident_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), whitetok=BlockTreeOrds(blocksize=128), side_s=BlockTreeOrds(blocksize=128)}, docValues:{expr_s=DocValuesFormat(name=Direct), name_s=DocValuesFormat(name=Memory), iteration_i=DocValuesFormat(name=Direct), terms_ss=DocValuesFormat(name=Lucene70), join2_s=DocValuesFormat(name=Lucene70), field_s=DocValuesFormat(name=Lucene70), multiDefault=DocValuesFormat(name=Lucene70), price_f=DocValuesFormat(name=Lucene70), a1_s=DocValuesFormat(name=Lucene70), intDefault=DocValuesFormat(name=Lucene70), a_f=DocValuesFormat(name=Lucene70), a2_s=DocValuesFormat(name=Memory), join1_i=DocValuesFormat(name=Direct), falseNegative_i=DocValuesFormat(name=Lucene70), a_i=DocValuesFormat(name=Direct), test_i=DocValuesFormat(name=Lucene70), join3_i=DocValuesFormat(name=Memory), falsePositive_i=DocValuesFormat(name=Lucene70), field_i=DocValuesFormat(name=Lucene70), level1_s=DocValuesFormat(name=Direct), s_multi=DocValuesFormat(name=Memory), col_s=DocValuesFormat(name=Lucene70), a_ss=DocValuesFormat(name=Lucene70), level2_s=DocValuesFormat(name=Lucene70), a_s=DocValuesFormat(name=Memory), score_f=DocValuesFormat(name=Memory), term_s=DocValuesFormat(name=Lucene70), text_s=DocValuesFormat(name=Direct), id=DocValuesFormat(name=Direct), timestamp=DocValuesFormat(name=Lucene70), i_multi=DocValuesFormat(name=Direct), featureSet_s=DocValuesFormat(name=Lucene70), trueNegative_i=DocValuesFormat(name=Memory), index_i=DocValuesFormat(name=Lucene70), b_ls=DocValuesFormat(name=Lucene70), test_dt=DocValuesFormat(name=Direct), out_i=DocValuesFormat(name=Lucene70), checkpoint_ss=DocValuesFormat(name=Memory), ident_s=DocValuesFormat(name=Lucene70), order_i=DocValuesFormat(name=Lucene70), _version_=DocValuesFormat(name=Lucene70), side_s=DocValuesFormat(name=Memory), truePositive_i=DocValuesFormat(name=Direct), miles_i=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=1727, maxMBSortInHeap=7.640102589879356, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@573d22c5), locale=sr, timezone=Asia/Anadyr
[junit4] 2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=3,threads=1,free=81589704,total=192425984
[junit4] 2> NOTE: All tests run in this JVM: [JsonValidatorTest, SolrExceptionTest, SolrExampleJettyTest, GreaterThanEqualToEvaluatorTest, TestConfigSetAdminRequest, StreamExpressionTest]
[junit4] Completed [47/146 (1!)] on J1 in 127.62s, 125 tests, 1 failure <<< FAILURES!
[...truncated 46600 lines...]
[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build #
1500 - Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1500/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC
2 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation
Error Message:
2 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation: 1) Thread[id=18321, name=jetty-launcher-2650-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037) at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328) at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277) at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105) at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41) at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244) at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44) at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61) at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505) 2) Thread[id=18331, name=jetty-launcher-2650-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037) at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328) at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277) at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105) at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41) at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244) at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44) at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61) at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
1) Thread[id=18321, name=jetty-launcher-2650-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
2) Thread[id=18331, name=jetty-launcher-2650-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
at __randomizedtesting.SeedInfo.seed([4DE8E6CD4434BA82]:0)
FAILED: org.apache.solr.security.hadoop.TestDelegationWithHadoopAuth.testDelegationTokenRenew
Error Message:
expected:<200> but was:<403>
Stack Trace:
java.lang.AssertionError: expected:<200> but was:<403>
at __randomizedtesting.SeedInfo.seed([4DE8E6CD4434BA82:7A7312D37CF86726]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.security.hadoop.TestDelegationWithHadoopAuth.renewDelegationToken(TestDelegationWithHadoopAuth.java:118)
at org.apache.solr.security.hadoop.TestDelegationWithHadoopAuth.verifyDelegationTokenRenew(TestDelegationWithHadoopAuth.java:302)
at org.apache.solr.security.hadoop.TestDelegationWithHadoopAuth.testDelegationTokenRenew(TestDelegationWithHadoopAuth.java:319)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 11903 lines...]
[junit4] Suite: org.apache.solr.security.hadoop.TestDelegationWithHadoopAuth
[junit4] 2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.security.hadoop.TestDelegationWithHadoopAuth_4DE8E6CD4434BA82-001/init-core-data-001
[junit4] 2> 1504384 WARN (SUITE-TestDelegationWithHadoopAuth-seed#[4DE8E6CD4434BA82]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=773 numCloses=773
[junit4] 2> 1504384 INFO (SUITE-TestDelegationWithHadoopAuth-seed#[4DE8E6CD4434BA82]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 1504387 INFO (SUITE-TestDelegationWithHadoopAuth-seed#[4DE8E6CD4434BA82]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 1504387 INFO (SUITE-TestDelegationWithHadoopAuth-seed#[4DE8E6CD4434BA82]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 1504389 INFO (SUITE-TestDelegationWithHadoopAuth-seed#[4DE8E6CD4434BA82]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.security.hadoop.TestDelegationWithHadoopAuth_4DE8E6CD4434BA82-001/tempDir-001
[junit4] 2> 1504389 INFO (SUITE-TestDelegationWithHadoopAuth-seed#[4DE8E6CD4434BA82]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1504389 INFO (Thread-2296) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1504389 INFO (Thread-2296) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1504391 ERROR (Thread-2296) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1504494 INFO (SUITE-TestDelegationWithHadoopAuth-seed#[4DE8E6CD4434BA82]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:33586
[junit4] 2> 1504655 INFO (jetty-launcher-1152-thread-1) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 1504656 INFO (jetty-launcher-1152-thread-2) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 1504658 INFO (jetty-launcher-1152-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@666a13b0{/solr,null,AVAILABLE}
[junit4] 2> 1504659 INFO (jetty-launcher-1152-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@6699caaf{HTTP/1.1,[http/1.1]}{127.0.0.1:64078}
[junit4] 2> 1504660 INFO (jetty-launcher-1152-thread-1) [ ] o.e.j.s.Server Started @1506899ms
[junit4] 2> 1504660 INFO (jetty-launcher-1152-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=64078}
[junit4] 2> 1504660 ERROR (jetty-launcher-1152-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1504660 INFO (jetty-launcher-1152-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1504660 INFO (jetty-launcher-1152-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1504660 INFO (jetty-launcher-1152-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1504660 INFO (jetty-launcher-1152-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-10-31T11:36:13.475Z
[junit4] 2> 1504663 INFO (jetty-launcher-1152-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@213be628{/solr,null,AVAILABLE}
[junit4] 2> 1504663 INFO (jetty-launcher-1152-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@55743bca{HTTP/1.1,[http/1.1]}{127.0.0.1:64624}
[junit4] 2> 1504663 INFO (jetty-launcher-1152-thread-2) [ ] o.e.j.s.Server Started @1506903ms
[junit4] 2> 1504663 INFO (jetty-launcher-1152-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=64624}
[junit4] 2> 1504664 ERROR (jetty-launcher-1152-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1504664 INFO (jetty-launcher-1152-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1504664 INFO (jetty-launcher-1152-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1504664 INFO (jetty-launcher-1152-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1504664 INFO (jetty-launcher-1152-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-10-31T11:36:13.479Z
[junit4] 2> 1504665 INFO (jetty-launcher-1152-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1504676 INFO (jetty-launcher-1152-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1504726 INFO (jetty-launcher-1152-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33586/solr
[junit4] 2> 1504739 INFO (jetty-launcher-1152-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33586/solr
[junit4] 2> 1505322 INFO (jetty-launcher-1152-thread-2) [n:127.0.0.1:64624_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1505324 INFO (jetty-launcher-1152-thread-2) [n:127.0.0.1:64624_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:64624_solr
[junit4] 2> 1505325 INFO (jetty-launcher-1152-thread-2) [n:127.0.0.1:64624_solr ] o.a.s.c.Overseer Overseer (id=98923300336828422-127.0.0.1:64624_solr-n_0000000000) starting
[junit4] 2> 1505354 INFO (jetty-launcher-1152-thread-2) [n:127.0.0.1:64624_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:64624_solr
[junit4] 2> 1505374 INFO (zkCallback-1164-thread-1-processing-n:127.0.0.1:64624_solr) [n:127.0.0.1:64624_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1505399 INFO (jetty-launcher-1152-thread-1) [n:127.0.0.1:64078_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1505405 INFO (jetty-launcher-1152-thread-1) [n:127.0.0.1:64078_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1505450 INFO (jetty-launcher-1152-thread-1) [n:127.0.0.1:64078_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:64078_solr
[junit4] 2> 1505455 INFO (zkCallback-1164-thread-1-processing-n:127.0.0.1:64624_solr) [n:127.0.0.1:64624_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1505466 INFO (zkCallback-1160-thread-1-processing-n:127.0.0.1:64078_solr) [n:127.0.0.1:64078_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1505497 INFO (jetty-launcher-1152-thread-2) [n:127.0.0.1:64624_solr ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.security.HadoopAuthPlugin
[junit4] 2> 1505517 INFO (jetty-launcher-1152-thread-2) [n:127.0.0.1:64624_solr ] o.a.s.s.HadoopAuthPlugin Params: {delegation-token.token-kind=solr-dt, cookie.domain=127.0.0.1, zk-dt-secret-manager.znodeWorkingPath=solr/security/zkdtsm, signer.secret.provider.zookeeper.path=/token, type=simple, zk-dt-secret-manager.enable=true, signer.secret.provider=zookeeper}
[junit4] 2> 1505648 INFO (jetty-launcher-1152-thread-1) [n:127.0.0.1:64078_solr ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.security.HadoopAuthPlugin
[junit4] 2> 1505648 INFO (jetty-launcher-1152-thread-1) [n:127.0.0.1:64078_solr ] o.a.s.s.HadoopAuthPlugin Params: {delegation-token.token-kind=solr-dt, cookie.domain=127.0.0.1, zk-dt-secret-manager.znodeWorkingPath=solr/security/zkdtsm, signer.secret.provider.zookeeper.path=/token, type=simple, zk-dt-secret-manager.enable=true, signer.secret.provider=zookeeper}
[junit4] 2> 1505675 INFO (jetty-launcher-1152-thread-2) [n:127.0.0.1:64624_solr ] o.a.c.f.i.CuratorFrameworkImpl Starting
[junit4] 2> 1505678 INFO (jetty-launcher-1152-thread-1) [n:127.0.0.1:64078_solr ] o.a.c.f.i.CuratorFrameworkImpl Starting
[junit4] 2> 1505766 INFO (jetty-launcher-1152-thread-2-EventThread) [n:127.0.0.1:64624_solr ] o.a.c.f.s.ConnectionStateManager State change: CONNECTED
[junit4] 2> 1505776 INFO (jetty-launcher-1152-thread-1-EventThread) [n:127.0.0.1:64078_solr ] o.a.c.f.s.ConnectionStateManager State change: CONNECTED
[junit4] 2> 1506112 INFO (jetty-launcher-1152-thread-2) [n:127.0.0.1:64624_solr ] o.a.s.c.CoreContainer PKIAuthenticationPlugin is managing internode requests
[junit4] 2> 1506202 INFO (jetty-launcher-1152-thread-1) [n:127.0.0.1:64078_solr ] o.a.s.c.CoreContainer PKIAuthenticationPlugin is managing internode requests
[junit4] 2> 1506266 INFO (jetty-launcher-1152-thread-2) [n:127.0.0.1:64624_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64624.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@39be70d3
[junit4] 2> 1506335 INFO (jetty-launcher-1152-thread-1) [n:127.0.0.1:64078_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64078.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@39be70d3
[junit4] 2> 1506339 INFO (jetty-launcher-1152-thread-2) [n:127.0.0.1:64624_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64624.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@39be70d3
[junit4] 2> 1506339 INFO (jetty-launcher-1152-thread-2) [n:127.0.0.1:64624_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64624.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@39be70d3
[junit4] 2> 1506340 INFO (jetty-launcher-1152-thread-2) [n:127.0.0.1:64624_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.security.hadoop.TestDelegationWithHadoopAuth_4DE8E6CD4434BA82-001/tempDir-001/node2/.
[junit4] 2> 1506363 INFO (jetty-launcher-1152-thread-1) [n:127.0.0.1:64078_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64078.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@39be70d3
[junit4] 2> 1506364 INFO (jetty-launcher-1152-thread-1) [n:127.0.0.1:64078_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64078.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@39be70d3
[junit4] 2> 1506365 INFO (jetty-launcher-1152-thread-1) [n:127.0.0.1:64078_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.security.hadoop.TestDelegationWithHadoopAuth_4DE8E6CD4434BA82-001/tempDir-001/node1/.
[junit4] 2> 1506523 INFO (TEST-TestDelegationWithHadoopAuth.testDelegationTokenRenewFail-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testDelegationTokenRenewFail
[junit4] 2> 1506625 WARN (qtp248131144-10659) [ ] o.a.h.s.a.s.AuthenticationFilter Authentication exception: org.apache.hadoop.security.AccessControlException: bar tries to renew a token with renewer foo
[junit4] 2> 1506626 WARN (qtp248131144-10659) [ ] o.a.s.s.HadoopAuthPlugin Could not find org.apache.solr.security.authentication.requestcontinues
[junit4] 2> 1506633 WARN (TEST-TestDelegationWithHadoopAuth.testDelegationTokenRenewFail-seed#[4DE8E6CD4434BA82]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Domain=127.0.0.1; Expires= am, 01-Jan-1970 00:00:00 GMT; HttpOnly". Invalid 'expires' attribute: am, 01-Jan-1970 00:00:00 GMT
[junit4] 2> 1506639 WARN (qtp1671684462-10648) [ ] o.a.h.s.a.s.AuthenticationFilter Authentication exception: org.apache.hadoop.security.AccessControlException: bar tries to renew a token with renewer foo
[junit4] 2> 1506640 WARN (qtp1671684462-10648) [ ] o.a.s.s.HadoopAuthPlugin Could not find org.apache.solr.security.authentication.requestcontinues
[junit4] 2> 1506642 WARN (TEST-TestDelegationWithHadoopAuth.testDelegationTokenRenewFail-seed#[4DE8E6CD4434BA82]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Domain=127.0.0.1; Expires= am, 01-Jan-1970 00:00:00 GMT; HttpOnly". Invalid 'expires' attribute: am, 01-Jan-1970 00:00:00 GMT
[junit4] 2> 1506690 WARN (qtp248131144-10658) [ ] o.a.h.s.a.s.AuthenticationFilter Authentication exception: org.apache.hadoop.security.AccessControlException: notRenewUser tries to renew a token with renewer renewUser
[junit4] 2> 1506691 WARN (qtp248131144-10658) [ ] o.a.s.s.HadoopAuthPlugin Could not find org.apache.solr.security.authentication.requestcontinues
[junit4] 2> 1506691 WARN (TEST-TestDelegationWithHadoopAuth.testDelegationTokenRenewFail-seed#[4DE8E6CD4434BA82]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Domain=127.0.0.1; Expires= am, 01-Jan-1970 00:00:00 GMT; HttpOnly". Invalid 'expires' attribute: am, 01-Jan-1970 00:00:00 GMT
[junit4] 2> 1506692 WARN (qtp1671684462-10648) [ ] o.a.h.s.a.s.AuthenticationFilter Authentication exception: org.apache.hadoop.security.AccessControlException: notRenewUser tries to renew a token with renewer renewUser
[junit4] 2> 1506693 WARN (qtp1671684462-10648) [ ] o.a.s.s.HadoopAuthPlugin Could not find org.apache.solr.security.authentication.requestcontinues
[junit4] 2> 1506698 WARN (TEST-TestDelegationWithHadoopAuth.testDelegationTokenRenewFail-seed#[4DE8E6CD4434BA82]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Domain=127.0.0.1; Expires= am, 01-Jan-1970 00:00:00 GMT; HttpOnly". Invalid 'expires' attribute: am, 01-Jan-1970 00:00:00 GMT
[junit4] 2> 1506698 INFO (TEST-TestDelegationWithHadoopAuth.testDelegationTokenRenewFail-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testDelegationTokenRenewFail
[junit4] 2> 1506783 INFO (TEST-TestDelegationWithHadoopAuth.testDelegationTokenCancel-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testDelegationTokenCancel
[junit4] 2> 1506916 INFO (TEST-TestDelegationWithHadoopAuth.testDelegationTokenCancel-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1506918 INFO (TEST-TestDelegationWithHadoopAuth.testDelegationTokenCancel-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33586/solr ready
[junit4] 2> 1506929 ERROR (qtp1671684462-10654) [ ] o.a.h.s.t.d.ZKDelegationTokenSecretManager No node in path [/ZKDTSMTokensRoot/DT_3]
[junit4] 2> 1506964 INFO (TEST-TestDelegationWithHadoopAuth.testDelegationTokenCancel-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1506966 INFO (TEST-TestDelegationWithHadoopAuth.testDelegationTokenCancel-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33586/solr ready
[junit4] 2> 1506968 WARN (TEST-TestDelegationWithHadoopAuth.testDelegationTokenCancel-seed#[4DE8E6CD4434BA82]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Domain=127.0.0.1; Expires= am, 01-Jan-1970 00:00:00 GMT; HttpOnly". Invalid 'expires' attribute: am, 01-Jan-1970 00:00:00 GMT
[junit4] 2> 1507042 INFO (TEST-TestDelegationWithHadoopAuth.testDelegationTokenCancel-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1507070 INFO (TEST-TestDelegationWithHadoopAuth.testDelegationTokenCancel-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33586/solr ready
[junit4] 2> 1507078 WARN (TEST-TestDelegationWithHadoopAuth.testDelegationTokenCancel-seed#[4DE8E6CD4434BA82]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Domain=127.0.0.1; Expires= am, 01-Jan-1970 00:00:00 GMT; HttpOnly". Invalid 'expires' attribute: am, 01-Jan-1970 00:00:00 GMT
[junit4] 2> 1507121 ERROR (qtp1671684462-10655) [ ] o.a.h.s.t.d.ZKDelegationTokenSecretManager No node in path [/ZKDTSMTokensRoot/DT_4]
[junit4] 2> 1507158 INFO (TEST-TestDelegationWithHadoopAuth.testDelegationTokenCancel-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1507168 INFO (TEST-TestDelegationWithHadoopAuth.testDelegationTokenCancel-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33586/solr ready
[junit4] 2> 1507175 WARN (TEST-TestDelegationWithHadoopAuth.testDelegationTokenCancel-seed#[4DE8E6CD4434BA82]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Domain=127.0.0.1; Expires= am, 01-Jan-1970 00:00:00 GMT; HttpOnly". Invalid 'expires' attribute: am, 01-Jan-1970 00:00:00 GMT
[junit4] 2> 1507215 INFO (TEST-TestDelegationWithHadoopAuth.testDelegationTokenCancel-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1507226 INFO (TEST-TestDelegationWithHadoopAuth.testDelegationTokenCancel-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33586/solr ready
[junit4] 2> 1507238 WARN (TEST-TestDelegationWithHadoopAuth.testDelegationTokenCancel-seed#[4DE8E6CD4434BA82]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Domain=127.0.0.1; Expires= am, 01-Jan-1970 00:00:00 GMT; HttpOnly". Invalid 'expires' attribute: am, 01-Jan-1970 00:00:00 GMT
[junit4] 2> 1507248 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15f7237b096000e, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1507254 INFO (TEST-TestDelegationWithHadoopAuth.testDelegationTokenCancel-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testDelegationTokenCancel
[junit4] 2> 1507481 INFO (TEST-TestDelegationWithHadoopAuth.testDelegationTokenRenew-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testDelegationTokenRenew
[junit4] 2> 1507500 WARN (qtp248131144-10659) [ ] o.a.h.s.a.s.AuthenticationFilter Authentication exception: org.apache.hadoop.security.AccessControlException: foo is trying to renew a token with wrong password
[junit4] 2> 1507500 WARN (qtp248131144-10659) [ ] o.a.s.s.HadoopAuthPlugin Could not find org.apache.solr.security.authentication.requestcontinues
[junit4] 2> 1507501 WARN (TEST-TestDelegationWithHadoopAuth.testDelegationTokenRenew-seed#[4DE8E6CD4434BA82]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Domain=127.0.0.1; Expires= am, 01-Jan-1970 00:00:00 GMT; HttpOnly". Invalid 'expires' attribute: am, 01-Jan-1970 00:00:00 GMT
[junit4] 2> 1507501 INFO (TEST-TestDelegationWithHadoopAuth.testDelegationTokenRenew-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testDelegationTokenRenew
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestDelegationWithHadoopAuth -Dtests.method=testDelegationTokenRenew -Dtests.seed=4DE8E6CD4434BA82 -Dtests.slow=true -Dtests.locale=mt -Dtests.timezone=America/Eirunepe -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] FAILURE 0.25s J0 | TestDelegationWithHadoopAuth.testDelegationTokenRenew <<<
[junit4] > Throwable #1: java.lang.AssertionError: expected:<200> but was:<403>
[junit4] > at __randomizedtesting.SeedInfo.seed([4DE8E6CD4434BA82:7A7312D37CF86726]:0)
[junit4] > at org.apache.solr.security.hadoop.TestDelegationWithHadoopAuth.renewDelegationToken(TestDelegationWithHadoopAuth.java:118)
[junit4] > at org.apache.solr.security.hadoop.TestDelegationWithHadoopAuth.verifyDelegationTokenRenew(TestDelegationWithHadoopAuth.java:302)
[junit4] > at org.apache.solr.security.hadoop.TestDelegationWithHadoopAuth.testDelegationTokenRenew(TestDelegationWithHadoopAuth.java:319)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1507641 INFO (TEST-TestDelegationWithHadoopAuth.testZNodePaths-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testZNodePaths
[junit4] 2> 1507651 INFO (TEST-TestDelegationWithHadoopAuth.testZNodePaths-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testZNodePaths
[junit4] IGNOR/A 0.00s J0 | TestDelegationWithHadoopAuth.testDelegationTokenCancelFail
[junit4] > Assumption #1: 'awaitsfix' test group is disabled (@AwaitsFix(bugUrl=https://issues.apache.org/jira/browse/HADOOP-14044))
[junit4] 2> 1507758 INFO (TEST-TestDelegationWithHadoopAuth.testDelegationTokenVerify-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testDelegationTokenVerify
[junit4] 2> 1507776 INFO (TEST-TestDelegationWithHadoopAuth.testDelegationTokenVerify-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1507777 INFO (TEST-TestDelegationWithHadoopAuth.testDelegationTokenVerify-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33586/solr ready
[junit4] 2> 1507793 INFO (qtp1671684462-10650) [n:127.0.0.1:64078_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&delegation=GgADZm9vA2ZvbwCKAV9yN73BigFflkRBwQcDFHnlikDQY4YN9Bg3WUHd0ecBymq4B3NvbHItZHQA&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1507793 INFO (qtp1671684462-10650) [n:127.0.0.1:64078_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&delegation=GgADZm9vA2ZvbwCKAV9yN73BigFflkRBwQcDFHnlikDQY4YN9Bg3WUHd0ecBymq4B3NvbHItZHQA&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1507873 INFO (TEST-TestDelegationWithHadoopAuth.testDelegationTokenVerify-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1507882 INFO (TEST-TestDelegationWithHadoopAuth.testDelegationTokenVerify-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33586/solr ready
[junit4] 2> 1507884 WARN (TEST-TestDelegationWithHadoopAuth.testDelegationTokenVerify-seed#[4DE8E6CD4434BA82]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Domain=127.0.0.1; Expires= am, 01-Jan-1970 00:00:00 GMT; HttpOnly". Invalid 'expires' attribute: am, 01-Jan-1970 00:00:00 GMT
[junit4] 2> 1507888 INFO (qtp1671684462-10652) [n:127.0.0.1:64078_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&delegation=GgADZm9vA2ZvbwCKAV9yN73BigFflkRBwQcDFHnlikDQY4YN9Bg3WUHd0ecBymq4B3NvbHItZHQA&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1507888 INFO (qtp1671684462-10652) [n:127.0.0.1:64078_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&delegation=GgADZm9vA2ZvbwCKAV9yN73BigFflkRBwQcDFHnlikDQY4YN9Bg3WUHd0ecBymq4B3NvbHItZHQA&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1507924 INFO (TEST-TestDelegationWithHadoopAuth.testDelegationTokenVerify-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1507958 INFO (TEST-TestDelegationWithHadoopAuth.testDelegationTokenVerify-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33586/solr ready
[junit4] 2> 1507966 WARN (TEST-TestDelegationWithHadoopAuth.testDelegationTokenVerify-seed#[4DE8E6CD4434BA82]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Domain=127.0.0.1; Expires= am, 01-Jan-1970 00:00:00 GMT; HttpOnly". Invalid 'expires' attribute: am, 01-Jan-1970 00:00:00 GMT
[junit4] 2> 1507974 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15f7237b0960012, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1507987 INFO (TEST-TestDelegationWithHadoopAuth.testDelegationTokenVerify-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testDelegationTokenVerify
[junit4] 2> 1508089 INFO (TEST-TestDelegationWithHadoopAuth.testDelegationOtherOp-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testDelegationOtherOp
[junit4] 2> 1508092 INFO (qtp248131144-10662) [n:127.0.0.1:64624_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params op=someSolrOperation&action=LIST&user.name=foo&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1508093 INFO (qtp248131144-10662) [n:127.0.0.1:64624_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={op=someSolrOperation&action=LIST&user.name=foo&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1508093 WARN (TEST-TestDelegationWithHadoopAuth.testDelegationOtherOp-seed#[4DE8E6CD4434BA82]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=foo&p=foo&t=simple-dt&e=1509485776907&s=IO2zHfDUb2BYtLiGJNruCVlQvZs="; Domain=127.0.0.1; Expires=Tli, 31-Ott-2017 21:36:16 GMT; HttpOnly". Invalid 'expires' attribute: Tli, 31-Ott-2017 21:36:16 GMT
[junit4] 2> 1508094 INFO (TEST-TestDelegationWithHadoopAuth.testDelegationOtherOp-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testDelegationOtherOp
[junit4] 2> 1508154 INFO (TEST-TestDelegationWithHadoopAuth.testDelegationTokenSolrClient-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testDelegationTokenSolrClient
[junit4] 2> 1508172 WARN (TEST-TestDelegationWithHadoopAuth.testDelegationTokenSolrClient-seed#[4DE8E6CD4434BA82]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Domain=127.0.0.1; Expires= am, 01-Jan-1970 00:00:00 GMT; HttpOnly". Invalid 'expires' attribute: am, 01-Jan-1970 00:00:00 GMT
[junit4] 2> 1508174 INFO (qtp248131144-10660) [n:127.0.0.1:64624_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&delegation=GgADZm9vA2ZvbwCKAV9yN79NigFflkRDTQgDFJLJCJ_OO-_HfLu1uoMJFFxNk-GoB3NvbHItZHQA&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1508174 INFO (qtp248131144-10660) [n:127.0.0.1:64624_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&delegation=GgADZm9vA2ZvbwCKAV9yN79NigFflkRDTQgDFJLJCJ_OO-_HfLu1uoMJFFxNk-GoB3NvbHItZHQA&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1508174 INFO (TEST-TestDelegationWithHadoopAuth.testDelegationTokenSolrClient-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testDelegationTokenSolrClient
[junit4] 2> 1508195 INFO (jetty-closer-1153-thread-2) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@6699caaf{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 1508195 INFO (jetty-closer-1153-thread-2) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=531910238
[junit4] 2> 1508196 INFO (jetty-closer-1153-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 1508196 INFO (jetty-closer-1153-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4e1645e7: rootName = solr_64078, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@6b66c555
[junit4] 2> 1508198 INFO (jetty-closer-1153-thread-1) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@55743bca{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 1508198 INFO (jetty-closer-1153-thread-1) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1561340807
[junit4] 2> 1508201 INFO (jetty-closer-1153-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 1508201 INFO (jetty-closer-1153-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@38ab3f9: rootName = solr_64624, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@5e6a2e3a
[junit4] 2> 1508221 INFO (jetty-closer-1153-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 1508221 INFO (jetty-closer-1153-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7ac1a528: rootName = solr_64624, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@41f23fb9
[junit4] 2> 1508225 INFO (jetty-closer-1153-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 1508225 INFO (jetty-closer-1153-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1e24ce5e: rootName = solr_64624, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@45cdb35f
[junit4] 2> 1508228 INFO (jetty-closer-1153-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 1508228 INFO (jetty-closer-1153-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@61e02075: rootName = solr_64078, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@41f23fb9
[junit4] 2> 1508232 INFO (jetty-closer-1153-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 1508232 INFO (jetty-closer-1153-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@24d26fab: rootName = solr_64078, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@45cdb35f
[junit4] 2> 1508246 INFO (jetty-closer-1153-thread-1) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:64624_solr
[junit4] 2> 1508252 INFO (zkCallback-1160-thread-1-processing-n:127.0.0.1:64078_solr) [n:127.0.0.1:64078_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 1508254 INFO (jetty-closer-1153-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 1508254 INFO (jetty-closer-1153-thread-1) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1508255 INFO (jetty-closer-1153-thread-1) [ ] o.a.s.c.Overseer Overseer (id=98923300336828422-127.0.0.1:64624_solr-n_0000000000) closing
[junit4] 2> 1508255 INFO (OverseerStateUpdate-98923300336828422-127.0.0.1:64624_solr-n_0000000000) [n:127.0.0.1:64624_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:64624_solr
[junit4] 2> 1508255 INFO (zkCallback-1164-thread-1-processing-n:127.0.0.1:64624_solr) [n:127.0.0.1:64624_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 1508256 INFO (jetty-closer-1153-thread-2) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:64078_solr
[junit4] 2> 1508263 WARN (OverseerAutoScalingTriggerThread-98923300336828422-127.0.0.1:64624_solr-n_0000000000) [n:127.0.0.1:64624_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1508272 INFO (zkCallback-1160-thread-1-processing-n:127.0.0.1:64078_solr) [n:127.0.0.1:64078_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1508274 INFO (zkCallback-1164-thread-1-processing-n:127.0.0.1:64624_solr) [n:127.0.0.1:64624_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1508274 INFO (jetty-closer-1153-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 1508274 INFO (jetty-closer-1153-thread-2) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1508274 INFO (zkCallback-1160-thread-1-processing-n:127.0.0.1:64078_solr) [n:127.0.0.1:64078_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:64078_solr
[junit4] 2> 1508279 ERROR (Thread[Thread-2313,5,TGRP-TestDelegationWithHadoopAuth]) [n:127.0.0.1:64624_solr ] o.a.h.s.t.d.AbstractDelegationTokenSecretManager ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
[junit4] 2> 1508306 INFO (jetty-closer-1153-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@213be628{/solr,null,UNAVAILABLE}
[junit4] 2> 1509775 ERROR (Thread[Thread-2315,5,TGRP-TestDelegationWithHadoopAuth]) [n:127.0.0.1:64078_solr ] o.a.h.s.t.d.AbstractDelegationTokenSecretManager ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
[junit4] 2> 1509777 INFO (jetty-closer-1153-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@666a13b0{/solr,null,UNAVAILABLE}
[junit4] 2> 1509780 ERROR (SUITE-TestDelegationWithHadoopAuth-seed#[4DE8E6CD4434BA82]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1509780 INFO (SUITE-TestDelegationWithHadoopAuth-seed#[4DE8E6CD4434BA82]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:33586 33586
[junit4] 2> 1519875 INFO (Thread-2296) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:33586 33586
[junit4] 2> 1519876 WARN (Thread-2296) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 10 /solr/aliases.json
[junit4] 2> 10 /solr/clusterprops.json
[junit4] 2> 2 /solr/security.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 10 /solr/clusterstate.json
[junit4] 2> 3 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMKeyIdRoot
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_7
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_6
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_8
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_1
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_1
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_3
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_3
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_2
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_5
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_4
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_4
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMSeqNumRoot
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 10 /solr/live_nodes
[junit4] 2> 10 /solr/collections
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.security.hadoop.TestDelegationWithHadoopAuth_4DE8E6CD4434BA82-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70), sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@78be3e18), locale=mt, timezone=America/Eirunepe
[junit4] 2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=3,threads=1,free=216815888,total=363012096
[junit4] 2> NOTE: All tests run in this JVM: [BigEndianAscendingWordSerializerTest, TestSegmentSorting, AutoAddReplicasPlanActionTest, TestSizeLimitedDistributedMap, TestPHPSerializedResponseWriter, TemplateUpdateProcessorTest, TestHdfsCloudBackupRestore, NoCacheHeaderTest, TestDownShardTolerantSearch, MergeStrategyTest, TestFiltering, TestSimpleQParserPlugin, SolrCmdDistributorTest, TestCloudPseudoReturnFields, BlockDirectoryTest, TestSchemaResource, CSVRequestHandlerTest, DirectoryFactoryTest, ExplicitHLLTest, SegmentsInfoRequestHandlerTest, TestSolr4Spatial2, TestDistributedSearch, TestLegacyNumericUtils, TestRecovery, TestMaxTokenLenTokenizer, CleanupOldIndexTest, TestSolrCoreProperties, TestUseDocValuesAsStored, NumericFieldsTest, SpatialRPTFieldTypeTest, TestPullReplicaErrorHandling, CoreAdminRequestStatusTest, TestReversedWildcardFilterFactory, DistributedFacetPivotSmallTest, TestLMDirichletSimilarityFactory, HdfsDirectoryTest, TestSubQueryTransformerCrossCore, ConfigureRecoveryStrategyTest, TestSlowCompositeReaderWrapper, ChaosMonkeyNothingIsSafeTest, DistributedDebugComponentTest, AnalyticsMergeStrategyTest, HdfsThreadLeakTest, TestPerFieldSimilarityWithDefaultOverride, TestDefaultStatsCache, WrapperMergePolicyFactoryTest, RegexBytesRefFilterTest, TestSolrDeletionPolicy2, TestXIncludeConfig, TestCustomDocTransformer, ChaosMonkeySafeLeaderWithPullReplicasTest, TestNumericTerms32, DataDrivenBlockJoinTest, TestConfigSetImmutable, TestAtomicUpdateErrorCases, TestMultiWordSynonyms, TestSolrXml, TestImplicitCoreProperties, SolrGraphiteReporterTest, AddSchemaFieldsUpdateProcessorFactoryTest, ShufflingReplicaListTransformerTest, HLLSerializationTest, TestPseudoReturnFields, TestDocTermOrds, FastVectorHighlighterTest, SolrMetricsIntegrationTest, TestSurroundQueryParser, TestFieldCacheReopen, TestConfigSetsAPI, TestRequestStatusCollectionAPI, BasicDistributedZkTest, StatsComponentTest, TestEmbeddedSolrServerSchemaAPI, TestDeleteCollectionOnDownNodes, TestBinaryField, TestChildDocTransformer, SliceStateTest, ReturnFieldsTest, HdfsRestartWhileUpdatingTest, InfixSuggestersTest, TestCSVResponseWriter, CursorMarkTest, DistributedTermsComponentTest, TestReloadAndDeleteDocs, TestDynamicFieldResource, HdfsChaosMonkeyNothingIsSafeTest, TestCloudJSONFacetJoinDomain, DistributedSuggestComponentTest, TestTolerantUpdateProcessorRandomCloud, TestDelegationWithHadoopAuth]
[junit4] Completed [128/747 (1!)] on J0 in 15.53s, 8 tests, 1 failure, 1 skipped <<< FAILURES!
[...truncated 543 lines...]
[junit4] Suite: org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation
[junit4] 2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_4DE8E6CD4434BA82-001/init-core-data-001
[junit4] 2> 2637744 WARN (SUITE-TestSolrCloudWithSecureImpersonation-seed#[4DE8E6CD4434BA82]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=5 numCloses=5
[junit4] 2> 2637744 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[4DE8E6CD4434BA82]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 2637745 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[4DE8E6CD4434BA82]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 2637745 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[4DE8E6CD4434BA82]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 2637774 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[4DE8E6CD4434BA82]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_4DE8E6CD4434BA82-001/tempDir-001
[junit4] 2> 2637774 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[4DE8E6CD4434BA82]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2637774 INFO (Thread-3841) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2637774 INFO (Thread-3841) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2637776 ERROR (Thread-3841) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2637874 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[4DE8E6CD4434BA82]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:50583
[junit4] 2> 2637899 INFO (jetty-launcher-2650-thread-1) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 2637899 INFO (jetty-launcher-2650-thread-2) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 2637901 INFO (jetty-launcher-2650-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@31647e90{/solr,null,AVAILABLE}
[junit4] 2> 2637901 INFO (jetty-launcher-2650-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7eee509{HTTP/1.1,[http/1.1]}{127.0.0.1:40157}
[junit4] 2> 2637901 INFO (jetty-launcher-2650-thread-1) [ ] o.e.j.s.Server Started @2640283ms
[junit4] 2> 2637901 INFO (jetty-launcher-2650-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40157}
[junit4] 2> 2637902 ERROR (jetty-launcher-2650-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2637902 INFO (jetty-launcher-2650-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@74f3c413{/solr,null,AVAILABLE}
[junit4] 2> 2637902 INFO (jetty-launcher-2650-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 2637902 INFO (jetty-launcher-2650-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2637902 INFO (jetty-launcher-2650-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2637902 INFO (jetty-launcher-2650-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-10-31T11:55:06.717Z
[junit4] 2> 2637902 INFO (jetty-launcher-2650-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1980466c{HTTP/1.1,[http/1.1]}{127.0.0.1:35647}
[junit4] 2> 2637902 INFO (jetty-launcher-2650-thread-2) [ ] o.e.j.s.Server Started @2640285ms
[junit4] 2> 2637902 INFO (jetty-launcher-2650-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35647}
[junit4] 2> 2637902 ERROR (jetty-launcher-2650-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2637903 INFO (jetty-launcher-2650-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 2637903 INFO (jetty-launcher-2650-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2637903 INFO (jetty-launcher-2650-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2637903 INFO (jetty-launcher-2650-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-10-31T11:55:06.718Z
[junit4] 2> 2637905 INFO (jetty-launcher-2650-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2637906 INFO (jetty-launcher-2650-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2637912 INFO (jetty-launcher-2650-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:50583/solr
[junit4] 2> 2637912 INFO (jetty-launcher-2650-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:50583/solr
[junit4] 2> 2638046 INFO (jetty-launcher-2650-thread-2) [n:127.0.0.1:35647_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2638046 INFO (jetty-launcher-2650-thread-1) [n:127.0.0.1:40157_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2638046 INFO (jetty-launcher-2650-thread-2) [n:127.0.0.1:35647_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:35647_solr
[junit4] 2> 2638047 INFO (jetty-launcher-2650-thread-1) [n:127.0.0.1:40157_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40157_solr
[junit4] 2> 2638048 INFO (jetty-launcher-2650-thread-2) [n:127.0.0.1:35647_solr ] o.a.s.c.Overseer Overseer (id=98923374614347781-127.0.0.1:35647_solr-n_0000000000) starting
[junit4] 2> 2638049 INFO (zkCallback-2661-thread-1-processing-n:127.0.0.1:35647_solr) [n:127.0.0.1:35647_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2638049 INFO (zkCallback-2662-thread-1-processing-n:127.0.0.1:40157_solr) [n:127.0.0.1:40157_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2638058 INFO (jetty-launcher-2650-thread-2) [n:127.0.0.1:35647_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35647_solr
[junit4] 2> 2638061 INFO (zkCallback-2662-thread-1-processing-n:127.0.0.1:40157_solr) [n:127.0.0.1:40157_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2638061 INFO (zkCallback-2661-thread-1-processing-n:127.0.0.1:35647_solr) [n:127.0.0.1:35647_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2638075 INFO (jetty-launcher-2650-thread-1) [n:127.0.0.1:40157_solr ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.security.HttpParamDelegationTokenPlugin
[junit4] 2> 2638075 WARN (jetty-launcher-2650-thread-1) [n:127.0.0.1:40157_solr ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
[junit4] 2> 2638075 INFO (jetty-launcher-2650-thread-1) [n:127.0.0.1:40157_solr ] o.a.s.s.KerberosPlugin Params: {solr.kerberos.impersonator.user.noGroups.hosts=*, solr.kerberos.impersonator.user.bogusGroup.groups=__some_bogus_group, token.valid=30, cookie.domain=127.0.0.1, solr.kerberos.impersonator.user.anyHostAnyUser.groups=*, solr.kerberos.impersonator.user.bogusGroup.hosts=*, solr.kerberos.impersonator.user.anyHostAnyUser.hosts=*, token.validity=36000, solr.kerberos.impersonator.user.noHosts.groups=*, type=kerberos, solr.kerberos.impersonator.user.localHostAnyGroup.groups=*, delegation-token.token-kind=solr-dt, solr.kerberos.impersonator.user.wrongHost.hosts=1.1.1.1.1.1, cookie.path=/, zk-dt-secret-manager.znodeWorkingPath=solr/security/zkdtsm, solr.kerberos.impersonator.user.localHostAnyGroup.hosts=serv1-vm3,localhost,127.0.0.1, signer.secret.provider.zookeeper.path=/token, solr.kerberos.impersonator.user.anyHostUsersGroup.groups=staff, zk-dt-secret-manager.enable=true, solr.kerberos.impersonator.user.wrongHost.groups=*, kerberos.name.rules=DEFAULT, signer.secret.provider=zookeeper, solr.kerberos.impersonator.user.anyHostUsersGroup.hosts=*}
[junit4] 2> 2638076 INFO (jetty-launcher-2650-thread-1) [n:127.0.0.1:40157_solr ] o.a.c.f.i.CuratorFrameworkImpl Starting
[junit4] 2> 2638082 INFO (jetty-launcher-2650-thread-1-EventThread) [n:127.0.0.1:40157_solr ] o.a.c.f.s.ConnectionStateManager State change: CONNECTED
[junit4] 2> 2638099 WARN (jetty-launcher-2650-thread-1) [n:127.0.0.1:40157_solr ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
[junit4] 2> 2638132 INFO (jetty-launcher-2650-thread-1) [n:127.0.0.1:40157_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40157.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@39be70d3
[junit4] 2> 2638136 INFO (jetty-launcher-2650-thread-2) [n:127.0.0.1:35647_solr ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.security.HttpParamDelegationTokenPlugin
[junit4] 2> 2638136 WARN (jetty-launcher-2650-thread-2) [n:127.0.0.1:35647_solr ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
[junit4] 2> 2638136 INFO (jetty-launcher-2650-thread-2) [n:127.0.0.1:35647_solr ] o.a.s.s.KerberosPlugin Params: {solr.kerberos.impersonator.user.noGroups.hosts=*, solr.kerberos.impersonator.user.bogusGroup.groups=__some_bogus_group, token.valid=30, cookie.domain=127.0.0.1, solr.kerberos.impersonator.user.anyHostAnyUser.groups=*, solr.kerberos.impersonator.user.bogusGroup.hosts=*, solr.kerberos.impersonator.user.anyHostAnyUser.hosts=*, token.validity=36000, solr.kerberos.impersonator.user.noHosts.groups=*, type=kerberos, solr.kerberos.impersonator.user.localHostAnyGroup.groups=*, delegation-token.token-kind=solr-dt, solr.kerberos.impersonator.user.wrongHost.hosts=1.1.1.1.1.1, cookie.path=/, zk-dt-secret-manager.znodeWorkingPath=solr/security/zkdtsm, solr.kerberos.impersonator.user.localHostAnyGroup.hosts=serv1-vm3,localhost,127.0.0.1, signer.secret.provider.zookeeper.path=/token, solr.kerberos.impersonator.user.anyHostUsersGroup.groups=staff, zk-dt-secret-manager.enable=true, solr.kerberos.impersonator.user.wrongHost.groups=*, kerberos.name.rules=DEFAULT, signer.secret.provider=zookeeper, solr.kerberos.impersonator.user.anyHostUsersGroup.hosts=*}
[junit4] 2> 2638137 INFO (jetty-launcher-2650-thread-2) [n:127.0.0.1:35647_solr ] o.a.c.f.i.CuratorFrameworkImpl Starting
[junit4] 2> 2638138 INFO (jetty-launcher-2650-thread-1) [n:127.0.0.1:40157_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40157.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@39be70d3
[junit4] 2> 2638138 INFO (jetty-launcher-2650-thread-1) [n:127.0.0.1:40157_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40157.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@39be70d3
[junit4] 2> 2638139 INFO (jetty-launcher-2650-thread-1) [n:127.0.0.1:40157_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_4DE8E6CD4434BA82-001/tempDir-001/node1/.
[junit4] 2> 2638139 INFO (jetty-launcher-2650-thread-2-EventThread) [n:127.0.0.1:35647_solr ] o.a.c.f.s.ConnectionStateManager State change: CONNECTED
[junit4] 2> 2638157 WARN (jetty-launcher-2650-thread-2) [n:127.0.0.1:35647_solr ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
[junit4] 2> 2638184 INFO (jetty-launcher-2650-thread-2) [n:127.0.0.1:35647_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35647.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@39be70d3
[junit4] 2> 2638190 INFO (jetty-launcher-2650-thread-2) [n:127.0.0.1:35647_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35647.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@39be70d3
[junit4] 2> 2638190 INFO (jetty-launcher-2650-thread-2) [n:127.0.0.1:35647_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35647.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@39be70d3
[junit4] 2> 2638190 INFO (jetty-launcher-2650-thread-2) [n:127.0.0.1:35647_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_4DE8E6CD4434BA82-001/tempDir-001/node2/.
[junit4] 2> 2638228 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNullProxyUser-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNullProxyUser
[junit4] 2> 2638231 INFO (qtp352836052-18276) [ ] o.a.s.s.SolrDispatchFilter Error authenticating
[junit4] 2> java.lang.IllegalArgumentException: userName is NULL, empty or contains a '&'
[junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.checkForIllegalArgument(AuthToken.java:87)
[junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.<init>(AuthToken.java:70)
[junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationToken.<init>(AuthenticationToken.java:58)
[junit4] 2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamDelegationTokenAuthenticationHandler$HttpParamAuthenticationHandler.authenticate(HttpParamDelegationTokenPlugin.java:213)
[junit4] 2> at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.authenticate(DelegationTokenAuthenticationHandler.java:348)
[junit4] 2> at org.apache.solr.security.RequestContinuesRecorderAuthenticationHandler.authenticate(RequestContinuesRecorderAuthenticationHandler.java:69)
[junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:518)
[junit4] 2> at org.apache.solr.security.DelegationTokenKerberosFilter.doFilter(DelegationTokenKerberosFilter.java:134)
[junit4] 2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamToRequestFilter.doFilter(HttpParamDelegationTokenPlugin.java:245)
[junit4] 2> at org.apache.solr.security.KerberosPlugin.doAuthenticate(KerberosPlugin.java:243)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:451)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 2638232 WARN (qtp352836052-18276) [ ] o.e.j.s.ServletHandler /solr/admin/collections
[junit4] 2> org.apache.solr.common.SolrException: Error during request authentication,
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:457)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: java.lang.IllegalArgumentException: userName is NULL, empty or contains a '&'
[junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.checkForIllegalArgument(AuthToken.java:87)
[junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.<init>(AuthToken.java:70)
[junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationToken.<init>(AuthenticationToken.java:58)
[junit4] 2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamDelegationTokenAuthenticationHandler$HttpParamAuthenticationHandler.authenticate(HttpParamDelegationTokenPlugin.java:213)
[junit4] 2> at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.authenticate(DelegationTokenAuthenticationHandler.java:348)
[junit4] 2> at org.apache.solr.security.RequestContinuesRecorderAuthenticationHandler.authenticate(RequestContinuesRecorderAuthenticationHandler.java:69)
[junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:518)
[junit4] 2> at org.apache.solr.security.DelegationTokenKerberosFilter.doFilter(DelegationTokenKerberosFilter.java:134)
[junit4] 2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamToRequestFilter.doFilter(HttpParamDelegationTokenPlugin.java:245)
[junit4] 2> at org.apache.solr.security.KerberosPlugin.doAuthenticate(KerberosPlugin.java:243)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:451)
[junit4] 2> ... 26 more
[junit4] 2> 2638232 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNullProxyUser-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyNullProxyUser
[junit4] 2> 2638252 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidProxyUser-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidProxyUser
[junit4] 2> 2638256 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidProxyUser-seed#[4DE8E6CD4434BA82]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=bar&p=bar&t=test&e=1509486907068&s=uuP35UKt4dL9bsEvF3VUqghdA8k="; Path=/; Domain=127.0.0.1; Expires=Út, 31- íj-2017 21:55:07 GMT; HttpOnly". Invalid 'expires' attribute: Út, 31- íj-2017 21:55:07 GMT
[junit4] 2> 2638256 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidProxyUser-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidProxyUser
[junit4] 2> 2638281 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidRemote-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidRemote
[junit4] 2> 2638287 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidRemote-seed#[4DE8E6CD4434BA82]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=localHostAnyGroup&p=localHostAnyGroup&t=test&e=1509486907096&s=pD91X2V1rbyxec2/wo8GYyuwUMU="; Path=/; Domain=127.0.0.1; Expires=Út, 31- íj-2017 21:55:07 GMT; HttpOnly". Invalid 'expires' attribute: Út, 31- íj-2017 21:55:07 GMT
[junit4] 2> 2638288 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidRemote-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidRemote
[junit4] 2> 2638311 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyWrongHost-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyWrongHost
[junit4] 2> 2653358 WARN (jetty-launcher-2650-thread-1-SendThread(127.0.0.1:50583)) [n:127.0.0.1:40157_solr ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 15196ms for sessionid 0x15f7248fbdf0007
[junit4] 2> 2653358 WARN (jetty-launcher-2650-thread-1-SendThread(127.0.0.1:50583)) [n:127.0.0.1:40157_solr ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 15219ms for sessionid 0x15f7248fbdf0006
[junit4] 2> 2653358 WARN (jetty-launcher-2650-thread-2-SendThread(127.0.0.1:50583)) [n:127.0.0.1:35647_solr ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 15196ms for sessionid 0x15f7248fbdf0008
[junit4] 2> 2653359 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15f7248fbdf0008, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 2653359 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15f7248fbdf0006, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 2653359 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15f7248fbdf0007, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 2653360 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyWrongHost-seed#[4DE8E6CD4434BA82]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=wrongHost&p=wrongHost&t=test&e=1509486907126&s=6N8of4Oc6rJSu+d45GPMghVE0ck="; Path=/; Domain=127.0.0.1; Expires=Út, 31- íj-2017 21:55:07 GMT; HttpOnly". Invalid 'expires' attribute: Út, 31- íj-2017 21:55:07 GMT
[junit4] 2> 2653359 WARN (jetty-launcher-2650-thread-2-SendThread(127.0.0.1:50583)) [n:127.0.0.1:35647_solr ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 15168ms for sessionid 0x15f7248fbdf0005
[junit4] 2> 2653360 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyWrongHost-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyWrongHost
[junit4] 2> 2653360 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15f7248fbdf0005, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 2653403 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateHost-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateHost
[junit4] 2> 2653404 INFO (qtp352836052-18273) [n:127.0.0.1:40157_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=bar&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2653404 INFO (qtp352836052-18273) [n:127.0.0.1:40157_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=bar&action=LIST&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2653405 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateHost-seed#[4DE8E6CD4434BA82]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=localHostAnyGroup&p=localHostAnyGroup&t=test&e=1509486922219&s=rfIr8TYD1guipoLtvfbzJltuMjA="; Path=/; Domain=127.0.0.1; Expires=Út, 31- íj-2017 21:55:22 GMT; HttpOnly". Invalid 'expires' attribute: Út, 31- íj-2017 21:55:22 GMT
[junit4] 2> 2653405 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateHost-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateHost
[junit4] 2> 2653438 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateGroup-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateGroup
[junit4] 2> 2653439 INFO (qtp352836052-18276) [n:127.0.0.1:40157_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=jenkins&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2653439 INFO (qtp352836052-18276) [n:127.0.0.1:40157_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=jenkins&action=LIST&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2653440 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateGroup-seed#[4DE8E6CD4434BA82]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=anyHostUsersGroup&p=anyHostUsersGroup&t=test&e=1509486922254&s=EMK7GVi1sY96qP9mkXh7rxhk24M="; Path=/; Domain=127.0.0.1; Expires=Út, 31- íj-2017 21:55:22 GMT; HttpOnly". Invalid 'expires' attribute: Út, 31- íj-2017 21:55:22 GMT
[junit4] 2> 2653440 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateGroup-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateGroup
[junit4] 2> 2653468 WARN (zkCallback-2662-thread-5-processing-n:127.0.0.1:40157_solr) [n:127.0.0.1:40157_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7746eb07 name: ZooKeeperConnection Watcher:127.0.0.1:50583/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
[junit4] 2> 2653469 WARN (zkCallback-2661-thread-1-processing-n:127.0.0.1:35647_solr) [n:127.0.0.1:35647_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@53585654 name: ZooKeeperConnection Watcher:127.0.0.1:50583/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
[junit4] 2> 2653479 WARN (zkCallback-2661-thread-1-processing-n:127.0.0.1:35647_solr) [n:127.0.0.1:35647_solr ] o.a.s.c.c.ConnectionManager zkClient has disconnected
[junit4] 2> 2653481 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateAnyHostAnyUser-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateAnyHostAnyUser
[junit4] 2> 2653473 WARN (zkCallback-2662-thread-5-processing-n:127.0.0.1:40157_solr) [n:127.0.0.1:40157_solr ] o.a.s.c.c.ConnectionManager zkClient has disconnected
[junit4] 2> 2653482 INFO (qtp352836052-18277) [n:127.0.0.1:40157_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=bar&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2653482 INFO (qtp352836052-18277) [n:127.0.0.1:40157_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=bar&action=LIST&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2653482 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateAnyHostAnyUser-seed#[4DE8E6CD4434BA82]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=anyHostAnyUser&p=anyHostAnyUser&t=test&e=1509486922296&s=OM8nd7qc9TJE143nY5p3TqXaiqY="; Path=/; Domain=127.0.0.1; Expires=Út, 31- íj-2017 21:55:22 GMT; HttpOnly". Invalid 'expires' attribute: Út, 31- íj-2017 21:55:22 GMT
[junit4] 2> 2653483 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateAnyHostAnyUser-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateAnyHostAnyUser
[junit4] 2> 2653511 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testForwarding
[junit4] 2> 2654173 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2654174 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[4DE8E6CD4434BA82]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:50583/solr ready
[junit4] 2> 2654178 INFO (qtp1171478510-18285) [n:127.0.0.1:35647_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=0&replicationFactor=1&collection.configName=conf1&maxShardsPerNode=1&name=forwardingCollection&nrtReplicas=1&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2655114 INFO (OverseerThreadFactory-6884-thread-1-processing-n:127.0.0.1:35647_solr) [n:127.0.0.1:35647_solr ] o.a.s.c.CreateCollectionCmd Create collection forwardingCollection
[junit4] 2> 2655159 INFO (jetty-launcher-2650-thread-1-EventThread) [n:127.0.0.1:40157_solr ] o.a.c.f.s.ConnectionStateManager State change: SUSPENDED
[junit4] 2> 2655234 INFO (OverseerStateUpdate-98923374614347781-127.0.0.1:35647_solr-n_0000000000) [n:127.0.0.1:35647_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"forwardingCollection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"forwardingCollection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:40157/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2655344 INFO (jetty-launcher-2650-thread-2-EventThread) [n:127.0.0.1:35647_solr ] o.a.c.f.s.ConnectionStateManager State change: SUSPENDED
[junit4] 2> 2655438 INFO (qtp352836052-18278) [n:127.0.0.1:40157_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf1&newCollection=true&name=forwardingCollection_shard1_replica_n1&action=CREATE&numShards=1&collection=forwardingCollection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2655438 INFO (qtp352836052-18278) [n:127.0.0.1:40157_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2655544 INFO (zkCallback-2662-thread-1-processing-n:127.0.0.1:40157_solr) [n:127.0.0.1:40157_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2655544 INFO (zkCallback-2662-thread-6-processing-n:127.0.0.1:40157_solr) [n:127.0.0.1:40157_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2656463 INFO (qtp352836052-18278) [n:127.0.0.1:40157_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2656487 INFO (qtp352836052-18278) [n:127.0.0.1:40157_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.IndexSchema [forwardingCollection_shard1_replica_n1] Schema name=test
[junit4] 2> 2656594 INFO (qtp352836052-18278) [n:127.0.0.1:40157_solr c:forwardingCollection s:shard1 r:core_no
[...truncated too long message...]
530)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
[junit4] 2> Oct 31, 2017 11:55:50 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: All leaked threads terminated.
[junit4] 2> NOTE: test params are: codec=HighCompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=18213, maxDocsPerChunk=268, blockSize=736), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=18213, blockSize=736)), sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@41c74418), locale=cs, timezone=Indian/Christmas
[junit4] 2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=3,threads=1,free=125113856,total=373157888
[junit4] 2> NOTE: All tests run in this JVM: [BigEndianAscendingWordSerializerTest, TestSegmentSorting, AutoAddReplicasPlanActionTest, TestSizeLimitedDistributedMap, TestPHPSerializedResponseWriter, TemplateUpdateProcessorTest, TestHdfsCloudBackupRestore, NoCacheHeaderTest, TestDownShardTolerantSearch, MergeStrategyTest, TestFiltering, TestSimpleQParserPlugin, SolrCmdDistributorTest, TestCloudPseudoReturnFields, BlockDirectoryTest, TestSchemaResource, CSVRequestHandlerTest, DirectoryFactoryTest, ExplicitHLLTest, SegmentsInfoRequestHandlerTest, TestSolr4Spatial2, TestDistributedSearch, TestLegacyNumericUtils, TestRecovery, TestMaxTokenLenTokenizer, CleanupOldIndexTest, TestSolrCoreProperties, TestUseDocValuesAsStored, NumericFieldsTest, SpatialRPTFieldTypeTest, TestPullReplicaErrorHandling, CoreAdminRequestStatusTest, TestReversedWildcardFilterFactory, DistributedFacetPivotSmallTest, TestLMDirichletSimilarityFactory, HdfsDirectoryTest, TestSubQueryTransformerCrossCore, ConfigureRecoveryStrategyTest, TestSlowCompositeReaderWrapper, ChaosMonkeyNothingIsSafeTest, DistributedDebugComponentTest, AnalyticsMergeStrategyTest, HdfsThreadLeakTest, TestPerFieldSimilarityWithDefaultOverride, TestDefaultStatsCache, WrapperMergePolicyFactoryTest, RegexBytesRefFilterTest, TestSolrDeletionPolicy2, TestXIncludeConfig, TestCustomDocTransformer, ChaosMonkeySafeLeaderWithPullReplicasTest, TestNumericTerms32, DataDrivenBlockJoinTest, TestConfigSetImmutable, TestAtomicUpdateErrorCases, TestMultiWordSynonyms, TestSolrXml, TestImplicitCoreProperties, SolrGraphiteReporterTest, AddSchemaFieldsUpdateProcessorFactoryTest, ShufflingReplicaListTransformerTest, HLLSerializationTest, TestPseudoReturnFields, TestDocTermOrds, FastVectorHighlighterTest, SolrMetricsIntegrationTest, TestSurroundQueryParser, TestFieldCacheReopen, TestConfigSetsAPI, TestRequestStatusCollectionAPI, BasicDistributedZkTest, StatsComponentTest, TestEmbeddedSolrServerSchemaAPI, TestDeleteCollectionOnDownNodes, TestBinaryField, TestChildDocTransformer, SliceStateTest, ReturnFieldsTest, HdfsRestartWhileUpdatingTest, InfixSuggestersTest, TestCSVResponseWriter, CursorMarkTest, DistributedTermsComponentTest, TestReloadAndDeleteDocs, TestDynamicFieldResource, HdfsChaosMonkeyNothingIsSafeTest, TestCloudJSONFacetJoinDomain, DistributedSuggestComponentTest, TestTolerantUpdateProcessorRandomCloud, TestDelegationWithHadoopAuth, TestHighlightDedupGrouping, DistribJoinFromCollectionTest, TimeZoneUtilsTest, MigrateRouteKeyTest, TestTestInjection, PrimUtilsTest, TestLocalFSCloudBackupRestore, ClusterStateTest, BadIndexSchemaTest, TriLevelCompositeIdRoutingTest, TestFieldCacheSortRandom, RegexBoostProcessorTest, TestSolrFieldCacheBean, MetricsConfigTest, TestJsonFacetRefinement, DefaultValueUpdateProcessorTest, ComputePlanActionTest, ConfigSetsAPITest, TestSolrCloudWithDelegationTokens, SmileWriterTest, TestCustomStream, TestPhraseSuggestions, SolrMetricManagerTest, ConvertedLegacyTest, TestRealTimeGet, TestRawResponseWriter, SaslZkACLProviderTest, CursorPagingTest, UUIDFieldTest, TestSchemaManager, UniqFieldsUpdateProcessorFactoryTest, TestRestManager, SolrCloudReportersTest, TestCloudSearcherWarming, OverseerModifyCollectionTest, ZkFailoverTest, TestGraphTermsQParserPlugin, SpellCheckComponentTest, RulesTest, SpellCheckCollatorTest, TestQuerySenderNoQuery, SignatureUpdateProcessorFactoryTest, SolrRequestParserTest, TestSolrConfigHandlerConcurrent, TestInitParams, DateMathParserTest, CreateCollectionCleanupTest, HdfsBasicDistributedZk2Test, ShowFileRequestHandlerTest, TestJsonRequest, TestExceedMaxTermLength, OverseerStatusTest, TestUniqueKeyFieldResource, TestPayloadScoreQParserPlugin, TestJavabinTupleStreamParser, TestStressInPlaceUpdates, DateFieldTest, TestRandomRequestDistribution, SolrXmlInZkTest, TestSortByMinMaxFunction, AutoscalingHistoryHandlerTest, HdfsRecoverLeaseTest, SimpleMLTQParserTest, TestSolr4Spatial, PrimitiveFieldTypeTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestReloadDeadlock, CdcrBootstrapTest, AnalysisErrorHandlingTest, TestFileDictionaryLookup, DistanceUnitsTest, TestAuthorizationFramework, TestTlogReplica, TestReRankQParserPlugin, DistributedSpellCheckComponentTest, TestReplicationHandlerBackup, TestConfigsApi, TestIBSimilarityFactory, OutputWriterTest, TestAnalyzeInfixSuggestions, IndexBasedSpellCheckerTest, TestOverriddenPrefixQueryForCustomFieldType, TestCursorMarkWithoutUniqueKey, DisMaxRequestHandlerTest, ResponseLogComponentTest, DeleteNodeTest, TestSolrCloudWithSecureImpersonation]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestSolrCloudWithSecureImpersonation -Dtests.seed=4DE8E6CD4434BA82 -Dtests.slow=true -Dtests.locale=cs -Dtests.timezone=Indian/Christmas -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.00s J0 | TestSolrCloudWithSecureImpersonation (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
[junit4] > 1) Thread[id=18321, name=jetty-launcher-2650-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
[junit4] > at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
[junit4] > at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
[junit4] > at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
[junit4] > at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
[junit4] > 2) Thread[id=18331, name=jetty-launcher-2650-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
[junit4] > at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
[junit4] > at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
[junit4] > at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
[junit4] > at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
[junit4] > at __randomizedtesting.SeedInfo.seed([4DE8E6CD4434BA82]:0)
[junit4] Completed [301/747 (2!)] on J0 in 44.00s, 12 tests, 1 error <<< FAILURES!
[...truncated 48789 lines...]
[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build #
1499 - Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1499/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseParallelGC
1 tests failed.
FAILED: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI
Error Message:
Error from server at http://127.0.0.1:33704/solr/awhollynewcollection_0: {"awhollynewcollection_0":5}
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:33704/solr/awhollynewcollection_0: {"awhollynewcollection_0":5}
at __randomizedtesting.SeedInfo.seed([7FDC08545B534090:37A97CE05D606F05]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:626)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1096)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:875)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:982)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:982)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:982)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:982)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:982)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:808)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:460)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12701 lines...]
[junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
[junit4] 2> 1979444 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[7FDC08545B534090]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7FDC08545B534090-001/init-core-data-001
[junit4] 2> 1979445 WARN (SUITE-CollectionsAPIDistributedZkTest-seed#[7FDC08545B534090]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=46 numCloses=46
[junit4] 2> 1979445 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[7FDC08545B534090]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 1979446 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[7FDC08545B534090]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 1979447 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[7FDC08545B534090]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7FDC08545B534090-001/tempDir-001
[junit4] 2> 1979447 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[7FDC08545B534090]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1979447 INFO (Thread-6560) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1979447 INFO (Thread-6560) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1979449 ERROR (Thread-6560) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1979547 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[7FDC08545B534090]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:57396
[junit4] 2> 1979554 INFO (jetty-launcher-3194-thread-1) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 1979554 INFO (jetty-launcher-3194-thread-4) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 1979554 INFO (jetty-launcher-3194-thread-3) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 1979555 INFO (jetty-launcher-3194-thread-2) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 1979556 INFO (jetty-launcher-3194-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3291a3a5{/solr,null,AVAILABLE}
[junit4] 2> 1979557 INFO (jetty-launcher-3194-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4e5eb8f4{/solr,null,AVAILABLE}
[junit4] 2> 1979557 INFO (jetty-launcher-3194-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@fc7413{/solr,null,AVAILABLE}
[junit4] 2> 1979557 INFO (jetty-launcher-3194-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3cdf52b1{HTTP/1.1,[http/1.1]}{127.0.0.1:51819}
[junit4] 2> 1979557 INFO (jetty-launcher-3194-thread-2) [ ] o.e.j.s.Server Started @1981506ms
[junit4] 2> 1979557 INFO (jetty-launcher-3194-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=51819}
[junit4] 2> 1979557 ERROR (jetty-launcher-3194-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1979557 INFO (jetty-launcher-3194-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1979557 INFO (jetty-launcher-3194-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1979557 INFO (jetty-launcher-3194-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1979558 INFO (jetty-launcher-3194-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-10-31T01:17:42.506Z
[junit4] 2> 1979558 INFO (jetty-launcher-3194-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@726550d1{HTTP/1.1,[http/1.1]}{127.0.0.1:48992}
[junit4] 2> 1979558 INFO (jetty-launcher-3194-thread-1) [ ] o.e.j.s.Server Started @1981506ms
[junit4] 2> 1979558 INFO (jetty-launcher-3194-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=48992}
[junit4] 2> 1979558 ERROR (jetty-launcher-3194-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1979558 INFO (jetty-launcher-3194-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1979558 INFO (jetty-launcher-3194-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1979558 INFO (jetty-launcher-3194-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1979558 INFO (jetty-launcher-3194-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-10-31T01:17:42.506Z
[junit4] 2> 1979561 INFO (jetty-launcher-3194-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5789ffb0{HTTP/1.1,[http/1.1]}{127.0.0.1:41084}
[junit4] 2> 1979561 INFO (jetty-launcher-3194-thread-3) [ ] o.e.j.s.Server Started @1981509ms
[junit4] 2> 1979561 INFO (jetty-launcher-3194-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=41084}
[junit4] 2> 1979561 ERROR (jetty-launcher-3194-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1979561 INFO (jetty-launcher-3194-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1a436554{/solr,null,AVAILABLE}
[junit4] 2> 1979561 INFO (jetty-launcher-3194-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1979561 INFO (jetty-launcher-3194-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1979561 INFO (jetty-launcher-3194-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1979561 INFO (jetty-launcher-3194-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-10-31T01:17:42.509Z
[junit4] 2> 1979561 INFO (jetty-launcher-3194-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@292e44e3{HTTP/1.1,[http/1.1]}{127.0.0.1:33704}
[junit4] 2> 1979561 INFO (jetty-launcher-3194-thread-4) [ ] o.e.j.s.Server Started @1981510ms
[junit4] 2> 1979561 INFO (jetty-launcher-3194-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33704}
[junit4] 2> 1979561 ERROR (jetty-launcher-3194-thread-4) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1979561 INFO (jetty-launcher-3194-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1979561 INFO (jetty-launcher-3194-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1979561 INFO (jetty-launcher-3194-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1979561 INFO (jetty-launcher-3194-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-10-31T01:17:42.509Z
[junit4] 2> 1979564 INFO (jetty-launcher-3194-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1979564 INFO (jetty-launcher-3194-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1979565 INFO (jetty-launcher-3194-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1979566 INFO (jetty-launcher-3194-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1979567 INFO (jetty-launcher-3194-thread-1) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 1979567 INFO (jetty-launcher-3194-thread-1) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 1979568 INFO (jetty-launcher-3194-thread-2) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 1979568 INFO (jetty-launcher-3194-thread-2) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 1979568 INFO (jetty-launcher-3194-thread-4) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 1979568 INFO (jetty-launcher-3194-thread-4) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 1979569 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15f70016c7c0003, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1979569 INFO (jetty-launcher-3194-thread-3) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 1979569 INFO (jetty-launcher-3194-thread-3) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 1979571 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15f70016c7c0001, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1979571 INFO (jetty-launcher-3194-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:57396/solr
[junit4] 2> 1979571 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15f70016c7c0004, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1979572 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15f70016c7c0002, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1979572 INFO (jetty-launcher-3194-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:57396/solr
[junit4] 2> 1979572 INFO (jetty-launcher-3194-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:57396/solr
[junit4] 2> 1979574 INFO (jetty-launcher-3194-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:57396/solr
[junit4] 2> 1979579 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15f70016c7c0008, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1979763 INFO (jetty-launcher-3194-thread-1) [n:127.0.0.1:48992_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1979764 INFO (jetty-launcher-3194-thread-1) [n:127.0.0.1:48992_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:48992_solr
[junit4] 2> 1979767 INFO (jetty-launcher-3194-thread-1) [n:127.0.0.1:48992_solr ] o.a.s.c.Overseer Overseer (id=98920868242784265-127.0.0.1:48992_solr-n_0000000000) starting
[junit4] 2> 1979776 INFO (jetty-launcher-3194-thread-3) [n:127.0.0.1:41084_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1979778 INFO (jetty-launcher-3194-thread-3) [n:127.0.0.1:41084_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41084_solr
[junit4] 2> 1979780 INFO (zkCallback-3216-thread-1-processing-n:127.0.0.1:41084_solr) [n:127.0.0.1:41084_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1979780 INFO (jetty-launcher-3194-thread-4) [n:127.0.0.1:33704_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1979781 INFO (zkCallback-3215-thread-1-processing-n:127.0.0.1:51819_solr) [n:127.0.0.1:51819_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1979781 INFO (zkCallback-3213-thread-1-processing-n:127.0.0.1:48992_solr) [n:127.0.0.1:48992_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1979781 INFO (jetty-launcher-3194-thread-2) [n:127.0.0.1:51819_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1979786 INFO (jetty-launcher-3194-thread-4) [n:127.0.0.1:33704_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1979786 INFO (jetty-launcher-3194-thread-2) [n:127.0.0.1:51819_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51819_solr
[junit4] 2> 1979788 INFO (jetty-launcher-3194-thread-1) [n:127.0.0.1:48992_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48992_solr
[junit4] 2> 1979789 INFO (zkCallback-3213-thread-1-processing-n:127.0.0.1:48992_solr) [n:127.0.0.1:48992_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1979789 INFO (zkCallback-3215-thread-1-processing-n:127.0.0.1:51819_solr) [n:127.0.0.1:51819_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1979789 INFO (zkCallback-3216-thread-1-processing-n:127.0.0.1:41084_solr) [n:127.0.0.1:41084_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1979790 INFO (zkCallback-3214-thread-1-processing-n:127.0.0.1:33704_solr) [n:127.0.0.1:33704_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1979795 INFO (zkCallback-3216-thread-1-processing-n:127.0.0.1:41084_solr) [n:127.0.0.1:41084_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1979795 INFO (zkCallback-3214-thread-1-processing-n:127.0.0.1:33704_solr) [n:127.0.0.1:33704_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1979795 INFO (zkCallback-3215-thread-1-processing-n:127.0.0.1:51819_solr) [n:127.0.0.1:51819_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1979797 INFO (zkCallback-3213-thread-1-processing-n:127.0.0.1:48992_solr) [n:127.0.0.1:48992_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1979797 INFO (jetty-launcher-3194-thread-4) [n:127.0.0.1:33704_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33704_solr
[junit4] 2> 1979827 INFO (jetty-launcher-3194-thread-3) [n:127.0.0.1:41084_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e3f0961
[junit4] 2> 1979839 INFO (zkCallback-3216-thread-1-processing-n:127.0.0.1:41084_solr) [n:127.0.0.1:41084_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1979849 INFO (zkCallback-3214-thread-1-processing-n:127.0.0.1:33704_solr) [n:127.0.0.1:33704_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1979849 INFO (zkCallback-3215-thread-1-processing-n:127.0.0.1:51819_solr) [n:127.0.0.1:51819_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1979849 INFO (zkCallback-3213-thread-1-processing-n:127.0.0.1:48992_solr) [n:127.0.0.1:48992_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1979850 INFO (jetty-launcher-3194-thread-3) [n:127.0.0.1:41084_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e3f0961
[junit4] 2> 1979850 INFO (jetty-launcher-3194-thread-3) [n:127.0.0.1:41084_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e3f0961
[junit4] 2> 1979850 INFO (jetty-launcher-3194-thread-3) [n:127.0.0.1:41084_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7FDC08545B534090-001/tempDir-001/node4
[junit4] 2> 1979860 INFO (jetty-launcher-3194-thread-2) [n:127.0.0.1:51819_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e3f0961
[junit4] 2> 1979875 INFO (jetty-launcher-3194-thread-2) [n:127.0.0.1:51819_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e3f0961
[junit4] 2> 1979875 INFO (jetty-launcher-3194-thread-2) [n:127.0.0.1:51819_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e3f0961
[junit4] 2> 1979876 INFO (jetty-launcher-3194-thread-2) [n:127.0.0.1:51819_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7FDC08545B534090-001/tempDir-001/node1
[junit4] 2> 1979896 INFO (jetty-launcher-3194-thread-4) [n:127.0.0.1:33704_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e3f0961
[junit4] 2> 1979902 INFO (jetty-launcher-3194-thread-4) [n:127.0.0.1:33704_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e3f0961
[junit4] 2> 1979902 INFO (jetty-launcher-3194-thread-4) [n:127.0.0.1:33704_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e3f0961
[junit4] 2> 1979902 INFO (jetty-launcher-3194-thread-4) [n:127.0.0.1:33704_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7FDC08545B534090-001/tempDir-001/node3
[junit4] 2> 1979917 INFO (jetty-launcher-3194-thread-1) [n:127.0.0.1:48992_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e3f0961
[junit4] 2> 1979923 INFO (jetty-launcher-3194-thread-1) [n:127.0.0.1:48992_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e3f0961
[junit4] 2> 1979923 INFO (jetty-launcher-3194-thread-1) [n:127.0.0.1:48992_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e3f0961
[junit4] 2> 1979924 INFO (jetty-launcher-3194-thread-1) [n:127.0.0.1:48992_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7FDC08545B534090-001/tempDir-001/node2
[junit4] 2> 1979950 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[7FDC08545B534090]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 1979951 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[7FDC08545B534090]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:57396/solr ready
[junit4] 2> 1979978 INFO (TEST-CollectionsAPIDistributedZkTest.addReplicaTest-seed#[7FDC08545B534090]) [ ] o.a.s.SolrTestCaseJ4 ###Starting addReplicaTest
[junit4] 2> 1979979 INFO (TEST-CollectionsAPIDistributedZkTest.addReplicaTest-seed#[7FDC08545B534090]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 1979983 INFO (qtp29335138-18888) [n:127.0.0.1:33704_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf&maxShardsPerNode=4&name=addReplicaColl&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1979985 INFO (OverseerThreadFactory-6563-thread-1-processing-n:127.0.0.1:48992_solr) [n:127.0.0.1:48992_solr ] o.a.s.c.CreateCollectionCmd Create collection addReplicaColl
[junit4] 2> 1980104 INFO (OverseerStateUpdate-98920868242784265-127.0.0.1:48992_solr-n_0000000000) [n:127.0.0.1:48992_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"addReplicaColl",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"addReplicaColl_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:33704/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1980110 INFO (OverseerStateUpdate-98920868242784265-127.0.0.1:48992_solr-n_0000000000) [n:127.0.0.1:48992_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"addReplicaColl",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"addReplicaColl_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:41084/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1980116 INFO (OverseerStateUpdate-98920868242784265-127.0.0.1:48992_solr-n_0000000000) [n:127.0.0.1:48992_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"addReplicaColl",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"addReplicaColl_shard2_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:48992/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1980124 INFO (OverseerStateUpdate-98920868242784265-127.0.0.1:48992_solr-n_0000000000) [n:127.0.0.1:48992_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"addReplicaColl",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"addReplicaColl_shard2_replica_n6",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:51819/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1980333 INFO (qtp29335138-18889) [n:127.0.0.1:33704_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=addReplicaColl_shard1_replica_n1&action=CREATE&numShards=2&collection=addReplicaColl&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1980336 INFO (qtp29335138-18889) [n:127.0.0.1:33704_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1980337 INFO (qtp681167845-18871) [n:127.0.0.1:51819_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf&newCollection=true&name=addReplicaColl_shard2_replica_n6&action=CREATE&numShards=2&collection=addReplicaColl&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1980337 INFO (qtp137164455-18861) [n:127.0.0.1:41084_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=conf&newCollection=true&name=addReplicaColl_shard1_replica_n2&action=CREATE&numShards=2&collection=addReplicaColl&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1980337 INFO (qtp681167845-18871) [n:127.0.0.1:51819_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1980338 INFO (qtp137164455-18861) [n:127.0.0.1:41084_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1980344 INFO (qtp162414955-18853) [n:127.0.0.1:48992_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf&newCollection=true&name=addReplicaColl_shard2_replica_n4&action=CREATE&numShards=2&collection=addReplicaColl&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1980344 INFO (qtp162414955-18853) [n:127.0.0.1:48992_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1980457 INFO (zkCallback-3213-thread-1-processing-n:127.0.0.1:48992_solr) [n:127.0.0.1:48992_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1980457 INFO (zkCallback-3214-thread-1-processing-n:127.0.0.1:33704_solr) [n:127.0.0.1:33704_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1980457 INFO (zkCallback-3215-thread-1-processing-n:127.0.0.1:51819_solr) [n:127.0.0.1:51819_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1980458 INFO (zkCallback-3214-thread-2-processing-n:127.0.0.1:33704_solr) [n:127.0.0.1:33704_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1980458 INFO (zkCallback-3215-thread-2-processing-n:127.0.0.1:51819_solr) [n:127.0.0.1:51819_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1980458 INFO (zkCallback-3213-thread-2-processing-n:127.0.0.1:48992_solr) [n:127.0.0.1:48992_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1980458 INFO (zkCallback-3216-thread-1-processing-n:127.0.0.1:41084_solr) [n:127.0.0.1:41084_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1980458 INFO (zkCallback-3216-thread-2-processing-n:127.0.0.1:41084_solr) [n:127.0.0.1:41084_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1981376 INFO (qtp162414955-18853) [n:127.0.0.1:48992_solr c:addReplicaColl s:shard2 r:core_node7 x:addReplicaColl_shard2_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 1981377 INFO (qtp137164455-18861) [n:127.0.0.1:41084_solr c:addReplicaColl s:shard1 r:core_node5 x:addReplicaColl_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 1981389 INFO (qtp681167845-18871) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard2 r:core_node8 x:addReplicaColl_shard2_replica_n6] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 1981394 INFO (qtp29335138-18889) [n:127.0.0.1:33704_solr c:addReplicaColl s:shard1 r:core_node3 x:addReplicaColl_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 1981397 INFO (qtp137164455-18861) [n:127.0.0.1:41084_solr c:addReplicaColl s:shard1 r:core_node5 x:addReplicaColl_shard1_replica_n2] o.a.s.s.IndexSchema [addReplicaColl_shard1_replica_n2] Schema name=minimal
[junit4] 2> 1981397 INFO (qtp162414955-18853) [n:127.0.0.1:48992_solr c:addReplicaColl s:shard2 r:core_node7 x:addReplicaColl_shard2_replica_n4] o.a.s.s.IndexSchema [addReplicaColl_shard2_replica_n4] Schema name=minimal
[junit4] 2> 1981398 INFO (qtp681167845-18871) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard2 r:core_node8 x:addReplicaColl_shard2_replica_n6] o.a.s.s.IndexSchema [addReplicaColl_shard2_replica_n6] Schema name=minimal
[junit4] 2> 1981399 INFO (qtp137164455-18861) [n:127.0.0.1:41084_solr c:addReplicaColl s:shard1 r:core_node5 x:addReplicaColl_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1981399 INFO (qtp681167845-18871) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard2 r:core_node8 x:addReplicaColl_shard2_replica_n6] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1981399 INFO (qtp162414955-18853) [n:127.0.0.1:48992_solr c:addReplicaColl s:shard2 r:core_node7 x:addReplicaColl_shard2_replica_n4] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1981399 INFO (qtp681167845-18871) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard2 r:core_node8 x:addReplicaColl_shard2_replica_n6] o.a.s.c.CoreContainer Creating SolrCore 'addReplicaColl_shard2_replica_n6' using configuration from collection addReplicaColl, trusted=true
[junit4] 2> 1981399 INFO (qtp162414955-18853) [n:127.0.0.1:48992_solr c:addReplicaColl s:shard2 r:core_node7 x:addReplicaColl_shard2_replica_n4] o.a.s.c.CoreContainer Creating SolrCore 'addReplicaColl_shard2_replica_n4' using configuration from collection addReplicaColl, trusted=true
[junit4] 2> 1981399 INFO (qtp137164455-18861) [n:127.0.0.1:41084_solr c:addReplicaColl s:shard1 r:core_node5 x:addReplicaColl_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'addReplicaColl_shard1_replica_n2' using configuration from collection addReplicaColl, trusted=true
[junit4] 2> 1981400 INFO (qtp681167845-18871) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard2 r:core_node8 x:addReplicaColl_shard2_replica_n6] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.addReplicaColl.shard2.replica_n6' (registry 'solr.core.addReplicaColl.shard2.replica_n6') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e3f0961
[junit4] 2> 1981400 INFO (qtp137164455-18861) [n:127.0.0.1:41084_solr c:addReplicaColl s:shard1 r:core_node5 x:addReplicaColl_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.addReplicaColl.shard1.replica_n2' (registry 'solr.core.addReplicaColl.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e3f0961
[junit4] 2> 1981400 INFO (qtp681167845-18871) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard2 r:core_node8 x:addReplicaColl_shard2_replica_n6] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1981400 INFO (qtp162414955-18853) [n:127.0.0.1:48992_solr c:addReplicaColl s:shard2 r:core_node7 x:addReplicaColl_shard2_replica_n4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.addReplicaColl.shard2.replica_n4' (registry 'solr.core.addReplicaColl.shard2.replica_n4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e3f0961
[junit4] 2> 1981400 INFO (qtp681167845-18871) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard2 r:core_node8 x:addReplicaColl_shard2_replica_n6] o.a.s.c.SolrCore [[addReplicaColl_shard2_replica_n6] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7FDC08545B534090-001/tempDir-001/node1/addReplicaColl_shard2_replica_n6], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7FDC08545B534090-001/tempDir-001/node1/addReplicaColl_shard2_replica_n6/data/]
[junit4] 2> 1981400 INFO (qtp137164455-18861) [n:127.0.0.1:41084_solr c:addReplicaColl s:shard1 r:core_node5 x:addReplicaColl_shard1_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1981400 INFO (qtp137164455-18861) [n:127.0.0.1:41084_solr c:addReplicaColl s:shard1 r:core_node5 x:addReplicaColl_shard1_replica_n2] o.a.s.c.SolrCore [[addReplicaColl_shard1_replica_n2] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7FDC08545B534090-001/tempDir-001/node4/addReplicaColl_shard1_replica_n2], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7FDC08545B534090-001/tempDir-001/node4/addReplicaColl_shard1_replica_n2/data/]
[junit4] 2> 1981402 INFO (qtp29335138-18889) [n:127.0.0.1:33704_solr c:addReplicaColl s:shard1 r:core_node3 x:addReplicaColl_shard1_replica_n1] o.a.s.s.IndexSchema [addReplicaColl_shard1_replica_n1] Schema name=minimal
[junit4] 2> 1981404 INFO (qtp29335138-18889) [n:127.0.0.1:33704_solr c:addReplicaColl s:shard1 r:core_node3 x:addReplicaColl_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1981404 INFO (qtp29335138-18889) [n:127.0.0.1:33704_solr c:addReplicaColl s:shard1 r:core_node3 x:addReplicaColl_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'addReplicaColl_shard1_replica_n1' using configuration from collection addReplicaColl, trusted=true
[junit4] 2> 1981404 INFO (qtp29335138-18889) [n:127.0.0.1:33704_solr c:addReplicaColl s:shard1 r:core_node3 x:addReplicaColl_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.addReplicaColl.shard1.replica_n1' (registry 'solr.core.addReplicaColl.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e3f0961
[junit4] 2> 1981405 INFO (qtp29335138-18889) [n:127.0.0.1:33704_solr c:addReplicaColl s:shard1 r:core_node3 x:addReplicaColl_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1981405 INFO (qtp29335138-18889) [n:127.0.0.1:33704_solr c:addReplicaColl s:shard1 r:core_node3 x:addReplicaColl_shard1_replica_n1] o.a.s.c.SolrCore [[addReplicaColl_shard1_replica_n1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7FDC08545B534090-001/tempDir-001/node3/addReplicaColl_shard1_replica_n1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7FDC08545B534090-001/tempDir-001/node3/addReplicaColl_shard1_replica_n1/data/]
[junit4] 2> 1981411 INFO (qtp162414955-18853) [n:127.0.0.1:48992_solr c:addReplicaColl s:shard2 r:core_node7 x:addReplicaColl_shard2_replica_n4] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1981411 INFO (qtp162414955-18853) [n:127.0.0.1:48992_solr c:addReplicaColl s:shard2 r:core_node7 x:addReplicaColl_shard2_replica_n4] o.a.s.c.SolrCore [[addReplicaColl_shard2_replica_n4] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7FDC08545B534090-001/tempDir-001/node2/addReplicaColl_shard2_replica_n4], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7FDC08545B534090-001/tempDir-001/node2/addReplicaColl_shard2_replica_n4/data/]
[junit4] 2> 1981451 INFO (qtp681167845-18871) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard2 r:core_node8 x:addReplicaColl_shard2_replica_n6] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1981451 INFO (qtp681167845-18871) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard2 r:core_node8 x:addReplicaColl_shard2_replica_n6] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1981452 INFO (qtp681167845-18871) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard2 r:core_node8 x:addReplicaColl_shard2_replica_n6] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1981452 INFO (qtp681167845-18871) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard2 r:core_node8 x:addReplicaColl_shard2_replica_n6] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1981467 INFO (qtp681167845-18871) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard2 r:core_node8 x:addReplicaColl_shard2_replica_n6] o.a.s.s.SolrIndexSearcher Opening [Searcher@39760dc8[addReplicaColl_shard2_replica_n6] main]
[junit4] 2> 1981469 INFO (qtp681167845-18871) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard2 r:core_node8 x:addReplicaColl_shard2_replica_n6] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 1981469 INFO (qtp681167845-18871) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard2 r:core_node8 x:addReplicaColl_shard2_replica_n6] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 1981469 INFO (qtp681167845-18871) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard2 r:core_node8 x:addReplicaColl_shard2_replica_n6] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1981470 INFO (qtp681167845-18871) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard2 r:core_node8 x:addReplicaColl_shard2_replica_n6] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1582733894004768768
[junit4] 2> 1981472 INFO (searcherExecutor-6572-thread-1-processing-n:127.0.0.1:51819_solr x:addReplicaColl_shard2_replica_n6 s:shard2 c:addReplicaColl r:core_node8) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard2 r:core_node8 x:addReplicaColl_shard2_replica_n6] o.a.s.c.SolrCore [addReplicaColl_shard2_replica_n6] Registered new searcher Searcher@39760dc8[addReplicaColl_shard2_replica_n6] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1981477 INFO (qtp681167845-18871) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard2 r:core_node8 x:addReplicaColl_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard2: total=2 found=1 timeoutin=9999ms
[junit4] 2> 1981477 INFO (qtp29335138-18889) [n:127.0.0.1:33704_solr c:addReplicaColl s:shard1 r:core_node3 x:addReplicaColl_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1981477 INFO (qtp29335138-18889) [n:127.0.0.1:33704_solr c:addReplicaColl s:shard1 r:core_node3 x:addReplicaColl_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1981478 INFO (qtp29335138-18889) [n:127.0.0.1:33704_solr c:addReplicaColl s:shard1 r:core_node3 x:addReplicaColl_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1981478 INFO (qtp29335138-18889) [n:127.0.0.1:33704_solr c:addReplicaColl s:shard1 r:core_node3 x:addReplicaColl_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1981479 INFO (qtp29335138-18889) [n:127.0.0.1:33704_solr c:addReplicaColl s:shard1 r:core_node3 x:addReplicaColl_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@782a37e8[addReplicaColl_shard1_replica_n1] main]
[junit4] 2> 1981481 INFO (qtp29335138-18889) [n:127.0.0.1:33704_solr c:addReplicaColl s:shard1 r:core_node3 x:addReplicaColl_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 1981481 INFO (qtp29335138-18889) [n:127.0.0.1:33704_solr c:addReplicaColl s:shard1 r:core_node3 x:addReplicaColl_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 1981482 INFO (qtp29335138-18889) [n:127.0.0.1:33704_solr c:addReplicaColl s:shard1 r:core_node3 x:addReplicaColl_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1981482 INFO (searcherExecutor-6575-thread-1-processing-n:127.0.0.1:33704_solr x:addReplicaColl_shard1_replica_n1 s:shard1 c:addReplicaColl r:core_node3) [n:127.0.0.1:33704_solr c:addReplicaColl s:shard1 r:core_node3 x:addReplicaColl_shard1_replica_n1] o.a.s.c.SolrCore [addReplicaColl_shard1_replica_n1] Registered new searcher Searcher@782a37e8[addReplicaColl_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1981482 INFO (qtp29335138-18889) [n:127.0.0.1:33704_solr c:addReplicaColl s:shard1 r:core_node3 x:addReplicaColl_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1582733894017351680
[junit4] 2> 1981483 INFO (qtp137164455-18861) [n:127.0.0.1:41084_solr c:addReplicaColl s:shard1 r:core_node5 x:addReplicaColl_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1981483 INFO (qtp137164455-18861) [n:127.0.0.1:41084_solr c:addReplicaColl s:shard1 r:core_node5 x:addReplicaColl_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1981484 INFO (qtp137164455-18861) [n:127.0.0.1:41084_solr c:addReplicaColl s:shard1 r:core_node5 x:addReplicaColl_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1981484 INFO (qtp137164455-18861) [n:127.0.0.1:41084_solr c:addReplicaColl s:shard1 r:core_node5 x:addReplicaColl_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1981485 INFO (qtp162414955-18853) [n:127.0.0.1:48992_solr c:addReplicaColl s:shard2 r:core_node7 x:addReplicaColl_shard2_replica_n4] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1981485 INFO (qtp162414955-18853) [n:127.0.0.1:48992_solr c:addReplicaColl s:shard2 r:core_node7 x:addReplicaColl_shard2_replica_n4] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1981485 INFO (qtp137164455-18861) [n:127.0.0.1:41084_solr c:addReplicaColl s:shard1 r:core_node5 x:addReplicaColl_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@19c938d5[addReplicaColl_shard1_replica_n2] main]
[junit4] 2> 1981486 INFO (qtp162414955-18853) [n:127.0.0.1:48992_solr c:addReplicaColl s:shard2 r:core_node7 x:addReplicaColl_shard2_replica_n4] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1981486 INFO (qtp137164455-18861) [n:127.0.0.1:41084_solr c:addReplicaColl s:shard1 r:core_node5 x:addReplicaColl_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 1981487 INFO (qtp162414955-18853) [n:127.0.0.1:48992_solr c:addReplicaColl s:shard2 r:core_node7 x:addReplicaColl_shard2_replica_n4] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1981487 INFO (qtp137164455-18861) [n:127.0.0.1:41084_solr c:addReplicaColl s:shard1 r:core_node5 x:addReplicaColl_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 1981487 INFO (qtp137164455-18861) [n:127.0.0.1:41084_solr c:addReplicaColl s:shard1 r:core_node5 x:addReplicaColl_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1981488 INFO (qtp137164455-18861) [n:127.0.0.1:41084_solr c:addReplicaColl s:shard1 r:core_node5 x:addReplicaColl_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1582733894023643136
[junit4] 2> 1981489 INFO (qtp29335138-18889) [n:127.0.0.1:33704_solr c:addReplicaColl s:shard1 r:core_node3 x:addReplicaColl_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
[junit4] 2> 1981489 INFO (searcherExecutor-6574-thread-1-processing-n:127.0.0.1:41084_solr x:addReplicaColl_shard1_replica_n2 s:shard1 c:addReplicaColl r:core_node5) [n:127.0.0.1:41084_solr c:addReplicaColl s:shard1 r:core_node5 x:addReplicaColl_shard1_replica_n2] o.a.s.c.SolrCore [addReplicaColl_shard1_replica_n2] Registered new searcher Searcher@19c938d5[addReplicaColl_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1981490 INFO (qtp162414955-18853) [n:127.0.0.1:48992_solr c:addReplicaColl s:shard2 r:core_node7 x:addReplicaColl_shard2_replica_n4] o.a.s.s.SolrIndexSearcher Opening [Searcher@211c7b39[addReplicaColl_shard2_replica_n4] main]
[junit4] 2> 1981492 INFO (qtp162414955-18853) [n:127.0.0.1:48992_solr c:addReplicaColl s:shard2 r:core_node7 x:addReplicaColl_shard2_replica_n4] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 1981492 INFO (qtp162414955-18853) [n:127.0.0.1:48992_solr c:addReplicaColl s:shard2 r:core_node7 x:addReplicaColl_shard2_replica_n4] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 1981492 INFO (qtp162414955-18853) [n:127.0.0.1:48992_solr c:addReplicaColl s:shard2 r:core_node7 x:addReplicaColl_shard2_replica_n4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1981492 INFO (qtp162414955-18853) [n:127.0.0.1:48992_solr c:addReplicaColl s:shard2 r:core_node7 x:addReplicaColl_shard2_replica_n4] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1582733894027837440
[junit4] 2> 1981493 INFO (searcherExecutor-6573-thread-1-processing-n:127.0.0.1:48992_solr x:addReplicaColl_shard2_replica_n4 s:shard2 c:addReplicaColl r:core_node7) [n:127.0.0.1:48992_solr c:addReplicaColl s:shard2 r:core_node7 x:addReplicaColl_shard2_replica_n4] o.a.s.c.SolrCore [addReplicaColl_shard2_replica_n4] Registered new searcher Searcher@211c7b39[addReplicaColl_shard2_replica_n4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1981592 INFO (zkCallback-3215-thread-2-processing-n:127.0.0.1:51819_solr) [n:127.0.0.1:51819_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1981592 INFO (zkCallback-3216-thread-2-processing-n:127.0.0.1:41084_solr) [n:127.0.0.1:41084_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1981592 INFO (zkCallback-3216-thread-1-processing-n:127.0.0.1:41084_solr) [n:127.0.0.1:41084_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1981593 INFO (zkCallback-3214-thread-2-processing-n:127.0.0.1:33704_solr) [n:127.0.0.1:33704_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1981593 INFO (zkCallback-3213-thread-1-processing-n:127.0.0.1:48992_solr) [n:127.0.0.1:48992_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1981593 INFO (zkCallback-3215-thread-1-processing-n:127.0.0.1:51819_solr) [n:127.0.0.1:51819_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1981593 INFO (zkCallback-3213-thread-2-processing-n:127.0.0.1:48992_solr) [n:127.0.0.1:48992_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1981593 INFO (zkCallback-3214-thread-1-processing-n:127.0.0.1:33704_solr) [n:127.0.0.1:33704_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1981979 INFO (qtp681167845-18871) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard2 r:core_node8 x:addReplicaColl_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1981979 INFO (qtp681167845-18871) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard2 r:core_node8 x:addReplicaColl_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1981979 INFO (qtp681167845-18871) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard2 r:core_node8 x:addReplicaColl_shard2_replica_n6] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:51819/solr/addReplicaColl_shard2_replica_n6/
[junit4] 2> 1981979 INFO (qtp681167845-18871) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard2 r:core_node8 x:addReplicaColl_shard2_replica_n6] o.a.s.u.PeerSync PeerSync: core=addReplicaColl_shard2_replica_n6 url=http://127.0.0.1:51819/solr START replicas=[http://127.0.0.1:48992/solr/addReplicaColl_shard2_replica_n4/] nUpdates=100
[junit4] 2> 1981980 INFO (qtp162414955-18857) [n:127.0.0.1:48992_solr c:addReplicaColl s:shard2 r:core_node7 x:addReplicaColl_shard2_replica_n4] o.a.s.c.S.Request [addReplicaColl_shard2_replica_n4] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1981989 INFO (qtp29335138-18889) [n:127.0.0.1:33704_solr c:addReplicaColl s:shard1 r:core_node3 x:addReplicaColl_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1981989 INFO (qtp29335138-18889) [n:127.0.0.1:33704_solr c:addReplicaColl s:shard1 r:core_node3 x:addReplicaColl_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1981989 INFO (qtp29335138-18889) [n:127.0.0.1:33704_solr c:addReplicaColl s:shard1 r:core_node3 x:addReplicaColl_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:33704/solr/addReplicaColl_shard1_replica_n1/
[junit4] 2> 1981990 INFO (qtp29335138-18889) [n:127.0.0.1:33704_solr c:addReplicaColl s:shard1 r:core_node3 x:addReplicaColl_shard1_replica_n1] o.a.s.u.PeerSync PeerSync: core=addReplicaColl_shard1_replica_n1 url=http://127.0.0.1:33704/solr START replicas=[http://127.0.0.1:41084/solr/addReplicaColl_shard1_replica_n2/] nUpdates=100
[junit4] 2> 1981991 INFO (qtp137164455-18866) [n:127.0.0.1:41084_solr c:addReplicaColl s:shard1 r:core_node5 x:addReplicaColl_shard1_replica_n2] o.a.s.c.S.Request [addReplicaColl_shard1_replica_n2] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1982279 INFO (qtp681167845-18871) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard2 r:core_node8 x:addReplicaColl_shard2_replica_n6] o.a.s.u.PeerSync PeerSync: core=addReplicaColl_shard2_replica_n6 url=http://127.0.0.1:51819/solr DONE. We have no versions. sync failed.
[junit4] 2> 1982279 INFO (qtp681167845-18871) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard2 r:core_node8 x:addReplicaColl_shard2_replica_n6] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 1982280 INFO (qtp681167845-18871) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard2 r:core_node8 x:addReplicaColl_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 1982280 INFO (qtp681167845-18871) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard2 r:core_node8 x:addReplicaColl_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1982290 INFO (qtp29335138-18889) [n:127.0.0.1:33704_solr c:addReplicaColl s:shard1 r:core_node3 x:addReplicaColl_shard1_replica_n1] o.a.s.u.PeerSync PeerSync: core=addReplicaColl_shard1_replica_n1 url=http://127.0.0.1:33704/solr DONE. We have no versions. sync failed.
[junit4] 2> 1982290 INFO (qtp29335138-18889) [n:127.0.0.1:33704_solr c:addReplicaColl s:shard1 r:core_node3 x:addReplicaColl_shard1_replica_n1] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 1982290 INFO (qtp29335138-18889) [n:127.0.0.1:33704_solr c:addReplicaColl s:shard1 r:core_node3 x:addReplicaColl_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 1982290 INFO (qtp29335138-18889) [n:127.0.0.1:33704_solr c:addReplicaColl s:shard1 r:core_node3 x:addReplicaColl_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1982293 INFO (qtp681167845-18871) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard2 r:core_node8 x:addReplicaColl_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:51819/solr/addReplicaColl_shard2_replica_n6/ shard2
[junit4] 2> 1982300 INFO (qtp29335138-18889) [n:127.0.0.1:33704_solr c:addReplicaColl s:shard1 r:core_node3 x:addReplicaColl_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:33704/solr/addReplicaColl_shard1_replica_n1/ shard1
[junit4] 2> 1982405 INFO (zkCallback-3215-thread-1-processing-n:127.0.0.1:51819_solr) [n:127.0.0.1:51819_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1982405 INFO (zkCallback-3215-thread-2-processing-n:127.0.0.1:51819_solr) [n:127.0.0.1:51819_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1982405 INFO (zkCallback-3213-thread-2-processing-n:127.0.0.1:48992_solr) [n:127.0.0.1:48992_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1982405 INFO (zkCallback-3216-thread-2-processing-n:127.0.0.1:41084_solr) [n:127.0.0.1:41084_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1982405 INFO (zkCallback-3214-thread-1-processing-n:127.0.0.1:33704_solr) [n:127.0.0.1:33704_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1982406 INFO (zkCallback-3216-thread-1-processing-n:127.0.0.1:41084_solr) [n:127.0.0.1:41084_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1982406 INFO (zkCallback-3213-thread-1-processing-n:127.0.0.1:48992_solr) [n:127.0.0.1:48992_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1982407 INFO (zkCallback-3214-thread-2-processing-n:127.0.0.1:33704_solr) [n:127.0.0.1:33704_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1982445 INFO (qtp681167845-18871) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard2 r:core_node8 x:addReplicaColl_shard2_replica_n6] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1982447 INFO (qtp681167845-18871) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard2 r:core_node8 x:addReplicaColl_shard2_replica_n6] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf&newCollection=true&name=addReplicaColl_shard2_replica_n6&action=CREATE&numShards=2&collection=addReplicaColl&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2110
[junit4] 2> 1982459 INFO (qtp29335138-18889) [n:127.0.0.1:33704_solr c:addReplicaColl s:shard1 r:core_node3 x:addReplicaColl_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1982461 INFO (qtp29335138-18889) [n:127.0.0.1:33704_solr c:addReplicaColl s:shard1 r:core_node3 x:addReplicaColl_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=addReplicaColl_shard1_replica_n1&action=CREATE&numShards=2&collection=addReplicaColl&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2128
[junit4] 2> 1982495 INFO (qtp137164455-18861) [n:127.0.0.1:41084_solr c:addReplicaColl s:shard1 r:core_node5 x:addReplicaColl_shard1_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node5&collection.configName=conf&newCollection=true&name=addReplicaColl_shard1_replica_n2&action=CREATE&numShards=2&collection=addReplicaColl&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2158
[junit4] 2> 1982498 INFO (qtp162414955-18853) [n:127.0.0.1:48992_solr c:addReplicaColl s:shard2 r:core_node7 x:addReplicaColl_shard2_replica_n4] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf&newCollection=true&name=addReplicaColl_shard2_replica_n4&action=CREATE&numShards=2&collection=addReplicaColl&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2160
[junit4] 2> 1982502 INFO (qtp29335138-18888) [n:127.0.0.1:33704_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 1982600 INFO (zkCallback-3213-thread-1-processing-n:127.0.0.1:48992_solr) [n:127.0.0.1:48992_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1982600 INFO (zkCallback-3216-thread-1-processing-n:127.0.0.1:41084_solr) [n:127.0.0.1:41084_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1982600 INFO (zkCallback-3213-thread-2-processing-n:127.0.0.1:48992_solr) [n:127.0.0.1:48992_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1982600 INFO (zkCallback-3214-thread-2-processing-n:127.0.0.1:33704_solr) [n:127.0.0.1:33704_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1982600 INFO (zkCallback-3214-thread-1-processing-n:127.0.0.1:33704_solr) [n:127.0.0.1:33704_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1982601 INFO (zkCallback-3216-thread-2-processing-n:127.0.0.1:41084_solr) [n:127.0.0.1:41084_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1982601 INFO (zkCallback-3215-thread-2-processing-n:127.0.0.1:51819_solr) [n:127.0.0.1:51819_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1982601 INFO (zkCallback-3215-thread-1-processing-n:127.0.0.1:51819_solr) [n:127.0.0.1:51819_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1983502 INFO (qtp29335138-18888) [n:127.0.0.1:33704_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=2&collection.configName=conf&maxShardsPerNode=4&name=addReplicaColl&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=3520
[junit4] 2> 1983504 INFO (qtp29335138-18891) [n:127.0.0.1:33704_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:51819_solr&action=ADDREPLICA&collection=addReplicaColl&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1983511 INFO (OverseerCollectionConfigSetProcessor-98920868242784265-127.0.0.1:48992_solr-n_0000000000) [n:127.0.0.1:48992_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1983512 INFO (OverseerThreadFactory-6563-thread-2-processing-n:127.0.0.1:48992_solr) [n:127.0.0.1:48992_solr ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:51819_solr for creating new replica
[junit4] 2> 1983521 INFO (OverseerStateUpdate-98920868242784265-127.0.0.1:48992_solr-n_0000000000) [n:127.0.0.1:48992_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"addreplica",
[junit4] 2> "collection":"addReplicaColl",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"addReplicaColl_shard1_replica_n9",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:51819/solr",
[junit4] 2> "node_name":"127.0.0.1:51819_solr",
[junit4] 2> "type":"NRT"}
[junit4] 2> 1983630 INFO (zkCallback-3216-thread-2-processing-n:127.0.0.1:41084_solr) [n:127.0.0.1:41084_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1983630 INFO (zkCallback-3216-thread-1-processing-n:127.0.0.1:41084_solr) [n:127.0.0.1:41084_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1983630 INFO (zkCallback-3213-thread-1-processing-n:127.0.0.1:48992_solr) [n:127.0.0.1:48992_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1983630 INFO (zkCallback-3215-thread-1-processing-n:127.0.0.1:51819_solr) [n:127.0.0.1:51819_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1983630 INFO (zkCallback-3215-thread-2-processing-n:127.0.0.1:51819_solr) [n:127.0.0.1:51819_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1983630 INFO (zkCallback-3214-thread-1-processing-n:127.0.0.1:33704_solr) [n:127.0.0.1:33704_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1983630 INFO (zkCallback-3213-thread-2-processing-n:127.0.0.1:48992_solr) [n:127.0.0.1:48992_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1983631 INFO (zkCallback-3214-thread-2-processing-n:127.0.0.1:33704_solr) [n:127.0.0.1:33704_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1983724 INFO (qtp681167845-18874) [n:127.0.0.1:51819_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node10&collection.configName=conf&name=addReplicaColl_shard1_replica_n9&action=CREATE&collection=addReplicaColl&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1983737 INFO (qtp681167845-18874) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard1 r:core_node10 x:addReplicaColl_shard1_replica_n9] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 1983742 INFO (qtp681167845-18874) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard1 r:core_node10 x:addReplicaColl_shard1_replica_n9] o.a.s.s.IndexSchema [addReplicaColl_shard1_replica_n9] Schema name=minimal
[junit4] 2> 1983745 INFO (qtp681167845-18874) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard1 r:core_node10 x:addReplicaColl_shard1_replica_n9] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1983745 INFO (qtp681167845-18874) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard1 r:core_node10 x:addReplicaColl_shard1_replica_n9] o.a.s.c.CoreContainer Creating SolrCore 'addReplicaColl_shard1_replica_n9' using configuration from collection addReplicaColl, trusted=true
[junit4] 2> 1983745 INFO (qtp681167845-18874) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard1 r:core_node10 x:addReplicaColl_shard1_replica_n9] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.addReplicaColl.shard1.replica_n9' (registry 'solr.core.addReplicaColl.shard1.replica_n9') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e3f0961
[junit4] 2> 1983745 INFO (qtp681167845-18874) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard1 r:core_node10 x:addReplicaColl_shard1_replica_n9] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1983745 INFO (qtp681167845-18874) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard1 r:core_node10 x:addReplicaColl_shard1_replica_n9] o.a.s.c.SolrCore [[addReplicaColl_shard1_replica_n9] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7FDC08545B534090-001/tempDir-001/node1/addReplicaColl_shard1_replica_n9], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7FDC08545B534090-001/tempDir-001/node1/addReplicaColl_shard1_replica_n9/data/]
[junit4] 2> 1983786 INFO (qtp681167845-18874) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard1 r:core_node10 x:addReplicaColl_shard1_replica_n9] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1983786 INFO (qtp681167845-18874) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard1 r:core_node10 x:addReplicaColl_shard1_replica_n9] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1983787 INFO (qtp681167845-18874) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard1 r:core_node10 x:addReplicaColl_shard1_replica_n9] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1983787 INFO (qtp681167845-18874) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard1 r:core_node10 x:addReplicaColl_shard1_replica_n9] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1983788 INFO (qtp681167845-18874) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard1 r:core_node10 x:addReplicaColl_shard1_replica_n9] o.a.s.s.SolrIndexSearcher Opening [Searcher@4d2537ec[addReplicaColl_shard1_replica_n9] main]
[junit4] 2> 1983789 INFO (qtp681167845-18874) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard1 r:core_node10 x:addReplicaColl_shard1_replica_n9] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 1983790 INFO (qtp681167845-18874) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard1 r:core_node10 x:addReplicaColl_shard1_replica_n9] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 1983790 INFO (qtp681167845-18874) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard1 r:core_node10 x:addReplicaColl_shard1_replica_n9] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1983790 INFO (qtp681167845-18874) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard1 r:core_node10 x:addReplicaColl_shard1_replica_n9] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1582733896437465088
[junit4] 2> 1983791 INFO (searcherExecutor-6592-thread-1-processing-n:127.0.0.1:51819_solr x:addReplicaColl_shard1_replica_n9 s:shard1 c:addReplicaColl r:core_node10) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard1 r:core_node10 x:addReplicaColl_shard1_replica_n9] o.a.s.c.SolrCore [addReplicaColl_shard1_replica_n9] Registered new searcher Searcher@4d2537ec[addReplicaColl_shard1_replica_n9] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1983794 INFO (qtp681167845-18874) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard1 r:core_node10 x:addReplicaColl_shard1_replica_n9] o.a.s.c.ZkController Core needs to recover:addReplicaColl_shard1_replica_n9
[junit4] 2> 1983794 INFO (updateExecutor-3205-thread-1-processing-n:127.0.0.1:51819_solr x:addReplicaColl_shard1_replica_n9 s:shard1 c:addReplicaColl r:core_node10) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard1 r:core_node10 x:addReplicaColl_shard1_replica_n9] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 1983795 INFO (recoveryExecutor-3208-thread-1-processing-n:127.0.0.1:51819_solr x:addReplicaColl_shard1_replica_n9 s:shard1 c:addReplicaColl r:core_node10) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard1 r:core_node10 x:addReplicaColl_shard1_replica_n9] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
[junit4] 2> 1983795 INFO (recoveryExecutor-3208-thread-1-processing-n:127.0.0.1:51819_solr x:addReplicaColl_shard1_replica_n9 s:shard1 c:addReplicaColl r:core_node10) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard1 r:core_node10 x:addReplicaColl_shard1_replica_n9] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 1983795 INFO (recoveryExecutor-3208-thread-1-processing-n:127.0.0.1:51819_solr x:addReplicaColl_shard1_replica_n9 s:shard1 c:addReplicaColl r:core_node10) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard1 r:core_node10 x:addReplicaColl_shard1_replica_n9] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[addReplicaColl_shard1_replica_n9]
[junit4] 2> 1983795 INFO (recoveryExecutor-3208-thread-1-processing-n:127.0.0.1:51819_solr x:addReplicaColl_shard1_replica_n9 s:shard1 c:addReplicaColl r:core_node10) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard1 r:core_node10 x:addReplicaColl_shard1_replica_n9] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 1983795 INFO (recoveryExecutor-3208-thread-1-processing-n:127.0.0.1:51819_solr x:addReplicaColl_shard1_replica_n9 s:shard1 c:addReplicaColl r:core_node10) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard1 r:core_node10 x:addReplicaColl_shard1_replica_n9] o.a.s.c.RecoveryStrategy Publishing state of core [addReplicaColl_shard1_replica_n9] as recovering, leader is [http://127.0.0.1:33704/solr/addReplicaColl_shard1_replica_n1/] and I am [http://127.0.0.1:51819/solr/addReplicaColl_shard1_replica_n9/]
[junit4] 2> 1983797 INFO (recoveryExecutor-3208-thread-1-processing-n:127.0.0.1:51819_solr x:addReplicaColl_shard1_replica_n9 s:shard1 c:addReplicaColl r:core_node10) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard1 r:core_node10 x:addReplicaColl_shard1_replica_n9] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:33704/solr]; [WaitForState: action=PREPRECOVERY&core=addReplicaColl_shard1_replica_n1&nodeName=127.0.0.1:51819_solr&coreNodeName=core_node10&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
[junit4] 2> 1983797 INFO (qtp681167845-18874) [n:127.0.0.1:51819_solr c:addReplicaColl s:shard1 r:core_node10 x:addReplicaColl_shard1_replica_n9] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node10&collection.configName=conf&name=addReplicaColl_shard1_replica_n9&action=CREATE&collection=addReplicaColl&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=73
[junit4] 2> 1983798 INFO (qtp29335138-18890) [n:127.0.0.1:33704_solr ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node10, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true, maxTime: 183 s
[junit4] 2> 1983799 INFO (qtp29335138-18890) [n:127.0.0.1:33704_solr ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=addReplicaColl, shard=shard1, thisCore=addReplicaColl_shard1_replica_n1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:51819_solr, coreNodeName=core_node10, onlyIfActiveCheckResult=false, nodeProps: core_node10:{"core":"addReplicaColl_shard1_replica_n9","base_url":"http://127.0.0.1:51819/solr","state":"down","node_name":"127.0.0.1:51819_solr","type":"NRT"}
[junit4] 2> 1983801 INFO (qtp29335138-18891) [n:127.0.0.1:33704_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:51819_solr&action=ADDREPLICA&collection=addReplicaColl&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=296
[junit4] 2> 1983802 INFO (qtp29335138-18891) [n:127.0.0.1:33704_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params action=ADDREPLICA&collection=addReplicaColl&shard=shard1&type=NRT&wt=javabin&version=2&property.instanceDir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7FDC08545B534090-001/tempDir-002 and sendToOCPQueue=true
[junit4] 2> 1983805 INFO (OverseerCollectionConfigSetProcessor-98920868242784265-127.0.0.1:48992_solr-n_0000000000) [n:127.0.0.1:48992_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1983806 INFO (OverseerThreadFactory-6563-thread-3-processing-n:127.0.0.1:48992_solr) [n:127.0.0.1:48992_solr ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:33704_solr for creating new replica
[junit4] 2> 1983809 INFO (OverseerStateUpdate-98920868242784265-127.0.0.1:48992_solr-n_0000000000) [n:127.0.0.1:48992_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"addreplica",
[junit4] 2> "collection":"addReplicaColl",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"addReplicaColl_shard1_replica_n11",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:33704/solr",
[junit4] 2> "node_name":"127.0.0.1:33704_solr",
[junit4] 2> "type":"NRT"}
[junit4] 2> 1983911 INFO (zkCallback-3213-thread-1-processing-n:127.0.0.1:48992_solr) [n:127.0.0.1:48992_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1983911 INFO (zkCallback-3213-thread-2-processing-n:127.0.0.1:48992_solr) [n:127.0.0.1:48992_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1983912 INFO (zkCallback-3214-thread-1-processing-n:127.0.0.1:33704_solr) [n:127.0.0.1:33704_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1983912 INFO (zkCallback-3216-thread-1-processing-n:127.0.0.1:41084_solr) [n:127.0.0.1:41084_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1983912 INFO (zkCallback-3215-thread-2-processing-n:127.0.0.1:51819_solr) [n:127.0.0.1:51819_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1983911 INFO (zkCallback-3215-thread-1-processing-n:127.0.0.1:51819_solr) [n:127.0.0.1:51819_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating..
[...truncated too long message...]
ectionMessageHandler.java:870)
[junit4] 2> at org.apache.solr.cloud.OverseerCollectionMessageHandler.processResponses(OverseerCollectionMessageHandler.java:749)
[junit4] 2> at org.apache.solr.cloud.CreateCollectionCmd.call(CreateCollectionCmd.java:297)
[junit4] 2> ... 6 more
[junit4] 2>
[junit4] 2> 2080585 INFO (OverseerThreadFactory-6563-thread-5-processing-n:127.0.0.1:48992_solr) [n:127.0.0.1:48992_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000081 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2080586 WARN (OverseerAutoScalingTriggerThread-98920868242784265-127.0.0.1:48992_solr-n_0000000000) [n:127.0.0.1:48992_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 2080590 INFO (jetty-closer-3195-thread-4) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3291a3a5{/solr,null,UNAVAILABLE}
[junit4] 2> 2081682 INFO (zkCallback-3215-thread-11-processing-n:127.0.0.1:51819_solr) [n:127.0.0.1:51819_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:51819_solr
[junit4] 2> 2081682 INFO (zkCallback-3215-thread-4-processing-n:127.0.0.1:51819_solr) [n:127.0.0.1:51819_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (1)
[junit4] 2> 2081771 WARN (zkCallback-3216-thread-1-processing-n:127.0.0.1:41084_solr) [n:127.0.0.1:41084_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 2081775 INFO (jetty-closer-3195-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@fc7413{/solr,null,UNAVAILABLE}
[junit4] 2> 2081857 INFO (jetty-closer-3195-thread-2) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:51819_solr
[junit4] 2> 2081858 INFO (jetty-closer-3195-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 2081859 INFO (zkCallback-3215-thread-11-processing-n:127.0.0.1:51819_solr) [n:127.0.0.1:51819_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 2081862 INFO (jetty-closer-3195-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4e5eb8f4{/solr,null,UNAVAILABLE}
[junit4] 2> 2081864 ERROR (SUITE-CollectionsAPIDistributedZkTest-seed#[7FDC08545B534090]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2081865 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[7FDC08545B534090]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:57396 57396
[junit4] 2> 2093812 INFO (Thread-6560) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:57396 57396
[junit4] 2> 2093814 WARN (Thread-6560) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 25 /solr/aliases.json
[junit4] 2> 25 /solr/clusterprops.json
[junit4] 2> 22 /solr/configs/conf
[junit4] 2> 5 /solr/security.json
[junit4] 2> 4 /solr/configs/conf2
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 40 /solr/collections/addReplicaColl/state.json
[junit4] 2> 39 /solr/collections/reloaded_collection/state.json
[junit4] 2> 38 /solr/collections/nodes_used_collection/state.json
[junit4] 2> 25 /solr/clusterstate.json
[junit4] 2> 15 /solr/collections/acollectionafterbaddelete/state.json
[junit4] 2> 13 /solr/collections/awhollynewcollection_0/state.json
[junit4] 2> 13 /solr/collections/halfdeletedcollection/state.json
[junit4] 2> 10 /solr/collections/onlyinzk/state.json
[junit4] 2> 5 /solr/collections/created_and_deleted/state.json
[junit4] 2> 5 /solr/collections/withconfigset2/state.json
[junit4] 2> 4 /solr/collections/halfcollectionblocker/state.json
[junit4] 2> 4 /solr/collections/halfcollection/state.json
[junit4] 2> 4 /solr/collections/halfcollectionblocker2/state.json
[junit4] 2> 2 /solr/overseer_elect/election/98920868242784267-127.0.0.1:51819_solr-n_0000000002
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 320 /solr/collections
[junit4] 2> 63 /solr/live_nodes
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_7FDC08545B534090-001
[junit4] 2> NOTE: test params are: codec=DummyCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=DUMMY, chunkSize=541, maxDocsPerChunk=9, blockSize=993), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=DUMMY, chunkSize=541, blockSize=993)), sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@40283e34), locale=da, timezone=SystemV/AST4ADT
[junit4] 2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=3,threads=1,free=203850952,total=517996544
[junit4] 2> NOTE: All tests run in this JVM: [TestRawTransformer, CdcrReplicationDistributedZkTest, TestHdfsBackupRestoreCore, TestDistributedGrouping, TestStressCloudBlindAtomicUpdates, TestSha256AuthenticationProvider, BlockJoinFacetRandomTest, SystemInfoHandlerTest, HdfsRecoveryZkTest, CustomCollectionTest, TestFieldCache, DOMUtilTest, SolrCoreTest, HdfsLockFactoryTest, UninvertDocValuesMergePolicyTest, FileUtilsTest, TestFieldCollectionResource, TestLeaderElectionWithEmptyReplica, DistributedFacetPivotSmallAdvancedTest, TestHashQParserPlugin, TestDistributedMissingSort, AddBlockUpdateTest, ReplaceNodeTest, DocumentAnalysisRequestHandlerTest, TestHashPartitioner, TestCloudRecovery, BlockCacheTest, UUIDUpdateProcessorFallbackTest, SolrCoreMetricManagerTest, TestFieldTypeCollectionResource, CacheHeaderTest, ZkSolrClientTest, DocValuesTest, TestLegacyNumericRangeQueryBuilder, TestSolrJ, TestCollectionsAPIViaSolrCloudCluster, NumberUtilsTest, V2StandaloneTest, TestCloudSchemaless, ManagedSchemaRoundRobinCloudTest, TestStandardQParsers, DeleteLastCustomShardedReplicaTest, SolrIndexSplitterTest, TestIntervalFaceting, TermVectorComponentTest, TestQueryUtils, RuleEngineTest, TestFilteredDocIdSet, MoveReplicaHDFSTest, HdfsNNFailoverTest, AnalysisAfterCoreReloadTest, TestPerFieldSimilarity, TestConfigSetsAPIExclusivity, EchoParamsTest, DocExpirationUpdateProcessorFactoryTest, TestBM25SimilarityFactory, TestSolrDeletionPolicy1, SampleTest, PreAnalyzedFieldTest, TestNumericRangeQuery32, TestNumericTokenStream, TestExportWriter, TestQueryWrapperFilter, TestFaceting, CdcrUpdateLogTest, TestPushWriter, SolrGangliaReporterTest, PKIAuthenticationIntegrationTest, TestSubQueryTransformer, TestCloudPivotFacet, LeaderElectionContextKeyTest, TestTrieFacet, OpenExchangeRatesOrgProviderTest, TestPivotHelperCode, TestUnifiedSolrHighlighter, ClassificationUpdateProcessorFactoryTest, CoreAdminHandlerTest, CoreSorterTest, SOLR749Test, TestTolerantUpdateProcessorCloud, TestComplexPhraseLeadingWildcard, TestSimpleTrackingShardHandler, ChaosMonkeySafeLeaderTest, TestInfoStreamLogging, CheckHdfsIndexTest, TestZkChroot, TermVectorComponentDistributedTest, PluginInfoTest, TestFoldingMultitermQuery, AutoScalingHandlerTest, TestCryptoKeys, MoreLikeThisHandlerTest, PolyFieldTest, TestTrie, TestPostingsSolrHighlighter, DistributedFacetPivotWhiteBoxTest, VMParamsZkACLAndCredentialsProvidersTest, TestLegacyFieldReuse, ChaosMonkeyNothingIsSafeWithPullReplicasTest, TestClassicSimilarityFactory, CdcrRequestHandlerTest, TestSolrCoreParser, CurrencyFieldTypeTest, SSLMigrationTest, TestSearcherReuse, DocumentBuilderTest, SolrCoreCheckLockOnStartupTest, QueryEqualityTest, TestJettySolrRunner, SolrSlf4jReporterTest, SpellCheckCollatorWithCollapseTest, TestSuggestSpellingConverter, CoreAdminCreateDiscoverTest, MBeansHandlerTest, TestPointFields, HdfsSyncSliceTest, TestReqParamsAPI, TestSolrQueryParser, TestSolrConfigHandler, TestManagedSynonymGraphFilterFactory, TestLeaderInitiatedRecoveryThread, TestAddFieldRealTimeGet, TestLRUCache, RankQueryTest, HdfsChaosMonkeySafeLeaderTest, TestDocBasedVersionConstraints, SolrJmxReporterCloudTest, TestNonDefinedSimilarityFactory, TestPKIAuthenticationPlugin, SoftAutoCommitTest, ZkStateWriterTest, QueryElevationComponentTest, TestBulkSchemaConcurrent, DistributedMLTComponentTest, TestZkAclsWithHadoopAuth, TestReload, ResponseHeaderTest, TriggerIntegrationTest, TestSchemaSimilarityResource, TestLuceneIndexBackCompat, SynonymTokenizerTest, FullHLLTest, TestStressReorder, CoreMergeIndexesAdminHandlerTest, TestRTGBase, SolrCloudExampleTest, TestFreeTextSuggestions, DeleteStatusTest, TestBulkSchemaAPI, TestNestedDocsSort, MetricUtilsTest, RestartWhileUpdatingTest, TestCloudInspectUtil, JvmMetricsTest, IndexSchemaRuntimeFieldTest, TestDocSet, AliasIntegrationTest, CollectionsAPIAsyncDistributedZkTest, TestCustomSort, BasicAuthIntegrationTest, TestClusterStateMutator, SolrMetricReporterTest, TestGroupingSearch, TestElisionMultitermQuery, SearchHandlerTest, TestSerializedLuceneMatchVersion, SpellingQueryConverterTest, TestSolrCLIRunExample, TestSearchPerf, TestCrossCoreJoin, SolrIndexConfigTest, TestRandomFlRTGCloud, LeaderInitiatedRecoveryOnShardRestartTest, TestExtendedDismaxParser, ShardRoutingCustomTest, TestFunctionQuery, PingRequestHandlerTest, SpatialHeatmapFacetsTest, TestNamedUpdateProcessors, TestDistribDocBasedVersion, BasicDistributedZk2Test, CollectionsAPIDistributedZkTest]
[junit4] Completed [354/747 (1!)] on J1 in 114.38s, 19 tests, 1 error <<< FAILURES!
[...truncated 48492 lines...]
[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build #
1498 - Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1498/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseG1GC
1 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation
Error Message:
2 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation: 1) Thread[id=33101, name=jetty-launcher-4685-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037) at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328) at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277) at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105) at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41) at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244) at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44) at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61) at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505) 2) Thread[id=33103, name=jetty-launcher-4685-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037) at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328) at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277) at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105) at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41) at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244) at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44) at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61) at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
1) Thread[id=33101, name=jetty-launcher-4685-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
2) Thread[id=33103, name=jetty-launcher-4685-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
at __randomizedtesting.SeedInfo.seed([8770C2BCC91D7DAF]:0)
Build Log:
[...truncated 13401 lines...]
[junit4] Suite: org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation
[junit4] 2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_8770C2BCC91D7DAF-001/init-core-data-001
[junit4] 2> 4578110 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[8770C2BCC91D7DAF]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 4578112 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[8770C2BCC91D7DAF]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 4578112 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[8770C2BCC91D7DAF]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 4578155 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[8770C2BCC91D7DAF]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_8770C2BCC91D7DAF-001/tempDir-001
[junit4] 2> 4578156 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[8770C2BCC91D7DAF]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 4578156 INFO (Thread-7515) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 4578156 INFO (Thread-7515) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 4578159 ERROR (Thread-7515) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 4578256 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[8770C2BCC91D7DAF]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:40456
[junit4] 2> 4578263 INFO (jetty-launcher-4685-thread-1) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 4578264 INFO (jetty-launcher-4685-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@567c7e77{/solr,null,AVAILABLE}
[junit4] 2> 4578269 INFO (jetty-launcher-4685-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@51b64f1b{HTTP/1.1,[http/1.1]}{127.0.0.1:37593}
[junit4] 2> 4578269 INFO (jetty-launcher-4685-thread-1) [ ] o.e.j.s.Server Started @4580832ms
[junit4] 2> 4578269 INFO (jetty-launcher-4685-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37593}
[junit4] 2> 4578269 ERROR (jetty-launcher-4685-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 4578270 INFO (jetty-launcher-4685-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 4578270 INFO (jetty-launcher-4685-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 4578270 INFO (jetty-launcher-4685-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 4578270 INFO (jetty-launcher-4685-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-10-30T15:26:03.225Z
[junit4] 2> 4578272 INFO (jetty-launcher-4685-thread-2) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 4578273 INFO (jetty-launcher-4685-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2ad7ddeb{/solr,null,AVAILABLE}
[junit4] 2> 4578274 INFO (jetty-launcher-4685-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7de82f5d{HTTP/1.1,[http/1.1]}{127.0.0.1:55633}
[junit4] 2> 4578274 INFO (jetty-launcher-4685-thread-2) [ ] o.e.j.s.Server Started @4580837ms
[junit4] 2> 4578274 INFO (jetty-launcher-4685-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=55633}
[junit4] 2> 4578274 ERROR (jetty-launcher-4685-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 4578274 INFO (jetty-launcher-4685-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 4578274 INFO (jetty-launcher-4685-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 4578274 INFO (jetty-launcher-4685-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 4578275 INFO (jetty-launcher-4685-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-10-30T15:26:03.229Z
[junit4] 2> 4578278 INFO (jetty-launcher-4685-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 4578284 INFO (jetty-launcher-4685-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40456/solr
[junit4] 2> 4578288 INFO (jetty-launcher-4685-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 4578293 INFO (jetty-launcher-4685-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40456/solr
[junit4] 2> 4578374 INFO (jetty-launcher-4685-thread-1) [n:127.0.0.1:37593_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 4578375 INFO (jetty-launcher-4685-thread-1) [n:127.0.0.1:37593_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37593_solr
[junit4] 2> 4578376 INFO (jetty-launcher-4685-thread-1) [n:127.0.0.1:37593_solr ] o.a.s.c.Overseer Overseer (id=98918541761708036-127.0.0.1:37593_solr-n_0000000000) starting
[junit4] 2> 4578378 INFO (jetty-launcher-4685-thread-2) [n:127.0.0.1:55633_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 4578380 INFO (jetty-launcher-4685-thread-2) [n:127.0.0.1:55633_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55633_solr
[junit4] 2> 4578382 INFO (zkCallback-4698-thread-1-processing-n:127.0.0.1:55633_solr) [n:127.0.0.1:55633_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 4578383 INFO (zkCallback-4693-thread-1-processing-n:127.0.0.1:37593_solr) [n:127.0.0.1:37593_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 4578389 INFO (jetty-launcher-4685-thread-1) [n:127.0.0.1:37593_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37593_solr
[junit4] 2> 4578394 INFO (zkCallback-4693-thread-2-processing-n:127.0.0.1:37593_solr) [n:127.0.0.1:37593_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 4578397 INFO (zkCallback-4698-thread-1-processing-n:127.0.0.1:55633_solr) [n:127.0.0.1:55633_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 4578466 INFO (jetty-launcher-4685-thread-1) [n:127.0.0.1:37593_solr ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.security.HttpParamDelegationTokenPlugin
[junit4] 2> 4578467 WARN (jetty-launcher-4685-thread-1) [n:127.0.0.1:37593_solr ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
[junit4] 2> 4578473 INFO (jetty-launcher-4685-thread-1) [n:127.0.0.1:37593_solr ] o.a.s.s.KerberosPlugin Params: {solr.kerberos.impersonator.user.noGroups.hosts=*, solr.kerberos.impersonator.user.bogusGroup.groups=__some_bogus_group, token.valid=30, cookie.domain=127.0.0.1, solr.kerberos.impersonator.user.anyHostAnyUser.groups=*, solr.kerberos.impersonator.user.bogusGroup.hosts=*, solr.kerberos.impersonator.user.anyHostAnyUser.hosts=*, token.validity=36000, solr.kerberos.impersonator.user.noHosts.groups=*, type=kerberos, solr.kerberos.impersonator.user.localHostAnyGroup.groups=*, delegation-token.token-kind=solr-dt, solr.kerberos.impersonator.user.wrongHost.hosts=1.1.1.1.1.1, cookie.path=/, zk-dt-secret-manager.znodeWorkingPath=solr/security/zkdtsm, solr.kerberos.impersonator.user.localHostAnyGroup.hosts=serv1-vm3,localhost,127.0.0.1, signer.secret.provider.zookeeper.path=/token, solr.kerberos.impersonator.user.anyHostUsersGroup.groups=staff, zk-dt-secret-manager.enable=true, solr.kerberos.impersonator.user.wrongHost.groups=*, kerberos.name.rules=DEFAULT, signer.secret.provider=zookeeper, solr.kerberos.impersonator.user.anyHostUsersGroup.hosts=*}
[junit4] 2> 4578506 INFO (jetty-launcher-4685-thread-2) [n:127.0.0.1:55633_solr ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.security.HttpParamDelegationTokenPlugin
[junit4] 2> 4578506 WARN (jetty-launcher-4685-thread-2) [n:127.0.0.1:55633_solr ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
[junit4] 2> 4578506 INFO (jetty-launcher-4685-thread-2) [n:127.0.0.1:55633_solr ] o.a.s.s.KerberosPlugin Params: {solr.kerberos.impersonator.user.noGroups.hosts=*, solr.kerberos.impersonator.user.bogusGroup.groups=__some_bogus_group, token.valid=30, cookie.domain=127.0.0.1, solr.kerberos.impersonator.user.anyHostAnyUser.groups=*, solr.kerberos.impersonator.user.bogusGroup.hosts=*, solr.kerberos.impersonator.user.anyHostAnyUser.hosts=*, token.validity=36000, solr.kerberos.impersonator.user.noHosts.groups=*, type=kerberos, solr.kerberos.impersonator.user.localHostAnyGroup.groups=*, delegation-token.token-kind=solr-dt, solr.kerberos.impersonator.user.wrongHost.hosts=1.1.1.1.1.1, cookie.path=/, zk-dt-secret-manager.znodeWorkingPath=solr/security/zkdtsm, solr.kerberos.impersonator.user.localHostAnyGroup.hosts=serv1-vm3,localhost,127.0.0.1, signer.secret.provider.zookeeper.path=/token, solr.kerberos.impersonator.user.anyHostUsersGroup.groups=staff, zk-dt-secret-manager.enable=true, solr.kerberos.impersonator.user.wrongHost.groups=*, kerberos.name.rules=DEFAULT, signer.secret.provider=zookeeper, solr.kerberos.impersonator.user.anyHostUsersGroup.hosts=*}
[junit4] 2> 4578540 INFO (jetty-launcher-4685-thread-2) [n:127.0.0.1:55633_solr ] o.a.c.f.i.CuratorFrameworkImpl Starting
[junit4] 2> 4578540 INFO (jetty-launcher-4685-thread-1) [n:127.0.0.1:37593_solr ] o.a.c.f.i.CuratorFrameworkImpl Starting
[junit4] 2> 4578560 INFO (jetty-launcher-4685-thread-1-EventThread) [n:127.0.0.1:37593_solr ] o.a.c.f.s.ConnectionStateManager State change: CONNECTED
[junit4] 2> 4578560 INFO (jetty-launcher-4685-thread-2-EventThread) [n:127.0.0.1:55633_solr ] o.a.c.f.s.ConnectionStateManager State change: CONNECTED
[junit4] 2> 4578679 WARN (jetty-launcher-4685-thread-2) [n:127.0.0.1:55633_solr ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
[junit4] 2> 4578685 WARN (jetty-launcher-4685-thread-1) [n:127.0.0.1:37593_solr ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
[junit4] 2> 4578724 INFO (jetty-launcher-4685-thread-2) [n:127.0.0.1:55633_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55633.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2bba2c6
[junit4] 2> 4578724 INFO (jetty-launcher-4685-thread-1) [n:127.0.0.1:37593_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37593.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2bba2c6
[junit4] 2> 4578730 INFO (jetty-launcher-4685-thread-2) [n:127.0.0.1:55633_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55633.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2bba2c6
[junit4] 2> 4578730 INFO (jetty-launcher-4685-thread-2) [n:127.0.0.1:55633_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55633.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2bba2c6
[junit4] 2> 4578730 INFO (jetty-launcher-4685-thread-1) [n:127.0.0.1:37593_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37593.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2bba2c6
[junit4] 2> 4578730 INFO (jetty-launcher-4685-thread-1) [n:127.0.0.1:37593_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37593.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2bba2c6
[junit4] 2> 4578731 INFO (jetty-launcher-4685-thread-2) [n:127.0.0.1:55633_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_8770C2BCC91D7DAF-001/tempDir-001/node2/.
[junit4] 2> 4578731 INFO (jetty-launcher-4685-thread-1) [n:127.0.0.1:37593_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_8770C2BCC91D7DAF-001/tempDir-001/node1/.
[junit4] 2> 4578785 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyUnknownRemote-seed#[8770C2BCC91D7DAF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyUnknownRemote
[junit4] 2> 4593402 WARN (jetty-launcher-4685-thread-2-SendThread(127.0.0.1:40456)) [n:127.0.0.1:55633_solr ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 14671ms for sessionid 0x15f6de3bf680006
[junit4] 2> 4593402 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15f6de3bf680006, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 4593402 WARN (jetty-launcher-4685-thread-1-SendThread(127.0.0.1:40456)) [n:127.0.0.1:37593_solr ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 14694ms for sessionid 0x15f6de3bf680007
[junit4] 2> 4593405 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15f6de3bf680007, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 4593404 WARN (jetty-launcher-4685-thread-2-SendThread(127.0.0.1:40456)) [n:127.0.0.1:55633_solr ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 14708ms for sessionid 0x15f6de3bf680008
[junit4] 2> 4593406 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15f6de3bf680008, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 4593406 WARN (jetty-launcher-4685-thread-1-SendThread(127.0.0.1:40456)) [n:127.0.0.1:37593_solr ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 14674ms for sessionid 0x15f6de3bf680004
[junit4] 2> 4593409 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15f6de3bf680004, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 4593451 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyUnknownRemote-seed#[8770C2BCC91D7DAF]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=localHostAnyGroup&p=localHostAnyGroup&t=test&e=1509413163751&s=pyb4gSUQcZzR+fcjpbzQK5GY9ZA="; Path=/; Domain=127.0.0.1; Expires= , 31-10-29 01:26:03 GMT; HttpOnly". Invalid 'expires' attribute: , 31-10-29 01:26:03 GMT
[junit4] 2> 4593451 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyUnknownRemote-seed#[8770C2BCC91D7DAF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyUnknownRemote
[junit4] 2> 4593513 WARN (zkCallback-4698-thread-1-processing-n:127.0.0.1:55633_solr) [n:127.0.0.1:55633_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@77cb20f3 name: ZooKeeperConnection Watcher:127.0.0.1:40456/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
[junit4] 2> 4593513 WARN (zkCallback-4698-thread-1-processing-n:127.0.0.1:55633_solr) [n:127.0.0.1:55633_solr ] o.a.s.c.c.ConnectionManager zkClient has disconnected
[junit4] 2> 4593514 WARN (zkCallback-4693-thread-3-processing-n:127.0.0.1:37593_solr) [n:127.0.0.1:37593_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3e1ba311 name: ZooKeeperConnection Watcher:127.0.0.1:40456/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
[junit4] 2> 4593515 WARN (zkCallback-4693-thread-3-processing-n:127.0.0.1:37593_solr) [n:127.0.0.1:37593_solr ] o.a.s.c.c.ConnectionManager zkClient has disconnected
[junit4] 2> 4593577 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNullProxyUser-seed#[8770C2BCC91D7DAF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNullProxyUser
[junit4] 2> 4593579 INFO (qtp1148788974-33070) [ ] o.a.s.s.SolrDispatchFilter Error authenticating
[junit4] 2> java.lang.IllegalArgumentException: userName is NULL, empty or contains a '&'
[junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.checkForIllegalArgument(AuthToken.java:87)
[junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.<init>(AuthToken.java:70)
[junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationToken.<init>(AuthenticationToken.java:58)
[junit4] 2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamDelegationTokenAuthenticationHandler$HttpParamAuthenticationHandler.authenticate(HttpParamDelegationTokenPlugin.java:213)
[junit4] 2> at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.authenticate(DelegationTokenAuthenticationHandler.java:348)
[junit4] 2> at org.apache.solr.security.RequestContinuesRecorderAuthenticationHandler.authenticate(RequestContinuesRecorderAuthenticationHandler.java:69)
[junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:518)
[junit4] 2> at org.apache.solr.security.DelegationTokenKerberosFilter.doFilter(DelegationTokenKerberosFilter.java:134)
[junit4] 2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamToRequestFilter.doFilter(HttpParamDelegationTokenPlugin.java:245)
[junit4] 2> at org.apache.solr.security.KerberosPlugin.doAuthenticate(KerberosPlugin.java:243)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:451)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 4593580 WARN (qtp1148788974-33070) [ ] o.e.j.s.ServletHandler /solr/admin/collections
[junit4] 2> org.apache.solr.common.SolrException: Error during request authentication,
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:457)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: java.lang.IllegalArgumentException: userName is NULL, empty or contains a '&'
[junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.checkForIllegalArgument(AuthToken.java:87)
[junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.<init>(AuthToken.java:70)
[junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationToken.<init>(AuthenticationToken.java:58)
[junit4] 2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamDelegationTokenAuthenticationHandler$HttpParamAuthenticationHandler.authenticate(HttpParamDelegationTokenPlugin.java:213)
[junit4] 2> at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.authenticate(DelegationTokenAuthenticationHandler.java:348)
[junit4] 2> at org.apache.solr.security.RequestContinuesRecorderAuthenticationHandler.authenticate(RequestContinuesRecorderAuthenticationHandler.java:69)
[junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:518)
[junit4] 2> at org.apache.solr.security.DelegationTokenKerberosFilter.doFilter(DelegationTokenKerberosFilter.java:134)
[junit4] 2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamToRequestFilter.doFilter(HttpParamDelegationTokenPlugin.java:245)
[junit4] 2> at org.apache.solr.security.KerberosPlugin.doAuthenticate(KerberosPlugin.java:243)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:451)
[junit4] 2> ... 26 more
[junit4] 2> 4593583 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNullProxyUser-seed#[8770C2BCC91D7DAF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyNullProxyUser
[junit4] 2> 4593692 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateGroup-seed#[8770C2BCC91D7DAF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateGroup
[junit4] 2> 4593742 INFO (qtp1148788974-33069) [n:127.0.0.1:55633_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=jenkins&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 4593743 INFO (qtp1148788974-33069) [n:127.0.0.1:55633_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=jenkins&action=LIST&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 4593745 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateGroup-seed#[8770C2BCC91D7DAF]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=anyHostUsersGroup&p=anyHostUsersGroup&t=test&e=1509413178647&s=iVdhjV5LgjBVjsa1k/HcG4TzSi4="; Path=/; Domain=127.0.0.1; Expires= , 31-10-29 01:26:18 GMT; HttpOnly". Invalid 'expires' attribute: , 31-10-29 01:26:18 GMT
[junit4] 2> 4593745 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateGroup-seed#[8770C2BCC91D7DAF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateGroup
[junit4] 2> 4593910 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateAnyHostAnyUser-seed#[8770C2BCC91D7DAF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateAnyHostAnyUser
[junit4] 2> 4593914 INFO (qtp1148788974-33061) [n:127.0.0.1:55633_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=bar&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 4593915 INFO (qtp1148788974-33061) [n:127.0.0.1:55633_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=bar&action=LIST&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 4593915 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateAnyHostAnyUser-seed#[8770C2BCC91D7DAF]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=anyHostAnyUser&p=anyHostAnyUser&t=test&e=1509413178869&s=1oMxS2bsyksvRxBzfeUBeodHEmY="; Path=/; Domain=127.0.0.1; Expires= , 31-10-29 01:26:18 GMT; HttpOnly". Invalid 'expires' attribute: , 31-10-29 01:26:18 GMT
[junit4] 2> 4593915 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateAnyHostAnyUser-seed#[8770C2BCC91D7DAF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateAnyHostAnyUser
[junit4] 2> 4594096 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigGroups-seed#[8770C2BCC91D7DAF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNoConfigGroups
[junit4] 2> 4594136 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigGroups-seed#[8770C2BCC91D7DAF]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=noGroups&p=noGroups&t=test&e=1509413179052&s=3VYqDwp3/hh3Bz9QbALd26WkXfQ="; Path=/; Domain=127.0.0.1; Expires= , 31-10-29 01:26:19 GMT; HttpOnly". Invalid 'expires' attribute: , 31-10-29 01:26:19 GMT
[junit4] 2> 4594137 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigGroups-seed#[8770C2BCC91D7DAF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyNoConfigGroups
[junit4] 2> 4594351 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidProxyUser-seed#[8770C2BCC91D7DAF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidProxyUser
[junit4] 2> 4594364 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidProxyUser-seed#[8770C2BCC91D7DAF]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=bar&p=bar&t=test&e=1509413179307&s=jjynIoxqWvq/s136+lkLPvN6mHU="; Path=/; Domain=127.0.0.1; Expires= , 31-10-29 01:26:19 GMT; HttpOnly". Invalid 'expires' attribute: , 31-10-29 01:26:19 GMT
[junit4] 2> 4594364 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidProxyUser-seed#[8770C2BCC91D7DAF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidProxyUser
[junit4] 2> 4594495 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidGroup-seed#[8770C2BCC91D7DAF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidGroup
[junit4] 2> 4594510 INFO (jetty-launcher-4685-thread-2-EventThread) [n:127.0.0.1:55633_solr ] o.a.c.f.s.ConnectionStateManager State change: SUSPENDED
[junit4] 2> 4594537 WARN (qtp1148788974-33059) [ ] o.a.h.s.ShellBasedUnixGroupsMapping got exception trying to get groups for user bar: id: invalid user name: "bar"
[junit4] 2> id: invalid user name: "bar"
[junit4] 2>
[junit4] 2> 4594540 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidGroup-seed#[8770C2BCC91D7DAF]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=bogusGroup&p=bogusGroup&t=test&e=1509413179452&s=IDkyKjNchSAYqsCb0Eo9NVOCXrY="; Path=/; Domain=127.0.0.1; Expires= , 31-10-29 01:26:19 GMT; HttpOnly". Invalid 'expires' attribute: , 31-10-29 01:26:19 GMT
[junit4] 2> 4594540 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidGroup-seed#[8770C2BCC91D7DAF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidGroup
[junit4] 2> 4594633 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateHost-seed#[8770C2BCC91D7DAF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateHost
[junit4] 2> 4594634 INFO (qtp1148788974-33068) [n:127.0.0.1:55633_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=bar&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 4594634 INFO (qtp1148788974-33068) [n:127.0.0.1:55633_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=bar&action=LIST&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 4594635 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateHost-seed#[8770C2BCC91D7DAF]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=localHostAnyGroup&p=localHostAnyGroup&t=test&e=1509413179588&s=U+rBWy9OopTNDk3oxXmWF4hxJXg="; Path=/; Domain=127.0.0.1; Expires= , 31-10-29 01:26:19 GMT; HttpOnly". Invalid 'expires' attribute: , 31-10-29 01:26:19 GMT
[junit4] 2> 4594635 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateHost-seed#[8770C2BCC91D7DAF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateHost
[junit4] 2> 4594774 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigHosts-seed#[8770C2BCC91D7DAF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNoConfigHosts
[junit4] 2> 4594779 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigHosts-seed#[8770C2BCC91D7DAF]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=noHosts&p=noHosts&t=test&e=1509413179730&s=h5xV4dq3DtWefcC+MLoIK95PNpc="; Path=/; Domain=127.0.0.1; Expires= , 31-10-29 01:26:19 GMT; HttpOnly". Invalid 'expires' attribute: , 31-10-29 01:26:19 GMT
[junit4] 2> 4594780 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigHosts-seed#[8770C2BCC91D7DAF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyNoConfigHosts
[junit4] 2> 4594923 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyWrongHost-seed#[8770C2BCC91D7DAF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyWrongHost
[junit4] 2> 4594950 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyWrongHost-seed#[8770C2BCC91D7DAF]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=wrongHost&p=wrongHost&t=test&e=1509413179879&s=KV3TS3p9aeLnntKGMw96Q1JM36U="; Path=/; Domain=127.0.0.1; Expires= , 31-10-29 01:26:19 GMT; HttpOnly". Invalid 'expires' attribute: , 31-10-29 01:26:19 GMT
[junit4] 2> 4594951 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyWrongHost-seed#[8770C2BCC91D7DAF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyWrongHost
[junit4] 2> 4595080 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[8770C2BCC91D7DAF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testForwarding
[junit4] 2> 4595444 INFO (jetty-launcher-4685-thread-1-EventThread) [n:127.0.0.1:37593_solr ] o.a.c.f.s.ConnectionStateManager State change: SUSPENDED
[junit4] 2> 4596641 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[8770C2BCC91D7DAF]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 4596643 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[8770C2BCC91D7DAF]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40456/solr ready
[junit4] 2> 4596651 INFO (qtp1615567036-33049) [n:127.0.0.1:37593_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=0&replicationFactor=1&collection.configName=conf1&maxShardsPerNode=1&name=forwardingCollection&nrtReplicas=1&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 4596656 INFO (OverseerThreadFactory-12749-thread-1-processing-n:127.0.0.1:37593_solr) [n:127.0.0.1:37593_solr ] o.a.s.c.CreateCollectionCmd Create collection forwardingCollection
[junit4] 2> 4596784 INFO (OverseerStateUpdate-98918541761708036-127.0.0.1:37593_solr-n_0000000000) [n:127.0.0.1:37593_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"forwardingCollection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"forwardingCollection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:37593/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 4596990 INFO (qtp1615567036-33057) [n:127.0.0.1:37593_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf1&newCollection=true&name=forwardingCollection_shard1_replica_n1&action=CREATE&numShards=1&collection=forwardingCollection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 4596991 INFO (qtp1615567036-33057) [n:127.0.0.1:37593_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 4597101 INFO (zkCallback-4693-thread-11-processing-n:127.0.0.1:37593_solr) [n:127.0.0.1:37593_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 4597101 INFO (zkCallback-4693-thread-9-processing-n:127.0.0.1:37593_solr) [n:127.0.0.1:37593_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 4598049 INFO (qtp1615567036-33057) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 4598096 INFO (qtp1615567036-33057) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.IndexSchema [forwardingCollection_shard1_replica_n1] Schema name=test
[junit4] 2> 4598351 INFO (qtp1615567036-33057) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 4598393 INFO (qtp1615567036-33057) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'forwardingCollection_shard1_replica_n1' using configuration from collection forwardingCollection, trusted=true
[junit4] 2> 4598394 INFO (qtp1615567036-33057) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37593.solr.core.forwardingCollection.shard1.replica_n1' (registry 'solr.core.forwardingCollection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2bba2c6
[junit4] 2> 4598394 INFO (qtp1615567036-33057) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 4598394 INFO (qtp1615567036-33057) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore [[forwardingCollection_shard1_replica_n1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_8770C2BCC91D7DAF-001/tempDir-001/node1/forwardingCollection_shard1_replica_n1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_8770C2BCC91D7DAF-001/tempDir-001/node1/./forwardingCollection_shard1_replica_n1/data/]
[junit4] 2> 4598397 INFO (qtp1615567036-33057) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=44, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 4598624 INFO (qtp1615567036-33057) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 4598624 INFO (qtp1615567036-33057) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 4598697 INFO (qtp1615567036-33057) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 4598698 INFO (qtp1615567036-33057) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 4598699 INFO (qtp1615567036-33057) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=45, maxMergeAtOnceExplicit=42, maxMergedSegmentMB=43.3408203125, floorSegmentMB=1.7060546875, forceMergeDeletesPctAllowed=16.73041714684591, segmentsPerTier=35.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 4598699 INFO (qtp1615567036-33057) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@71b0015e[forwardingCollection_shard1_replica_n1] main]
[junit4] 2> 4598702 INFO (qtp1615567036-33057) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
[junit4] 2> 4598749 INFO (qtp1615567036-33057) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4] 2> 4598781 INFO (qtp1615567036-33057) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 4598781 INFO (searcherExecutor-12754-thread-1-processing-n:127.0.0.1:37593_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
[junit4] 2> 4598781 INFO (qtp1615567036-33057) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1582696689813159936
[junit4] 2> 4598785 INFO (searcherExecutor-12754-thread-1-processing-n:127.0.0.1:37593_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
[junit4] 2> 4598785 INFO (searcherExecutor-12754-thread-1-processing-n:127.0.0.1:37593_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
[junit4] 2> 4598785 INFO (searcherExecutor-12754-thread-1-processing-n:127.0.0.1:37593_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
[junit4] 2> 4598786 INFO (searcherExecutor-12754-thread-1-processing-n:127.0.0.1:37593_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
[junit4] 2> 4598787 INFO (searcherExecutor-12754-thread-1-processing-n:127.0.0.1:37593_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
[junit4] 2> 4598788 INFO (searcherExecutor-12754-thread-1-processing-n:127.0.0.1:37593_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
[junit4] 2> 4598788 INFO (searcherExecutor-12754-thread-1-processing-n:127.0.0.1:37593_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
[junit4] 2> 4598789 INFO (searcherExecutor-12754-thread-1-processing-n:127.0.0.1:37593_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
[junit4] 2> 4598791 INFO (qtp1615567036-33057) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 4598791 INFO (qtp1615567036-33057) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 4598791 INFO (qtp1615567036-33057) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:37593/solr/forwardingCollection_shard1_replica_n1/
[junit4] 2> 4598791 INFO (searcherExecutor-12754-thread-1-processing-n:127.0.0.1:37593_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore [forwardingCollection_shard1_replica_n1] Registered new searcher Searcher@71b0015e[forwardingCollection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 4598791 INFO (qtp1615567036-33057) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 4598791 INFO (qtp1615567036-33057) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:37593/solr/forwardingCollection_shard1_replica_n1/ has no replicas
[junit4] 2> 4598792 INFO (qtp1615567036-33057) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 4598796 INFO (qtp1615567036-33057) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:37593/solr/forwardingCollection_shard1_replica_n1/ shard1
[junit4] 2> 4598899 INFO (zkCallback-4693-thread-9-processing-n:127.0.0.1:37593_solr) [n:127.0.0.1:37593_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 4598899 INFO (zkCallback-4693-thread-7-processing-n:127.0.0.1:37593_solr) [n:127.0.0.1:37593_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 4599049 INFO (qtp1615567036-33057) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 4599063 INFO (qtp1615567036-33057) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf1&newCollection=true&name=forwardingCollection_shard1_replica_n1&action=CREATE&numShards=1&collection=forwardingCollection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2073
[junit4] 2> 4599064 WARN (httpShardExecutor-12741-thread-1-processing-n:127.0.0.1:37593_solr [http:////127.0.0.1:37593//solr] http:////127.0.0.1:37593//solr) [n:127.0.0.1:37593_solr ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=$&p=$&t=test&e=1509413181943&s=hEvkTJPEmABX+S7eUEAGWTgW/rU="; Path=/; Domain=127.0.0.1; Expires= , 31-10-29 01:26:21 GMT; HttpOnly". Invalid 'expires' attribute: , 31-10-29 01:26:21 GMT
[junit4] 2> 4599072 INFO (qtp1615567036-33049) [n:127.0.0.1:37593_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 4599170 INFO (zkCallback-4693-thread-7-processing-n:127.0.0.1:37593_solr) [n:127.0.0.1:37593_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 4599170 INFO (zkCallback-4693-thread-9-processing-n:127.0.0.1:37593_solr) [n:127.0.0.1:37593_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 4600081 INFO (qtp1615567036-33049) [n:127.0.0.1:37593_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={pullReplicas=0&replicationFactor=1&collection.configName=conf1&maxShardsPerNode=1&name=forwardingCollection&nrtReplicas=1&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2} status=0 QTime=3430
[junit4] 2> 4600082 WARN (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[8770C2BCC91D7DAF]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=user&p=user&t=test&e=1509413181605&s=Ymu5fQhCd8uqrYcvKy3m5mmumB8="; Path=/; Domain=127.0.0.1; Expires= , 31-10-29 01:26:21 GMT; HttpOnly". Invalid 'expires' attribute: , 31-10-29 01:26:21 GMT
[junit4] 2> 4600083 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[8770C2BCC91D7DAF]) [ ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: forwardingCollection failOnTimeout:true timeout (sec):100
[junit4] 2> 4600083 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[8770C2BCC91D7DAF]) [ ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: forwardingCollection
[junit4] 2> 4600088 INFO (qtp1148788974-33069) [n:127.0.0.1:55633_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 4600103 INFO (qtp1615567036-33058) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.S.Request [forwardingCollection_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 4600104 WARN (qtp1148788974-33069) [n:127.0.0.1:55633_solr ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=$&p=$&t=test&e=1509413185056&s=RtTFajSA4m1btnIRQZeqWzgyfH8="; Path=/; Domain=127.0.0.1; Expires= , 31-10-29 01:26:25 GMT; HttpOnly". Invalid 'expires' attribute: , 31-10-29 01:26:25 GMT
[junit4] 2> 4600105 WARN (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[8770C2BCC91D7DAF]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=user&p=user&t=test&e=1509413185042&s=k4DdBo5waAnci1GAiA734OK6O3s="; Path=/; Domain=127.0.0.1; Expires= , 31-10-29 01:26:25 GMT; HttpOnly". Invalid 'expires' attribute: , 31-10-29 01:26:25 GMT
[junit4] 2> 4600105 WARN (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[8770C2BCC91D7DAF]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=$&p=$&t=test&e=1509413185056&s=RtTFajSA4m1btnIRQZeqWzgyfH8="; Path=/; Domain=127.0.0.1; Expires= , 31-10-29 01:26:25 GMT; HttpOnly". Invalid 'expires' attribute: , 31-10-29 01:26:25 GMT
[junit4] 2> 4600109 INFO (qtp1615567036-33054) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.S.Request [forwardingCollection_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 4600109 WARN (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[8770C2BCC91D7DAF]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=user&p=user&t=test&e=1509413185063&s=ek85tA877/4GCcXRtK0I/5QY69E="; Path=/; Domain=127.0.0.1; Expires= , 31-10-29 01:26:25 GMT; HttpOnly". Invalid 'expires' attribute: , 31-10-29 01:26:25 GMT
[junit4] 2> 4600110 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[8770C2BCC91D7DAF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testForwarding
[junit4] 2> 4600316 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidRemote-seed#[8770C2BCC91D7DAF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidRemote
[junit4] 2> 4600330 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidRemote-seed#[8770C2BCC91D7DAF]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=localHostAnyGroup&p=localHostAnyGroup&t=test&e=1509413185273&s=s3FPRRjeNVML4WymcFyhE6zVxRc="; Path=/; Domain=127.0.0.1; Expires= , 31-10-29 01:26:25 GMT; HttpOnly". Invalid 'expires' attribute: , 31-10-29 01:26:25 GMT
[junit4] 2> 4600331 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidRemote-seed#[8770C2BCC91D7DAF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidRemote
[junit4] 2> 4600338 INFO (jetty-closer-4686-thread-2) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@51b64f1b{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 4600371 INFO (jetty-closer-4686-thread-2) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1105574936
[junit4] 2> 4600372 INFO (jetty-closer-4686-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 4600372 INFO (jetty-closer-4686-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2d39ca7e: rootName = solr_37593, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@25911cc1
[junit4] 2> 4600377 INFO (jetty-closer-4686-thread-1) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@7de82f5d{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 4600378 INFO (jetty-closer-4686-thread-1) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1235480952
[junit4] 2> 4600378 INFO (jetty-closer-4686-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 4600378 INFO (jetty-closer-4686-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6521d473: rootName = solr_55633, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@36a0624e
[junit4] 2> 4600378 INFO (jetty-closer-4686-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 4600378 INFO (jetty-closer-4686-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@724cd49f: rootName = solr_37593, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@282e0f15
[junit4] 2> 4600386 INFO (jetty-closer-4686-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 4600386 INFO (jetty-closer-4686-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6947ef5a: rootName = solr_55633, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@282e0f15
[junit4] 2> 4600388 INFO (jetty-closer-4686-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 4600388 INFO (jetty-closer-4686-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@74798771: rootName = solr_37593, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@49554558
[junit4] 2> 4600390 INFO (jetty-closer-4686-thread-2) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:37593_solr
[junit4] 2> 4600391 INFO (jetty-closer-4686-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 4600392 INFO (jetty-closer-4686-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 4600392 INFO (jetty-closer-4686-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7b02c134: rootName = solr_55633, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@49554558
[junit4] 2> 4600394 INFO (zkCallback-4698-thread-6-processing-n:127.0.0.1:55633_solr) [n:127.0.0.1:55633_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 4600396 INFO (jetty-closer-4686-thread-1) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:55633_solr
[junit4] 2> 4600397 INFO (coreCloseExecutor-12759-thread-1) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore [forwardingCollection_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@4645063f
[junit4] 2> 4600397 INFO (jetty-closer-4686-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 4600397 INFO (coreCloseExecutor-12759-thread-1) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.forwardingCollection.shard1.replica_n1, tag=1178928703
[junit4] 2> 4600397 INFO (coreCloseExecutor-12759-thread-1) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@102f67de: rootName = solr_37593, domain = solr.core.forwardingCollection.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.forwardingCollection.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@1980c22a
[junit4] 2> 4600399 WARN (zkCallback-4698-thread-6-processing-n:127.0.0.1:55633_solr) [n:127.0.0.1:55633_solr ] o.a.s.c.ZkController Unable to read autoscaling.json
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /autoscaling.json
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:332)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:332)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.getAutoScalingConfig(ZkStateReader.java:185)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.getAutoScalingConfig(ZkStateReader.java:171)
[junit4] 2> at org.apache.solr.cloud.ZkController.lambda$registerLiveNodesListener$2(ZkController.java:871)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.lambda$refreshLiveNodes$1(ZkStateReader.java:749)
[junit4] 2> at java.util.concurrent.ConcurrentHashMap$KeySetView.forEach(ConcurrentHashMap.java:4649)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.refreshLiveNodes(ZkStateReader.java:748)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.access$1000(ZkStateReader.java:72)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader$LiveNodeWatcher.refreshAndWatch(ZkStateReader.java:1154)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader$LiveNodeWatcher.process(ZkStateReader.java:1149)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$1.lambda$process$0(SolrZkClient.java:262)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 4600409 INFO (zkCallback-4693-thread-7-processing-n:127.0.0.1:37593_solr) [n:127.0.0.1:37593_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (0)
[junit4] 2> 4600421 ERROR (Thread[Thread-7538,5,TGRP-TestSolrCloudWithSecureImpersonation]) [n:127.0.0.1:55633_solr ] o.a.h.s.t.d.AbstractDelegationTokenSecretManager ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
[junit4] 2> 4600424 INFO (coreCloseExecutor-12759-thread-1) [n:127.0.0.1:37593_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.forwardingCollection.shard1.leader, tag=1178928703
[junit4] 2> 4600476 INFO (jetty-closer-4686-thread-2) [ ] o.a.s.c.Overseer Overseer (id=98918541761708036-127.0.0.1:37593_solr-n_0000000000) closing
[junit4] 2> 4600478 WARN (OverseerAutoScalingTriggerThread-98918541761708036-127.0.0.1:37593_solr-n_0000000000) [n:127.0.0.1:37593_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 4600479 INFO (OverseerStateUpdate-98918541761708036-127.0.0.1:37593_solr-n_0000000000) [n:127.0.0.1:37593_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:37593_solr
[junit4] 2> 4600493 ERROR (Thread[Thread-7540,5,TGRP-TestSolrCloudWithSecureImpersonation]) [n:127.0.0.1:37593_solr ] o.a.h.s.t.d.AbstractDelegationTokenSecretManager ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
[junit4] 2> 4600499 INFO (jetty-closer-4686-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@567c7e77{/solr,null,UNAVAILABLE}
[junit4] 2> 4600500 INFO (jetty-closer-4686-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2ad7ddeb{/solr,null,UNAVAILABLE}
[junit4] 2> 4600511 ERROR (SUITE-TestSolrCloudWithSecureImpersonation-seed#[8770C2BCC91D7DAF]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 4600516 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[8770C2BCC91D7DAF]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:40456 40456
[junit4] 2> 4600607 INFO (Thread-7515) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:40456 40456
[junit4] 2> 4600609 WARN (Thread-7515) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/aliases.json
[junit4] 2> 3 /solr/clusterprops.json
[junit4] 2> 2 /solr/security.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMKeyIdRoot
[junit4] 2> 5 /solr/collections/forwardingCollection/state.json
[junit4] 2> 3 /solr/clusterstate.json
[junit4] 2> 3 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMSeqNumRoot
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_1
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_3
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_2
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_4
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/live_nodes
[junit4] 2> 3 /solr/collections
[junit4] 2> 3 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot
[junit4] 2> 3 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot
[junit4] 2>
[junit4] 2> 4600698 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[8770C2BCC91D7DAF]-worker) [ ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
[junit4] 2> Oct 30, 2017 3:26:25 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 3 leaked thread(s).
[junit4] 2> Oct 30, 2017 3:26:46 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 2 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
[junit4] 2> 1) Thread[id=33101, name=jetty-launcher-4685-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
[junit4] 2> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
[junit4] 2> at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
[junit4] 2> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
[junit4] 2> at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
[junit4] 2> 2) Thread[id=33103, name=jetty-launcher-4685-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
[junit4] 2> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
[junit4] 2> at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
[junit4] 2> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
[junit4] 2> at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
[junit4] 2> Oct 30, 2017 3:26:46 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=33101, name=jetty-launcher-4685-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] 2> 2) Thread[id=33103, name=jetty-launcher-4685-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] 2> 4621073 ERROR (jetty-launcher-4685-thread-2-EventThread) [n:127.0.0.1:55633_solr ] o.a.c.f.i.CuratorFrameworkImpl Watcher exception
[junit4] 2> java.lang.InterruptedException
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1039)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
[junit4] 2> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
[junit4] 2> at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
[junit4] 2> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
[junit4] 2> at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
[junit4] 2> 4621075 ERROR (jetty-launcher-4685-thread-1-EventThread) [n:127.0.0.1:37593_solr ] o.a.c.f.i.CuratorFrameworkImpl Watcher exception
[junit4] 2> java.lang.InterruptedException
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1039)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
[junit4] 2> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
[junit4] 2> at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
[junit4] 2> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
[junit4] 2> at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
[junit4] 2> Oct 30, 2017 3:26:46 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: All leaked threads terminated.
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=64, maxMBSortInHeap=7.495786127416174, sim=RandomSimilarity(queryNorm=false): {}, locale=ja-JP-u-ca-japanese-x-lvariant-JP, timezone=GMT
[junit4] 2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=3,threads=1,free=247437512,total=536870912
[junit4] 2> NOTE: All tests run in this JVM: [RecoveryZkTest, OverseerModifyCollectionTest, HighlighterMaxOffsetTest, UUIDUpdateProcessorFallbackTest, WrapperMergePolicyFactoryTest, LeaderFailoverAfterPartitionTest, DistanceFunctionTest, DistributedFacetPivotLongTailTest, SpellCheckCollatorWithCollapseTest, TestConfigReload, ReplicationFactorTest, TestPullReplicaErrorHandling, TestSimpleTextCodec, PeerSyncReplicationTest, TestStressInPlaceUpdates, TestFilteredDocIdSet, SearchRateTriggerTest, DeleteInactiveReplicaTest, TestSolrXml, ZkFailoverTest, TestEmbeddedSolrServerSchemaAPI, TestCodecSupport, TestLegacyFieldCache, TestRealTimeGet, TestInPlaceUpdatesStandalone, TestLocalFSCloudBackupRestore, TestConfigSetImmutable, TestQuerySenderListener, TestSlowCompositeReaderWrapper, TestUtils, TestFieldCache, TestMiniSolrCloudClusterSSL, TestHdfsUpdateLog, TestNonDefinedSimilarityFactory, OverriddenZkACLAndCredentialsProvidersTest, TestOmitPositions, TestRecovery, BitVectorTest, PropertiesRequestHandlerTest, TestUseDocValuesAsStored2, TestRangeQuery, AutoAddReplicasPlanActionTest, OverseerStatusTest, TestSchemaSimilarityResource, RemoteQueryErrorTest, HdfsCollectionsAPIDistributedZkTest, TestFieldCacheSort, DistributedVersionInfoTest, XsltUpdateRequestHandlerTest, TestConfigSetsAPI, TestMissingGroups, SuggesterTest, ForceLeaderTest, AsyncCallRequestStatusResponseTest, HttpPartitionTest, TestIBSimilarityFactory, TestSweetSpotSimilarityFactory, TestLazyCores, DirectSolrConnectionTest, DeleteNodeTest, TestPHPSerializedResponseWriter, SuggesterFSTTest, DistributedTermsComponentTest, RulesTest, SolrCloudReportersTest, ResponseLogComponentTest, TestNamedUpdateProcessors, TestSolrQueryResponse, HdfsRecoverLeaseTest, TestImplicitCoreProperties, TestLegacyNumericRangeQueryBuilder, TestComplexPhraseLeadingWildcard, DocExpirationUpdateProcessorFactoryTest, DisMaxRequestHandlerTest, DateMathParserTest, TestFieldCollectionResource, DistributedDebugComponentTest, TestCloudJSONFacetJoinDomain, SearchHandlerTest, PeerSyncWithIndexFingerprintCachingTest, IndexSchemaTest, V2ApiIntegrationTest, TestFreeTextSuggestions, SharedFSAutoReplicaFailoverTest, TestHdfsCloudBackupRestore, TestSQLHandlerNonCloud, ZkStateReaderTest, ClassificationUpdateProcessorTest, TestSolrCoreProperties, SuggestComponentContextFilterQueryTest, ConnectionReuseTest, TestHighlightDedupGrouping, CdcrVersionReplicationTest, ConfigureRecoveryStrategyTest, TestManagedStopFilterFactory, TestSerializedLuceneMatchVersion, HdfsAutoAddReplicasIntegrationTest, LeaderInitiatedRecoveryOnCommitTest, DistributedSuggestComponentTest, CleanupOldIndexTest, DeleteShardTest, TestAtomicUpdateErrorCases, CdcrUpdateLogTest, CurrencyFieldTypeTest, MoveReplicaHDFSFailoverTest, TestUpdate, TestIntervalFaceting, TestSubQueryTransformerDistrib, JSONWriterTest, TestWriterPerf, TestPolicyCloud, TestScoreJoinQPScore, HighlighterConfigTest, TestPhraseSuggestions, TestSchemaNameResource, LargeFieldTest, DefaultValueUpdateProcessorTest, IndexBasedSpellCheckerTest, TestCloudSchemaless, SolrXmlInZkTest, TestSolrCoreParser, SpellCheckCollatorTest, TestJoin, SolrCLIZkUtilsTest, SpellCheckComponentTest, TestSolrConfigHandlerConcurrent, ZkControllerTest, TestDistributedStatsComponentCardinality, TestTrieFacet, TestXIncludeConfig, DistribCursorPagingTest, CreateCollectionCleanupTest, SolrJmxReporterTest, InfixSuggestersTest, ParsingFieldUpdateProcessorsTest, TestSurroundQueryParser, SignatureUpdateProcessorFactoryTest, ScriptEngineTest, TestSegmentSorting, DirectUpdateHandlerTest, TestSchemalessBufferedUpdates, TriLevelCompositeIdRoutingTest, TestSubQueryTransformer, BlockCacheTest, PrimitiveFieldTypeTest, TestSmileRequest, RankQueryTest, UnloadDistributedZkTest, MetricsConfigTest, AnalyticsQueryTest, TestTolerantSearch, NumericFieldsTest, TestStressRecovery, TestHttpShardHandlerFactory, BasicAuthIntegrationTest, DistributedSpellCheckComponentTest, ConvertedLegacyTest, DistributedFacetPivotSmallTest, TestRestoreCore, CSVRequestHandlerTest, SolrMetricManagerTest, TestSolrCoreSnapshots, TestConfigsApi, AddBlockUpdateTest, SolrGraphiteReporterTest, TestExactStatsCache, ReturnFieldsTest, HdfsUnloadDistributedZkTest, DateFieldTest, CollectionReloadTest, NodeMutatorTest, TestRandomRequestDistribution, TestCloudDeleteByQuery, ZkCLITest, TestReplicationHandler, BlockJoinFacetRandomTest, AddReplicaTest, PrimUtilsTest, ManagedSchemaRoundRobinCloudTest, TestCollectionAPI, TestJsonFacets, SpatialFilterTest, BasicDistributedZkTest, BasicDistributedZk2Test, SyncSliceTest, ShardRoutingTest, BasicZkTest, ClusterStateUpdateTest, LeaderElectionTest, TestZkChroot, ShardRoutingCustomTest, TestDistributedSearch, SolrCoreTest, SolrCmdDistributorTest, BadIndexSchemaTest, TestFiltering, BasicFunctionalityTest, TestBadConfig, HighlighterTest, ShowFileRequestHandlerTest, DistributedQueryElevationComponentTest, TestCoreDiscovery, TestExtendedDismaxParser, CoreAdminHandlerTest, SolrRequestParserTest, TestFoldingMultitermQuery, SuggesterTSTTest, TestStressLucene, SuggesterWFSTTest, PolyFieldTest, NoCacheHeaderTest, WordBreakSolrSpellCheckerTest, SchemaVersionSpecificBehaviorTest, SolrCoreCheckLockOnStartupTest, TestPseudoReturnFields, TestWordDelimiterFilterFactory, DirectUpdateHandlerOptimizeTest, SortByFunctionTest, TestSolrDeletionPolicy1, SolrInfoBeanTest, LukeRequestHandlerTest, TestReversedWildcardFilterFactory, FileBasedSpellCheckerTest, TermsComponentTest, XmlUpdateRequestHandlerTest, RequestHandlersTest, PathHierarchyTokenizerFactoryTest, TestIndexingPerformance, RequiredFieldsTest, FieldAnalysisRequestHandlerTest, TestSolrQueryParser, FastVectorHighlighterTest, LoggingHandlerTest, RegexBoostProcessorTest, SolrPluginUtilsTest, TestJmxIntegration, UpdateRequestProcessorFactoryTest, TestCSVResponseWriter, MBeansHandlerTest, QueryParsingTest, TestAnalyzedSuggestions, UniqFieldsUpdateProcessorFactoryTest, BinaryUpdateRequestHandlerTest, TestPartialUpdateDeduplication, PingRequestHandlerTest, TestLFUCache, SOLR749Test, UpdateParamsTest, AlternateDirectoryTest, TestQuerySenderNoQuery, CopyFieldTest, BadComponentTest, TestMergePolicyConfig, TestSolrDeletionPolicy2, MultiTermTest, SampleTest, MinimalSchemaTest, OutputWriterTest, TestPostingsSolrHighlighter, NotRequiredUniqueKeyTest, SynonymTokenizerTest, TestDFRSimilarityFactory, TestPerFieldSimilarity, TestBM25SimilarityFactory, TimeZoneUtilsTest, OpenExchangeRatesOrgProviderTest, URLClassifyProcessorTest, ChaosMonkeyNothingIsSafeTest, CdcrReplicationHandlerTest, CdcrRequestHandlerTest, ChaosMonkeyNothingIsSafeWithPullReplicasTest, CloudExitableDirectoryReaderTest, CollectionTooManyReplicasTest, CollectionsAPISolrJTest, DistribJoinFromCollectionTest, HealthCheckHandlerTest, MigrateRouteKeyTest, MoveReplicaTest, SSLMigrationTest, SaslZkACLProviderTest, SolrCloudExampleTest, TestCloudPivotFacet, TestCloudPseudoReturnFields, TestCloudSearcherWarming, TestCollectionsAPIViaSolrCloudCluster, TestConfigSetsAPIExclusivity, TestConfigSetsAPIZkFailure, TestDeleteCollectionOnDownNodes, TestDownShardTolerantSearch, TestExclusionRuleCollectionAccess, TestLeaderElectionWithEmptyReplica, TestLeaderElectionZkExpiry, TestLockTree, TestPrepRecovery, TestRequestStatusCollectionAPI, TestSizeLimitedDistributedMap, TestSolrCloudWithSecureImpersonation]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestSolrCloudWithSecureImpersonation -Dtests.seed=8770C2BCC91D7DAF -Dtests.slow=true -Dtests.locale=ja-JP-u-ca-japanese-x-lvariant-JP -Dtests.timezone=GMT -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.00s J0 | TestSolrCloudWithSecureImpersonation (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
[junit4] > 1) Thread[id=33101, name=jetty-launcher-4685-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
[junit4] > at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
[junit4] > at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
[junit4] > at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
[junit4] > at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
[junit4] > 2) Thread[id=33103, name=jetty-launcher-4685-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
[junit4] > at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
[junit4] > at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
[junit4] > at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
[junit4] > at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
[junit4] > at __randomizedtesting.SeedInfo.seed([8770C2BCC91D7DAF]:0)
[junit4] Completed [555/746 (1!)] on J0 in 43.00s, 12 tests, 1 error <<< FAILURES!
[...truncated 47824 lines...]
[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build #
1497 - Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1497/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseSerialGC
1 tests failed.
FAILED: org.apache.solr.cloud.TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([E851EAF348116DFC:3E8ECBA97E0C7E64]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry(TestLeaderElectionZkExpiry.java:97)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 11693 lines...]
[junit4] Suite: org.apache.solr.cloud.TestLeaderElectionZkExpiry
[junit4] 2> 375845 INFO (SUITE-TestLeaderElectionZkExpiry-seed#[E851EAF348116DFC]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestLeaderElectionZkExpiry_E851EAF348116DFC-001/init-core-data-001
[junit4] 2> 375846 WARN (SUITE-TestLeaderElectionZkExpiry-seed#[E851EAF348116DFC]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=353 numCloses=353
[junit4] 2> 375846 INFO (SUITE-TestLeaderElectionZkExpiry-seed#[E851EAF348116DFC]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 375847 INFO (SUITE-TestLeaderElectionZkExpiry-seed#[E851EAF348116DFC]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 375855 INFO (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testLeaderElectionWithZkExpiry
[junit4] 2> 375869 INFO (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@730c929d, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 375922 INFO (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@730c929d
[junit4] 2> 375934 INFO (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@730c929d
[junit4] 2> 375935 INFO (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@730c929d
[junit4] 2> 375936 INFO (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC]) [ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestLeaderElectionZkExpiry_E851EAF348116DFC-001/testLeaderElectionWithZkExpiry-solr-001
[junit4] 2> 375937 INFO (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 375937 INFO (Thread-1101) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 375947 INFO (Thread-1101) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 375949 ERROR (Thread-1101) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 376038 INFO (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC]) [ ] o.a.s.c.ZkTestServer start zk server on port:58040
[junit4] 2> 376261 INFO (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC]) [n:dummy.host.com:8984_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 376262 INFO (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC]) [n:dummy.host.com:8984_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader dummy.host.com:8984_solr
[junit4] 2> 376267 INFO (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC]) [n:dummy.host.com:8984_solr ] o.a.s.c.Overseer Overseer (id=98916024352047106-dummy.host.com:8984_solr-n_0000000000) starting
[junit4] 2> 376288 INFO (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC]) [n:dummy.host.com:8984_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/dummy.host.com:8984_solr
[junit4] 2> 376291 INFO (zkCallback-393-thread-1-processing-n:dummy.host.com:8984_solr) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 376395 WARN (zkCallback-393-thread-2-processing-n:dummy.host.com:8984_solr) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2d684005 name: ZooKeeperConnection Watcher:127.0.0.1:58040/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
[junit4] 2> 376397 WARN (zkCallback-393-thread-2-processing-n:dummy.host.com:8984_solr) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager zkClient has disconnected
[junit4] 2> 377867 WARN (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC]-SendThread(127.0.0.1:58040)) [n:dummy.host.com:8984_solr ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x15f6b999e190002 has expired
[junit4] 2> 377868 WARN (OverseerStateUpdate-98916024352047106-dummy.host.com:8984_solr-n_0000000000) [n:dummy.host.com:8984_solr ] o.a.s.c.Overseer Solr cannot talk to ZK, exiting Overseer main queue loop
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/queue
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1532)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getChildren$4(SolrZkClient.java:320)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:320)
[junit4] 2> at org.apache.solr.cloud.ZkDistributedQueue.fetchZkChildren(ZkDistributedQueue.java:410)
[junit4] 2> at org.apache.solr.cloud.ZkDistributedQueue.firstChild(ZkDistributedQueue.java:387)
[junit4] 2> at org.apache.solr.cloud.ZkDistributedQueue.peekElements(ZkDistributedQueue.java:441)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 377868 INFO (OverseerStateUpdate-98916024352047106-dummy.host.com:8984_solr-n_0000000000) [n:dummy.host.com:8984_solr ] o.a.s.c.Overseer Overseer Loop exiting : dummy.host.com:8984_solr
[junit4] 2> 377868 WARN (OverseerCollectionConfigSetProcessor-98916024352047106-dummy.host.com:8984_solr-n_0000000000) [n:dummy.host.com:8984_solr ] o.a.s.c.OverseerTaskProcessor Overseer cannot talk to ZK
[junit4] 2> 377868 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2d684005 name: ZooKeeperConnection Watcher:127.0.0.1:58040/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
[junit4] 2> 377868 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
[junit4] 2> 377868 INFO (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr) [n:dummy.host.com:8984_solr ] o.a.s.c.Overseer Overseer (id=98916024352047106-dummy.host.com:8984_solr-n_0000000000) closing
[junit4] 2> 377875 ERROR (OverseerExitThread) [n:dummy.host.com:8984_solr ] o.a.s.c.Overseer could not read the data
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:332)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:332)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:301)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 377894 WARN (zkCallback-393-thread-1-processing-n:dummy.host.com:8984_solr) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 377895 ERROR (OverseerAutoScalingTriggerThread-98916024352047106-dummy.host.com:8984_solr-n_0000000000) [n:dummy.host.com:8984_solr ] o.a.s.c.a.ScheduledTriggers Failed to add trigger .auto_add_replicas
[junit4] 2> org.apache.solr.common.SolrException: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /autoscaling
[junit4] 2> at org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:124)
[junit4] 2> at org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:114)
[junit4] 2> at org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:110)
[junit4] 2> at org.apache.solr.cloud.ZkDistributedQueueFactory.makeQueue(ZkDistributedQueueFactory.java:36)
[junit4] 2> at org.apache.solr.cloud.autoscaling.TriggerEventQueue.<init>(TriggerEventQueue.java:49)
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$ScheduledTrigger.<init>(ScheduledTriggers.java:419)
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.add(ScheduledTriggers.java:161)
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:218)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /autoscaling
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1102)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$exists$2(SolrZkClient.java:296)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:296)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:475)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:97)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:78)
[junit4] 2> at org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:122)
[junit4] 2> ... 8 more
[junit4] 2> 377895 WARN (OverseerAutoScalingTriggerThread-98916024352047106-dummy.host.com:8984_solr-n_0000000000) [n:dummy.host.com:8984_solr ] o.a.s.c.a.OverseerTriggerThread Error removing old nodeAdded markers
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /autoscaling/nodeAdded
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1532)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getChildren$4(SolrZkClient.java:320)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:320)
[junit4] 2> at org.apache.solr.client.solrj.impl.ZkDistribStateManager.listData(ZkDistribStateManager.java:66)
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:245)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 377895 WARN (OverseerAutoScalingTriggerThread-98916024352047106-dummy.host.com:8984_solr-n_0000000000) [n:dummy.host.com:8984_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed, exiting.
[junit4] 2> 377896 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr) [n:dummy.host.com:8984_solr ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
[junit4] 2> 377897 INFO (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
[junit4] 2> 377897 INFO (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr) [n:dummy.host.com:8984_solr ] o.a.s.c.ZkController ZooKeeper session re-connected ... refreshing core states after session expiration.
[junit4] 2> 377899 INFO (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 377904 INFO (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr) [n:dummy.host.com:8984_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader dummy.host.com:8984_solr
[junit4] 2> 377905 INFO (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr) [n:dummy.host.com:8984_solr ] o.a.s.c.Overseer Overseer (id=98916024352047107-dummy.host.com:8984_solr-n_0000000001) starting
[junit4] 2> 378007 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2d684005 name: ZooKeeperConnection Watcher:127.0.0.1:58040/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
[junit4] 2> 378007 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager zkClient has disconnected
[junit4] 2> 379307 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-SendThread(127.0.0.1:58040)) [n:dummy.host.com:8984_solr ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x15f6b999e190003 has expired
[junit4] 2> 379308 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2d684005 name: ZooKeeperConnection Watcher:127.0.0.1:58040/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
[junit4] 2> 379308 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
[junit4] 2> 379511 ERROR (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr) [n:dummy.host.com:8984_solr ] o.a.s.c.Overseer Could not create Overseer node
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$create$7(SolrZkClient.java:357)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:357)
[junit4] 2> at org.apache.solr.cloud.Overseer.createOverseerNode(Overseer.java:771)
[junit4] 2> at org.apache.solr.cloud.Overseer.getStateUpdateQueue(Overseer.java:644)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.<init>(Overseer.java:107)
[junit4] 2> at org.apache.solr.cloud.Overseer.start(Overseer.java:518)
[junit4] 2> at org.apache.solr.cloud.OverseerElectionContext.runLeaderProcess(ElectionContext.java:799)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:216)
[junit4] 2> at org.apache.solr.cloud.ZkController$1.command(ZkController.java:353)
[junit4] 2> at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
[junit4] 2> at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
[junit4] 2> at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$1.lambda$process$0(SolrZkClient.java:262)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 379512 INFO (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.Overseer Overseer (id=98916024352047107-dummy.host.com:8984_solr-n_0000000001) closing
[junit4] 2> 379512 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
[junit4] 2> 379512 ERROR (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr) [n:dummy.host.com:8984_solr ] o.a.s.c.ZkController :java.lang.RuntimeException: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer
[junit4] 2> at org.apache.solr.cloud.Overseer.createOverseerNode(Overseer.java:780)
[junit4] 2> at org.apache.solr.cloud.Overseer.getStateUpdateQueue(Overseer.java:644)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.<init>(Overseer.java:107)
[junit4] 2> at org.apache.solr.cloud.Overseer.start(Overseer.java:518)
[junit4] 2> at org.apache.solr.cloud.OverseerElectionContext.runLeaderProcess(ElectionContext.java:799)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:216)
[junit4] 2> at org.apache.solr.cloud.ZkController$1.command(ZkController.java:353)
[junit4] 2> at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
[junit4] 2> at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
[junit4] 2> at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$1.lambda$process$0(SolrZkClient.java:262)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$create$7(SolrZkClient.java:357)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:357)
[junit4] 2> at org.apache.solr.cloud.Overseer.createOverseerNode(Overseer.java:771)
[junit4] 2> ... 19 more
[junit4] 2>
[junit4] 2> 379512 ERROR (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr) [n:dummy.host.com:8984_solr ] o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper failed:org.apache.solr.common.cloud.ZooKeeperException:
[junit4] 2> at org.apache.solr.cloud.ZkController$1.command(ZkController.java:413)
[junit4] 2> at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
[junit4] 2> at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
[junit4] 2> at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$1.lambda$process$0(SolrZkClient.java:262)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: java.lang.RuntimeException: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer
[junit4] 2> at org.apache.solr.cloud.Overseer.createOverseerNode(Overseer.java:780)
[junit4] 2> at org.apache.solr.cloud.Overseer.getStateUpdateQueue(Overseer.java:644)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.<init>(Overseer.java:107)
[junit4] 2> at org.apache.solr.cloud.Overseer.start(Overseer.java:518)
[junit4] 2> at org.apache.solr.cloud.OverseerElectionContext.runLeaderProcess(ElectionContext.java:799)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:216)
[junit4] 2> at org.apache.solr.cloud.ZkController$1.command(ZkController.java:353)
[junit4] 2> ... 10 more
[junit4] 2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$create$7(SolrZkClient.java:357)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:357)
[junit4] 2> at org.apache.solr.cloud.Overseer.createOverseerNode(Overseer.java:771)
[junit4] 2> ... 19 more
[junit4] 2>
[junit4] 2> 379512 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr) [n:dummy.host.com:8984_solr ] o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper failed
[junit4] 2> 379512 INFO (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Connected:false
[junit4] 2> 379514 INFO (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
[junit4] 2> 379514 INFO (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.ZkController ZooKeeper session re-connected ... refreshing core states after session expiration.
[junit4] 2> 379618 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2d684005 name: ZooKeeperConnection Watcher:127.0.0.1:58040/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
[junit4] 2> 379618 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager zkClient has disconnected
[junit4] 2> 381302 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-SendThread(127.0.0.1:58040)) [n:dummy.host.com:8984_solr ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x15f6b999e190004 has expired
[junit4] 2> 381302 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2d684005 name: ZooKeeperConnection Watcher:127.0.0.1:58040/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
[junit4] 2> 381302 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
[junit4] 2> 381303 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
[junit4] 2> 381304 INFO (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
[junit4] 2> 381304 INFO (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.ZkController ZooKeeper session re-connected ... refreshing core states after session expiration.
[junit4] 2> 381403 ERROR (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.ZkController :org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /aliases.json
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1102)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$exists$2(SolrZkClient.java:296)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:296)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:443)
[junit4] 2> at org.apache.solr.cloud.ZkController$1.command(ZkController.java:328)
[junit4] 2> at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
[junit4] 2> at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
[junit4] 2> at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
[junit4] 2>
[junit4] 2> 381403 ERROR (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper failed:org.apache.solr.common.cloud.ZooKeeperException:
[junit4] 2> at org.apache.solr.cloud.ZkController$1.command(ZkController.java:413)
[junit4] 2> at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
[junit4] 2> at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
[junit4] 2> at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
[junit4] 2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /aliases.json
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1102)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$exists$2(SolrZkClient.java:296)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:296)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:443)
[junit4] 2> at org.apache.solr.cloud.ZkController$1.command(ZkController.java:328)
[junit4] 2> ... 5 more
[junit4] 2>
[junit4] 2> 381403 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper failed
[junit4] 2> 381404 INFO (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Connected:true
[junit4] 2> 381413 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2d684005 name: ZooKeeperConnection Watcher:127.0.0.1:58040/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
[junit4] 2> 381413 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager zkClient has disconnected
[junit4] 2> 382709 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-SendThread(127.0.0.1:58040)) [n:dummy.host.com:8984_solr ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x15f6b999e190005 has expired
[junit4] 2> 382709 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2d684005 name: ZooKeeperConnection Watcher:127.0.0.1:58040/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
[junit4] 2> 382709 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
[junit4] 2> 382709 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
[junit4] 2> 382710 INFO (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
[junit4] 2> 382710 INFO (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.ZkController ZooKeeper session re-connected ... refreshing core states after session expiration.
[junit4] 2> 382719 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxnFactory Ignoring unexpected runtime exception
[junit4] 2> java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.readyOps(SelectionKeyImpl.java:87)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:182)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 382819 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2d684005 name: ZooKeeperConnection Watcher:127.0.0.1:58040/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
[junit4] 2> 382819 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager zkClient has disconnected
[junit4] 2> 384753 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-SendThread(127.0.0.1:58040)) [n:dummy.host.com:8984_solr ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x15f6b999e190006 has expired
[junit4] 2> 384753 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2d684005 name: ZooKeeperConnection Watcher:127.0.0.1:58040/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
[junit4] 2> 384753 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
[junit4] 2> 384753 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
[junit4] 2> 384755 INFO (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
[junit4] 2> 384755 INFO (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.ZkController ZooKeeper session re-connected ... refreshing core states after session expiration.
[junit4] 2> 384854 ERROR (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.ZkController :org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /clusterstate.json
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1102)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$exists$3(SolrZkClient.java:308)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:308)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:428)
[junit4] 2> at org.apache.solr.cloud.ZkController$1.command(ZkController.java:328)
[junit4] 2> at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
[junit4] 2> at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
[junit4] 2> at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
[junit4] 2>
[junit4] 2> 384854 ERROR (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper failed:org.apache.solr.common.cloud.ZooKeeperException:
[junit4] 2> at org.apache.solr.cloud.ZkController$1.command(ZkController.java:413)
[junit4] 2> at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
[junit4] 2> at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
[junit4] 2> at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
[junit4] 2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /clusterstate.json
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1102)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$exists$3(SolrZkClient.java:308)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:308)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:428)
[junit4] 2> at org.apache.solr.cloud.ZkController$1.command(ZkController.java:328)
[junit4] 2> ... 5 more
[junit4] 2>
[junit4] 2> 384854 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper failed
[junit4] 2> 384854 INFO (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Connected:true
[junit4] 2> 384857 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2d684005 name: ZooKeeperConnection Watcher:127.0.0.1:58040/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
[junit4] 2> 384857 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager zkClient has disconnected
[junit4] 2> 386735 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-SendThread(127.0.0.1:58040)) [n:dummy.host.com:8984_solr ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x15f6b999e190007 has expired
[junit4] 2> 386735 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2d684005 name: ZooKeeperConnection Watcher:127.0.0.1:58040/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
[junit4] 2> 386735 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
[junit4] 2> 386735 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
[junit4] 2> 386740 INFO (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
[junit4] 2> 386741 INFO (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.ZkController ZooKeeper session re-connected ... refreshing core states after session expiration.
[junit4] 2> 386839 ERROR (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.ZkController :org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /clusterstate.json
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1102)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$exists$3(SolrZkClient.java:308)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:308)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:428)
[junit4] 2> at org.apache.solr.cloud.ZkController$1.command(ZkController.java:328)
[junit4] 2> at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
[junit4] 2> at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
[junit4] 2> at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
[junit4] 2>
[junit4] 2> 386839 ERROR (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper failed:org.apache.solr.common.cloud.ZooKeeperException:
[junit4] 2> at org.apache.solr.cloud.ZkController$1.command(ZkController.java:413)
[junit4] 2> at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
[junit4] 2> at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
[junit4] 2> at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
[junit4] 2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /clusterstate.json
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1102)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$exists$3(SolrZkClient.java:308)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:308)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:428)
[junit4] 2> at org.apache.solr.cloud.ZkController$1.command(ZkController.java:328)
[junit4] 2> ... 5 more
[junit4] 2>
[junit4] 2> 386839 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper failed
[junit4] 2> 386840 INFO (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Connected:true
[junit4] 2> 386848 INFO (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader dummy.host.com:8984_solr
[junit4] 2> 386850 INFO (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.Overseer Overseer (id=98916024352047113-dummy.host.com:8984_solr-n_0000000002) starting
[junit4] 2> 386884 INFO (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.Overseer Overseer (id=98916024352047113-dummy.host.com:8984_solr-n_0000000002) closing
[junit4] 2> 386904 ERROR (OverseerCollectionConfigSetProcessor-98916024352047113-dummy.host.com:8984_solr-n_0000000002) [n:dummy.host.com:8984_solr ] o.a.s.c.OverseerTaskProcessor Unable to prioritize overseer
[junit4] 2> java.lang.InterruptedException
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Object.wait(Object.java:502)
[junit4] 2> at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1406)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1097)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$exists$3(SolrZkClient.java:308)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:308)
[junit4] 2> at org.apache.solr.cloud.OverseerNodePrioritizer.prioritizeOverseerNodes(OverseerNodePrioritizer.java:60)
[junit4] 2> at org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:178)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 386906 INFO (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/dummy.host.com:8984_solr
[junit4] 2> 386909 INFO (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.DefaultConnectionStrategy Reconnected to ZooKeeper
[junit4] 2> 386909 INFO (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Connected:true
[junit4] 2> 386909 INFO (zkCallback-393-thread-3-processing-n:dummy.host.com:8984_solr) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 386910 INFO (OverseerStateUpdate-98916024352047113-dummy.host.com:8984_solr-n_0000000002) [n:dummy.host.com:8984_solr ] o.a.s.c.Overseer Overseer Loop exiting : dummy.host.com:8984_solr
[junit4] 2> 386911 WARN (OverseerExitThread) [n:dummy.host.com:8984_solr ] o.a.s.c.Overseer I'm exiting, but I'm still the leader
[junit4] 2> 386913 INFO (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader dummy.host.com:8984_solr
[junit4] 2> 386914 INFO (OverseerExitThread) [n:dummy.host.com:8984_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader dummy.host.com:8984_solr
[junit4] 2> 386915 INFO (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.Overseer Overseer (id=98916024352047113-dummy.host.com:8984_solr-n_0000000003) starting
[junit4] 2> 386916 ERROR (OverseerExitThread) [n:dummy.host.com:8984_solr ] o.a.s.c.LeaderElector node exists
[junit4] 2> org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$makePath$8(SolrZkClient.java:488)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:487)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:430)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:417)
[junit4] 2> at org.apache.solr.cloud.OverseerElectionContext.runLeaderProcess(ElectionContext.java:788)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.retryElection(LeaderElector.java:393)
[junit4] 2> at org.apache.solr.cloud.ZkController.rejoinOverseerElection(ZkController.java:1967)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:326)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 386923 INFO (OverseerExitThread) [n:dummy.host.com:8984_solr ] o.a.s.c.Overseer Overseer (id=98916024352047113-dummy.host.com:8984_solr-n_0000000003) closing
[junit4] 2> 386923 INFO (OverseerCollectionConfigSetProcessor-98916024352047113-dummy.host.com:8984_solr-n_0000000003) [n:dummy.host.com:8984_solr ] o.a.s.c.OverseerTaskProcessor According to ZK I (id=98916024352047113-dummy.host.com:8984_solr-n_0000000003) am no longer a leader.
[junit4] 2> 386923 INFO (OverseerStateUpdate-98916024352047113-dummy.host.com:8984_solr-n_0000000003) [n:dummy.host.com:8984_solr ] o.a.s.c.Overseer According to ZK I (id=98916024352047113-dummy.host.com:8984_solr-n_0000000003) am no longer a leader.
[junit4] 2> 386923 INFO (OverseerStateUpdate-98916024352047113-dummy.host.com:8984_solr-n_0000000003) [n:dummy.host.com:8984_solr ] o.a.s.c.Overseer Overseer Loop exiting : dummy.host.com:8984_solr
[junit4] 2> 386923 ERROR (OverseerCollectionConfigSetProcessor-98916024352047113-dummy.host.com:8984_solr-n_0000000003) [n:dummy.host.com:8984_solr ] o.a.s.c.OverseerTaskProcessor Unable to prioritize overseer
[junit4] 2> java.lang.InterruptedException
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Object.wait(Object.java:502)
[junit4] 2> at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1406)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1097)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$exists$3(SolrZkClient.java:308)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:308)
[junit4] 2> at org.apache.solr.cloud.OverseerNodePrioritizer.prioritizeOverseerNodes(OverseerNodePrioritizer.java:60)
[junit4] 2> at org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:178)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 386923 WARN (OverseerAutoScalingTriggerThread-98916024352047113-dummy.host.com:8984_solr-n_0000000003) [n:dummy.host.com:8984_solr ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Object.wait(Object.java:502)
[junit4] 2> at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1406)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1210)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:332)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:332)
[junit4] 2> at org.apache.solr.client.solrj.impl.ZkDistribStateManager.getAutoScalingConfig(ZkDistribStateManager.java:153)
[junit4] 2> at org.apache.solr.client.solrj.cloud.autoscaling.DistribStateManager.getAutoScalingConfig(DistribStateManager.java:62)
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:117)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 386938 INFO (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/dummy.host.com:8984_solr
[junit4] 2> 386942 ERROR (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.ZkController :org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.multiInternal(ZooKeeper.java:1006)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.multi(ZooKeeper.java:910)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$multi$9(SolrZkClient.java:550)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.multi(SolrZkClient.java:550)
[junit4] 2> at org.apache.solr.cloud.ZkController.createEphemeralLiveNode(ZkController.java:976)
[junit4] 2> at org.apache.solr.cloud.ZkController.access$500(ZkController.java:138)
[junit4] 2> at org.apache.solr.cloud.ZkController$1.command(ZkController.java:361)
[junit4] 2> at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
[junit4] 2> at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
[junit4] 2> at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
[junit4] 2>
[junit4] 2> 386942 ERROR (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper failed:org.apache.solr.common.cloud.ZooKeeperException:
[junit4] 2> at org.apache.solr.cloud.ZkController$1.command(ZkController.java:413)
[junit4] 2> at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
[junit4] 2> at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
[junit4] 2> at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
[junit4] 2> Caused by: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.multiInternal(ZooKeeper.java:1006)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.multi(ZooKeeper.java:910)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$multi$9(SolrZkClient.java:550)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.multi(SolrZkClient.java:550)
[junit4] 2> at org.apache.solr.cloud.ZkController.createEphemeralLiveNode(ZkController.java:976)
[junit4] 2> at org.apache.solr.cloud.ZkController.access$500(ZkController.java:138)
[junit4] 2> at org.apache.solr.cloud.ZkController$1.command(ZkController.java:361)
[junit4] 2> ... 5 more
[junit4] 2>
[junit4] 2> 386942 WARN (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper failed
[junit4] 2> 386943 INFO (zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Connected:true
[junit4] 2> 386946 WARN (OverseerExitThread) [n:dummy.host.com:8984_solr ] o.a.s.c.Overseer Unable to rejoinElection
[junit4] 2> org.apache.solr.common.SolrException: Unable to rejoin election
[junit4] 2> at org.apache.solr.cloud.ZkController.rejoinOverseerElection(ZkController.java:1970)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:326)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election/98916024352047113-dummy.host.com:8984_solr-n_
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:361)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:258)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.retryElection(LeaderElector.java:393)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:138)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.retryElection(LeaderElector.java:393)
[junit4] 2> at org.apache.solr.cloud.ZkController.rejoinOverseerElection(ZkController.java:1967)
[junit4] 2> ... 2 more
[junit4] 2> 386946 ERROR (OverseerExitThread) [n:dummy.host.com:8984_solr ] o.a.s.c.Overseer could not read the data
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:332)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:332)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:301)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 388444 WARN (zkCallback-393-thread-3-processing-n:dummy.host.com:8984_solr) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 388444 WARN (zkCallback-393-thread-1-processing-n:dummy.host.com:8984_solr) [n:dummy.host.com:8984_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 446294 INFO (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC]) [n:dummy.host.com:8984_solr ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1880262316
[junit4] 2> 446294 INFO (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC]) [n:dummy.host.com:8984_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 446294 INFO (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC]) [n:dummy.host.com:8984_solr ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@9ca6743: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@acda2a9
[junit4] 2> 446297 INFO (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC]) [n:dummy.host.com:8984_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 446297 INFO (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC]) [n:dummy.host.com:8984_solr ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@c57ce4e: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@30fb61d4
[junit4] 2> 446303 INFO (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC]) [n:dummy.host.com:8984_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 446303 INFO (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC]) [n:dummy.host.com:8984_solr ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4e19af0e: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@39dbbf3
[junit4] 2> 446303 INFO (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC]) [n:dummy.host.com:8984_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 446304 ERROR (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC]) [n:dummy.host.com:8984_solr ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 446305 INFO (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC]) [n:dummy.host.com:8984_solr ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:58040 58040
[junit4] 2> 448829 INFO (Thread-1101) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:58040 58040
[junit4] 2> 448830 WARN (Thread-1101) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/clusterprops.json
[junit4] 2> 4 /solr/aliases.json
[junit4] 2> 4 /solr/security.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/clusterstate.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/live_nodes
[junit4] 2> 5 /solr/collections
[junit4] 2>
[junit4] 2> 448830 INFO (TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC]) [n:dummy.host.com:8984_solr ] o.a.s.SolrTestCaseJ4 ###Ending testLeaderElectionWithZkExpiry
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestLeaderElectionZkExpiry -Dtests.method=testLeaderElectionWithZkExpiry -Dtests.seed=E851EAF348116DFC -Dtests.slow=true -Dtests.locale=vi-VN -Dtests.timezone=Canada/Pacific -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] FAILURE 73.0s J0 | TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry <<<
[junit4] > Throwable #1: java.lang.AssertionError
[junit4] > at __randomizedtesting.SeedInfo.seed([E851EAF348116DFC:3E8ECBA97E0C7E64]:0)
[junit4] > at org.apache.solr.cloud.TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry(TestLeaderElectionZkExpiry.java:97)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 448832 INFO (SUITE-TestLeaderElectionZkExpiry-seed#[E851EAF348116DFC]-worker) [n:dummy.host.com:8984_solr ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> 448832 INFO (SUITE-TestLeaderElectionZkExpiry-seed#[E851EAF348116DFC]-worker) [n:dummy.host.com:8984_solr ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1880262316
[junit4] 2> 448832 INFO (SUITE-TestLeaderElectionZkExpiry-seed#[E851EAF348116DFC]-worker) [n:dummy.host.com:8984_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 448832 INFO (SUITE-TestLeaderElectionZkExpiry-seed#[E851EAF348116DFC]-worker) [n:dummy.host.com:8984_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 448832 INFO (SUITE-TestLeaderElectionZkExpiry-seed#[E851EAF348116DFC]-worker) [n:dummy.host.com:8984_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestLeaderElectionZkExpiry_E851EAF348116DFC-001
[junit4] 2> NOTE: test params are: codec=HighCompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=6055, maxDocsPerChunk=9, blockSize=877), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=6055, blockSize=877)), sim=RandomSimilarity(queryNorm=false): {}, locale=vi-VN, timezone=Canada/Pacific
[junit4] 2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=3,threads=1,free=111401648,total=292343808
[junit4] 2> NOTE: All tests run in this JVM: [TestLocalFSCloudBackupRestore, AnalyticsQueryTest, AutoAddReplicasPlanActionTest, AddReplicaTest, AutoscalingHistoryHandlerTest, TestEmbeddedSolrServerSchemaAPI, TestJsonFacets, TestSmileRequest, TestDistributedSearch, V2ApiIntegrationTest, SliceStateTest, DeleteNodeTest, MoveReplicaHDFSFailoverTest, TestTolerantUpdateProcessorRandomCloud, TestLeaderElectionZkExpiry]
[junit4] Completed [67/746 (1!)] on J0 in 73.00s, 1 test, 1 failure <<< FAILURES!
[...truncated 49504 lines...]