You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2013/06/08 15:08:35 UTC

[JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 1315 - Failure

Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/1315/

All tests passed

Build Log:
[...truncated 27421 lines...]
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6
  [javadoc] Loading source files for package org.apache.lucene...
  [javadoc] Loading source files for package org.apache.lucene.analysis...
  [javadoc] Loading source files for package org.apache.lucene.analysis.tokenattributes...
  [javadoc] Loading source files for package org.apache.lucene.codecs...
  [javadoc] Loading source files for package org.apache.lucene.codecs.compressing...
  [javadoc] Loading source files for package org.apache.lucene.codecs.lucene3x...
  [javadoc] Loading source files for package org.apache.lucene.codecs.lucene40...
  [javadoc] Loading source files for package org.apache.lucene.codecs.lucene41...
  [javadoc] Loading source files for package org.apache.lucene.codecs.lucene42...
  [javadoc] Loading source files for package org.apache.lucene.codecs.perfield...
  [javadoc] Loading source files for package org.apache.lucene.document...
  [javadoc] Loading source files for package org.apache.lucene.index...
  [javadoc] Loading source files for package org.apache.lucene.search...
  [javadoc] Loading source files for package org.apache.lucene.search.payloads...
  [javadoc] Loading source files for package org.apache.lucene.search.similarities...
  [javadoc] Loading source files for package org.apache.lucene.search.spans...
  [javadoc] Loading source files for package org.apache.lucene.store...
  [javadoc] Loading source files for package org.apache.lucene.util...
  [javadoc] Loading source files for package org.apache.lucene.util.automaton...
  [javadoc] Loading source files for package org.apache.lucene.util.fst...
  [javadoc] Loading source files for package org.apache.lucene.util.mutable...
  [javadoc] Loading source files for package org.apache.lucene.util.packed...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.7.0_21
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/core/org/apache/lucene/search/package-summary.html...
  [javadoc] Copying file /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/core/src/java/org/apache/lucene/search/doc-files/nrq-formula-2.png to directory /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/core/org/apache/lucene/search/doc-files...
  [javadoc] Copying file /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/core/src/java/org/apache/lucene/search/doc-files/nrq-formula-1.png to directory /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/core/org/apache/lucene/search/doc-files...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/core/help-doc.html...
  [javadoc] 1 warning

[...truncated 33 lines...]
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6
  [javadoc] Loading source files for package org.apache.lucene.analysis.ar...
  [javadoc] Loading source files for package org.apache.lucene.analysis.bg...
  [javadoc] Loading source files for package org.apache.lucene.analysis.br...
  [javadoc] Loading source files for package org.apache.lucene.analysis.ca...
  [javadoc] Loading source files for package org.apache.lucene.analysis.charfilter...
  [javadoc] Loading source files for package org.apache.lucene.analysis.cjk...
  [javadoc] Loading source files for package org.apache.lucene.analysis.cn...
  [javadoc] Loading source files for package org.apache.lucene.analysis.commongrams...
  [javadoc] Loading source files for package org.apache.lucene.analysis.compound...
  [javadoc] Loading source files for package org.apache.lucene.analysis.compound.hyphenation...
  [javadoc] Loading source files for package org.apache.lucene.analysis.core...
  [javadoc] Loading source files for package org.apache.lucene.analysis.cz...
  [javadoc] Loading source files for package org.apache.lucene.analysis.da...
  [javadoc] Loading source files for package org.apache.lucene.analysis.de...
  [javadoc] Loading source files for package org.apache.lucene.analysis.el...
  [javadoc] Loading source files for package org.apache.lucene.analysis.en...
  [javadoc] Loading source files for package org.apache.lucene.analysis.es...
  [javadoc] Loading source files for package org.apache.lucene.analysis.eu...
  [javadoc] Loading source files for package org.apache.lucene.analysis.fa...
  [javadoc] Loading source files for package org.apache.lucene.analysis.fi...
  [javadoc] Loading source files for package org.apache.lucene.analysis.fr...
  [javadoc] Loading source files for package org.apache.lucene.analysis.ga...
  [javadoc] Loading source files for package org.apache.lucene.analysis.gl...
  [javadoc] Loading source files for package org.apache.lucene.analysis.hi...
  [javadoc] Loading source files for package org.apache.lucene.analysis.hu...
  [javadoc] Loading source files for package org.apache.lucene.analysis.hunspell...
  [javadoc] Loading source files for package org.apache.lucene.analysis.hy...
  [javadoc] Loading source files for package org.apache.lucene.analysis.id...
  [javadoc] Loading source files for package org.apache.lucene.analysis.in...
  [javadoc] Loading source files for package org.apache.lucene.analysis.it...
  [javadoc] Loading source files for package org.apache.lucene.analysis.lv...
  [javadoc] Loading source files for package org.apache.lucene.analysis.miscellaneous...
  [javadoc] Loading source files for package org.apache.lucene.analysis.ngram...
  [javadoc] Loading source files for package org.apache.lucene.analysis.nl...
  [javadoc] Loading source files for package org.apache.lucene.analysis.no...
  [javadoc] Loading source files for package org.apache.lucene.analysis.path...
  [javadoc] Loading source files for package org.apache.lucene.analysis.pattern...
  [javadoc] Loading source files for package org.apache.lucene.analysis.payloads...
  [javadoc] Loading source files for package org.apache.lucene.analysis.position...
  [javadoc] Loading source files for package org.apache.lucene.analysis.pt...
  [javadoc] Loading source files for package org.apache.lucene.analysis.query...
  [javadoc] Loading source files for package org.apache.lucene.analysis.reverse...
  [javadoc] Loading source files for package org.apache.lucene.analysis.ro...
  [javadoc] Loading source files for package org.apache.lucene.analysis.ru...
  [javadoc] Loading source files for package org.apache.lucene.analysis.shingle...
  [javadoc] Loading source files for package org.apache.lucene.analysis.sinks...
  [javadoc] Loading source files for package org.apache.lucene.analysis.snowball...
  [javadoc] Loading source files for package org.apache.lucene.analysis.standard...
  [javadoc] Loading source files for package org.apache.lucene.analysis.standard.std31...
  [javadoc] Loading source files for package org.apache.lucene.analysis.standard.std34...
  [javadoc] Loading source files for package org.apache.lucene.analysis.standard.std36...
  [javadoc] Loading source files for package org.apache.lucene.analysis.sv...
  [javadoc] Loading source files for package org.apache.lucene.analysis.synonym...
  [javadoc] Loading source files for package org.apache.lucene.analysis.th...
  [javadoc] Loading source files for package org.apache.lucene.analysis.tr...
  [javadoc] Loading source files for package org.apache.lucene.analysis.util...
  [javadoc] Loading source files for package org.apache.lucene.analysis.wikipedia...
  [javadoc] Loading source files for package org.apache.lucene.collation...
  [javadoc] Loading source files for package org.apache.lucene.collation.tokenattributes...
  [javadoc] Loading source files for package org.tartarus.snowball...
  [javadoc] Loading source files for package org.tartarus.snowball.ext...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.7.0_21
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/analyzers-common/help-doc.html...
  [javadoc] 1 warning

[...truncated 28 lines...]
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6
  [javadoc] Loading source files for package org.apache.lucene.analysis.icu...
  [javadoc] Loading source files for package org.apache.lucene.analysis.icu.segmentation...
  [javadoc] Loading source files for package org.apache.lucene.analysis.icu.tokenattributes...
  [javadoc] Loading source files for package org.apache.lucene.collation...
  [javadoc] Loading source files for package org.apache.lucene.collation.tokenattributes...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.7.0_21
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/analyzers-icu/help-doc.html...
  [javadoc] Note: Custom tags that were not seen:  @lucene.internal
  [javadoc] 1 warning

[...truncated 27 lines...]
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6
  [javadoc] Loading source files for package org.apache.lucene.analysis.ja...
  [javadoc] Loading source files for package org.apache.lucene.analysis.ja.dict...
  [javadoc] Loading source files for package org.apache.lucene.analysis.ja.tokenattributes...
  [javadoc] Loading source files for package org.apache.lucene.analysis.ja.util...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.7.0_21
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/analyzers-kuromoji/help-doc.html...
  [javadoc] Note: Custom tags that were not seen:  @lucene.internal
  [javadoc] 1 warning

[...truncated 27 lines...]
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6
  [javadoc] Loading source files for package org.apache.lucene.analysis.morfologik...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.7.0_21
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/analyzers-morfologik/help-doc.html...
  [javadoc] Note: Custom tags that were not seen:  @lucene.internal, @lucene.experimental
  [javadoc] 1 warning

[...truncated 27 lines...]
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6
  [javadoc] Loading source files for package org.apache.lucene.analysis.phonetic...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.7.0_21
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/analyzers-phonetic/help-doc.html...
  [javadoc] Note: Custom tags that were not seen:  @lucene.internal
  [javadoc] 1 warning

[...truncated 27 lines...]
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6
  [javadoc] Loading source files for package org.apache.lucene.analysis.cn.smart...
  [javadoc] Loading source files for package org.apache.lucene.analysis.cn.smart.hhmm...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.7.0_21
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/analyzers-smartcn/help-doc.html...
  [javadoc] Note: Custom tags that were not seen:  @lucene.internal
  [javadoc] 1 warning

[...truncated 27 lines...]
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6
  [javadoc] Loading source files for package org.apache.lucene.analysis.pl...
  [javadoc] Loading source files for package org.apache.lucene.analysis.stempel...
  [javadoc] Loading source files for package org.egothor.stemmer...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.7.0_21
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/analyzers-stempel/help-doc.html...
  [javadoc] Note: Custom tags that were not seen:  @lucene.internal, @lucene.experimental
  [javadoc] 1 warning

[...truncated 27 lines...]
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6
  [javadoc] Loading source files for package org.apache.lucene.analysis.uima...
  [javadoc] Loading source files for package org.apache.lucene.analysis.uima.ae...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.7.0_21
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/analyzers-uima/help-doc.html...
  [javadoc] Note: Custom tags that were not seen:  @lucene.internal, @lucene.experimental
  [javadoc] 1 warning

[...truncated 30 lines...]
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6
  [javadoc] Loading source files for package org.apache.lucene.index.memory...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.7.0_21
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/memory/help-doc.html...
  [javadoc] Note: Custom tags that were not seen:  @lucene.internal, @lucene.experimental
  [javadoc] 1 warning

[...truncated 29 lines...]
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6
  [javadoc] Loading source files for package org.apache.lucene.search.highlight...
  [javadoc] Loading source files for package org.apache.lucene.search.postingshighlight...
  [javadoc] Loading source files for package org.apache.lucene.search.vectorhighlight...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.7.0_21
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/highlighter/help-doc.html...
  [javadoc] 1 warning

[...truncated 31 lines...]
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6
  [javadoc] Loading source files for package org.apache.lucene.queries...
  [javadoc] Loading source files for package org.apache.lucene.queries.function...
  [javadoc] Loading source files for package org.apache.lucene.queries.function.docvalues...
  [javadoc] Loading source files for package org.apache.lucene.queries.function.valuesource...
  [javadoc] Loading source files for package org.apache.lucene.queries.mlt...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.7.0_21
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/queries/help-doc.html...
  [javadoc] 1 warning

[...truncated 26 lines...]
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6
  [javadoc] Loading source files for package org.apache.lucene.sandbox.queries...
  [javadoc] Loading source files for package org.apache.lucene.sandbox.queries.regex...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.7.0_21
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/sandbox/help-doc.html...
  [javadoc] Note: Custom tags that were not seen:  @lucene.internal, @lucene.experimental
  [javadoc] 1 warning

[...truncated 26 lines...]
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6
  [javadoc] Loading source files for package org.apache.lucene.queryparser.analyzing...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.classic...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.complexPhrase...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.ext...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.flexible.core...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.flexible.core.builders...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.flexible.core.config...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.flexible.core.messages...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.flexible.core.nodes...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.flexible.core.parser...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.flexible.core.processors...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.flexible.core.util...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.flexible.messages...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.flexible.precedence...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.flexible.precedence.processors...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.flexible.standard...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.flexible.standard.builders...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.flexible.standard.config...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.flexible.standard.nodes...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.flexible.standard.parser...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.flexible.standard.processors...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.surround.parser...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.surround.query...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.xml...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.xml.builders...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.7.0_21
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/queryparser/help-doc.html...
  [javadoc] Note: Custom tags that were not seen:  @lucene.internal, @lucene.experimental
  [javadoc] 1 warning

[...truncated 26 lines...]
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6
  [javadoc] Loading source files for package org.apache.lucene.facet...
  [javadoc] Loading source files for package org.apache.lucene.facet.associations...
  [javadoc] Loading source files for package org.apache.lucene.facet.codecs.facet42...
  [javadoc] Loading source files for package org.apache.lucene.facet.collections...
  [javadoc] Loading source files for package org.apache.lucene.facet.complements...
  [javadoc] Loading source files for package org.apache.lucene.facet.encoding...
  [javadoc] Loading source files for package org.apache.lucene.facet.index...
  [javadoc] Loading source files for package org.apache.lucene.facet.params...
  [javadoc] Loading source files for package org.apache.lucene.facet.partitions...
  [javadoc] Loading source files for package org.apache.lucene.facet.range...
  [javadoc] Loading source files for package org.apache.lucene.facet.sampling...
  [javadoc] Loading source files for package org.apache.lucene.facet.search...
  [javadoc] Loading source files for package org.apache.lucene.facet.sortedset...
  [javadoc] Loading source files for package org.apache.lucene.facet.taxonomy...
  [javadoc] Loading source files for package org.apache.lucene.facet.taxonomy.directory...
  [javadoc] Loading source files for package org.apache.lucene.facet.taxonomy.writercache...
  [javadoc] Loading source files for package org.apache.lucene.facet.taxonomy.writercache.cl2o...
  [javadoc] Loading source files for package org.apache.lucene.facet.taxonomy.writercache.lru...
  [javadoc] Loading source files for package org.apache.lucene.facet.util...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.7.0_21
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/facet/org/apache/lucene/facet/package-summary.html...
  [javadoc] Copying file /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/facet/src/java/org/apache/lucene/facet/doc-files/prettify.css to directory /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/facet/org/apache/lucene/facet/doc-files...
  [javadoc] Copying file /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/facet/src/java/org/apache/lucene/facet/doc-files/userguide.html to directory /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/facet/org/apache/lucene/facet/doc-files...
  [javadoc] Copying file /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/facet/src/java/org/apache/lucene/facet/doc-files/prettify.js to directory /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/facet/org/apache/lucene/facet/doc-files...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/facet/help-doc.html...
  [javadoc] 1 warning

[...truncated 40 lines...]
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6
  [javadoc] Loading source files for package org.apache.lucene.benchmark...
  [javadoc] Loading source files for package org.apache.lucene.benchmark.byTask...
  [javadoc] Loading source files for package org.apache.lucene.benchmark.byTask.feeds...
  [javadoc] Loading source files for package org.apache.lucene.benchmark.byTask.programmatic...
  [javadoc] Loading source files for package org.apache.lucene.benchmark.byTask.stats...
  [javadoc] Loading source files for package org.apache.lucene.benchmark.byTask.tasks...
  [javadoc] Loading source files for package org.apache.lucene.benchmark.byTask.utils...
  [javadoc] Loading source files for package org.apache.lucene.benchmark.quality...
  [javadoc] Loading source files for package org.apache.lucene.benchmark.quality.trec...
  [javadoc] Loading source files for package org.apache.lucene.benchmark.quality.utils...
  [javadoc] Loading source files for package org.apache.lucene.benchmark.utils...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.7.0_21
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/benchmark/help-doc.html...
  [javadoc] Note: Custom tags that were not seen:  @lucene.internal
  [javadoc] 1 warning

[...truncated 29 lines...]
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6
  [javadoc] Loading source files for package org.apache.lucene.classification...
  [javadoc] Loading source files for package org.apache.lucene.classification.utils...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.7.0_21
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/classification/help-doc.html...
  [javadoc] Note: Custom tags that were not seen:  @lucene.internal
  [javadoc] 1 warning

[...truncated 24 lines...]
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6
  [javadoc] Loading source files for package org.apache.lucene.codecs.appending...
  [javadoc] Loading source files for package org.apache.lucene.codecs.blockterms...
  [javadoc] Loading source files for package org.apache.lucene.codecs.bloom...
  [javadoc] Loading source files for package org.apache.lucene.codecs.diskdv...
  [javadoc] Loading source files for package org.apache.lucene.codecs.intblock...
  [javadoc] Loading source files for package org.apache.lucene.codecs.memory...
  [javadoc] Loading source files for package org.apache.lucene.codecs.pulsing...
  [javadoc] Loading source files for package org.apache.lucene.codecs.sep...
  [javadoc] Loading source files for package org.apache.lucene.codecs.simpletext...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.7.0_21
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/codecs/help-doc.html...
  [javadoc] 1 warning

[...truncated 38 lines...]
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6
  [javadoc] Loading source files for package org.apache.lucene.demo...
  [javadoc] Loading source files for package org.apache.lucene.demo.facet...
  [javadoc] Loading source files for package org.apache.lucene.demo.xmlparser...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.7.0_21
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/demo/help-doc.html...
  [javadoc] Note: Custom tags that were not seen:  @lucene.internal
  [javadoc] 1 warning

[...truncated 23 lines...]
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6
  [javadoc] Loading source files for package org.apache.lucene.facet...
  [javadoc] Loading source files for package org.apache.lucene.facet.associations...
  [javadoc] Loading source files for package org.apache.lucene.facet.codecs.facet42...
  [javadoc] Loading source files for package org.apache.lucene.facet.collections...
  [javadoc] Loading source files for package org.apache.lucene.facet.complements...
  [javadoc] Loading source files for package org.apache.lucene.facet.encoding...
  [javadoc] Loading source files for package org.apache.lucene.facet.index...
  [javadoc] Loading source files for package org.apache.lucene.facet.params...
  [javadoc] Loading source files for package org.apache.lucene.facet.partitions...
  [javadoc] Loading source files for package org.apache.lucene.facet.range...
  [javadoc] Loading source files for package org.apache.lucene.facet.sampling...
  [javadoc] Loading source files for package org.apache.lucene.facet.search...
  [javadoc] Loading source files for package org.apache.lucene.facet.sortedset...
  [javadoc] Loading source files for package org.apache.lucene.facet.taxonomy...
  [javadoc] Loading source files for package org.apache.lucene.facet.taxonomy.directory...
  [javadoc] Loading source files for package org.apache.lucene.facet.taxonomy.writercache...
  [javadoc] Loading source files for package org.apache.lucene.facet.taxonomy.writercache.cl2o...
  [javadoc] Loading source files for package org.apache.lucene.facet.taxonomy.writercache.lru...
  [javadoc] Loading source files for package org.apache.lucene.facet.util...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.7.0_21
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/facet/org/apache/lucene/facet/package-summary.html...
  [javadoc] Copying file /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/facet/src/java/org/apache/lucene/facet/doc-files/prettify.css to directory /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/facet/org/apache/lucene/facet/doc-files...
  [javadoc] Copying file /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/facet/src/java/org/apache/lucene/facet/doc-files/userguide.html to directory /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/facet/org/apache/lucene/facet/doc-files...
  [javadoc] Copying file /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/facet/src/java/org/apache/lucene/facet/doc-files/prettify.js to directory /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/facet/org/apache/lucene/facet/doc-files...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/facet/help-doc.html...
  [javadoc] 1 warning

[...truncated 27 lines...]
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6
  [javadoc] Loading source files for package org.apache.lucene.search.grouping...
  [javadoc] Loading source files for package org.apache.lucene.search.grouping.function...
  [javadoc] Loading source files for package org.apache.lucene.search.grouping.term...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.7.0_21
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/grouping/help-doc.html...
  [javadoc] 1 warning

[...truncated 26 lines...]
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6
  [javadoc] Loading source files for package org.apache.lucene.search.highlight...
  [javadoc] Loading source files for package org.apache.lucene.search.postingshighlight...
  [javadoc] Loading source files for package org.apache.lucene.search.vectorhighlight...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.7.0_21
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/highlighter/help-doc.html...
  [javadoc] 1 warning

[...truncated 27 lines...]
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6
  [javadoc] Loading source files for package org.apache.lucene.search.join...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.7.0_21
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/join/help-doc.html...
  [javadoc] Note: Custom tags that were not seen:  @lucene.internal
  [javadoc] 1 warning

[...truncated 26 lines...]
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6
  [javadoc] Loading source files for package org.apache.lucene.index.memory...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.7.0_21
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/memory/help-doc.html...
  [javadoc] Note: Custom tags that were not seen:  @lucene.internal, @lucene.experimental
  [javadoc] 1 warning

[...truncated 24 lines...]
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6
  [javadoc] Loading source files for package org.apache.lucene.document...
  [javadoc] Loading source files for package org.apache.lucene.index...
  [javadoc] Loading source files for package org.apache.lucene.index.sorter...
  [javadoc] Loading source files for package org.apache.lucene.misc...
  [javadoc] Loading source files for package org.apache.lucene.store...
  [javadoc] Loading source files for package org.apache.lucene.util.fst...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.7.0_21
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/misc/org/apache/lucene/misc/package-summary.html...
  [javadoc] Copying file /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/misc/src/java/org/apache/lucene/misc/doc-files/ss.baselineTf.svg to directory /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/misc/org/apache/lucene/misc/doc-files...
  [javadoc] Copying file /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/misc/src/java/org/apache/lucene/misc/doc-files/ss.computeLengthNorm.svg to directory /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/misc/org/apache/lucene/misc/doc-files...
  [javadoc] Copying file /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/misc/src/java/org/apache/lucene/misc/doc-files/ss.hyperbolicTf.svg to directory /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/misc/org/apache/lucene/misc/doc-files...
  [javadoc] Copying file /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/misc/src/java/org/apache/lucene/misc/doc-files/ss.gnuplot to directory /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/misc/org/apache/lucene/misc/doc-files...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/misc/help-doc.html...
  [javadoc] 1 warning

[...truncated 23 lines...]
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6
  [javadoc] Loading source files for package org.apache.lucene.queries...
  [javadoc] Loading source files for package org.apache.lucene.queries.function...
  [javadoc] Loading source files for package org.apache.lucene.queries.function.docvalues...
  [javadoc] Loading source files for package org.apache.lucene.queries.function.valuesource...
  [javadoc] Loading source files for package org.apache.lucene.queries.mlt...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.7.0_21
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/queries/help-doc.html...
  [javadoc] 1 warning

[...truncated 30 lines...]
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6
  [javadoc] Loading source files for package org.apache.lucene.queryparser.analyzing...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.classic...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.complexPhrase...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.ext...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.flexible.core...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.flexible.core.builders...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.flexible.core.config...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.flexible.core.messages...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.flexible.core.nodes...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.flexible.core.parser...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.flexible.core.processors...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.flexible.core.util...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.flexible.messages...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.flexible.precedence...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.flexible.precedence.processors...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.flexible.standard...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.flexible.standard.builders...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.flexible.standard.config...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.flexible.standard.nodes...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.flexible.standard.parser...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.flexible.standard.processors...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.surround.parser...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.surround.query...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.xml...
  [javadoc] Loading source files for package org.apache.lucene.queryparser.xml.builders...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.7.0_21
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/queryparser/help-doc.html...
  [javadoc] Note: Custom tags that were not seen:  @lucene.internal, @lucene.experimental
  [javadoc] 1 warning

[...truncated 28 lines...]
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6
  [javadoc] Loading source files for package org.apache.lucene.replicator...
  [javadoc] Loading source files for package org.apache.lucene.replicator.http...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.7.0_21
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/replicator/help-doc.html...
  [javadoc] Note: Custom tags that were not seen:  @lucene.internal
  [javadoc] 1 warning

[...truncated 23 lines...]
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] Loading source files for package org.apache.lucene.sandbox.queries...
  [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6
  [javadoc] Loading source files for package org.apache.lucene.sandbox.queries.regex...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.7.0_21
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/sandbox/help-doc.html...
  [javadoc] Note: Custom tags that were not seen:  @lucene.internal, @lucene.experimental
  [javadoc] 1 warning

[...truncated 27 lines...]
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6
  [javadoc] Loading source files for package org.apache.lucene.spatial...
  [javadoc] Loading source files for package org.apache.lucene.spatial.prefix...
  [javadoc] Loading source files for package org.apache.lucene.spatial.prefix.tree...
  [javadoc] Loading source files for package org.apache.lucene.spatial.query...
  [javadoc] Loading source files for package org.apache.lucene.spatial.util...
  [javadoc] Loading source files for package org.apache.lucene.spatial.vector...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.7.0_21
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/spatial/help-doc.html...
  [javadoc] 1 warning

[...truncated 24 lines...]
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6
  [javadoc] Loading source files for package org.apache.lucene.search.spell...
  [javadoc] Loading source files for package org.apache.lucene.search.suggest...
  [javadoc] Loading source files for package org.apache.lucene.search.suggest.analyzing...
  [javadoc] Loading source files for package org.apache.lucene.search.suggest.fst...
  [javadoc] Loading source files for package org.apache.lucene.search.suggest.jaspell...
  [javadoc] Loading source files for package org.apache.lucene.search.suggest.tst...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.7.0_21
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/suggest/help-doc.html...
  [javadoc] 1 warning

[...truncated 17 lines...]
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6
  [javadoc] Loading source files for package org.apache.lucene.analysis...
  [javadoc] Loading source files for package org.apache.lucene.codecs.appending...
  [javadoc] Loading source files for package org.apache.lucene.codecs.asserting...
  [javadoc] Loading source files for package org.apache.lucene.codecs.bloom...
  [javadoc] Loading source files for package org.apache.lucene.codecs.cheapbastard...
  [javadoc] Loading source files for package org.apache.lucene.codecs.compressing...
  [javadoc] Loading source files for package org.apache.lucene.codecs.compressing.dummy...
  [javadoc] Loading source files for package org.apache.lucene.codecs.lucene3x...
  [javadoc] Loading source files for package org.apache.lucene.codecs.lucene40...
  [javadoc] Loading source files for package org.apache.lucene.codecs.lucene41...
  [javadoc] Loading source files for package org.apache.lucene.codecs.lucene41ords...
  [javadoc] Loading source files for package org.apache.lucene.codecs.mockintblock...
  [javadoc] Loading source files for package org.apache.lucene.codecs.mockrandom...
  [javadoc] Loading source files for package org.apache.lucene.codecs.mocksep...
  [javadoc] Loading source files for package org.apache.lucene.codecs.nestedpulsing...
  [javadoc] Loading source files for package org.apache.lucene.codecs.ramonly...
  [javadoc] Loading source files for package org.apache.lucene.index...
  [javadoc] Loading source files for package org.apache.lucene.search...
  [javadoc] Loading source files for package org.apache.lucene.store...
  [javadoc] Loading source files for package org.apache.lucene.util...
  [javadoc] Loading source files for package org.apache.lucene.util.automaton...
  [javadoc] Loading source files for package org.apache.lucene.util.fst...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.7.0_21
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/test-framework/help-doc.html...
  [javadoc] 1 warning

[...truncated 7 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:391: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:60: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build.xml:250: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build.xml:558: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:2109: java.net.ConnectException: Operation timed out
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
	at java.net.Socket.connect(Socket.java:579)
	at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618)
	at sun.security.ssl.BaseSSLSocketImpl.connect(BaseSSLSocketImpl.java:160)
	at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
	at sun.net.www.http.HttpClient.openServer(HttpClient.java:378)
	at sun.net.www.http.HttpClient.openServer(HttpClient.java:473)
	at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:270)
	at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:327)
	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)
	at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:931)
	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)
	at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:153)
	at org.apache.tools.ant.taskdefs.Get$GetThread.openConnection(Get.java:660)
	at org.apache.tools.ant.taskdefs.Get$GetThread.get(Get.java:579)
	at org.apache.tools.ant.taskdefs.Get$GetThread.run(Get.java:569)

Total time: 72 minutes 23 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 1317 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/1317/

2 tests failed.
REGRESSION:  org.apache.solr.cloud.ShardSplitTest.testDistribSearch

Error Message:
Server at http://127.0.0.1:21057/_h/dd returned non ok status:500, message:Server Error

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:21057/_h/dd returned non ok status:500, message:Server Error
	at __randomizedtesting.SeedInfo.seed([2EC0E81AD9280F18:AF266602AE776F24]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
	at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
	at org.apache.solr.cloud.ShardSplitTest.doTest(ShardSplitTest.java:150)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)


FAILED:  org.apache.solr.cloud.ChaosMonkeyShardSplitTest.testDistribSearch

Error Message:
Server at http://127.0.0.1:26184/v_ss/yt returned non ok status:500, message:Server Error

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:26184/v_ss/yt returned non ok status:500, message:Server Error
	at __randomizedtesting.SeedInfo.seed([2EC0E81AD9280F18:AF266602AE776F24]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
	at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
	at org.apache.solr.cloud.ChaosMonkeyShardSplitTest.doTest(ChaosMonkeyShardSplitTest.java:136)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)




Build Log:
[...truncated 9613 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.ChaosMonkeyShardSplitTest
[junit4:junit4]   2> 808190 T1125 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /v_ss/yt
[junit4:junit4]   2> 808196 T1125 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./solrtest-ChaosMonkeyShardSplitTest-1370756784469
[junit4:junit4]   2> 808197 T1125 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 808198 T1126 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 808299 T1125 oasc.ZkTestServer.run start zk server on port:26177
[junit4:junit4]   2> 808300 T1125 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 808304 T1132 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6da3d475 name:ZooKeeperConnection Watcher:127.0.0.1:26177 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 808304 T1125 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 808305 T1125 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 808319 T1125 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 808322 T1134 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@79dff24 name:ZooKeeperConnection Watcher:127.0.0.1:26177/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 808322 T1125 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 808322 T1125 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 808332 T1125 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 808336 T1125 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 808339 T1125 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 808343 T1125 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 808344 T1125 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 808356 T1125 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 808357 T1125 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 808461 T1125 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 808462 T1125 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 808473 T1125 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 808474 T1125 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 808477 T1125 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 808478 T1125 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 808482 T1125 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 808483 T1125 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 808486 T1125 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 808487 T1125 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 808490 T1125 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 808491 T1125 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 808495 T1125 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 808495 T1125 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 808761 T1125 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 808765 T1125 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:26180
[junit4:junit4]   2> 808766 T1125 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 808767 T1125 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 808767 T1125 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1370756784772
[junit4:junit4]   2> 808768 T1125 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1370756784772/solr.xml
[junit4:junit4]   2> 808768 T1125 oasc.CoreContainer.<init> New CoreContainer 1245496938
[junit4:junit4]   2> 808769 T1125 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1370756784772/'
[junit4:junit4]   2> 808770 T1125 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1370756784772/'
[junit4:junit4]   2> 808897 T1125 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 808898 T1125 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 808899 T1125 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 808900 T1125 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 808900 T1125 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 808901 T1125 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 808901 T1125 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 808902 T1125 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 808902 T1125 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 808903 T1125 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 808922 T1125 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 808923 T1125 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:26177/solr
[junit4:junit4]   2> 808923 T1125 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 808925 T1125 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 808927 T1145 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@17ee2a8f name:ZooKeeperConnection Watcher:127.0.0.1:26177 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 808928 T1125 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 808931 T1125 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 808945 T1125 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 808947 T1147 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@fd1b2a2 name:ZooKeeperConnection Watcher:127.0.0.1:26177/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 808947 T1125 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 808949 T1125 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4]   2> 808960 T1125 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4]   2> 808963 T1125 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 808970 T1125 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:26180_v_ss%2Fyt
[junit4:junit4]   2> 808972 T1125 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:26180_v_ss%2Fyt
[junit4:junit4]   2> 808976 T1125 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 808993 T1125 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 809000 T1125 oasc.Overseer.start Overseer (id=89833916627419139-127.0.0.1:26180_v_ss%2Fyt-n_0000000000) starting
[junit4:junit4]   2> 809010 T1125 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4]   2> 809021 T1149 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 809022 T1125 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 809025 T1125 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4]   2> 809027 T1125 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 809030 T1148 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 809035 T1150 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1370756784772/collection1
[junit4:junit4]   2> 809035 T1150 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 809036 T1150 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 809036 T1150 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 809038 T1150 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1370756784772/collection1/'
[junit4:junit4]   2> 809039 T1150 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1370756784772/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 809040 T1150 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1370756784772/collection1/lib/README' to classloader
[junit4:junit4]   2> 809092 T1150 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 809151 T1150 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 809253 T1150 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 809259 T1150 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 809830 T1150 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 809834 T1150 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 809836 T1150 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 809841 T1150 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 809868 T1150 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 809868 T1150 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1370756784772/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/control/data/
[junit4:junit4]   2> 809868 T1150 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@64b49aaa
[junit4:junit4]   2> 809869 T1150 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 809870 T1150 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/control/data
[junit4:junit4]   2> 809870 T1150 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/control/data/index/
[junit4:junit4]   2> 809870 T1150 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 809871 T1150 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/control/data/index
[junit4:junit4]   2> 809875 T1150 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@58606861 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6ff7c5ae),segFN=segments_1,generation=1}
[junit4:junit4]   2> 809876 T1150 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 809879 T1150 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 809879 T1150 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 809880 T1150 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 809881 T1150 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 809881 T1150 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 809882 T1150 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 809882 T1150 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 809882 T1150 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 809883 T1150 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 809899 T1150 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 809907 T1150 oass.SolrIndexSearcher.<init> Opening Searcher@711c2cee main
[junit4:junit4]   2> 809907 T1150 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/control/data/tlog
[junit4:junit4]   2> 809908 T1150 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 809909 T1150 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 809912 T1151 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@711c2cee main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 809914 T1150 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 809915 T1150 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 810535 T1148 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 810535 T1148 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:26180_v_ss%2Fyt",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:26180/v_ss/yt"}
[junit4:junit4]   2> 810536 T1148 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 810536 T1148 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4]   2> 810550 T1147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 810917 T1150 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 810917 T1150 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:26180/v_ss/yt collection:control_collection shard:shard1
[junit4:junit4]   2> 810918 T1150 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 810941 T1150 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 810944 T1150 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 810944 T1150 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 810944 T1150 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:26180/v_ss/yt/collection1/
[junit4:junit4]   2> 810944 T1150 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 810945 T1150 oasc.SyncStrategy.syncToMe http://127.0.0.1:26180/v_ss/yt/collection1/ has no replicas
[junit4:junit4]   2> 810945 T1150 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:26180/v_ss/yt/collection1/
[junit4:junit4]   2> 810945 T1150 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 812056 T1148 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 812081 T1147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 812125 T1150 oasc.ZkController.register We are http://127.0.0.1:26180/v_ss/yt/collection1/ and leader is http://127.0.0.1:26180/v_ss/yt/collection1/
[junit4:junit4]   2> 812125 T1150 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:26180/v_ss/yt
[junit4:junit4]   2> 812125 T1150 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 812125 T1150 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 812126 T1150 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 812128 T1150 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 812129 T1125 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 812130 T1125 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 812130 T1125 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 812147 T1125 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 812150 T1125 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 812152 T1154 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@25a05797 name:ZooKeeperConnection Watcher:127.0.0.1:26177/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 812153 T1125 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 812155 T1125 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 812158 T1125 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 812423 T1125 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 812427 T1125 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:26184
[junit4:junit4]   2> 812428 T1125 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 812428 T1125 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 812429 T1125 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1370756788430
[junit4:junit4]   2> 812429 T1125 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1370756788430/solr.xml
[junit4:junit4]   2> 812430 T1125 oasc.CoreContainer.<init> New CoreContainer 666639284
[junit4:junit4]   2> 812430 T1125 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1370756788430/'
[junit4:junit4]   2> 812431 T1125 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1370756788430/'
[junit4:junit4]   2> 812560 T1125 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 812561 T1125 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 812561 T1125 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 812562 T1125 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 812562 T1125 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 812563 T1125 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 812563 T1125 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 812564 T1125 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 812564 T1125 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 812565 T1125 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 812582 T1125 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 812583 T1125 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:26177/solr
[junit4:junit4]   2> 812583 T1125 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 812584 T1125 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 812588 T1165 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@538fc6b2 name:ZooKeeperConnection Watcher:127.0.0.1:26177 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 812588 T1125 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 812602 T1125 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 812615 T1125 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 812617 T1167 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@61e33e34 name:ZooKeeperConnection Watcher:127.0.0.1:26177/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 812617 T1125 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 812629 T1125 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 813588 T1148 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 813589 T1148 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:26180_v_ss%2Fyt_collection1",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:26180_v_ss%2Fyt",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:26180/v_ss/yt"}
[junit4:junit4]   2> 813603 T1147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 813603 T1167 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 813603 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 813633 T1125 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:26184_v_ss%2Fyt
[junit4:junit4]   2> 813635 T1125 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:26184_v_ss%2Fyt
[junit4:junit4]   2> 813638 T1147 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 813638 T1154 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 813639 T1147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 813639 T1167 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 813639 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 813639 T1167 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 813646 T1168 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1370756788430/collection1
[junit4:junit4]   2> 813647 T1168 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 813648 T1168 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 813648 T1168 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 813650 T1168 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1370756788430/collection1/'
[junit4:junit4]   2> 813651 T1168 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1370756788430/collection1/lib/README' to classloader
[junit4:junit4]   2> 813652 T1168 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1370756788430/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 813703 T1168 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 813763 T1168 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 813865 T1168 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 813870 T1168 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 814439 T1168 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 814443 T1168 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 814445 T1168 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 814451 T1168 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 814477 T1168 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 814477 T1168 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1370756788430/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/jetty1/
[junit4:junit4]   2> 814478 T1168 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@64b49aaa
[junit4:junit4]   2> 814478 T1168 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 814479 T1168 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/jetty1
[junit4:junit4]   2> 814479 T1168 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/jetty1/index/
[junit4:junit4]   2> 814480 T1168 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 814480 T1168 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/jetty1/index
[junit4:junit4]   2> 814484 T1168 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@f85ad37 lockFactory=org.apache.lucene.store.NativeFSLockFactory@76bc4f9b),segFN=segments_1,generation=1}
[junit4:junit4]   2> 814485 T1168 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 814488 T1168 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 814489 T1168 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 814489 T1168 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 814490 T1168 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 814491 T1168 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 814491 T1168 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 814491 T1168 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 814492 T1168 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 814493 T1168 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 814509 T1168 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 814516 T1168 oass.SolrIndexSearcher.<init> Opening Searcher@3cd2110 main
[junit4:junit4]   2> 814517 T1168 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/jetty1/tlog
[junit4:junit4]   2> 814518 T1168 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 814518 T1168 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 814522 T1169 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3cd2110 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 814525 T1168 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 814525 T1168 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 815110 T1148 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 815111 T1148 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:26184_v_ss%2Fyt",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:26184/v_ss/yt"}
[junit4:junit4]   2> 815111 T1148 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4]   2> 815111 T1148 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4]   2> 815124 T1147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 815124 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 815124 T1167 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 815639 T1168 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 815639 T1168 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:26184/v_ss/yt collection:collection1 shard:shard1
[junit4:junit4]   2> 815640 T1168 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 815750 T1168 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 815754 T1168 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 815754 T1168 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 815755 T1168 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:26184/v_ss/yt/collection1/
[junit4:junit4]   2> 815755 T1168 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 815755 T1168 oasc.SyncStrategy.syncToMe http://127.0.0.1:26184/v_ss/yt/collection1/ has no replicas
[junit4:junit4]   2> 815755 T1168 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:26184/v_ss/yt/collection1/
[junit4:junit4]   2> 815756 T1168 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 816630 T1148 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 816673 T1147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 816674 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 816674 T1167 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 816718 T1168 oasc.ZkController.register We are http://127.0.0.1:26184/v_ss/yt/collection1/ and leader is http://127.0.0.1:26184/v_ss/yt/collection1/
[junit4:junit4]   2> 816718 T1168 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:26184/v_ss/yt
[junit4:junit4]   2> 816718 T1168 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 816718 T1168 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 816719 T1168 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 816721 T1168 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 816722 T1125 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 816723 T1125 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 816724 T1125 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 817004 T1125 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 817007 T1125 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:26187
[junit4:junit4]   2> 817008 T1125 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 817009 T1125 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 817009 T1125 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1370756793013
[junit4:junit4]   2> 817010 T1125 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1370756793013/solr.xml
[junit4:junit4]   2> 817010 T1125 oasc.CoreContainer.<init> New CoreContainer 1235077470
[junit4:junit4]   2> 817011 T1125 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1370756793013/'
[junit4:junit4]   2> 817012 T1125 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1370756793013/'
[junit4:junit4]   2> 817139 T1125 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 817140 T1125 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 817141 T1125 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 817141 T1125 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 817142 T1125 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 817142 T1125 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 817143 T1125 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 817144 T1125 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 817144 T1125 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 817145 T1125 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 817162 T1125 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 817163 T1125 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:26177/solr
[junit4:junit4]   2> 817163 T1125 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 817164 T1125 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 817168 T1181 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@73b9c795 name:ZooKeeperConnection Watcher:127.0.0.1:26177 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 817168 T1125 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 817183 T1125 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 817197 T1125 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 817200 T1183 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@229ae253 name:ZooKeeperConnection Watcher:127.0.0.1:26177/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 817200 T1125 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 817212 T1125 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 818190 T1148 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 818191 T1148 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:26184_v_ss%2Fyt_collection1",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:26184_v_ss%2Fyt",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:26184/v_ss/yt"}
[junit4:junit4]   2> 818205 T1183 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 818205 T1147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 818205 T1167 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 818205 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 818216 T1125 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:26187_v_ss%2Fyt
[junit4:junit4]   2> 818218 T1125 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:26187_v_ss%2Fyt
[junit4:junit4]   2> 818221 T1154 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 818222 T1147 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 818221 T1167 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 818221 T1183 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 818222 T1167 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 818222 T1147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 818222 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 818222 T1183 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 818229 T1184 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1370756793013/collection1
[junit4:junit4]   2> 818229 T1184 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 818230 T1184 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 818231 T1184 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 818232 T1184 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1370756793013/collection1/'
[junit4:junit4]   2> 818234 T1184 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1370756793013/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 818235 T1184 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1370756793013/collection1/lib/README' to classloader
[junit4:junit4]   2> 818287 T1184 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 818348 T1184 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 818450 T1184 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 818455 T1184 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 819033 T1184 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 819037 T1184 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 819039 T1184 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 819045 T1184 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 819071 T1184 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 819072 T1184 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1370756793013/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/jetty2/
[junit4:junit4]   2> 819072 T1184 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@64b49aaa
[junit4:junit4]   2> 819073 T1184 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 819074 T1184 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/jetty2
[junit4:junit4]   2> 819074 T1184 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/jetty2/index/
[junit4:junit4]   2> 819075 T1184 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 819075 T1184 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/jetty2/index
[junit4:junit4]   2> 819079 T1184 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@599cf202 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6bc6c898),segFN=segments_1,generation=1}
[junit4:junit4]   2> 819079 T1184 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 819083 T1184 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 819083 T1184 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 819084 T1184 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 819085 T1184 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 819085 T1184 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 819086 T1184 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 819086 T1184 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 819087 T1184 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 819087 T1184 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 819104 T1184 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 819111 T1184 oass.SolrIndexSearcher.<init> Opening Searcher@7b9b673a main
[junit4:junit4]   2> 819112 T1184 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/jetty2/tlog
[junit4:junit4]   2> 819113 T1184 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 819113 T1184 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 819118 T1185 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7b9b673a main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 819121 T1184 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 819121 T1184 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 819711 T1148 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 819712 T1148 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:26187_v_ss%2Fyt",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:26187/v_ss/yt"}
[junit4:junit4]   2> 819712 T1148 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 819712 T1148 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4]   2> 819726 T1147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 819726 T1167 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 819726 T1183 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 819726 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 820124 T1184 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 820124 T1184 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:26187/v_ss/yt collection:collection1 shard:shard2
[junit4:junit4]   2> 820125 T1184 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 820142 T1184 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 820153 T1184 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 820154 T1184 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 820154 T1184 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:26187/v_ss/yt/collection1/
[junit4:junit4]   2> 820154 T1184 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 820154 T1184 oasc.SyncStrategy.syncToMe http://127.0.0.1:26187/v_ss/yt/collection1/ has no replicas
[junit4:junit4]   2> 820155 T1184 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:26187/v_ss/yt/collection1/
[junit4:junit4]   2> 820155 T1184 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   2> 821232 T1148 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 821253 T1147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 821253 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 821253 T1167 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 821253 T1183 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 821291 T1184 oasc.ZkController.register We are http://127.0.0.1:26187/v_ss/yt/collection1/ and leader is http://127.0.0.1:26187/v_ss/yt/collection1/
[junit4:junit4]   2> 821291 T1184 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:26187/v_ss/yt
[junit4:junit4]   2> 821291 T1184 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 821291 T1184 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 821292 T1184 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 821294 T1184 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 821295 T1125 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 821296 T1125 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 821297 T1125 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 821577 T1125 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 821581 T1125 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:26195
[junit4:junit4]   2> 821582 T1125 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 821583 T1125 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 821583 T1125 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1370756797586
[junit4:junit4]   2> 821584 T1125 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1370756797586/solr.xml
[junit4:junit4]   2> 821584 T1125 oasc.CoreContainer.<init> New CoreContainer 500217951
[junit4:junit4]   2> 821585 T1125 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1370756797586/'
[junit4:junit4]   2> 821585 T1125 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1370756797586/'
[junit4:junit4]   2> 821713 T1125 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 821714 T1125 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 821715 T1125 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 821715 T1125 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 821716 T1125 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 821716 T1125 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 821717 T1125 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 821717 T1125 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 821718 T1125 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 821718 T1125 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 821736 T1125 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 821736 T1125 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:26177/solr
[junit4:junit4]   2> 821737 T1125 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 821738 T1125 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 821741 T1197 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@76d4f1c7 name:ZooKeeperConnection Watcher:127.0.0.1:26177 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 821742 T1125 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 821756 T1125 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 821768 T1125 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 821770 T1199 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@32e2660a name:ZooKeeperConnection Watcher:127.0.0.1:26177/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 821771 T1125 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 821776 T1125 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 822767 T1148 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 822768 T1148 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:26187_v_ss%2Fyt_collection1",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:26187_v_ss%2Fyt",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:26187/v_ss/yt"}
[junit4:junit4]   2> 822783 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 822783 T1125 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:26195_v_ss%2Fyt
[junit4:junit4]   2> 822783 T1147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 822783 T1199 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 822783 T1167 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 822783 T1183 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 822785 T1125 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:26195_v_ss%2Fyt
[junit4:junit4]   2> 822794 T1183 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 822794 T1199 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 822795 T1147 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 822794 T1154 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 822795 T1147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 822795 T1199 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 822794 T1167 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 822794 T1183 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 822796 T1167 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 822795 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 822803 T1200 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1370756797586/collection1
[junit4:junit4]   2> 822803 T1200 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 822804 T1200 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 822804 T1200 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 822806 T1200 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1370756797586/collection1/'
[junit4:junit4]   2> 822807 T1200 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1370756797586/collection1/lib/README' to classloader
[junit4:junit4]   2> 822808 T1200 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1370756797586/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 822860 T1200 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 822920 T1200 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 823021 T1200 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 823026 T1200 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 823590 T1200 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 823593 T1200 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 823596 T1200 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 823601 T1200 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 823627 T1200 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 823628 T1200 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1370756797586/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/jetty3/
[junit4:junit4]   2> 823628 T1200 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@64b49aaa
[junit4:junit4]   2> 823628 T1200 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 823629 T1200 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/jetty3
[junit4:junit4]   2> 823629 T1200 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/jetty3/index/
[junit4:junit4]   2> 823630 T1200 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 823630 T1200 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/jetty3/index
[junit4:junit4]   2> 823634 T1200 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50f85f07 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4d90b8a3),segFN=segments_1,generation=1}
[junit4:junit4]   2> 823634 T1200 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 823637 T1200 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 823638 T1200 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 823638 T1200 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 823639 T1200 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 823640 T1200 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 823640 T1200 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 823640 T1200 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 823641 T1200 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 823641 T1200 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 823657 T1200 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 823663 T1200 oass.SolrIndexSearcher.<init> Opening Searcher@57e3b528 main
[junit4:junit4]   2> 823664 T1200 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/jetty3/tlog
[junit4:junit4]   2> 823665 T1200 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 823665 T1200 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 823669 T1201 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@57e3b528 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 823672 T1200 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 823672 T1200 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 824295 T1148 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 824295 T1148 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:26195_v_ss%2Fyt",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:26195/v_ss/yt"}
[junit4:junit4]   2> 824296 T1148 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 824296 T1148 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4]   2> 824310 T1183 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 824310 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 824310 T1147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 824310 T1199 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 824310 T1167 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 824675 T1200 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 824675 T1200 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:26195/v_ss/yt collection:collection1 shard:shard1
[junit4:junit4]   2> 824679 T1200 oasc.ZkController.register We are http://127.0.0.1:26195/v_ss/yt/collection1/ and leader is http://127.0.0.1:26184/v_ss/yt/collection1/
[junit4:junit4]   2> 824679 T1200 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:26195/v_ss/yt
[junit4:junit4]   2> 824679 T1200 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 824680 T1200 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C362 name=collection1 org.apache.solr.core.SolrCore@398f1866 url=http://127.0.0.1:26195/v_ss/yt/collection1 node=127.0.0.1:26195_v_ss%2Fyt C362_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:26195_v_ss%2Fyt, base_url=http://127.0.0.1:26195/v_ss/yt}
[junit4:junit4]   2> 824680 T1202 C362 P26195 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 824680 T1200 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 824681 T1202 C362 P26195 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 824682 T1202 C362 P26195 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 824682 T1202 C362 P26195 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 824683 T1125 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 824683 T1125 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 824683 T1202 C362 P26195 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 824684 T1125 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 824705 T1157 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:26195_v_ss%2Fyt_collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4]   2> 824975 T1125 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 824978 T1125 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:26199
[junit4:junit4]   2> 824979 T1125 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 824980 T1125 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 824980 T1125 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1370756800975
[junit4:junit4]   2> 824981 T1125 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1370756800975/solr.xml
[junit4:junit4]   2> 824981 T1125 oasc.CoreContainer.<init> New CoreContainer 1105021883
[junit4:junit4]   2> 824982 T1125 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1370756800975/'
[junit4:junit4]   2> 824982 T1125 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1370756800975/'
[junit4:junit4]   2> 825111 T1125 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 825112 T1125 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 825112 T1125 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 825113 T1125 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 825113 T1125 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 825114 T1125 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 825115 T1125 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 825115 T1125 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 825116 T1125 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 825116 T1125 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 825133 T1125 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 825134 T1125 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:26177/solr
[junit4:junit4]   2> 825135 T1125 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 825136 T1125 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 825139 T1214 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@35416116 name:ZooKeeperConnection Watcher:127.0.0.1:26177 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 825139 T1125 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 825152 T1125 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 825165 T1125 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 825167 T1216 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@40e4dd1a name:ZooKeeperConnection Watcher:127.0.0.1:26177/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 825167 T1125 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 825173 T1125 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 825816 T1148 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 825817 T1148 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:26195_v_ss%2Fyt_collection1",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:26195_v_ss%2Fyt",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:26195/v_ss/yt"}
[junit4:junit4]   2> 825954 T1183 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 825954 T1147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 825954 T1216 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 825954 T1199 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 825954 T1167 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 825954 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 826177 T1125 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:26199_v_ss%2Fyt
[junit4:junit4]   2> 826179 T1125 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:26199_v_ss%2Fyt
[junit4:junit4]   2> 826220 T1183 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 826221 T1154 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 826221 T1216 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 826221 T1183 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 826221 T1147 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 826221 T1167 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 826220 T1199 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 826222 T1167 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 826222 T1147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 826221 T1216 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 826221 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 826222 T1199 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 826229 T1217 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1370756800975/collection1
[junit4:junit4]   2> 826229 T1217 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 826230 T1217 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 826230 T1217 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 826232 T1217 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1370756800975/collection1/'
[junit4:junit4]   2> 826234 T1217 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1370756800975/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 826234 T1217 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1370756800975/collection1/lib/README' to classloader
[junit4:junit4]   2> 826286 T1217 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 826346 T1217 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 826448 T1217 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 826453 T1217 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 826707 T1157 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:26195_v_ss%2Fyt_collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4]   2> 826707 T1157 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:26195_v_ss%252Fyt_collection1&state=recovering&nodeName=127.0.0.1:26195_v_ss%252Fyt&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2002 
[junit4:junit4]   2> 827020 T1217 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 827024 T1217 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 827026 T1217 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 827032 T1217 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 827058 T1217 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 827058 T1217 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1370756800975/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/jetty4/
[junit4:junit4]   2> 827058 T1217 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@64b49aaa
[junit4:junit4]   2> 827059 T1217 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 827060 T1217 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/jetty4
[junit4:junit4]   2> 827060 T1217 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/jetty4/index/
[junit4:junit4]   2> 827060 T1217 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 827061 T1217 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/jetty4/index
[junit4:junit4]   2> 827065 T1217 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4c5e9ece lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ddd9220),segFN=segments_1,generation=1}
[junit4:junit4]   2> 827065 T1217 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 827068 T1217 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 827068 T1217 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 827069 T1217 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 827070 T1217 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 827070 T1217 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 827071 T1217 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 827071 T1217 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 827072 T1217 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 827072 T1217 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 827087 T1217 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 827094 T1217 oass.SolrIndexSearcher.<init> Opening Searcher@15b3a83d main
[junit4:junit4]   2> 827094 T1217 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/jetty4/tlog
[junit4:junit4]   2> 827096 T1217 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 827096 T1217 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 827101 T1218 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@15b3a83d main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 827103 T1217 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 827103 T1217 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 827460 T1148 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 827461 T1148 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:26199_v_ss%2Fyt",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:26199/v_ss/yt"}
[junit4:junit4]   2> 827461 T1148 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 827461 T1148 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4]   2> 827475 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 827475 T1147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 827475 T1216 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 827475 T1167 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 827475 T1199 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 827475 T1183 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 828131 T1217 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 828131 T1217 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:26199/v_ss/yt collection:collection1 shard:shard2
[junit4:junit4]   2> 828135 T1217 oasc.ZkController.register We are http://127.0.0.1:26199/v_ss/yt/collection1/ and leader is http://127.0.0.1:26187/v_ss/yt/collection1/
[junit4:junit4]   2> 828135 T1217 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:26199/v_ss/yt
[junit4:junit4]   2> 828136 T1217 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 828136 T1217 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C363 name=collection1 org.apache.solr.core.SolrCore@5922b8de url=http://127.0.0.1:26199/v_ss/yt/collection1 node=127.0.0.1:26199_v_ss%2Fyt C363_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:26199_v_ss%2Fyt, base_url=http://127.0.0.1:26199/v_ss/yt}
[junit4:junit4]   2> 828136 T1219 C363 P26199 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 828137 T1217 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 828137 T1219 C363 P26199 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 828138 T1219 C363 P26199 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 828138 T1219 C363 P26199 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 828139 T1125 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 828139 T1125 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 828140 T1125 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 828140 T1219 C363 P26199 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 828160 T1125 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 828161 T1173 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:26199_v_ss%2Fyt_collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4]   2> 828162 T1125 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4]   2> 828163 T1125 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4]   2> 828164 T1125 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> ASYNC  NEW_CORE C364 name=collection1 org.apache.solr.core.SolrCore@398f1866 url=http://127.0.0.1:26195/v_ss/yt/collection1 node=127.0.0.1:26195_v_ss%2Fyt C364_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:26195_v_ss%2Fyt, base_url=http://127.0.0.1:26195/v_ss/yt}
[junit4:junit4]   2> 828710 T1202 C364 P26195 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:26184/v_ss/yt/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 828710 T1202 C364 P26195 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:26195/v_ss/yt START replicas=[http://127.0.0.1:26184/v_ss/yt/collection1/] nUpdates=100
[junit4:junit4]   2> 828711 T1202 C364 P26195 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 828712 T1202 C364 P26195 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 828712 T1202 C364 P26195 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 828712 T1202 C364 P26195 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 828712 T1202 C364 P26195 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 828713 T1202 C364 P26195 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:26184/v_ss/yt/collection1/. core=collection1
[junit4:junit4]   2> 828713 T1202 C364 P26195 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C365 name=collection1 org.apache.solr.core.SolrCore@70659414 url=http://127.0.0.1:26184/v_ss/yt/collection1 node=127.0.0.1:26184_v_ss%2Fyt C365_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:26184_v_ss%2Fyt, base_url=http://127.0.0.1:26184/v_ss/yt, leader=true}
[junit4:junit4]   2> 828720 T1158 C365 P26184 oasc.SolrCore.execute [collection1] webapp=/v_ss/yt path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 828726 T1159 C365 P26184 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 828729 T1159 C365 P26184 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@f85ad37 lockFactory=org.apache.lucene.store.NativeFSLockFactory@76bc4f9b),segFN=segments_1,generation=1}
[junit4:junit4]   2> 828730 T1159 C365 P26184 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 828730 T1159 C365 P26184 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@f85ad37 lockFactory=org.apache.lucene.store.NativeFSLockFactory@76bc4f9b),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@f85ad37 lockFactory=org.apache.lucene.store.NativeFSLockFactory@76bc4f9b),segFN=segments_2,generation=2}
[junit4:junit4]   2> 828731 T1159 C365 P26184 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 828731 T1159 C365 P26184 oass.SolrIndexSearcher.<init> Opening Searcher@144cbe9e realtime
[junit4:junit4]   2> 828732 T1159 C365 P26184 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 828732 T1159 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4]   2> 828733 T1202 C364 P26195 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 828734 T1202 C364 P26195 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 828736 T1160 C365 P26184 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 828736 T1160 C365 P26184 oasc.SolrCore.execute [collection1] webapp=/v_ss/yt path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 828737 T1202 C364 P26195 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 828737 T1202 C364 P26195 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 828738 T1202 C364 P26195 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 828740 T1160 C365 P26184 oasc.SolrCore.execute [collection1] webapp=/v_ss/yt path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0 
[junit4:junit4]   2> 828741 T1202 C364 P26195 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 828742 T1202 C364 P26195 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/jetty3/index.20130609024645014
[junit4:junit4]   2> 828743 T1202 C364 P26195 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@2086f3e6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@704129bf) fullCopy=false
[junit4:junit4]   2> 828745 T1160 C365 P26184 oasc.SolrCore.execute [collection1] webapp=/v_ss/yt path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 828747 T1202 C364 P26195 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 828747 T1202 C364 P26195 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 828748 T1202 C364 P26195 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 828750 T1202 C364 P26195 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50f85f07 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4d90b8a3),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50f85f07 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4d90b8a3),segFN=segments_2,generation=2}
[junit4:junit4]   2> 828750 T1202 C364 P26195 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 828751 T1202 C364 P26195 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 828751 T1202 C364 P26195 oass.SolrIndexSearcher.<init> Opening Searcher@38bb27f main
[junit4:junit4]   2> 828752 T1201 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@38bb27f main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 828753 T1202 C364 P26195 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/jetty3/index.20130609024645014 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/jetty3/index.20130609024645014;done=true>>]
[junit4:junit4]   2> 828753 T1202 C364 P26195 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/jetty3/index.20130609024645014
[junit4:junit4]   2> 828753 T1202 C364 P26195 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/jetty3/index.20130609024645014
[junit4:junit4]   2> 828754 T1202 C364 P26195 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 828754 T1202 C364 P26195 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 828754 T1202 C364 P26195 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 828754 T1202 C364 P26195 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 828779 T1202 C364 P26195 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 828981 T1148 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 828982 T1148 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:26199_v_ss%2Fyt_collection1",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:26199_v_ss%2Fyt",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:26199/v_ss/yt"}
[junit4:junit4]   2> 828987 T1148 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:26195_v_ss%2Fyt_collection1",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:26195_v_ss%2Fyt",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:26195/v_ss/yt"}
[junit4:junit4]   2> 828993 T1183 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 828993 T1167 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 828993 T1216 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 828993 T1147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 828993 T1199 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 828993 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 829163 T1173 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:26199_v_ss%2Fyt_collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4]   2> 829163 T1173 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:26199_v_ss%252Fyt_collection1&state=recovering&nodeName=127.0.0.1:26199_v_ss%252Fyt&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1002 
[junit4:junit4]   2> 829166 T1125 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 830168 T1125 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C363_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:26199_v_ss%2Fyt, base_url=http://127.0.0.1:26199/v_ss/yt}
[junit4:junit4]   2> 831165 T1219 C363 P26199 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:26187/v_ss/yt/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 831165 T1219 C363 P26199 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:26199/v_ss/yt START replicas=[http://127.0.0.1:26187/v_ss/yt/collection1/] nUpdates=100
[junit4:junit4]   2> 831166 T1219 C363 P26199 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 831166 T1219 C363 P26199 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 831166 T1219 C363 P26199 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 831167 T1219 C363 P26199 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 831167 T1219 C363 P26199 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 831167 T1219 C363 P26199 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:26187/v_ss/yt/collection1/. core=collection1
[junit4:junit4]   2> 831167 T1219 C363 P26199 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 831171 T1125 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> ASYNC  NEW_CORE C366 name=collection1 org.apache.solr.core.SolrCore@2ca908f4 url=http://127.0.0.1:26187/v_ss/yt/collection1 node=127.0.0.1:26187_v_ss%2Fyt C366_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:26187_v_ss%2Fyt, base_url=http://127.0.0.1:26187/v_ss/yt, leader=true}
[junit4:junit4]   2> 831176 T1174 C366 P26187 oasc.SolrCore.execute [collection1] webapp=/v_ss/yt path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1 
[junit4:junit4]   2> 831181 T1175 C366 P26187 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 831184 T1175 C366 P26187 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@599cf202 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6bc6c898),segFN=segments_1,generation=1}
[junit4:junit4]   2> 831185 T1175 C366 P26187 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 831185 T1175 C366 P26187 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@599cf202 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6bc6c898),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@599cf202 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6bc6c898),segFN=segments_2,generation=2}
[junit4:junit4]   2> 831186 T1175 C366 P26187 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 831186 T1175 C366 P26187 oass.SolrIndexSearcher.<init> Opening Searcher@33d9ff78 realtime
[junit4:junit4]   2> 831187 T1175 C366 P26187 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 831187 T1175 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4]   2> 831188 T1219 C363 P26199 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 831189 T1219 C363 P26199 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 831191 T1176 C366 P26187 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 831191 T1176 C366 P26187 oasc.SolrCore.execute [collection1] webapp=/v_ss/yt path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 831192 T1219 C363 P26199 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 831192 T1219 C363 P26199 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 831193 T1219 C363 P26199 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 831195 T1176 C366 P26187 oasc.SolrCore.execute [collection1] webapp=/v_ss/yt path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0 
[junit4:junit4]   2> 831196 T1219 C363 P26199 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 831197 T1219 C363 P26199 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/jetty4/index.20130609024647469
[junit4:junit4]   2> 831198 T1219 C363 P26199 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@7b6c959a lockFactory=org.apache.lucene.store.NativeFSLockFactory@21e6d3a7) fullCopy=false
[junit4:junit4]   2> 831200 T1176 C366 P26187 oasc.SolrCore.execute [collection1] webapp=/v_ss/yt path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 831202 T1219 C363 P26199 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 831202 T1219 C363 P26199 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 831203 T1219 C363 P26199 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 831204 T1219 C363 P26199 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4c5e9ece lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ddd9220),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4c5e9ece lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ddd9220),segFN=segments_2,generation=2}
[junit4:junit4]   2> 831205 T1219 C363 P26199 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 831205 T1219 C363 P26199 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 831206 T1219 C363 P26199 oass.SolrIndexSearcher.<init> Opening Searcher@8bd45be main
[junit4:junit4]   2> 831207 T1218 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@8bd45be main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 831207 T1219 C363 P26199 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/jetty4/index.20130609024647469 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/jetty4/index.20130609024647469;done=true>>]
[junit4:junit4]   2> 831208 T1219 C363 P26199 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/jetty4/index.20130609024647469
[junit4:junit4]   2> 831208 T1219 C363 P26199 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370756784468/jetty4/index.20130609024647469
[junit4:junit4]   2> 831208 T1219 C363 P26199 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 831208 T1219 C363 P26199 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 831209 T1219 C363 P26199 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 831209 T1219 C363 P26199 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 831213 T1219 C363 P26199 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 832010 T1148 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 832011 T1148 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:26199_v_ss%2Fyt_collection1",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:26199_v_ss%2Fyt",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:26199/v_ss/yt"}
[junit4:junit4]   2> 832042 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 832042 T1216 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 832042 T1167 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 832042 T1147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 832042 T1199 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 832042 T1183 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 832174 T1125 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 832175 T1125 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4]   2> ASYNC  NEW_CORE C367 name=collection1 org.apache.solr.core.SolrCore@a3c7785 url=http://127.0.0.1:26180/v_ss/yt/collection1 node=127.0.0.1:26180_v_ss%2Fyt C367_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:26180_v_ss%2Fyt, base_url=http://127.0.0.1:26180/v_ss/yt, leader=true}
[junit4:junit4]   2> 832184 T1137 C367 P26180 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 832189 T1137 C367 P26180 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@58606861 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6ff7c5ae),segFN=segments_1,generation=1}
[junit4:junit4]   2> 832189 T1137 C367 P26180 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 832190 T1137 C367 P26180 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@58606861 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6ff7c5ae),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@58606861 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6ff7c5ae),segFN=segments_2,generation=2}
[junit4:junit4]   2> 832191 T1137 C367 P26180 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 832191 T1137 C367 P26180 oass.SolrIndexSearcher.<init> Opening Searcher@63674a0 main
[junit4:junit4]   2> 832192 T1137 C367 P26180 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 832193 T1151 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@63674a0 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 832193 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4]   2> 832222 T1177 C366 P26187 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 832223 T1177 C366 P26187 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@599cf202 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6bc6c898),segFN=segments_2,generation=2}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@599cf202 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6bc6c898),segFN=segments_3,generation=3}
[junit4:junit4]   2> 832223 T1177 C366 P26187 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4]   2> 832224 T1177 C366 P26187 oass.SolrIndexSearcher.<init> Opening Searcher@5d44a48b main
[junit4:junit4]   2> 832225 T1177 C366 P26187 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 832225 T1185 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5d44a48b main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 832226 T1177 C366 P26187 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:26184/v_ss/yt/collection1/, StdNode: http://127.0.0.1:26195/v_ss/yt/collection1/, StdNode: http://127.0.0.1:26199/v_ss/yt/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4]   2> 832229 T1161 C365 P26184 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 832230 T1161 C365 P26184 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@f85ad37 lockFactory=org.apache.lucene.store.NativeFSLockFactory@76bc4f9b),segFN=segments_2,generation=2}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@f85ad37 lockFactory=org.apache.lucene.store.NativeFSLockFactory@76bc4f9b),segFN=segments_3,generation=3}
[junit4:junit4]   2> 832231 T1161 C365 P26184 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4]   2> 832231 T1161 C365 P26184 oass.SolrIndexSearcher.<init> Opening Searcher@67ffa19d main
[junit4:junit4]   2> 832232 T1161 C365 P26184 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 832233 T1169 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@67ffa19d main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 832233 T1161 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 4
[junit4:junit4]   2>  C364_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:26195_v_ss%2Fyt, base_url=http://127.0.0.1:26195/v_ss/yt}
[junit4:junit4]   2> 832236 T1189 C364 P26195 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2>  C363_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:26199_v_ss%2Fyt, base_url=http://127.0.0.1:26199/v_ss/yt}
[junit4:junit4]   2> 832236 T1206 C363 P26199 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 832238 T1189 C364 P26195 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50f85f07 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4d90b8a3),segFN=segments_2,generation=2}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50f85f07 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4d90b8a3),segFN=segments_3,generation=3}
[junit4:junit4]   2> 832238 T1206 C363 P26199 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4c5e9ece lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ddd9220),segFN=segments_2,generation=2}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4c5e9ece lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ddd9220),segFN=segments_3,generation=3}
[junit4:junit4]   2> 832238 T1189 C364 P26195 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4]   2> 832239 T1206 C363 P26199 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4]   2> 832240 T1189 C364 P26195 oass.SolrIndexSearcher.<init> Opening Searcher@6813dff1 main
[junit4:junit4]   2> 832240 T1206 C363 P26199 oass.SolrIndexSearcher.<init> Opening Searcher@27c0fb2c main
[junit4:junit4]   2> 832241 T1189 C364 P26195 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 832241 T1201 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6813dff1 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 832241 T1206 C363 P26199 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 832242 T1189 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4]   2> 832242 T1218 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@27c0fb2c main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 832243 T1206 C363 P26199 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4]   2> 832244 T1177 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 23
[junit4:junit4]   2> 832245 T1125 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 832248 T1162 C365 P26184 oasc.SolrCore.execute [collection1] webapp=/v_ss/yt path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0 
[junit4:junit4]   2> 832251 T1190 C364 P26195 oasc.SolrCore.execute [collection1] webapp=/v_ss/yt path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0 
[junit4:junit4]   2> 832253 T1178 C366 P26187 oasc.SolrCore.execute [collection1] webapp=/v_ss/yt path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0 
[junit4:junit4]   2> 832256 T1207 C363 P26199 oasc.SolrCore.execute [collection1] webapp=/v_ss/yt path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0 
[junit4:junit4]   2> 834263 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1437342693361451008)} 0 2
[junit4:junit4]   2> 834272 T1191 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={update.distrib=FROMLEADER&_version_=-1437342693366693888&update.from=http://127.0.0.1:26184/v_ss/yt/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1437342693366693888)} 0 2
[junit4:junit4]   2> 834276 T1206 C363 P26199 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={update.distrib=FROMLEADER&_version_=-1437342693370888192&update.from=http://127.0.0.1:26187/v_ss/yt/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1437342693370888192)} 0 2
[junit4:junit4]   2> 834277 T1179 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1437342693370888192)} 0 7
[junit4:junit4]   2> 834278 T1163 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1437342693366693888)} 0 13
[junit4:junit4]   2> 834282 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[0 (1437342693382422528)]} 0 1
[junit4:junit4]   2> 834292 T1191 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1437342693390811136)]} 0 1
[junit4:junit4]   2> 834293 T1161 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[0 (1437342693390811136)]} 0 5
[junit4:junit4]   2> 834294 T1177 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[0]} 0 9
[junit4:junit4]   2> 834298 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[1 (1437342693400248320)]} 0 1
[junit4:junit4]   2> 834308 T1191 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1437342693406539776)]} 0 1
[junit4:junit4]   2> 834309 T1161 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1437342693406539776)]} 0 5
[junit4:junit4]   2> 834309 T1177 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[1]} 0 8
[junit4:junit4]   2> 834313 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[2 (1437342693414928384)]} 0 1
[junit4:junit4]   2> 834323 T1206 C363 P26199 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1437342693422268416)]} 0 1
[junit4:junit4]   2> 834324 T1179 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[2 (1437342693422268416)]} 0 5
[junit4:junit4]   2> 834324 T1163 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[2]} 0 8
[junit4:junit4]   2> 834328 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[3 (1437342693430657024)]} 0 1
[junit4:junit4]   2> 834335 T1206 C363 P26199 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1437342693434851328)]} 0 1
[junit4:junit4]   2> 834336 T1177 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[3 (1437342693434851328)]} 0 5
[junit4:junit4]   2> 834340 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[4 (1437342693443239936)]} 0 1
[junit4:junit4]   2> 834347 T1191 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1437342693447434240)]} 0 1
[junit4:junit4]   2> 834348 T1163 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[4 (1437342693447434240)]} 0 5
[junit4:junit4]   2> 834351 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[5 (1437342693454774272)]} 0 1
[junit4:junit4]   2> 834361 T1206 C363 P26199 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[5 (1437342693462114304)]} 0 1
[junit4:junit4]   2> 834362 T1179 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[5 (1437342693462114304)]} 0 5
[junit4:junit4]   2> 834362 T1163 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[5]} 0 8
[junit4:junit4]   2> 834366 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[6 (1437342693470502912)]} 0 1
[junit4:junit4]   2> 834376 T1206 C363 P26199 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[6 (1437342693477842944)]} 0 1
[junit4:junit4]   2> 834377 T1179 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[6 (1437342693477842944)]} 0 5
[junit4:junit4]   2> 834377 T1163 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[6]} 0 8
[junit4:junit4]   2> 834381 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[7 (1437342693486231552)]} 0 1
[junit4:junit4]   2> 834388 T1206 C363 P26199 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[7 (1437342693490425856)]} 0 1
[junit4:junit4]   2> 834389 T1177 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[7 (1437342693490425856)]} 0 5
[junit4:junit4]   2> 834393 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[8 (1437342693498814464)]} 0 1
[junit4:junit4]   2> 834403 T1191 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1437342693506154496)]} 0 1
[junit4:junit4]   2> 834404 T1161 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[8 (1437342693506154496)]} 0 5
[junit4:junit4]   2> 834404 T1177 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[8]} 0 8
[junit4:junit4]   2> 834408 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[9 (1437342693514543104)]} 0 1
[junit4:junit4]   2> 834418 T1206 C363 P26199 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[9 (1437342693522931712)]} 0 1
[junit4:junit4]   2> 834419 T1179 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[9 (1437342693522931712)]} 0 5
[junit4:junit4]   2> 834419 T1163 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[9]} 0 8
[junit4:junit4]   2> 834423 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[10 (1437342693530271744)]} 0 1
[junit4:junit4]   2> 834433 T1191 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10 (1437342693538660352)]} 0 1
[junit4:junit4]   2> 834434 T1161 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[10 (1437342693538660352)]} 0 5
[junit4:junit4]   2> 834434 T1177 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[10]} 0 8
[junit4:junit4]   2> 834438 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[11 (1437342693546000384)]} 0 1
[junit4:junit4]   2> 834445 T1191 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[11 (1437342693550194688)]} 0 1
[junit4:junit4]   2> 834446 T1163 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[11 (1437342693550194688)]} 0 5
[junit4:junit4]   2> 834450 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[12 (1437342693558583296)]} 0 1
[junit4:junit4]   2> 834460 T1191 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[12 (1437342693565923328)]} 0 1
[junit4:junit4]   2> 834461 T1161 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[12 (1437342693565923328)]} 0 5
[junit4:junit4]   2> 834461 T1177 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[12]} 0 8
[junit4:junit4]   2> 834465 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[13 (1437342693574311936)]} 0 1
[junit4:junit4]   2> 834475 T1191 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[13 (1437342693581651968)]} 0 1
[junit4:junit4]   2> 834476 T1161 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[13 (1437342693581651968)]} 0 5
[junit4:junit4]   2> 834476 T1177 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[13]} 0 8
[junit4:junit4]   2> 834480 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[14 (1437342693591089152)]} 0 0
[junit4:junit4]   2> 834490 T1191 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[14 (1437342693597380608)]} 0 1
[junit4:junit4]   2> 834490 T1161 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[14 (1437342693597380608)]} 0 4
[junit4:junit4]   2> 834491 T1177 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[14]} 0 8
[junit4:junit4]   2> 834495 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[15 (1437342693605769216)]} 0 1
[junit4:junit4]   2> 834505 T1191 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[15 (1437342693613109248)]} 0 1
[junit4:junit4]   2> 834505 T1161 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[15 (1437342693613109248)]} 0 4
[junit4:junit4]   2> 834506 T1177 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[15]} 0 8
[junit4:junit4]   2> 834510 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[16 (1437342693621497856)]} 0 1
[junit4:junit4]   2> 834520 T1191 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[16 (1437342693628837888)]} 0 1
[junit4:junit4]   2> 834521 T1161 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[16 (1437342693628837888)]} 0 5
[junit4:junit4]   2> 834521 T1177 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[16]} 0 8
[junit4:junit4]   2> 834525 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[17 (1437342693637226496)]} 0 1
[junit4:junit4]   2> 834535 T1206 C363 P26199 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[17 (1437342693644566528)]} 0 1
[junit4:junit4]   2> 834536 T1179 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[17 (1437342693644566528)]} 0 5
[junit4:junit4]   2> 834536 T1163 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[17]} 0 8
[junit4:junit4]   2> 834540 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[18 (1437342693652955136)]} 0 1
[junit4:junit4]   2> 834547 T1206 C363 P26199 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[18 (1437342693657149440)]} 0 1
[junit4:junit4]   2> 834548 T1177 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[18 (1437342693657149440)]} 0 5
[junit4:junit4]   2> 834551 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[19 (1437342693664489472)]} 0 1
[junit4:junit4]   2> 834561 T1206 C363 P26199 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[19 (1437342693671829504)]} 0 1
[junit4:junit4]   2> 834561 T1179 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[19 (1437342693671829504)]} 0 4
[junit4:junit4]   2> 834562 T1163 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[19]} 0 8
[junit4:junit4]   2> 834566 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[20 (1437342693680218112)]} 0 1
[junit4:junit4]   2> 834573 T1191 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[20 (1437342693684412416)]} 0 1
[junit4:junit4]   2> 834574 T1163 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[20 (1437342693684412416)]} 0 5
[junit4:junit4]   2> 834578 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[21 (1437342693692801024)]} 0 1
[junit4:junit4]   2> 834585 T1206 C363 P26199 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[21 (1437342693696995328)]} 0 1
[junit4:junit4]   2> 834585 T1177 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[21 (1437342693696995328)]} 0 4
[junit4:junit4]   2> 834589 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[22 (1437342693704335360)]} 0 1
[junit4:junit4]   2> 834599 T1191 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[22 (1437342693711675392)]} 0 1
[junit4:junit4]   2> 834600 T1161 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[22 (1437342693711675392)]} 0 5
[junit4:junit4]   2> 834600 T1177 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[22]} 0 8
[junit4:junit4]   2> 834604 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[23 (1437342693720064000)]} 0 1
[junit4:junit4]   2> 834614 T1206 C363 P26199 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[23 (1437342693727404032)]} 0 1
[junit4:junit4]   2> 834614 T1179 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[23 (1437342693727404032)]} 0 4
[junit4:junit4]   2> 834615 T1163 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[23]} 0 8
[junit4:junit4]   2> 834619 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[24 (1437342693735792640)]} 0 1
[junit4:junit4]   2> 834629 T1191 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[24 (1437342693743132672)]} 0 1
[junit4:junit4]   2> 834630 T1161 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[24 (1437342693743132672)]} 0 5
[junit4:junit4]   2> 834630 T1177 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[24]} 0 8
[junit4:junit4]   2> 834634 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[25 (1437342693751521280)]} 0 1
[junit4:junit4]   2> 834641 T1206 C363 P26199 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[25 (1437342693755715584)]} 0 1
[junit4:junit4]   2> 834641 T1177 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[25 (1437342693755715584)]} 0 4
[junit4:junit4]   2> 834645 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[26 (1437342693763055616)]} 0 1
[junit4:junit4]   2> 834652 T1206 C363 P26199 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[26 (1437342693767249920)]} 0 1
[junit4:junit4]   2> 834653 T1177 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[26 (1437342693767249920)]} 0 5
[junit4:junit4]   2> 834656 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[27 (1437342693774589952)]} 0 1
[junit4:junit4]   2> 834666 T1191 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[27 (1437342693781929984)]} 0 1
[junit4:junit4]   2> 834667 T1161 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[27 (1437342693781929984)]} 0 5
[junit4:junit4]   2> 834667 T1177 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[27]} 0 8
[junit4:junit4]   2> 834671 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[28 (1437342693790318592)]} 0 1
[junit4:junit4]   2> 834681 T1191 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[28 (1437342693797658624)]} 0 1
[junit4:junit4]   2> 834682 T1161 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[28 (1437342693797658624)]} 0 5
[junit4:junit4]   2> 834682 T1177 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[28]} 0 8
[junit4:junit4]   2> 834686 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[29 (1437342693806047232)]} 0 1
[junit4:junit4]   2> 834693 T1206 C363 P26199 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[29 (1437342693810241536)]} 0 1
[junit4:junit4]   2> 834693 T1177 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[29 (1437342693810241536)]} 0 4
[junit4:junit4]   2> 834697 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[30 (1437342693817581568)]} 0 1
[junit4:junit4]   2> 834704 T1206 C363 P26199 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[30 (1437342693821775872)]} 0 1
[junit4:junit4]   2> 834704 T1177 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[30 (1437342693821775872)]} 0 4
[junit4:junit4]   2> 834708 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[31 (1437342693829115904)]} 0 1
[junit4:junit4]   2> 834718 T1206 C363 P26199 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[31 (1437342693836455936)]} 0 1
[junit4:junit4]   2> 834718 T1179 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[31 (1437342693836455936)]} 0 4
[junit4:junit4]   2> 834719 T1163 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[31]} 0 8
[junit4:junit4]   2> 834723 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[32 (1437342693844844544)]} 0 1
[junit4:junit4]   2> 834733 T1191 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[32 (1437342693852184576)]} 0 1
[junit4:junit4]   2> 834733 T1161 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[32 (1437342693852184576)]} 0 4
[junit4:junit4]   2> 834734 T1177 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[32]} 0 8
[junit4:junit4]   2> 834738 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[33 (1437342693860573184)]} 0 1
[junit4:junit4]   2> 834745 T1206 C363 P26199 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[33 (1437342693864767488)]} 0 1
[junit4:junit4]   2> 834745 T1177 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[33 (1437342693864767488)]} 0 4
[junit4:junit4]   2> 834749 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[34 (1437342693872107520)]} 0 1
[junit4:junit4]   2> 834756 T1191 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[34 (1437342693876301824)]} 0 1
[junit4:junit4]   2> 834757 T1163 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[34 (1437342693876301824)]} 0 5
[junit4:junit4]   2> 834761 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[35 (1437342693884690432)]} 0 1
[junit4:junit4]   2> 834768 T1191 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[35 (1437342693888884736)]} 0 1
[junit4:junit4]   2> 834769 T1163 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[35 (1437342693888884736)]} 0 5
[junit4:junit4]   2> 834773 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[36 (1437342693897273344)]} 0 1
[junit4:junit4]   2> 834783 T1191 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[36 (1437342693904613376)]} 0 1
[junit4:junit4]   2> 834784 T1161 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[36 (1437342693904613376)]} 0 5
[junit4:junit4]   2> 834784 T1177 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[36]} 0 8
[junit4:junit4]   2> 834788 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[37 (1437342693913001984)]} 0 1
[junit4:junit4]   2> 834795 T1206 C363 P26199 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[37 (1437342693917196288)]} 0 1
[junit4:junit4]   2> 834795 T1177 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[37 (1437342693917196288)]} 0 4
[junit4:junit4]   2> 834799 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[38 (1437342693924536320)]} 0 1
[junit4:junit4]   2> 834806 T1191 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[38 (1437342693928730624)]} 0 1
[junit4:junit4]   2> 834806 T1163 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[38 (1437342693928730624)]} 0 4
[junit4:junit4]   2> 834810 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[39 (1437342693936070656)]} 0 1
[junit4:junit4]   2> 834820 T1206 C363 P26199 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[39 (1437342693943410688)]} 0 1
[junit4:junit4]   2> 834820 T1179 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[39 (1437342693943410688)]} 0 4
[junit4:junit4]   2> 834821 T1163 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[39]} 0 8
[junit4:junit4]   2> 834825 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[40 (1437342693951799296)]} 0 1
[junit4:junit4]   2> 834835 T1191 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[40 (1437342693959139328)]} 0 1
[junit4:junit4]   2> 834835 T1161 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[40 (1437342693959139328)]} 0 4
[junit4:junit4]   2> 834836 T1177 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[40]} 0 8
[junit4:junit4]   2> 834840 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[41 (1437342693967527936)]} 0 1
[junit4:junit4]   2> 834850 T1191 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[41 (1437342693974867968)]} 0 1
[junit4:junit4]   2> 834851 T1161 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[41 (1437342693974867968)]} 0 5
[junit4:junit4]   2> 834851 T1177 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[41]} 0 8
[junit4:junit4]   2> 834855 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[42 (1437342693983256576)]} 0 1
[junit4:junit4]   2> 834865 T1191 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[42 (1437342693990596608)]} 0 1
[junit4:junit4]   2> 834865 T1161 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[42 (1437342693990596608)]} 0 4
[junit4:junit4]   2> 834866 T1177 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[42]} 0 8
[junit4:junit4]   2> 834870 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[43 (1437342693998985216)]} 0 1
[junit4:junit4]   2> 834877 T1191 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[43 (1437342694003179520)]} 0 1
[junit4:junit4]   2> 834878 T1163 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[43 (1437342694003179520)]} 0 5
[junit4:junit4]   2> 834881 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[44 (1437342694010519552)]} 0 1
[junit4:junit4]   2> 834891 T1191 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[44 (1437342694017859584)]} 0 1
[junit4:junit4]   2> 834892 T1161 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[44 (1437342694017859584)]} 0 5
[junit4:junit4]   2> 834892 T1177 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[44]} 0 8
[junit4:junit4]   2> 834896 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[45 (1437342694026248192)]} 0 1
[junit4:junit4]   2> 834903 T1206 C363 P26199 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[45 (1437342694030442496)]} 0 1
[junit4:junit4]   2> 834903 T1177 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[45 (1437342694030442496)]} 0 4
[junit4:junit4]   2> 834908 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[46 (1437342694038831104)]} 0 1
[junit4:junit4]   2> 834918 T1191 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[46 (1437342694046171136)]} 0 1
[junit4:junit4]   2> 834919 T1161 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[46 (1437342694046171136)]} 0 5
[junit4:junit4]   2> 834919 T1177 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[46]} 0 8
[junit4:junit4]   2> 834923 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[47 (1437342694054559744)]} 0 1
[junit4:junit4]   2> 834930 T1191 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[47 (1437342694058754048)]} 0 1
[junit4:junit4]   2> 834931 T1163 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[47 (1437342694058754048)]} 0 5
[junit4:junit4]   2> 834934 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[48 (1437342694066094080)]} 0 1
[junit4:junit4]   2> 834941 T1191 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[48 (1437342694070288384)]} 0 1
[junit4:junit4]   2> 834941 T1163 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[48 (1437342694070288384)]} 0 4
[junit4:junit4]   2> 834945 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[49 (1437342694077628416)]} 0 1
[junit4:junit4]   2> 834952 T1191 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[49 (1437342694081822720)]} 0 1
[junit4:junit4]   2> 834953 T1163 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[49 (1437342694081822720)]} 0 5
[junit4:junit4]   2> 834956 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[50 (1437342694089162752)]} 0 1
[junit4:junit4]   2> 834963 T1191 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[50 (1437342694093357056)]} 0 1
[junit4:junit4]   2> 834964 T1163 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[50 (1437342694093357056)]} 0 5
[junit4:junit4]   2> 834967 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[51 (1437342694100697088)]} 0 1
[junit4:junit4]   2> 834977 T1191 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[51 (1437342694108037120)]} 0 1
[junit4:junit4]   2> 834978 T1161 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[51 (1437342694108037120)]} 0 5
[junit4:junit4]   2> 834978 T1177 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[51]} 0 8
[junit4:junit4]   2> 834982 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[52 (1437342694116425728)]} 0 1
[junit4:junit4]   2> 834989 T1191 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[52 (1437342694120620032)]} 0 1
[junit4:junit4]   2> 834990 T1163 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[52 (1437342694120620032)]} 0 5
[junit4:junit4]   2> 834993 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[53 (1437342694127960064)]} 0 1
[junit4:junit4]   2> 835000 T1206 C363 P26199 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[53 (1437342694132154368)]} 0 1
[junit4:junit4]   2> 835001 T1177 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[53 (1437342694132154368)]} 0 5
[junit4:junit4]   2> 835005 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[54 (1437342694140542976)]} 0 1
[junit4:junit4]   2> 835012 T1191 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[54 (1437342694144737280)]} 0 1
[junit4:junit4]   2> 835012 T1163 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[54 (1437342694144737280)]} 0 4
[junit4:junit4]   2> 835016 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[55 (1437342694152077312)]} 0 1
[junit4:junit4]   2> 835023 T1191 C364 P26195 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[55 (1437342694156271616)]} 0 1
[junit4:junit4]   2> 835023 T1163 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[55 (1437342694156271616)]} 0 4
[junit4:junit4]   2> 835027 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[56 (1437342694163611648)]} 0 1
[junit4:junit4]   2> 835037 T1206 C363 P26199 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[56 (1437342694170951680)]} 0 1
[junit4:junit4]   2> 835037 T1179 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[56 (1437342694170951680)]} 0 4
[junit4:junit4]   2> 835038 T1163 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[56]} 0 8
[junit4:junit4]   2> 835042 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[57 (1437342694179340288)]} 0 1
[junit4:junit4]   2> 835052 T1206 C363 P26199 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26187/v_ss/yt/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[57 (1437342694186680320)]} 0 1
[junit4:junit4]   2> 835053 T1179 C366 P26187 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={distrib.from=http://127.0.0.1:26184/v_ss/yt/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[57 (1437342694186680320)]} 0 5
[junit4:junit4]   2> 835053 T1163 C365 P26184 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_ss/yt path=/update params={wt=javabin&version=2} {add=[57]} 0 8
[junit4:junit4]   2> 835057 T1137 C367 P26180 oasup.LogUpdateProcessor.finish [c

[...truncated too long message...]

on expired for /overseer_elect/election
[junit4:junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:235)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:232)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:232)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:84)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4]   2> 
[junit4:junit4]   2> 1269751 T3153 oasu.PeerSync.handleResponse WARN PeerSync: core=collection1 url=http://127.0.0.1:21073/_h/dd  couldn't connect to http://127.0.0.1:21060/_h/dd/collection1/, counting as success
[junit4:junit4]   2> 1269751 T3153 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:21073/_h/dd DONE. sync succeeded
[junit4:junit4]   2> 1269751 T3153 oasc.SyncStrategy.syncReplicas We have been closed, won't attempt to sync replicas back to leader
[junit4:junit4]   2> 1269752 T3153 oasc.ShardLeaderElectionContext.rejoinLeaderElection Not rejoining election because CoreContainer is shutdown
[junit4:junit4]   2> 1269753 T3153 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@4e44f976
[junit4:junit4]   2> 1269759 T3153 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=2,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=805,adds=805,deletesById=389,deletesByQuery=0,errors=0,cumulative_adds=847,cumulative_deletesById=389,cumulative_deletesByQuery=1,cumulative_errors=0}
[junit4:junit4]   2> 1269759 T3153 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 1269759 T3153 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 1269760 T3153 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> ASYNC  NEW_CORE C919 name=collection1 org.apache.solr.core.SolrCore@4e44f976 url=http://127.0.0.1:21073/_h/dd/collection1 node=127.0.0.1:21073__h%2Fdd C919_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:21073__h%2Fdd, base_url=http://127.0.0.1:21073/_h/dd}
[junit4:junit4]   2> 1269773 T3153 C919 P21073 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3438a7c5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4a06ff55),segFN=segments_4,generation=4}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3438a7c5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4a06ff55),segFN=segments_5,generation=5}
[junit4:junit4]   2> 1269773 T3153 C919 P21073 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4]   2> 1269777 T3153 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 1269778 T3153 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4]   2> 1269778 T3153 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ShardSplitTest-1370756980247/jetty4/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ShardSplitTest-1370756980247/jetty4/index;done=false>>]
[junit4:junit4]   2> 1269778 T3153 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ShardSplitTest-1370756980247/jetty4/index
[junit4:junit4]   2> 1269778 T3153 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ShardSplitTest-1370756980247/jetty4 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ShardSplitTest-1370756980247/jetty4;done=false>>]
[junit4:junit4]   2> 1269779 T3153 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ShardSplitTest-1370756980247/jetty4
[junit4:junit4]   2> 1269779 T3153 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 1269779 T3153 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 1269780 T3153 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 1269780 T3153 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4:junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:235)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:232)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:232)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:84)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4]   2> 
[junit4:junit4]   2> NOTE: test params are: codec=Lucene41, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=ar_SA, timezone=America/Rosario
[junit4:junit4]   2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=16,threads=3,free=278017520,total=394264576
[junit4:junit4]   2> NOTE: All tests run in this JVM: [TestCloudManagedSchema, LeaderElectionIntegrationTest, FieldAnalysisRequestHandlerTest, SliceStateUpdateTest, ShowFileRequestHandlerTest, TestOmitPositions, DistributedTermsComponentTest, TestPluginEnable, FastVectorHighlighterTest, ExternalFileFieldSortTest, DocumentBuilderTest, TestSweetSpotSimilarityFactory, TestConfig, TestBadConfig, TestDistributedGrouping, RegexBoostProcessorTest, HighlighterTest, TestShardHandlerFactory, TestPHPSerializedResponseWriter, TestUtils, DistributedSpellCheckComponentTest, TestDFRSimilarityFactory, TestCharFilters, JSONWriterTest, TestDefaultSimilarityFactory, ResourceLoaderTest, BasicDistributedZkTest, SpellCheckCollatorTest, SolrPluginUtilsTest, TestSchemaSimilarityResource, ShardRoutingCustomTest, TestSolrDeletionPolicy1, TestFastOutputStream, TestDocSet, TestBinaryField, TestFaceting, RequiredFieldsTest, CopyFieldTest, DistanceFunctionTest, TestXIncludeConfig, TestSolrQueryParserDefaultOperatorResource, SuggesterTSTTest, TestPseudoReturnFields, IndexReaderFactoryTest, TestQueryUtils, TestAddFieldRealTimeGet, CollectionsAPIDistributedZkTest, TestCopyFieldCollectionResource, SpatialFilterTest, HighlighterConfigTest, SynonymTokenizerTest, TestWriterPerf, RecoveryZkTest, TestDocumentBuilder, TestQuerySenderListener, PathHierarchyTokenizerFactoryTest, HardAutoCommitTest, BadCopyFieldTest, TermsComponentTest, TestStressRecovery, SpellCheckComponentTest, CachingDirectoryFactoryTest, ChaosMonkeySafeLeaderTest, ChaosMonkeyNothingIsSafeTest, TestValueSourceCache, TestReplicationHandler, TestRemoteStreaming, OpenExchangeRatesOrgProviderTest, CurrencyFieldXmlFileTest, OutputWriterTest, BadComponentTest, ShardSplitTest]
[junit4:junit4] Completed on J0 in 266.29s, 1 test, 1 error <<< FAILURES!

[...truncated 459 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:391: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:371: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:181: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/common-build.xml:437: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:1243: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:887: There were test failures: 297 suites, 1246 tests, 2 errors, 13 ignored (7 assumptions)

Total time: 50 minutes 16 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 1316 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/1316/

4 tests failed.
REGRESSION:  org.apache.solr.cloud.ChaosMonkeyShardSplitTest.testDistribSearch

Error Message:
Server at http://127.0.0.1:54300/xp_ah returned non ok status:500, message:Server Error

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:54300/xp_ah returned non ok status:500, message:Server Error
	at __randomizedtesting.SeedInfo.seed([EC4264284E50992E:6DA4EA30390FF912]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
	at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
	at org.apache.solr.cloud.ChaosMonkeyShardSplitTest.doTest(ChaosMonkeyShardSplitTest.java:136)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)


REGRESSION:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch

Error Message:
There are still nodes recoverying - waited for 230 seconds

Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 230 seconds
	at __randomizedtesting.SeedInfo.seed([EC4264284E50992E:6DA4EA30390FF912]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:173)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:131)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:126)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:512)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:146)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
2 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest:     1) Thread[id=2596, name=recoveryCmdExecutor-1089-thread-1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at java.net.PlainSocketImpl.socketConnect(Native Method)         at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)         at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)         at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)         at java.net.Socket.connect(Socket.java:579)         at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)         at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)         at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)         at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)         at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)         at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)         at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)         at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)         at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)         at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)         at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:298)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=2598, name=recoveryCmdExecutor-1089-thread-2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at java.net.PlainSocketImpl.socketConnect(Native Method)         at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)         at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)         at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)         at java.net.Socket.connect(Socket.java:579)         at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)         at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)         at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)         at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)         at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)         at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)         at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)         at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)         at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)         at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)         at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:298)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest: 
   1) Thread[id=2596, name=recoveryCmdExecutor-1089-thread-1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
        at java.net.Socket.connect(Socket.java:579)
        at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
        at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
        at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
        at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
        at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
        at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
        at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
        at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:298)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=2598, name=recoveryCmdExecutor-1089-thread-2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
        at java.net.Socket.connect(Socket.java:579)
        at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
        at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
        at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
        at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
        at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
        at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
        at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
        at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:298)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([EC4264284E50992E]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=2596, name=recoveryCmdExecutor-1089-thread-1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at java.net.PlainSocketImpl.socketConnect(Native Method)         at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)         at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)         at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)         at java.net.Socket.connect(Socket.java:579)         at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)         at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)         at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)         at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)         at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)         at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)         at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)         at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)         at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)         at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)         at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:298)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=2598, name=recoveryCmdExecutor-1089-thread-2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]         at java.net.PlainSocketImpl.socketConnect(Native Method)         at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)         at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)         at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)         at java.net.Socket.connect(Socket.java:579)         at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)         at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)         at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)         at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)         at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)         at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)         at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)         at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)         at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)         at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)         at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:298)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=2596, name=recoveryCmdExecutor-1089-thread-1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
        at java.net.Socket.connect(Socket.java:579)
        at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
        at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
        at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
        at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
        at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
        at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
        at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
        at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:298)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=2598, name=recoveryCmdExecutor-1089-thread-2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
        at java.net.Socket.connect(Socket.java:579)
        at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
        at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
        at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
        at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
        at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
        at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
        at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
        at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:298)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([EC4264284E50992E]:0)




Build Log:
[...truncated 9550 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.ChaosMonkeyShardSplitTest
[junit4:junit4]   2> 817506 T2746 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /xp_ah/
[junit4:junit4]   2> 817513 T2746 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./solrtest-ChaosMonkeyShardSplitTest-1370724290916
[junit4:junit4]   2> 817515 T2746 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 817515 T2747 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 817616 T2746 oasc.ZkTestServer.run start zk server on port:42447
[junit4:junit4]   2> 817616 T2746 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 817647 T2753 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9775f30 name:ZooKeeperConnection Watcher:127.0.0.1:42447 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 817647 T2746 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 817648 T2746 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 817660 T2746 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 817661 T2755 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f1c981f name:ZooKeeperConnection Watcher:127.0.0.1:42447/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 817662 T2746 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 817662 T2746 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 817673 T2746 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 817675 T2746 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 817678 T2746 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 817687 T2746 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 817687 T2746 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 817700 T2746 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 817700 T2746 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 817805 T2746 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 817806 T2746 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 817818 T2746 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 817818 T2746 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 817821 T2746 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 817822 T2746 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 817825 T2746 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 817826 T2746 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 817829 T2746 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 817829 T2746 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 817832 T2746 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 817833 T2746 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 817835 T2746 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 817836 T2746 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 818105 T2746 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 818110 T2746 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:43889
[junit4:junit4]   2> 818111 T2746 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 818111 T2746 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 818112 T2746 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1370724291249
[junit4:junit4]   2> 818112 T2746 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1370724291249/solr.xml
[junit4:junit4]   2> 818113 T2746 oasc.CoreContainer.<init> New CoreContainer 1945132172
[junit4:junit4]   2> 818113 T2746 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1370724291249/'
[junit4:junit4]   2> 818114 T2746 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1370724291249/'
[junit4:junit4]   2> 818249 T2746 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 818250 T2746 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 818251 T2746 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 818252 T2746 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 818252 T2746 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 818253 T2746 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 818253 T2746 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 818254 T2746 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 818254 T2746 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 818255 T2746 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 818276 T2746 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 818277 T2746 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:42447/solr
[junit4:junit4]   2> 818278 T2746 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 818279 T2746 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 818282 T2766 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@72f183c3 name:ZooKeeperConnection Watcher:127.0.0.1:42447 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 818282 T2746 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 818284 T2746 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 818297 T2746 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 818299 T2768 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7778c2c2 name:ZooKeeperConnection Watcher:127.0.0.1:42447/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 818299 T2746 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 818301 T2746 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4]   2> 818311 T2746 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4]   2> 818315 T2746 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 818317 T2746 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43889_xp_ah
[junit4:junit4]   2> 818326 T2746 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:43889_xp_ah
[junit4:junit4]   2> 818329 T2746 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 818346 T2746 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 818348 T2746 oasc.Overseer.start Overseer (id=89831787129929731-127.0.0.1:43889_xp_ah-n_0000000000) starting
[junit4:junit4]   2> 818361 T2746 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4]   2> 818372 T2770 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 818373 T2746 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 818375 T2746 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4]   2> 818383 T2746 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 818385 T2769 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 818390 T2771 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1370724291249/collection1
[junit4:junit4]   2> 818390 T2771 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 818391 T2771 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 818391 T2771 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 818392 T2771 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1370724291249/collection1/'
[junit4:junit4]   2> 818394 T2771 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1370724291249/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 818394 T2771 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1370724291249/collection1/lib/README' to classloader
[junit4:junit4]   2> 818448 T2771 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 818509 T2771 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 818610 T2771 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 818617 T2771 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 819204 T2771 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 819208 T2771 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 819211 T2771 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 819216 T2771 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 819238 T2771 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 819239 T2771 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1370724291249/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/control/data/
[junit4:junit4]   2> 819239 T2771 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7539b974
[junit4:junit4]   2> 819240 T2771 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 819240 T2771 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/control/data
[junit4:junit4]   2> 819241 T2771 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/control/data/index/
[junit4:junit4]   2> 819241 T2771 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 819241 T2771 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/control/data/index
[junit4:junit4]   2> 819246 T2771 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6b281ea7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@23f30a9c),segFN=segments_1,generation=1}
[junit4:junit4]   2> 819246 T2771 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 819249 T2771 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 819250 T2771 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 819250 T2771 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 819251 T2771 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 819251 T2771 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 819252 T2771 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 819252 T2771 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 819253 T2771 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 819253 T2771 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 819268 T2771 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 819275 T2771 oass.SolrIndexSearcher.<init> Opening Searcher@59b34d9b main
[junit4:junit4]   2> 819275 T2771 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/control/data/tlog
[junit4:junit4]   2> 819277 T2771 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 819277 T2771 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 819280 T2772 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@59b34d9b main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 819282 T2771 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 819282 T2771 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 819888 T2769 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 819889 T2769 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:43889_xp_ah",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:43889/xp_ah"}
[junit4:junit4]   2> 819889 T2769 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 819890 T2769 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4]   2> 819957 T2768 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 820311 T2771 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 820311 T2771 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:43889/xp_ah collection:control_collection shard:shard1
[junit4:junit4]   2> 820312 T2771 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 820335 T2771 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 820345 T2771 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 820346 T2771 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 820346 T2771 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:43889/xp_ah/collection1/
[junit4:junit4]   2> 820346 T2771 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 820347 T2771 oasc.SyncStrategy.syncToMe http://127.0.0.1:43889/xp_ah/collection1/ has no replicas
[junit4:junit4]   2> 820347 T2771 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:43889/xp_ah/collection1/
[junit4:junit4]   2> 820347 T2771 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 821461 T2769 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 821480 T2768 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 821532 T2771 oasc.ZkController.register We are http://127.0.0.1:43889/xp_ah/collection1/ and leader is http://127.0.0.1:43889/xp_ah/collection1/
[junit4:junit4]   2> 821532 T2771 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:43889/xp_ah
[junit4:junit4]   2> 821532 T2771 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 821533 T2771 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 821533 T2771 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 821535 T2771 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 821536 T2746 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 821536 T2746 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 821537 T2746 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 821553 T2746 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 821555 T2746 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 821557 T2775 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@535c0c4e name:ZooKeeperConnection Watcher:127.0.0.1:42447/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 821557 T2746 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 821558 T2746 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 821560 T2746 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 821826 T2746 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 821830 T2746 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:54300
[junit4:junit4]   2> 821831 T2746 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 821831 T2746 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 821832 T2746 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1370724294963
[junit4:junit4]   2> 821832 T2746 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1370724294963/solr.xml
[junit4:junit4]   2> 821833 T2746 oasc.CoreContainer.<init> New CoreContainer 480687307
[junit4:junit4]   2> 821833 T2746 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1370724294963/'
[junit4:junit4]   2> 821834 T2746 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1370724294963/'
[junit4:junit4]   2> 821982 T2746 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 821983 T2746 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 821983 T2746 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 821984 T2746 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 821984 T2746 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 821985 T2746 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 821986 T2746 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 821986 T2746 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 821987 T2746 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 821987 T2746 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 822003 T2746 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 822003 T2746 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:42447/solr
[junit4:junit4]   2> 822004 T2746 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 822005 T2746 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 822008 T2786 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@420a2bf1 name:ZooKeeperConnection Watcher:127.0.0.1:42447 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 822009 T2746 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 822020 T2746 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 822031 T2746 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 822033 T2788 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@412abc64 name:ZooKeeperConnection Watcher:127.0.0.1:42447/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 822033 T2746 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 822042 T2746 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 822985 T2769 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 822986 T2769 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:43889_xp_ah_collection1",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:43889_xp_ah",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:43889/xp_ah"}
[junit4:junit4]   2> 823000 T2775 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 823001 T2788 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 823001 T2768 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 823045 T2746 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54300_xp_ah
[junit4:junit4]   2> 823047 T2746 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:54300_xp_ah
[junit4:junit4]   2> 823049 T2775 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 823049 T2768 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 823049 T2788 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 823050 T2788 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 823050 T2775 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 823051 T2768 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 823064 T2789 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1370724294963/collection1
[junit4:junit4]   2> 823064 T2789 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 823064 T2789 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 823065 T2789 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 823066 T2789 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1370724294963/collection1/'
[junit4:junit4]   2> 823068 T2789 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1370724294963/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 823068 T2789 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1370724294963/collection1/lib/README' to classloader
[junit4:junit4]   2> 823121 T2789 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 823182 T2789 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 823283 T2789 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 823290 T2789 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 823872 T2789 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 823876 T2789 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 823878 T2789 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 823884 T2789 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 823906 T2789 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 823906 T2789 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1370724294963/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/jetty1/
[junit4:junit4]   2> 823906 T2789 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7539b974
[junit4:junit4]   2> 823907 T2789 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 823908 T2789 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/jetty1
[junit4:junit4]   2> 823908 T2789 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/jetty1/index/
[junit4:junit4]   2> 823908 T2789 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 823909 T2789 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/jetty1/index
[junit4:junit4]   2> 823913 T2789 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@60e024ad lockFactory=org.apache.lucene.store.NativeFSLockFactory@1781716d),segFN=segments_1,generation=1}
[junit4:junit4]   2> 823913 T2789 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 823916 T2789 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 823917 T2789 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 823917 T2789 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 823918 T2789 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 823919 T2789 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 823919 T2789 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 823919 T2789 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 823920 T2789 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 823921 T2789 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 823937 T2789 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 823944 T2789 oass.SolrIndexSearcher.<init> Opening Searcher@7a711036 main
[junit4:junit4]   2> 823945 T2789 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/jetty1/tlog
[junit4:junit4]   2> 823946 T2789 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 823946 T2789 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 823951 T2790 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7a711036 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 823953 T2789 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 823953 T2789 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 824505 T2769 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 824506 T2769 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:54300_xp_ah",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:54300/xp_ah"}
[junit4:junit4]   2> 824506 T2769 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4]   2> 824506 T2769 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4]   2> 824531 T2768 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 824531 T2775 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 824531 T2788 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 824956 T2789 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 824956 T2789 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:54300/xp_ah collection:collection1 shard:shard1
[junit4:junit4]   2> 824957 T2789 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 824976 T2789 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 824987 T2789 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 824988 T2789 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 824988 T2789 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:54300/xp_ah/collection1/
[junit4:junit4]   2> 824988 T2789 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 824988 T2789 oasc.SyncStrategy.syncToMe http://127.0.0.1:54300/xp_ah/collection1/ has no replicas
[junit4:junit4]   2> 824989 T2789 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:54300/xp_ah/collection1/
[junit4:junit4]   2> 824989 T2789 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 826035 T2769 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 826080 T2768 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 826080 T2788 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 826080 T2775 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 826125 T2789 oasc.ZkController.register We are http://127.0.0.1:54300/xp_ah/collection1/ and leader is http://127.0.0.1:54300/xp_ah/collection1/
[junit4:junit4]   2> 826125 T2789 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:54300/xp_ah
[junit4:junit4]   2> 826125 T2789 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 826125 T2789 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 826126 T2789 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 826127 T2789 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 826128 T2746 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 826129 T2746 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 826130 T2746 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 826407 T2746 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 826411 T2746 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:63735
[junit4:junit4]   2> 826411 T2746 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 826412 T2746 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 826413 T2746 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1370724299548
[junit4:junit4]   2> 826413 T2746 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1370724299548/solr.xml
[junit4:junit4]   2> 826414 T2746 oasc.CoreContainer.<init> New CoreContainer 1548568110
[junit4:junit4]   2> 826414 T2746 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1370724299548/'
[junit4:junit4]   2> 826415 T2746 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1370724299548/'
[junit4:junit4]   2> 826544 T2746 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 826545 T2746 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 826546 T2746 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 826546 T2746 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 826547 T2746 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 826547 T2746 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 826548 T2746 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 826548 T2746 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 826549 T2746 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 826550 T2746 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 826567 T2746 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 826568 T2746 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:42447/solr
[junit4:junit4]   2> 826568 T2746 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 826569 T2746 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 826572 T2802 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ecee2e2 name:ZooKeeperConnection Watcher:127.0.0.1:42447 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 826573 T2746 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 826586 T2746 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 826598 T2746 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 826600 T2804 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1907596d name:ZooKeeperConnection Watcher:127.0.0.1:42447/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 826600 T2746 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 826612 T2746 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 827594 T2769 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 827595 T2769 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:54300_xp_ah_collection1",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:54300_xp_ah",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:54300/xp_ah"}
[junit4:junit4]   2> 827610 T2775 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 827610 T2768 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 827610 T2804 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 827610 T2788 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 827615 T2746 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63735_xp_ah
[junit4:junit4]   2> 827617 T2746 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:63735_xp_ah
[junit4:junit4]   2> 827619 T2775 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 827619 T2768 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 827619 T2804 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 827619 T2788 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 827620 T2804 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 827620 T2788 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 827620 T2775 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 827628 T2768 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 827634 T2805 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1370724299548/collection1
[junit4:junit4]   2> 827634 T2805 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 827635 T2805 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 827635 T2805 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 827636 T2805 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1370724299548/collection1/'
[junit4:junit4]   2> 827638 T2805 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1370724299548/collection1/lib/README' to classloader
[junit4:junit4]   2> 827638 T2805 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1370724299548/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 827692 T2805 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 827755 T2805 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 827856 T2805 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 827863 T2805 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 828461 T2805 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 828465 T2805 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 828467 T2805 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 828473 T2805 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 828494 T2805 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 828495 T2805 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1370724299548/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/jetty2/
[junit4:junit4]   2> 828495 T2805 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7539b974
[junit4:junit4]   2> 828496 T2805 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 828496 T2805 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/jetty2
[junit4:junit4]   2> 828497 T2805 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/jetty2/index/
[junit4:junit4]   2> 828497 T2805 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 828497 T2805 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/jetty2/index
[junit4:junit4]   2> 828501 T2805 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5ba0731b lockFactory=org.apache.lucene.store.NativeFSLockFactory@25e08757),segFN=segments_1,generation=1}
[junit4:junit4]   2> 828502 T2805 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 828505 T2805 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 828505 T2805 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 828506 T2805 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 828507 T2805 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 828507 T2805 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 828508 T2805 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 828508 T2805 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 828509 T2805 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 828509 T2805 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 828525 T2805 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 828533 T2805 oass.SolrIndexSearcher.<init> Opening Searcher@892598b main
[junit4:junit4]   2> 828533 T2805 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/jetty2/tlog
[junit4:junit4]   2> 828535 T2805 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 828535 T2805 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 828539 T2806 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@892598b main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 828541 T2805 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 828541 T2805 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 829114 T2769 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 829115 T2769 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:63735_xp_ah",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:63735/xp_ah"}
[junit4:junit4]   2> 829115 T2769 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 829115 T2769 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4]   2> 829134 T2804 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 829134 T2775 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 829134 T2788 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 829134 T2768 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 829544 T2805 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 829544 T2805 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:63735/xp_ah collection:collection1 shard:shard2
[junit4:junit4]   2> 829545 T2805 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 829560 T2805 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 829562 T2805 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 829563 T2805 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 829563 T2805 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:63735/xp_ah/collection1/
[junit4:junit4]   2> 829563 T2805 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 829564 T2805 oasc.SyncStrategy.syncToMe http://127.0.0.1:63735/xp_ah/collection1/ has no replicas
[junit4:junit4]   2> 829564 T2805 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:63735/xp_ah/collection1/
[junit4:junit4]   2> 829564 T2805 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   2> 830638 T2769 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 830659 T2768 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 830659 T2804 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 830659 T2788 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 830659 T2775 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 830678 T2805 oasc.ZkController.register We are http://127.0.0.1:63735/xp_ah/collection1/ and leader is http://127.0.0.1:63735/xp_ah/collection1/
[junit4:junit4]   2> 830678 T2805 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:63735/xp_ah
[junit4:junit4]   2> 830678 T2805 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 830678 T2805 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 830679 T2805 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 830680 T2805 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 830682 T2746 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 830682 T2746 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 830683 T2746 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 830959 T2746 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 830962 T2746 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:36192
[junit4:junit4]   2> 830963 T2746 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 830963 T2746 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 830964 T2746 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1370724304101
[junit4:junit4]   2> 830964 T2746 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1370724304101/solr.xml
[junit4:junit4]   2> 830965 T2746 oasc.CoreContainer.<init> New CoreContainer 1227315234
[junit4:junit4]   2> 830966 T2746 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1370724304101/'
[junit4:junit4]   2> 830966 T2746 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1370724304101/'
[junit4:junit4]   2> 831095 T2746 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 831096 T2746 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 831096 T2746 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 831097 T2746 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 831097 T2746 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 831098 T2746 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 831098 T2746 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 831099 T2746 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 831099 T2746 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 831100 T2746 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 831115 T2746 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 831116 T2746 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:42447/solr
[junit4:junit4]   2> 831117 T2746 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 831118 T2746 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 831124 T2818 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1fdfc96e name:ZooKeeperConnection Watcher:127.0.0.1:42447 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 831124 T2746 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 831136 T2746 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 831147 T2746 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 831149 T2820 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@362634e2 name:ZooKeeperConnection Watcher:127.0.0.1:42447/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 831150 T2746 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 831159 T2746 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 832162 T2746 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36192_xp_ah
[junit4:junit4]   2> 832192 T2746 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:36192_xp_ah
[junit4:junit4]   2> 832193 T2769 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 832194 T2769 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:63735_xp_ah_collection1",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:63735_xp_ah",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:63735/xp_ah"}
[junit4:junit4]   2> 832195 T2775 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 832195 T2768 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 832204 T2788 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 832204 T2820 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 832204 T2804 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 832204 T2788 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 832205 T2775 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 832205 T2804 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 832216 T2775 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 832216 T2788 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 832216 T2820 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 832216 T2804 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 832219 T2768 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 832220 T2768 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 832222 T2821 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1370724304101/collection1
[junit4:junit4]   2> 832222 T2821 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 832223 T2821 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 832223 T2821 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 832224 T2821 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1370724304101/collection1/'
[junit4:junit4]   2> 832226 T2821 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1370724304101/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 832226 T2821 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1370724304101/collection1/lib/README' to classloader
[junit4:junit4]   2> 832280 T2821 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 832340 T2821 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 832442 T2821 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 832450 T2821 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 833038 T2821 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 833042 T2821 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 833045 T2821 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 833050 T2821 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 833073 T2821 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 833073 T2821 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1370724304101/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/jetty3/
[junit4:junit4]   2> 833073 T2821 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7539b974
[junit4:junit4]   2> 833074 T2821 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 833075 T2821 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/jetty3
[junit4:junit4]   2> 833075 T2821 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/jetty3/index/
[junit4:junit4]   2> 833075 T2821 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 833076 T2821 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/jetty3/index
[junit4:junit4]   2> 833080 T2821 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@25168fea lockFactory=org.apache.lucene.store.NativeFSLockFactory@551d59a0),segFN=segments_1,generation=1}
[junit4:junit4]   2> 833080 T2821 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 833083 T2821 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 833084 T2821 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 833084 T2821 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 833085 T2821 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 833086 T2821 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 833086 T2821 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 833087 T2821 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 833087 T2821 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 833088 T2821 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 833104 T2821 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 833112 T2821 oass.SolrIndexSearcher.<init> Opening Searcher@40b7960d main
[junit4:junit4]   2> 833112 T2821 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/jetty3/tlog
[junit4:junit4]   2> 833113 T2821 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 833114 T2821 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 833118 T2822 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@40b7960d main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 833120 T2821 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 833121 T2821 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 833720 T2769 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 833721 T2769 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:36192_xp_ah",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:36192/xp_ah"}
[junit4:junit4]   2> 833722 T2769 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 833722 T2769 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4]   2> 833738 T2768 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 833738 T2804 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 833738 T2788 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 833738 T2820 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 833738 T2775 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 834123 T2821 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 834123 T2821 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:36192/xp_ah collection:collection1 shard:shard1
[junit4:junit4]   2> 834126 T2821 oasc.ZkController.register We are http://127.0.0.1:36192/xp_ah/collection1/ and leader is http://127.0.0.1:54300/xp_ah/collection1/
[junit4:junit4]   2> 834126 T2821 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:36192/xp_ah
[junit4:junit4]   2> 834127 T2821 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 834127 T2821 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C1674 name=collection1 org.apache.solr.core.SolrCore@4e6b8c64 url=http://127.0.0.1:36192/xp_ah/collection1 node=127.0.0.1:36192_xp_ah C1674_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:36192_xp_ah, base_url=http://127.0.0.1:36192/xp_ah}
[junit4:junit4]   2> 834127 T2823 C1674 P36192 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 834128 T2821 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 834129 T2823 C1674 P36192 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 834130 T2823 C1674 P36192 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 834130 T2823 C1674 P36192 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 834130 T2746 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 834131 T2746 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 834131 T2823 C1674 P36192 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 834132 T2746 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 834151 T2778 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:36192_xp_ah_collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4]   2> 834416 T2746 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 834419 T2746 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:19607
[junit4:junit4]   2> 834420 T2746 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 834420 T2746 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 834421 T2746 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1370724307551
[junit4:junit4]   2> 834421 T2746 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1370724307551/solr.xml
[junit4:junit4]   2> 834422 T2746 oasc.CoreContainer.<init> New CoreContainer 1863737581
[junit4:junit4]   2> 834423 T2746 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1370724307551/'
[junit4:junit4]   2> 834423 T2746 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1370724307551/'
[junit4:junit4]   2> 834552 T2746 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 834553 T2746 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 834554 T2746 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 834554 T2746 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 834555 T2746 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 834555 T2746 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 834556 T2746 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 834556 T2746 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 834557 T2746 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 834557 T2746 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 834577 T2746 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 834578 T2746 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:42447/solr
[junit4:junit4]   2> 834579 T2746 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 834580 T2746 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 834606 T2835 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7aa4ae96 name:ZooKeeperConnection Watcher:127.0.0.1:42447 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 834607 T2746 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 834660 T2746 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 834673 T2746 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 834693 T2837 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7f5d6a68 name:ZooKeeperConnection Watcher:127.0.0.1:42447/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 834693 T2746 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 834716 T2746 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 835242 T2769 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 835243 T2769 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:36192_xp_ah_collection1",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:36192_xp_ah",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:36192/xp_ah"}
[junit4:junit4]   2> 835260 T2775 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 835260 T2788 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 835260 T2820 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 835260 T2804 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 835260 T2768 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 835260 T2837 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 835719 T2746 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:19607_xp_ah
[junit4:junit4]   2> 835721 T2746 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:19607_xp_ah
[junit4:junit4]   2> 835723 T2768 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 835723 T2775 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 835724 T2788 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 835724 T2837 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 835724 T2804 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 835723 T2820 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 835725 T2768 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 835724 T2804 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 835724 T2837 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 835724 T2788 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 835725 T2820 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 835734 T2775 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 835740 T2838 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1370724307551/collection1
[junit4:junit4]   2> 835740 T2838 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 835741 T2838 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 835741 T2838 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 835742 T2838 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1370724307551/collection1/'
[junit4:junit4]   2> 835744 T2838 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1370724307551/collection1/lib/README' to classloader
[junit4:junit4]   2> 835745 T2838 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1370724307551/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 835818 T2838 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 835879 T2838 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 835980 T2838 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 835987 T2838 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 836154 T2778 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:36192_xp_ah_collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4]   2> 836154 T2778 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:36192_xp_ah_collection1&state=recovering&nodeName=127.0.0.1:36192_xp_ah&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2003 
[junit4:junit4]   2> 836565 T2838 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 836570 T2838 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 836572 T2838 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 836577 T2838 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 836599 T2838 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 836600 T2838 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1370724307551/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/jetty4/
[junit4:junit4]   2> 836600 T2838 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7539b974
[junit4:junit4]   2> 836601 T2838 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 836601 T2838 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/jetty4
[junit4:junit4]   2> 836602 T2838 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/jetty4/index/
[junit4:junit4]   2> 836602 T2838 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 836602 T2838 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/jetty4/index
[junit4:junit4]   2> 836606 T2838 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@64e7405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@62ca97ee),segFN=segments_1,generation=1}
[junit4:junit4]   2> 836607 T2838 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 836610 T2838 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 836610 T2838 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 836611 T2838 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 836612 T2838 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 836613 T2838 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 836613 T2838 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 836613 T2838 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 836614 T2838 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 836614 T2838 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 836631 T2838 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 836639 T2838 oass.SolrIndexSearcher.<init> Opening Searcher@74056889 main
[junit4:junit4]   2> 836639 T2838 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/jetty4/tlog
[junit4:junit4]   2> 836641 T2838 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 836641 T2838 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 836645 T2839 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@74056889 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 836647 T2838 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 836648 T2838 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 836765 T2769 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 836766 T2769 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:19607_xp_ah",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:19607/xp_ah"}
[junit4:junit4]   2> 836766 T2769 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 836766 T2769 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4]   2> 836779 T2768 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 836780 T2820 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 836780 T2788 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 836780 T2804 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 836779 T2775 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 836779 T2837 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 837652 T2838 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 837652 T2838 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:19607/xp_ah collection:collection1 shard:shard2
[junit4:junit4]   2> 837655 T2838 oasc.ZkController.register We are http://127.0.0.1:19607/xp_ah/collection1/ and leader is http://127.0.0.1:63735/xp_ah/collection1/
[junit4:junit4]   2> 837655 T2838 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:19607/xp_ah
[junit4:junit4]   2> 837656 T2838 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 837656 T2838 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C1675 name=collection1 org.apache.solr.core.SolrCore@1ae4d52 url=http://127.0.0.1:19607/xp_ah/collection1 node=127.0.0.1:19607_xp_ah C1675_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:19607_xp_ah, base_url=http://127.0.0.1:19607/xp_ah}
[junit4:junit4]   2> 837656 T2840 C1675 P19607 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 837657 T2838 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 837658 T2840 C1675 P19607 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 837658 T2840 C1675 P19607 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 837658 T2840 C1675 P19607 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 837659 T2746 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 837660 T2746 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 837660 T2746 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 837661 T2840 C1675 P19607 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 837678 T2746 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 837680 T2794 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:19607_xp_ah_collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4]   2> 837680 T2746 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4]   2> 837680 T2746 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4]   2> 837681 T2746 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> ASYNC  NEW_CORE C1676 name=collection1 org.apache.solr.core.SolrCore@4e6b8c64 url=http://127.0.0.1:36192/xp_ah/collection1 node=127.0.0.1:36192_xp_ah C1676_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:36192_xp_ah, base_url=http://127.0.0.1:36192/xp_ah}
[junit4:junit4]   2> 838156 T2823 C1676 P36192 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:54300/xp_ah/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 838156 T2823 C1676 P36192 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:36192/xp_ah START replicas=[http://127.0.0.1:54300/xp_ah/collection1/] nUpdates=100
[junit4:junit4]   2> 838156 T2823 C1676 P36192 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 838157 T2823 C1676 P36192 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 838157 T2823 C1676 P36192 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 838157 T2823 C1676 P36192 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 838157 T2823 C1676 P36192 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 838158 T2823 C1676 P36192 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:54300/xp_ah/collection1/. core=collection1
[junit4:junit4]   2> 838158 T2823 C1676 P36192 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C1677 name=collection1 org.apache.solr.core.SolrCore@1f815fc url=http://127.0.0.1:54300/xp_ah/collection1 node=127.0.0.1:54300_xp_ah C1677_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:54300_xp_ah, base_url=http://127.0.0.1:54300/xp_ah, leader=true}
[junit4:junit4]   2> 838165 T2779 C1677 P54300 oasc.SolrCore.execute [collection1] webapp=/xp_ah path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1 
[junit4:junit4]   2> 838170 T2780 C1677 P54300 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 838172 T2780 C1677 P54300 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@60e024ad lockFactory=org.apache.lucene.store.NativeFSLockFactory@1781716d),segFN=segments_1,generation=1}
[junit4:junit4]   2> 838173 T2780 C1677 P54300 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 838174 T2780 C1677 P54300 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@60e024ad lockFactory=org.apache.lucene.store.NativeFSLockFactory@1781716d),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@60e024ad lockFactory=org.apache.lucene.store.NativeFSLockFactory@1781716d),segFN=segments_2,generation=2}
[junit4:junit4]   2> 838174 T2780 C1677 P54300 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 838175 T2780 C1677 P54300 oass.SolrIndexSearcher.<init> Opening Searcher@6b5b27b0 realtime
[junit4:junit4]   2> 838175 T2780 C1677 P54300 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 838176 T2780 C1677 P54300 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4]   2> 838177 T2823 C1676 P36192 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 838177 T2823 C1676 P36192 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 838179 T2781 C1677 P54300 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 838180 T2781 C1677 P54300 oasc.SolrCore.execute [collection1] webapp=/xp_ah path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 838180 T2823 C1676 P36192 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 838181 T2823 C1676 P36192 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 838181 T2823 C1676 P36192 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 838183 T2781 C1677 P54300 oasc.SolrCore.execute [collection1] webapp=/xp_ah path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0 
[junit4:junit4]   2> 838184 T2823 C1676 P36192 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 838185 T2823 C1676 P36192 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/jetty3/index.20130609084511586
[junit4:junit4]   2> 838186 T2823 C1676 P36192 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@6a21e86b lockFactory=org.apache.lucene.store.NativeFSLockFactory@5394a0cf) fullCopy=false
[junit4:junit4]   2> 838188 T2781 C1677 P54300 oasc.SolrCore.execute [collection1] webapp=/xp_ah path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> ASYNC  NEW_CORE C1678 name=collection1 org.apache.solr.core.SolrCore@4e6b8c64 url=http://127.0.0.1:36192/xp_ah/collection1 node=127.0.0.1:36192_xp_ah C1678_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:36192_xp_ah, base_url=http://127.0.0.1:36192/xp_ah}
[junit4:junit4]   2> 838211 T2823 C1678 P36192 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 838212 T2823 C1678 P36192 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 838212 T2823 C1678 P36192 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 838214 T2823 C1678 P36192 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@25168fea lockFactory=org.apache.lucene.store.NativeFSLockFactory@551d59a0),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@25168fea lockFactory=org.apache.lucene.store.NativeFSLockFactory@551d59a0),segFN=segments_2,generation=2}
[junit4:junit4]   2> 838216 T2823 C1678 P36192 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 838216 T2823 C1678 P36192 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 838216 T2823 C1678 P36192 oass.SolrIndexSearcher.<init> Opening Searcher@e6ba2c2 main
[junit4:junit4]   2> 838218 T2822 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@e6ba2c2 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 838218 T2823 C1678 P36192 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/jetty3/index.20130609084511586 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/jetty3/index.20130609084511586;done=true>>]
[junit4:junit4]   2> 838219 T2823 C1678 P36192 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/jetty3/index.20130609084511586
[junit4:junit4]   2> 838219 T2823 C1678 P36192 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/jetty3/index.20130609084511586
[junit4:junit4]   2> 838219 T2823 C1678 P36192 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 838219 T2823 C1678 P36192 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 838219 T2823 C1678 P36192 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 838220 T2823 C1678 P36192 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 838221 T2823 C1678 P36192 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 838284 T2769 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 838285 T2769 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:19607_xp_ah_collection1",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:19607_xp_ah",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:19607/xp_ah"}
[junit4:junit4]   2> 838289 T2769 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:36192_xp_ah_collection1",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:36192_xp_ah",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:36192/xp_ah"}
[junit4:junit4]   2> 838312 T2768 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 838313 T2804 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 838313 T2788 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 838312 T2837 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 838312 T2820 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 838312 T2775 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 838681 T2794 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:19607_xp_ah_collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4]   2> 838681 T2794 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:19607_xp_ah_collection1&state=recovering&nodeName=127.0.0.1:19607_xp_ah&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1002 
[junit4:junit4]   2> 838683 T2746 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 839685 T2746 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> ASYNC  NEW_CORE C1679 name=collection1 org.apache.solr.core.SolrCore@1ae4d52 url=http://127.0.0.1:19607/xp_ah/collection1 node=127.0.0.1:19607_xp_ah C1679_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:19607_xp_ah, base_url=http://127.0.0.1:19607/xp_ah}
[junit4:junit4]   2> 840683 T2840 C1679 P19607 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:63735/xp_ah/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 840683 T2840 C1679 P19607 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:19607/xp_ah START replicas=[http://127.0.0.1:63735/xp_ah/collection1/] nUpdates=100
[junit4:junit4]   2> 840684 T2840 C1679 P19607 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 840684 T2840 C1679 P19607 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 840684 T2840 C1679 P19607 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 840684 T2840 C1679 P19607 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 840685 T2840 C1679 P19607 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 840685 T2840 C1679 P19607 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:63735/xp_ah/collection1/. core=collection1
[junit4:junit4]   2> 840685 T2840 C1679 P19607 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 840687 T2746 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> ASYNC  NEW_CORE C1680 name=collection1 org.apache.solr.core.SolrCore@f4eb84e url=http://127.0.0.1:63735/xp_ah/collection1 node=127.0.0.1:63735_xp_ah C1680_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:63735_xp_ah, base_url=http://127.0.0.1:63735/xp_ah, leader=true}
[junit4:junit4]   2> 840693 T2795 C1680 P63735 oasc.SolrCore.execute [collection1] webapp=/xp_ah path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1 
[junit4:junit4]   2> 840698 T2796 C1680 P63735 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 840700 T2796 C1680 P63735 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5ba0731b lockFactory=org.apache.lucene.store.NativeFSLockFactory@25e08757),segFN=segments_1,generation=1}
[junit4:junit4]   2> 840701 T2796 C1680 P63735 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 840702 T2796 C1680 P63735 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5ba0731b lockFactory=org.apache.lucene.store.NativeFSLockFactory@25e08757),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5ba0731b lockFactory=org.apache.lucene.store.NativeFSLockFactory@25e08757),segFN=segments_2,generation=2}
[junit4:junit4]   2> 840702 T2796 C1680 P63735 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 840703 T2796 C1680 P63735 oass.SolrIndexSearcher.<init> Opening Searcher@1fd149ab realtime
[junit4:junit4]   2> 840703 T2796 C1680 P63735 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 840704 T2796 C1680 P63735 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4]   2> 840704 T2840 C1679 P19607 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 840705 T2840 C1679 P19607 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 840707 T2797 C1680 P63735 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 840707 T2797 C1680 P63735 oasc.SolrCore.execute [collection1] webapp=/xp_ah path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 840708 T2840 C1679 P19607 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 840708 T2840 C1679 P19607 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 840708 T2840 C1679 P19607 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 840711 T2797 C1680 P63735 oasc.SolrCore.execute [collection1] webapp=/xp_ah path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0 
[junit4:junit4]   2> 840712 T2840 C1679 P19607 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 840713 T2840 C1679 P19607 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/jetty4/index.20130609084514114
[junit4:junit4]   2> 840713 T2840 C1679 P19607 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@6349336a lockFactory=org.apache.lucene.store.NativeFSLockFactory@23632c24) fullCopy=false
[junit4:junit4]   2> 840716 T2797 C1680 P63735 oasc.SolrCore.execute [collection1] webapp=/xp_ah path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 840717 T2840 C1679 P19607 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 840718 T2840 C1679 P19607 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 840718 T2840 C1679 P19607 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 840720 T2840 C1679 P19607 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@64e7405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@62ca97ee),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@64e7405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@62ca97ee),segFN=segments_2,generation=2}
[junit4:junit4]   2> 840720 T2840 C1679 P19607 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 840721 T2840 C1679 P19607 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 840721 T2840 C1679 P19607 oass.SolrIndexSearcher.<init> Opening Searcher@61d2876c main
[junit4:junit4]   2> 840722 T2839 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@61d2876c main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 840723 T2840 C1679 P19607 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/jetty4/index.20130609084514114 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/jetty4/index.20130609084514114;done=true>>]
[junit4:junit4]   2> 840723 T2840 C1679 P19607 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/jetty4/index.20130609084514114
[junit4:junit4]   2> 840723 T2840 C1679 P19607 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1370724290915/jetty4/index.20130609084514114
[junit4:junit4]   2> 840724 T2840 C1679 P19607 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 840724 T2840 C1679 P19607 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 840724 T2840 C1679 P19607 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 840724 T2840 C1679 P19607 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 840728 T2840 C1679 P19607 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 841328 T2769 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 841329 T2769 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:19607_xp_ah_collection1",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:19607_xp_ah",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:19607/xp_ah"}
[junit4:junit4]   2> 841369 T2775 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 841369 T2788 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 841369 T2768 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 841369 T2820 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 841369 T2837 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 841369 T2804 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 841689 T2746 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 841690 T2746 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4]   2> ASYNC  NEW_CORE C1681 name=collection1 org.apache.solr.core.SolrCore@25e71a51 url=http://127.0.0.1:43889/xp_ah/collection1 node=127.0.0.1:43889_xp_ah C1681_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:43889_xp_ah, base_url=http://127.0.0.1:43889/xp_ah, leader=true}
[junit4:junit4]   2> 841699 T2758 C1681 P43889 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 841703 T2758 C1681 P43889 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6b281ea7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@23f30a9c),segFN=segments_1,generation=1}
[junit4:junit4]   2> 841703 T2758 C1681 P43889 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 841704 T2758 C1681 P43889 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6b281ea7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@23f30a9c),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6b281ea7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@23f30a9c),segFN=segments_2,generation=2}
[junit4:junit4]   2> 841705 T2758 C1681 P43889 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 841705 T2758 C1681 P43889 oass.SolrIndexSearcher.<init> Opening Searcher@75c3c6be main
[junit4:junit4]   2> 841706 T2758 C1681 P43889 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 841706 T2772 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@75c3c6be main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 841707 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 8
[junit4:junit4]   2> ASYNC  NEW_CORE C1682 name=collection1 org.apache.solr.core.SolrCore@1f815fc url=http://127.0.0.1:54300/xp_ah/collection1 node=127.0.0.1:54300_xp_ah C1682_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:54300_xp_ah, base_url=http://127.0.0.1:54300/xp_ah, leader=true}
[junit4:junit4]   2> 841720 T2782 C1682 P54300 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 841721 T2782 C1682 P54300 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@60e024ad lockFactory=org.apache.lucene.store.NativeFSLockFactory@1781716d),segFN=segments_2,generation=2}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@60e024ad lockFactory=org.apache.lucene.store.NativeFSLockFactory@1781716d),segFN=segments_3,generation=3}
[junit4:junit4]   2> 841722 T2782 C1682 P54300 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4]   2> 841722 T2782 C1682 P54300 oass.SolrIndexSearcher.<init> Opening Searcher@50ad5eb2 main
[junit4:junit4]   2> 841723 T2782 C1682 P54300 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 841723 T2790 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@50ad5eb2 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 841724 T2782 C1682 P54300 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:36192/xp_ah/collection1/, StdNode: http://127.0.0.1:63735/xp_ah/collection1/, StdNode: http://127.0.0.1:19607/xp_ah/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4]   2> 841727 T2798 C1680 P63735 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 841728 T2798 C1680 P63735 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5ba0731b lockFactory=org.apache.lucene.store.NativeFSLockFactory@25e08757),segFN=segments_2,generation=2}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5ba0731b lockFactory=org.apache.lucene.store.NativeFSLockFactory@25e08757),segFN=segments_3,generation=3}
[junit4:junit4]   2> 841729 T2798 C1680 P63735 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4]   2> 841729 T2798 C1680 P63735 oass.SolrIndexSearcher.<init> Opening Searcher@6c64a9fb main
[junit4:junit4]   2> 841730 T2798 C1680 P63735 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 841731 T2806 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6c64a9fb main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 841731 T2798 C1680 P63735 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 4
[junit4:junit4]   2>  C1678_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:36192_xp_ah, base_url=http://127.0.0.1:36192/xp_ah}
[junit4:junit4]   2> 841734 T2810 C1678 P36192 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2>  C1679_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:19607_xp_ah, base_url=http://127.0.0.1:19607/xp_ah}
[junit4:junit4]   2> 841734 T2827 C1679 P19607 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 841735 T2810 C1678 P36192 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@25168fea lockFactory=org.apache.lucene.store.NativeFSLockFactory@551d59a0),segFN=segments_2,generation=2}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@25168fea lockFactory=org.apache.lucene.store.NativeFSLockFactory@551d59a0),segFN=segments_3,generation=3}
[junit4:junit4]   2> 841736 T2827 C1679 P19607 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@64e7405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@62ca97ee),segFN=segments_2,generation=2}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@64e7405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@62ca97ee),segFN=segments_3,generation=3}
[junit4:junit4]   2> 841736 T2810 C1678 P36192 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4]   2> 841737 T2827 C1679 P19607 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4]   2> 841738 T2810 C1678 P36192 oass.SolrIndexSearcher.<init> Opening Searcher@7f16fca7 main
[junit4:junit4]   2> 841738 T2827 C1679 P19607 oass.SolrIndexSearcher.<init> Opening Searcher@73f44107 main
[junit4:junit4]   2> 841739 T2810 C1678 P36192 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 841739 T2822 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7f16fca7 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 841739 T2827 C1679 P19607 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 841740 T2810 C1678 P36192 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4]   2> 841740 T2839 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@73f44107 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 841741 T2827 C1679 P19607 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4]   2> 841742 T2782 C1682 P54300 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 22
[junit4:junit4]   2> 841743 T2746 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 841746 T2783 C1682 P54300 oasc.SolrCore.execute [collection1] webapp=/xp_ah path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 841748 T2811 C1678 P36192 oasc.SolrCore.execute [collection1] webapp=/xp_ah path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 841749 T2799 C1680 P63735 oasc.SolrCore.execute [collection1] webapp=/xp_ah path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0 
[junit4:junit4]   2> 841751 T2828 C1679 P19607 oasc.SolrCore.execute [collection1] webapp=/xp_ah path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0 
[junit4:junit4]   2> 843757 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1437308621587218432)} 0 2
[junit4:junit4]   2> 843766 T2810 C1678 P36192 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={update.distrib=FROMLEADER&_version_=-1437308621592461312&update.from=http://127.0.0.1:54300/xp_ah/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1437308621592461312)} 0 2
[junit4:junit4]   2> 843770 T2829 C1679 P19607 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={update.distrib=FROMLEADER&_version_=-1437308621596655616&update.from=http://127.0.0.1:63735/xp_ah/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1437308621596655616)} 0 2
[junit4:junit4]   2> 843770 T2798 C1680 P63735 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1437308621596655616)} 0 6
[junit4:junit4]   2> 843771 T2782 C1682 P54300 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1437308621592461312)} 0 11
[junit4:junit4]   2> 843775 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[0 (1437308621607141376)]} 0 1
[junit4:junit4]   2> 843781 T2810 C1678 P36192 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:54300/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1437308621610287104)]} 0 1
[junit4:junit4]   2> 843781 T2782 C1682 P54300 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[0 (1437308621610287104)]} 0 4
[junit4:junit4]   2> 843784 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[1 (1437308621617627136)]} 0 0
[junit4:junit4]   2> 843790 T2810 C1678 P36192 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:54300/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1437308621620772864)]} 0 0
[junit4:junit4]   2> 843791 T2784 C1682 P54300 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:63735/xp_ah/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1437308621620772864)]} 0 4
[junit4:junit4]   2> 843792 T2800 C1680 P63735 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[1]} 0 6
[junit4:junit4]   2> 843794 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[2 (1437308621628112896)]} 0 0
[junit4:junit4]   2> 843803 T2829 C1679 P19607 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:63735/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1437308621633355776)]} 0 1
[junit4:junit4]   2> 843803 T2798 C1680 P63735 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:54300/xp_ah/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[2 (1437308621633355776)]} 0 4
[junit4:junit4]   2> 843804 T2782 C1682 P54300 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[2]} 0 7
[junit4:junit4]   2> 843806 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[3 (1437308621640695808)]} 0 0
[junit4:junit4]   2> 843811 T2829 C1679 P19607 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:63735/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1437308621643841536)]} 0 0
[junit4:junit4]   2> 843812 T2800 C1680 P63735 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[3 (1437308621643841536)]} 0 3
[junit4:junit4]   2> 843815 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[4 (1437308621650132992)]} 0 0
[junit4:junit4]   2> 843822 T2810 C1678 P36192 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:54300/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1437308621655375872)]} 0 0
[junit4:junit4]   2> 843823 T2784 C1682 P54300 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:63735/xp_ah/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[4 (1437308621655375872)]} 0 3
[junit4:junit4]   2> 843824 T2800 C1680 P63735 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[4]} 0 6
[junit4:junit4]   2> 843826 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[5 (1437308621661667328)]} 0 0
[junit4:junit4]   2> 843833 T2829 C1679 P19607 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:63735/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[5 (1437308621666910208)]} 0 0
[junit4:junit4]   2> 843834 T2798 C1680 P63735 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:54300/xp_ah/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[5 (1437308621666910208)]} 0 3
[junit4:junit4]   2> 843835 T2782 C1682 P54300 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[5]} 0 6
[junit4:junit4]   2> 843837 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[6 (1437308621673201664)]} 0 0
[junit4:junit4]   2> 843842 T2829 C1679 P19607 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:63735/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[6 (1437308621676347392)]} 0 0
[junit4:junit4]   2> 843843 T2800 C1680 P63735 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[6 (1437308621676347392)]} 0 3
[junit4:junit4]   2> 843846 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[7 (1437308621682638848)]} 0 0
[junit4:junit4]   2> 843853 T2829 C1679 P19607 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:63735/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[7 (1437308621687881728)]} 0 0
[junit4:junit4]   2> 843854 T2798 C1680 P63735 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:54300/xp_ah/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[7 (1437308621687881728)]} 0 3
[junit4:junit4]   2> 843855 T2782 C1682 P54300 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[7]} 0 6
[junit4:junit4]   2> 843857 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[8 (1437308621694173184)]} 0 0
[junit4:junit4]   2> 843864 T2810 C1678 P36192 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:54300/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1437308621699416064)]} 0 0
[junit4:junit4]   2> 843865 T2784 C1682 P54300 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:63735/xp_ah/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[8 (1437308621699416064)]} 0 3
[junit4:junit4]   2> 843866 T2800 C1680 P63735 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[8]} 0 6
[junit4:junit4]   2> 843868 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[9 (1437308621705707520)]} 0 0
[junit4:junit4]   2> 843873 T2829 C1679 P19607 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:63735/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[9 (1437308621708853248)]} 0 0
[junit4:junit4]   2> 843874 T2800 C1680 P63735 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[9 (1437308621708853248)]} 0 3
[junit4:junit4]   2> 843877 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[10 (1437308621715144704)]} 0 0
[junit4:junit4]   2> 843882 T2810 C1678 P36192 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:54300/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10 (1437308621718290432)]} 0 0
[junit4:junit4]   2> 843883 T2782 C1682 P54300 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[10 (1437308621718290432)]} 0 3
[junit4:junit4]   2> 843886 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[11 (1437308621724581888)]} 0 0
[junit4:junit4]   2> 843891 T2810 C1678 P36192 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:54300/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[11 (1437308621727727616)]} 0 0
[junit4:junit4]   2> 843892 T2782 C1682 P54300 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[11 (1437308621727727616)]} 0 3
[junit4:junit4]   2> 843895 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[12 (1437308621734019072)]} 0 0
[junit4:junit4]   2> 843900 T2810 C1678 P36192 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:54300/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[12 (1437308621737164800)]} 0 0
[junit4:junit4]   2> 843901 T2782 C1682 P54300 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[12 (1437308621737164800)]} 0 3
[junit4:junit4]   2> 843904 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[13 (1437308621743456256)]} 0 0
[junit4:junit4]   2> 843912 T2810 C1678 P36192 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:54300/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[13 (1437308621748699136)]} 0 0
[junit4:junit4]   2> 843913 T2784 C1682 P54300 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:63735/xp_ah/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[13 (1437308621748699136)]} 0 4
[junit4:junit4]   2> 843914 T2800 C1680 P63735 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[13]} 0 7
[junit4:junit4]   2> 843916 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[14 (1437308621756039168)]} 0 0
[junit4:junit4]   2> 843921 T2810 C1678 P36192 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:54300/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[14 (1437308621759184896)]} 0 0
[junit4:junit4]   2> 843922 T2782 C1682 P54300 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[14 (1437308621759184896)]} 0 3
[junit4:junit4]   2> 843925 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[15 (1437308621765476352)]} 0 0
[junit4:junit4]   2> 843932 T2810 C1678 P36192 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:54300/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[15 (1437308621770719232)]} 0 0
[junit4:junit4]   2> 843933 T2784 C1682 P54300 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:63735/xp_ah/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[15 (1437308621770719232)]} 0 3
[junit4:junit4]   2> 843934 T2800 C1680 P63735 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[15]} 0 6
[junit4:junit4]   2> 843936 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[16 (1437308621777010688)]} 0 0
[junit4:junit4]   2> 843941 T2810 C1678 P36192 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:54300/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[16 (1437308621780156416)]} 0 0
[junit4:junit4]   2> 843942 T2782 C1682 P54300 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[16 (1437308621780156416)]} 0 3
[junit4:junit4]   2> 843945 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[17 (1437308621786447872)]} 0 0
[junit4:junit4]   2> 843953 T2829 C1679 P19607 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:63735/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[17 (1437308621791690752)]} 0 0
[junit4:junit4]   2> 843954 T2798 C1680 P63735 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:54300/xp_ah/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[17 (1437308621791690752)]} 0 4
[junit4:junit4]   2> 843955 T2782 C1682 P54300 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[17]} 0 7
[junit4:junit4]   2> 843957 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[18 (1437308621799030784)]} 0 0
[junit4:junit4]   2> 843962 T2829 C1679 P19607 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:63735/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[18 (1437308621802176512)]} 0 0
[junit4:junit4]   2> 843963 T2800 C1680 P63735 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[18 (1437308621802176512)]} 0 3
[junit4:junit4]   2> 843966 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[19 (1437308621808467968)]} 0 0
[junit4:junit4]   2> 843973 T2829 C1679 P19607 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:63735/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[19 (1437308621813710848)]} 0 0
[junit4:junit4]   2> 843974 T2798 C1680 P63735 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:54300/xp_ah/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[19 (1437308621813710848)]} 0 3
[junit4:junit4]   2> 843975 T2782 C1682 P54300 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[19]} 0 6
[junit4:junit4]   2> 843977 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[20 (1437308621820002304)]} 0 0
[junit4:junit4]   2> 843982 T2810 C1678 P36192 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:54300/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[20 (1437308621823148032)]} 0 0
[junit4:junit4]   2> 843983 T2782 C1682 P54300 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[20 (1437308621823148032)]} 0 3
[junit4:junit4]   2> 843986 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[21 (1437308621829439488)]} 0 0
[junit4:junit4]   2> 843994 T2829 C1679 P19607 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:63735/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[21 (1437308621834682368)]} 0 0
[junit4:junit4]   2> 843995 T2798 C1680 P63735 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:54300/xp_ah/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[21 (1437308621834682368)]} 0 4
[junit4:junit4]   2> 843995 T2782 C1682 P54300 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[21]} 0 6
[junit4:junit4]   2> 843998 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[22 (1437308621842022400)]} 0 0
[junit4:junit4]   2> 844003 T2810 C1678 P36192 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:54300/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[22 (1437308621845168128)]} 0 0
[junit4:junit4]   2> 844004 T2782 C1682 P54300 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[22 (1437308621845168128)]} 0 3
[junit4:junit4]   2> 844007 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[23 (1437308621851459584)]} 0 0
[junit4:junit4]   2> 844012 T2829 C1679 P19607 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:63735/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[23 (1437308621854605312)]} 0 0
[junit4:junit4]   2> 844013 T2800 C1680 P63735 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[23 (1437308621854605312)]} 0 3
[junit4:junit4]   2> 844016 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[24 (1437308621860896768)]} 0 0
[junit4:junit4]   2> 844023 T2810 C1678 P36192 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:54300/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[24 (1437308621866139648)]} 0 0
[junit4:junit4]   2> 844024 T2784 C1682 P54300 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:63735/xp_ah/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[24 (1437308621866139648)]} 0 3
[junit4:junit4]   2> 844024 T2800 C1680 P63735 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[24]} 0 5
[junit4:junit4]   2> 844027 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[25 (1437308621872431104)]} 0 0
[junit4:junit4]   2> 844032 T2829 C1679 P19607 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:63735/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[25 (1437308621875576832)]} 0 0
[junit4:junit4]   2> 844033 T2800 C1680 P63735 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[25 (1437308621875576832)]} 0 3
[junit4:junit4]   2> 844036 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[26 (1437308621881868288)]} 0 0
[junit4:junit4]   2> 844041 T2829 C1679 P19607 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:63735/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[26 (1437308621885014016)]} 0 0
[junit4:junit4]   2> 844042 T2800 C1680 P63735 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[26 (1437308621885014016)]} 0 3
[junit4:junit4]   2> 844043 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[27 (1437308621889208320)]} 0 0
[junit4:junit4]   2> 844050 T2810 C1678 P36192 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:54300/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[27 (1437308621894451200)]} 0 0
[junit4:junit4]   2> 844051 T2784 C1682 P54300 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:63735/xp_ah/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[27 (1437308621894451200)]} 0 3
[junit4:junit4]   2> 844051 T2800 C1680 P63735 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[27]} 0 5
[junit4:junit4]   2> 844054 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[28 (1437308621900742656)]} 0 0
[junit4:junit4]   2> 844061 T2810 C1678 P36192 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:54300/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[28 (1437308621905985536)]} 0 0
[junit4:junit4]   2> 844062 T2784 C1682 P54300 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:63735/xp_ah/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[28 (1437308621905985536)]} 0 3
[junit4:junit4]   2> 844063 T2800 C1680 P63735 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[28]} 0 6
[junit4:junit4]   2> 844065 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[29 (1437308621912276992)]} 0 0
[junit4:junit4]   2> 844070 T2829 C1679 P19607 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:63735/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[29 (1437308621915422720)]} 0 0
[junit4:junit4]   2> 844071 T2800 C1680 P63735 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[29 (1437308621915422720)]} 0 3
[junit4:junit4]   2> 844074 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[30 (1437308621921714176)]} 0 0
[junit4:junit4]   2> 844079 T2829 C1679 P19607 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:63735/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[30 (1437308621924859904)]} 0 0
[junit4:junit4]   2> 844080 T2800 C1680 P63735 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[30 (1437308621924859904)]} 0 3
[junit4:junit4]   2> 844083 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[31 (1437308621931151360)]} 0 0
[junit4:junit4]   2> 844091 T2829 C1679 P19607 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:63735/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[31 (1437308621936394240)]} 0 0
[junit4:junit4]   2> 844092 T2798 C1680 P63735 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:54300/xp_ah/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[31 (1437308621936394240)]} 0 4
[junit4:junit4]   2> 844093 T2782 C1682 P54300 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[31]} 0 7
[junit4:junit4]   2> 844095 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[32 (1437308621943734272)]} 0 0
[junit4:junit4]   2> 844102 T2810 C1678 P36192 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:54300/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[32 (1437308621948977152)]} 0 0
[junit4:junit4]   2> 844103 T2784 C1682 P54300 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:63735/xp_ah/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[32 (1437308621948977152)]} 0 3
[junit4:junit4]   2> 844103 T2800 C1680 P63735 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[32]} 0 5
[junit4:junit4]   2> 844106 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[33 (1437308621955268608)]} 0 0
[junit4:junit4]   2> 844111 T2829 C1679 P19607 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:63735/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[33 (1437308621958414336)]} 0 0
[junit4:junit4]   2> 844112 T2800 C1680 P63735 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[33 (1437308621958414336)]} 0 3
[junit4:junit4]   2> 844115 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[34 (1437308621964705792)]} 0 0
[junit4:junit4]   2> 844122 T2810 C1678 P36192 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:54300/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[34 (1437308621969948672)]} 0 0
[junit4:junit4]   2> 844123 T2784 C1682 P54300 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:63735/xp_ah/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[34 (1437308621969948672)]} 0 3
[junit4:junit4]   2> 844124 T2800 C1680 P63735 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[34]} 0 6
[junit4:junit4]   2> 844126 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[35 (1437308621976240128)]} 0 0
[junit4:junit4]   2> 844131 T2810 C1678 P36192 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:54300/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[35 (1437308621979385856)]} 0 0
[junit4:junit4]   2> 844132 T2782 C1682 P54300 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[35 (1437308621979385856)]} 0 3
[junit4:junit4]   2> 844135 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[36 (1437308621985677312)]} 0 0
[junit4:junit4]   2> 844142 T2810 C1678 P36192 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:54300/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[36 (1437308621990920192)]} 0 0
[junit4:junit4]   2> 844143 T2784 C1682 P54300 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:63735/xp_ah/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[36 (1437308621990920192)]} 0 3
[junit4:junit4]   2> 844143 T2800 C1680 P63735 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[36]} 0 5
[junit4:junit4]   2> 844146 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[37 (1437308621997211648)]} 0 0
[junit4:junit4]   2> 844151 T2829 C1679 P19607 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={distrib.from=http://127.0.0.1:63735/xp_ah/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[37 (1437308622000357376)]} 0 0
[junit4:junit4]   2> 844152 T2800 C1680 P63735 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/update params={wt=javabin&version=2} {add=[37 (1437308622000357376)]} 0 3
[junit4:junit4]   2> 844155 T2758 C1681 P43889 oasup.LogUpdateProcessor.finish [collection1] webapp=/xp_ah path=/upd

[...truncated too long message...]

ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4:junit4]    >         at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4:junit4]    >         at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4:junit4]    >         at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4]    >         at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4]    >         at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4]    >         at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4:junit4]    >         at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4]    >         at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:298)
[junit4:junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    2) Thread[id=2598, name=recoveryCmdExecutor-1089-thread-2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4:junit4]    >         at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4:junit4]    >         at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4:junit4]    >         at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4:junit4]    >         at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4:junit4]    >         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4:junit4]    >         at java.net.Socket.connect(Socket.java:579)
[junit4:junit4]    >         at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4:junit4]    >         at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4:junit4]    >         at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4:junit4]    >         at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4:junit4]    >         at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4:junit4]    >         at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4]    >         at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4]    >         at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4]    >         at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4:junit4]    >         at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4]    >         at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:298)
[junit4:junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([EC4264284E50992E]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4:junit4]    >    1) Thread[id=2596, name=recoveryCmdExecutor-1089-thread-1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4:junit4]    >         at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4:junit4]    >         at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4:junit4]    >         at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4:junit4]    >         at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4:junit4]    >         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4:junit4]    >         at java.net.Socket.connect(Socket.java:579)
[junit4:junit4]    >         at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4:junit4]    >         at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4:junit4]    >         at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4:junit4]    >         at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4:junit4]    >         at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4:junit4]    >         at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4]    >         at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4]    >         at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4]    >         at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4:junit4]    >         at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4]    >         at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:298)
[junit4:junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    2) Thread[id=2598, name=recoveryCmdExecutor-1089-thread-2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4:junit4]    >         at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4:junit4]    >         at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4:junit4]    >         at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4:junit4]    >         at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4:junit4]    >         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4:junit4]    >         at java.net.Socket.connect(Socket.java:579)
[junit4:junit4]    >         at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4:junit4]    >         at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4:junit4]    >         at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4:junit4]    >         at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4:junit4]    >         at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4:junit4]    >         at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4]    >         at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4]    >         at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4]    >         at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4:junit4]    >         at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4]    >         at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:298)
[junit4:junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([EC4264284E50992E]:0)
[junit4:junit4] Completed on J0 in 351.24s, 1 test, 1 failure, 2 errors <<< FAILURES!

[...truncated 426 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:391: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:371: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:181: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/common-build.xml:437: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:1243: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:887: There were test failures: 297 suites, 1246 tests, 2 suite-level errors, 1 error, 1 failure, 499 ignored (7 assumptions)

Total time: 53 minutes 14 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure