You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by to...@r322.com on 2011/06/09 22:25:53 UTC

pauses on simple .html pages

 Hello:

 We're having problems with tomcat when it's running solr on a Sun JVM. 
 I'm stuck and hoping for some direction.

 /opt/tomcat6/bin/version.sh
 Using CATALINA_BASE:   /opt/tomcat6
 Using CATALINA_HOME:   /opt/tomcat6
 Using CATALINA_TMPDIR: /opt/tomcat6/temp
 Using JRE_HOME:       /usr/lib/jvm/java-6-sun/jre
 Server version: Apache Tomcat/6.0.20
 Server built:   May 14 2009 01:13:50
 Server number:  6.0.20.0
 OS Name:        Linux
 OS Version:     2.6.31-14-server
 Architecture:   amd64
 JVM Version:    1.6.0_22-b04
 JVM Vendor:     Sun Microsystems Inc.


 I have a script setup does a wget hit to docs/config/valve.html every 1 
 second. If the wget call takes longer than 1 second it grabs a bunch of 
 stats -- iostat, vmstat, top, jstack, last 30 lines of jvm.log.

 A lot of these are caused by GC. However, there are a lot that aren't 
 and I am having a hardtime finding a correlation between the stats and 
 these timeouts. Looking at the stats I have, does anyone see anything? 
 Ideas on more stats I can grab?

 Here is the options I'm passing to the jvm:

 /usr/lib/jvm/java-6-sun/jre/bin/java 
 -Djava.util.logging.config.file=/opt/tomcat6/conf/logging.properties 
 -Dsolr.solr.home=/opt/solr 
 -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager 
 -Xms28G -Xmx28G -XX:+UseConcMarkSweepGC -XX:+UseParNewGC 
 -XX:+CMSParallelRemarkEnabled -XX:NewRatio=8 
 -XX:CMSInitiatingOccupancyFraction=70 -verbose:gc -XX:+PrintGCTimeStamps 
 -XX:+PrintGCDetails -Xloggc:/home/solr/jvm.log 
 -Djava.endorsed.dirs=/opt/tomcat6/endorsed -classpath 
 :/opt/tomcat6/bin/bootstrap.jar -Dcatalina.base=/opt/tomcat6 
 -Dcatalina.home=/opt/tomcat6 -Djava.io.tmpdir=/opt/tomcat6/temp 
 org.apache.catalina.startup.Bootstrap start


 Here is the output of my script.


 =============================================================
 tomcat failed on Thu Jun  9 16:01:06 EDT 2011. Response was 5 seconds 
 and returned 200.
 
 ========= iostat =========
 Linux 2.6.31-14-server 	06/09/11 	_x86_64_	(4 CPU)

 avg-cpu:  %user   %nice %system %iowait  %steal   %idle
            6.06    0.00    0.80   12.19    0.00   80.95

 Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s 
 avgrq-sz avgqu-sz   await  svctm  %util
 sdb               0.09     0.35    0.01    0.01     0.83     2.83   
 187.39     0.00   33.52  22.99   0.04
 sda               2.37     8.85    1.04    1.89    49.05    85.86    
 46.16     0.04   12.65   5.66   1.66

 avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           25.32    0.00    2.53   35.44    0.00   36.71

 Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s 
 avgrq-sz avgqu-sz   await  svctm  %util
 sdb               0.00     0.00    0.00    0.00     0.00     0.00     
 0.00     0.00    0.00   0.00   0.00
 sda               7.00     4.00    1.00    6.00    64.00    80.00    
 20.57     0.06    8.57   8.57   6.00

 avg-cpu:  %user   %nice %system %iowait  %steal   %idle
            4.20    0.00    0.56   33.05    0.00   62.18

 Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s 
 avgrq-sz avgqu-sz   await  svctm  %util
 sdb               0.00     0.00    0.00    0.00     0.00     0.00     
 0.00     0.00    0.00   0.00   0.00
 sda               0.00     9.00    0.00    3.00     0.00    96.00    
 32.00     0.02    6.67   3.33   1.00
 
 ========= vmstat =========
 procs -----------memory---------- ---swap-- -----io---- -system-- 
 ----cpu----
  r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy 
 id wa
  1  1 419564 198156  11016 12654716    3    3     6    11    4    2  6  
 1 81 12
  0  1 419564 204696  11008 12647248    0    0     0     0 1353  686  4  
 1 83 13
  1  2 419564 201752  11008 12650916   36    0    36     0 1648 1050  6  
 2 75 16
  0  0 419564 199908  11008 12652816   20    0    20     0 1062  630  6  
 1 86  8
 
 Top:
 top - 16:01:17 up 15 days,  5:49,  1 user,  load average: 1.66, 1.55, 
 1.51
 Tasks: 154 total,   1 running, 153 sleeping,   0 stopped,   0 zombie
 Cpu(s):  6.1%us,  0.5%sy,  0.0%ni, 81.0%id, 12.2%wa,  0.0%hi,  0.3%si,  
 0.0%st
 Mem:  41292720k total, 41092688k used,   200032k free,    11008k 
 buffers
 Swap: 18723708k total,   419564k used, 18304144k free, 12653120k cached

   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
   912 avahi     20   0 31924 1376 1128 S    2  0.0   0:48.54 
 avahi-daemon
  7146 root      20   0 31.1g  26g 5256 S    2 66.2 298:49.06 java
     1 root      20   0 19428 1468 1032 S    0  0.0   0:01.71 init
     2 root      15  -5     0    0    0 S    0  0.0   0:00.00 kthreadd
     3 root      RT  -5     0    0    0 S    0  0.0   0:15.51 
 migration/0
     4 root      15  -5     0    0    0 S    0  0.0   0:04.81 
 ksoftirqd/0
     5 root      RT  -5     0    0    0 S    0  0.0   0:00.00 watchdog/0
     6 root      RT  -5     0    0    0 S    0  0.0   0:20.13 
 migration/1
     7 root      15  -5     0    0    0 S    0  0.0   0:00.30 
 ksoftirqd/1
     8 root      RT  -5     0    0    0 S    0  0.0   0:00.00 watchdog/1
     9 root      RT  -5     0    0    0 S    0  0.0   0:23.71 
 migration/2
    10 root      15  -5     0    0    0 S    0  0.0   0:00.35 
 ksoftirqd/2
    11 root      RT  -5     0    0    0 S    0  0.0   0:00.00 watchdog/2
    12 root      RT  -5     0    0    0 S    0  0.0   0:21.48 
 migration/3
    13 root      15  -5     0    0    0 S    0  0.0   0:00.32 
 ksoftirqd/3
    14 root      RT  -5     0    0    0 S    0  0.0   0:00.00 watchdog/3
    15 root      15  -5     0    0    0 S    0  0.0   0:01.90 events/0
    16 root      15  -5     0    0    0 S    0  0.0   0:00.19 events/1
    17 root      15  -5     0    0    0 S    0  0.0   0:00.27 events/2
    18 root      15  -5     0    0    0 S    0  0.0   0:00.29 events/3
    19 root      15  -5     0    0    0 S    0  0.0   0:00.00 cpuset
    20 root      15  -5     0    0    0 S    0  0.0   0:00.00 khelper
    21 root      15  -5     0    0    0 S    0  0.0   0:00.00 netns
    22 root      15  -5     0    0    0 S    0  0.0   0:00.00 async/mgr
    23 root      15  -5     0    0    0 S    0  0.0   0:00.00 
 kintegrityd/0
    24 root      15  -5     0    0    0 S    0  0.0   0:00.00 
 kintegrityd/1
    25 root      15  -5     0    0    0 S    0  0.0   0:00.00 
 kintegrityd/2
    26 root      15  -5     0    0    0 S    0  0.0   0:00.00 
 kintegrityd/3
    27 root      15  -5     0    0    0 S    0  0.0   0:32.20 kblockd/0
    28 root      15  -5     0    0    0 S    0  0.0   0:43.28 kblockd/1
    29 root      15  -5     0    0    0 S    0  0.0   1:00.28 kblockd/2
    30 root      15  -5     0    0    0 S    0  0.0   0:54.29 kblockd/3
    31 root      15  -5     0    0    0 S    0  0.0   0:00.00 kacpid
    32 root      15  -5     0    0    0 S    0  0.0   0:00.00 
 kacpi_notify
    33 root      15  -5     0    0    0 S    0  0.0   0:00.00 
 kacpi_hotplug
    34 root      15  -5     0    0    0 S    0  0.0   0:00.00 ata/0
    35 root      15  -5     0    0    0 S    0  0.0   0:00.01 ata/1
    36 root      15  -5     0    0    0 S    0  0.0   0:00.00 ata/2
    37 root      15  -5     0    0    0 S    0  0.0   0:00.00 ata/3
    38 root      15  -5     0    0    0 S    0  0.0   0:00.00 ata_aux
    39 root      15  -5     0    0    0 S    0  0.0   0:00.00 
 ksuspend_usbd
    40 root      15  -5     0    0    0 S    0  0.0   0:00.00 khubd
    41 root      15  -5     0    0    0 S    0  0.0   0:00.00 kseriod
    42 root      15  -5     0    0    0 S    0  0.0   0:00.00 kmmcd
    43 root      15  -5     0    0    0 S    0  0.0   0:00.00 bluetooth
    44 root      20   0     0    0    0 S    0  0.0   0:00.04 khungtaskd
    45 root      20   0     0    0    0 S    0  0.0   0:00.00 pdflush
    46 root      20   0     0    0    0 S    0  0.0   5:25.19 pdflush
    47 root      15  -5     0    0    0 S    0  0.0  11:20.40 kswapd0
    48 root      15  -5     0    0    0 S    0  0.0   0:00.00 aio/0
    49 root      15  -5     0    0    0 S    0  0.0   0:00.00 aio/1
    50 root      15  -5     0    0    0 S    0  0.0   0:00.00 aio/2
    51 root      15  -5     0    0    0 S    0  0.0   0:00.00 aio/3
    52 root      15  -5     0    0    0 S    0  0.0   0:00.00 
 ecryptfs-kthrea
    53 root      15  -5     0    0    0 S    0  0.0   0:00.00 crypto/0
    54 root      15  -5     0    0    0 S    0  0.0   0:00.00 crypto/1
    55 root      15  -5     0    0    0 S    0  0.0   0:00.00 crypto/2
    56 root      15  -5     0    0    0 S    0  0.0   0:00.00 crypto/3
    92 root      15  -5     0    0    0 S    0  0.0   0:00.00 pciehpd
    95 root      15  -5     0    0    0 S    0  0.0   0:00.00 scsi_eh_0
    96 root      15  -5     0    0    0 S    0  0.0   0:00.02 scsi_eh_1
    97 root      15  -5     0    0    0 S    0  0.0   0:00.00 kstriped
    98 root      15  -5     0    0    0 S    0  0.0   0:00.00 kmpathd/0
    99 root      15  -5     0    0    0 S    0  0.0   0:00.00 kmpathd/1
   100 root      15  -5     0    0    0 S    0  0.0   0:00.00 kmpathd/2
   101 root      15  -5     0    0    0 S    0  0.0   0:00.00 kmpathd/3
   102 root      15  -5     0    0    0 S    0  0.0   0:00.00 
 kmpath_handlerd
   103 root      15  -5     0    0    0 S    0  0.0   0:00.00 ksnapd
   104 root      15  -5     0    0    0 S    0  0.0   0:00.00 
 kondemand/0
   105 root      15  -5     0    0    0 S    0  0.0   0:00.00 
 kondemand/1
   106 root      15  -5     0    0    0 S    0  0.0   0:00.00 
 kondemand/2
   107 root      15  -5     0    0    0 S    0  0.0   0:00.00 
 kondemand/3
   108 root      15  -5     0    0    0 S    0  0.0   0:00.00 
 kconservative/0
   109 root      15  -5     0    0    0 S    0  0.0   0:00.00 
 kconservative/1
   110 root      15  -5     0    0    0 S    0  0.0   0:00.00 
 kconservative/2
   111 root      15  -5     0    0    0 S    0  0.0   0:00.00 
 kconservative/3
   112 root      10 -10     0    0    0 S    0  0.0   0:00.00 krfcommd
   323 root      15  -5     0    0    0 S    0  0.0   0:13.60 mpt_poll_0
   325 root      15  -5     0    0    0 S    0  0.0   0:00.00 mpt/0
   360 root      15  -5     0    0    0 S    0  0.0   0:00.00 scsi_eh_2
   456 root      15  -5     0    0    0 S    0  0.0   2:07.63 kjournald2
   504 root      20   0 12768  312  308 S    0  0.0   0:00.01 
 upstart-udev-br
   509 root      16  -4 16836  236  232 S    0  0.0   0:00.05 udevd
   730 root      15  -5     0    0    0 S    0  0.0   0:00.00 kpsmoused
   840 root      15  -5     0    0    0 S    0  0.0   0:00.07 kjournald
   842 root      15  -5     0    0    0 S    0  0.0   1:50.10 kjournald2
   865 root      20   0  8192  460  456 S    0  0.0   0:00.04 dd
   872 syslog    20   0  184m 1184  832 S    0  0.0   0:42.56 rsyslogd
   873 daemon    20   0  8164  396  392 S    0  0.0   0:00.00 portmap
   894 statd     20   0 10300  568  564 S    0  0.0   0:00.00 rpc.statd
   899 messageb  20   0 23460  752  564 S    0  0.0   0:00.31 
 dbus-daemon
   913 avahi     20   0 31760  236  220 S    0  0.0   0:00.00 
 avahi-daemon
  1006 root      20   0  5988  492  488 S    0  0.0   0:00.00 getty
  1010 root      20   0  5988  492  488 S    0  0.0   0:00.00 getty
  1014 root      20   0  5988  492  488 S    0  0.0   0:00.00 getty
  1015 root      20   0  5988  492  488 S    0  0.0   0:00.00 getty
  1017 root      20   0  5988  492  488 S    0  0.0   0:00.00 getty
  1030 root      20   0 18708  832  716 S    0  0.0   0:35.33 cron
  1031 daemon    20   0 16512  304  288 S    0  0.0   0:00.00 atd
  1083 root      18  -2 17060  216  212 S    0  0.0   0:00.00 udevd
  1084 root      18  -2 17060  216  212 S    0  0.0   0:00.00 udevd
  1164 root      20   0     0    0    0 S    0  0.0   0:01.46 vmmemctl
  1285 root      20   0 23008 1120  952 S    0  0.0   3:02.82 
 vmware-guestd
  1316 root      20   0 49072  724  612 S    0  0.0   0:00.24 sshd
  1397 root      20   0 73416  940  680 S    0  0.0   0:02.07 
 sendmail-mta
  1409 root      20   0  5988  492  488 S    0  0.0   0:00.00 getty
  1428 root      20   0  115m 1972 1376 S    0  0.0   0:03.06 
 console-kit-dae
  1775 root      15  -5     0    0    0 S    0  0.0  32:55.58 rpciod/0
  1776 root      15  -5     0    0    0 S    0  0.0   0:03.71 rpciod/1
  1777 root      15  -5     0    0    0 S    0  0.0   0:01.85 rpciod/2
  1778 root      15  -5     0    0    0 S    0  0.0   0:01.23 rpciod/3
  1779 root      15  -5     0    0    0 S    0  0.0   9:43.54 nfsiod
  1780 root      15  -5     0    0    0 S    0  0.0   0:00.00 lockd
 20545 root      20   0 83036 3428 2644 S    0  0.0   0:00.13 sshd
 20683 solr      20   0 83036 1900 1048 S    0  0.0   0:00.01 sshd
 20684 solr      20   0  4004  616  508 S    0  0.0   0:00.01 sh
 20704 root      20   0 47424 1712 1324 S    0  0.0   0:00.02 su
 20712 root      20   0 19380 2400 1572 S    0  0.0   0:00.07 bash
 25699 root      20   0 42044 1224  896 S    0  0.0   0:00.00 cron
 25701 root      20   0 42044 1224  896 S    0  0.0   0:00.00 cron
 25702 root      20   0  4004  544  456 S    0  0.0   0:00.00 sh
 25703 root      20   0  4004  540  456 S    0  0.0   0:00.00 sh
 25705 root      20   0  4004  296  204 S    0  0.0   0:00.00 sh
 25706 root      20   0  4004  300  204 S    0  0.0   0:00.00 sh
 25707 root      20   0  8168  592  488 S    0  0.0   0:00.00 sleep
 25708 root      20   0  8168  592  488 S    0  0.0   0:00.00 sleep
 25711 root      20   0 42044 1224  896 S    0  0.0   0:00.00 cron
 25712 root      20   0  4004  544  456 S    0  0.0   0:00.00 sh
 25713 root      20   0 42044 1224  896 S    0  0.0   0:00.00 cron
 25714 root      20   0 42044 1224  896 S    0  0.0   0:00.00 cron
 25715 root      20   0  4004  300  204 S    0  0.0   0:00.00 sh
 25716 root      20   0  8168  592  488 S    0  0.0   0:00.00 sleep
 25717 root      20   0  4004  548  456 S    0  0.0   0:00.00 sh
 25718 root      20   0  4004  544  456 S    0  0.0   0:00.00 sh
 25719 root      20   0 42044 1224  896 S    0  0.0   0:00.00 cron
 25720 root      20   0 42044 1224  896 S    0  0.0   0:00.01 cron
 25721 root      20   0  4004  300  204 S    0  0.0   0:00.00 sh
 25722 root      20   0  8168  592  488 S    0  0.0   0:00.00 sleep
 25723 root      20   0  4004  544  456 S    0  0.0   0:00.00 sh
 25724 root      20   0  4004  544  456 S    0  0.0   0:00.00 sh
 25726 root      20   0  4004  300  204 S    0  0.0   0:00.01 sh
 25727 root      20   0  8168  592  488 S    0  0.0   0:00.00 sleep
 25728 root      20   0  4004  300  204 S    0  0.0   0:00.00 sh
 25729 root      20   0  8168  596  488 S    0  0.0   0:00.00 sleep
 25732 root      20   0  4004  304  204 S    0  0.0   0:00.00 sh
 25733 root      20   0  8168  596  488 S    0  0.0   0:00.00 sleep
 25735 root      20   0 42044 1224  896 S    0  0.0   0:00.00 cron
 25738 root      20   0  4004  548  456 S    0  0.0   0:00.00 sh
 25740 root      20   0  9172 1332 1072 S    0  0.0   0:00.00 checker.sh
 25743 root      20   0 42044 1224  896 S    0  0.0   0:00.00 cron
 25746 root      20   0  4004  544  456 S    0  0.0   0:00.00 sh
 25747 root      20   0  9116 1296 1092 S    0  0.0   0:00.00 
 stat_mon_homes.
 25795 root      20   0  8168  592  488 S    0  0.0   0:00.00 sleep
 25823 root      20   0 19128 1240  876 R    0  0.0   0:00.01 top
 
 ========= jstack =========
 2011-06-09 16:01:21
 Full thread dump Java HotSpot(TM) 64-Bit Server VM (17.1-b03 mixed 
 mode):

 "Attach Listener" daemon prio=10 tid=0x0000000040d5b800 nid=0x4408 
 waiting on condition [0x0000000000000000]
    java.lang.Thread.State: RUNNABLE

    Locked ownable synchronizers:
 	- None

 "http-80-148" daemon prio=10 tid=0x00007f3abc46b000 nid=0x1dcb in 
 Object.wait() [0x00007f3aabb38000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c53919bb8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c53919bb8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-147" daemon prio=10 tid=0x00007f3abc469800 nid=0x1dca in 
 Object.wait() [0x00007f3aabc39000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c46016e10> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c46016e10> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-146" daemon prio=10 tid=0x00007f3abc468000 nid=0x1dc9 in 
 Object.wait() [0x00007f3aabd3a000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c41a1a720> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c41a1a720> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-145" daemon prio=10 tid=0x00007f3abc467000 nid=0x1d83 in 
 Object.wait() [0x00007f3aabe3b000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c46016e40> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c46016e40> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-144" daemon prio=10 tid=0x00007f3abc466000 nid=0x1d82 runnable 
 [0x00007f3aabf3c000]
    java.lang.Thread.State: RUNNABLE
 	at sun.nio.ch.FileDispatcher.pread0(Native Method)
 	at sun.nio.ch.FileDispatcher.pread(FileDispatcher.java:31)
 	at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:234)
 	at sun.nio.ch.IOUtil.read(IOUtil.java:210)
 	at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:622)
 	at 
 org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(NIOFSDirectory.java:161)
 	at 
 org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:222)
 	at 
 org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:39)
 	at org.apache.lucene.store.DataInput.readVInt(DataInput.java:94)
 	at 
 org.apache.lucene.store.BufferedIndexInput.readVInt(BufferedIndexInput.java:190)
 	at org.apache.lucene.index.FieldsReader.doc(FieldsReader.java:214)
 	at 
 org.apache.lucene.index.SegmentReader.document(SegmentReader.java:842)
 	at 
 org.apache.lucene.index.DirectoryReader.document(DirectoryReader.java:565)
 	at 
 org.apache.solr.search.SolrIndexSearcher.doc(SolrIndexSearcher.java:427)
 	at 
 org.apache.solr.response.TextResponseWriter.writeDocuments(TextResponseWriter.java:248)
 	at 
 org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:153)
 	at 
 org.apache.solr.response.JSONWriter.writeNamedListAsMapWithDups(JSONResponseWriter.java:175)
 	at 
 org.apache.solr.response.JSONWriter.writeNamedList(JSONResponseWriter.java:291)
 	at 
 org.apache.solr.response.JSONWriter.writeResponse(JSONResponseWriter.java:88)
 	at 
 org.apache.solr.response.JSONResponseWriter.write(JSONResponseWriter.java:52)
 	at 
 org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:340)
 	at 
 org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:261)
 	at 
 org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
 	at 
 org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
 	at 
 org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
 	at 
 org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
 	at 
 org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
 	at 
 org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
 	at 
 org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:567)
 	at 
 org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
 	at 
 org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
 	at 
 org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:849)
 	at 
 org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:454)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-143" daemon prio=10 tid=0x00007f3abc671000 nid=0x1d6f in 
 Object.wait() [0x00007f3aac03d000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c46016ea0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c46016ea0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-142" daemon prio=10 tid=0x00007f3abc887000 nid=0x1d6e in 
 Object.wait() [0x00007f3aac13e000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c46016ed0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c46016ed0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-141" daemon prio=10 tid=0x00007f3abc885000 nid=0x1d6d in 
 Object.wait() [0x00007f3aac23f000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c46016f30> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c46016f30> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-140" daemon prio=10 tid=0x00007f3abc798000 nid=0x1d6c in 
 Object.wait() [0x00007f3aac340000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c4108bb00> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c4108bb00> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-139" daemon prio=10 tid=0x00007f3abc796800 nid=0x1d6b in 
 Object.wait() [0x00007f3aac441000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c4108bb30> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c4108bb30> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-138" daemon prio=10 tid=0x00007f3abc3b4800 nid=0x1d6a in 
 Object.wait() [0x00007f3aac542000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c4108baa0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c4108baa0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-137" daemon prio=10 tid=0x00007f3abc9e7000 nid=0x1d69 in 
 Object.wait() [0x00007f3aac643000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c4108bad0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c4108bad0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-136" daemon prio=10 tid=0x00007f3abc9e5000 nid=0x1d68 in 
 Object.wait() [0x00007f3aac744000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c4108ba40> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c4108ba40> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-135" daemon prio=10 tid=0x00007f3abc696000 nid=0x1d67 in 
 Object.wait() [0x00007f3aac845000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c4108ba70> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c4108ba70> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-134" daemon prio=10 tid=0x00007f3abc34b800 nid=0x1d66 in 
 Object.wait() [0x00007f3aac946000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c4108b9e0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c4108b9e0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-133" daemon prio=10 tid=0x00007f3abca16800 nid=0x1d65 runnable 
 [0x00007f3aaca47000]
    java.lang.Thread.State: RUNNABLE
 	at sun.nio.ch.FileDispatcher.pread0(Native Method)
 	at sun.nio.ch.FileDispatcher.pread(FileDispatcher.java:31)
 	at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:234)
 	at sun.nio.ch.IOUtil.read(IOUtil.java:210)
 	at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:622)
 	at 
 org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(NIOFSDirectory.java:161)
 	at 
 org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:222)
 	at 
 org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:39)
 	at org.apache.lucene.store.DataInput.readVInt(DataInput.java:94)
 	at 
 org.apache.lucene.store.BufferedIndexInput.readVInt(BufferedIndexInput.java:190)
 	at org.apache.lucene.index.FieldsReader.doc(FieldsReader.java:214)
 	at 
 org.apache.lucene.index.SegmentReader.document(SegmentReader.java:842)
 	at 
 org.apache.lucene.index.DirectoryReader.document(DirectoryReader.java:565)
 	at 
 org.apache.solr.search.SolrIndexSearcher.doc(SolrIndexSearcher.java:427)
 	at 
 org.apache.solr.response.TextResponseWriter.writeDocuments(TextResponseWriter.java:248)
 	at 
 org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:153)
 	at 
 org.apache.solr.response.JSONWriter.writeNamedListAsMapWithDups(JSONResponseWriter.java:175)
 	at 
 org.apache.solr.response.JSONWriter.writeNamedList(JSONResponseWriter.java:291)
 	at 
 org.apache.solr.response.JSONWriter.writeResponse(JSONResponseWriter.java:88)
 	at 
 org.apache.solr.response.JSONResponseWriter.write(JSONResponseWriter.java:52)
 	at 
 org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:340)
 	at 
 org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:261)
 	at 
 org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
 	at 
 org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
 	at 
 org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
 	at 
 org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
 	at 
 org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
 	at 
 org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
 	at 
 org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:567)
 	at 
 org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
 	at 
 org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
 	at 
 org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:849)
 	at 
 org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:454)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-132" daemon prio=10 tid=0x00007f3abc771000 nid=0x1d64 in 
 Object.wait() [0x00007f3aacb48000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c4108b980> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c4108b980> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-131" daemon prio=10 tid=0x00007f3abc826800 nid=0x1d63 in 
 Object.wait() [0x00007f3aacc49000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c4108b9b0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c4108b9b0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-130" daemon prio=10 tid=0x00007f3abc6cd800 nid=0x1d62 in 
 Object.wait() [0x00007f3aacd4a000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c4108b920> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c4108b920> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-129" daemon prio=10 tid=0x00007f3abc259000 nid=0x1d61 in 
 Object.wait() [0x00007f3aace4b000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c4108b950> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c4108b950> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-128" daemon prio=10 tid=0x00007f3abc977000 nid=0x1d60 in 
 Object.wait() [0x00007f3aacf4c000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c4108b8c0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c4108b8c0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-127" daemon prio=10 tid=0x00007f3abca40800 nid=0x1d5d in 
 Object.wait() [0x00007f3aad04d000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c3d456068> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c3d456068> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-126" daemon prio=10 tid=0x00007f3abc2ad800 nid=0x1d5c in 
 Object.wait() [0x00007f3aad14e000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c15815478> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c15815478> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-125" daemon prio=10 tid=0x00007f3abc93d800 nid=0x1d5b in 
 Object.wait() [0x00007f3aad24f000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c01a2f990> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c01a2f990> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-124" daemon prio=10 tid=0x00007f3abc3b7000 nid=0x1d5a in 
 Object.wait() [0x00007f3aad350000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c3dfa6718> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c3dfa6718> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-123" daemon prio=10 tid=0x00007f3abca6c800 nid=0x1d59 in 
 Object.wait() [0x00007f3aad451000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c3d456668> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c3d456668> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-122" daemon prio=10 tid=0x00007f3abcabe000 nid=0x1d46 in 
 Object.wait() [0x00007f3aad552000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c15815b98> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c15815b98> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-121" daemon prio=10 tid=0x00007f3abcabc800 nid=0x1d45 in 
 Object.wait() [0x00007f3aad653000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c3d4575f8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c3d4575f8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-120" daemon prio=10 tid=0x00007f3abcaba000 nid=0x1d44 in 
 Object.wait() [0x00007f3aad754000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c01990d28> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c01990d28> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-119" daemon prio=10 tid=0x00007f3abc91b000 nid=0x1d43 in 
 Object.wait() [0x00007f3aad855000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c3d457718> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c3d457718> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-118" daemon prio=10 tid=0x00007f3abc919800 nid=0x1d42 in 
 Object.wait() [0x00007f3aad956000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c15811248> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c15811248> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-117" daemon prio=10 tid=0x00007f3abc916800 nid=0x1d41 in 
 Object.wait() [0x00007f3aada57000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c01990a28> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c01990a28> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-116" daemon prio=10 tid=0x00007f3abca98000 nid=0x1d40 in 
 Object.wait() [0x00007f3aadb58000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c01990758> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c01990758> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-115" daemon prio=10 tid=0x00007f3abca96800 nid=0x1d3f in 
 Object.wait() [0x00007f3aadc59000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c158121d8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c158121d8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-114" daemon prio=10 tid=0x00007f3abc191000 nid=0x1d3e in 
 Object.wait() [0x00007f3aadd5a000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c3d3f8b18> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c3d3f8b18> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-113" daemon prio=10 tid=0x00007f3abc3b8000 nid=0x1d3d in 
 Object.wait() [0x00007f3aade5b000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c3d3f89f8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c3d3f89f8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-112" daemon prio=10 tid=0x00007f3abca51800 nid=0x1d26 in 
 Object.wait() [0x00007f3aadf5c000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c158132b8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c158132b8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-111" daemon prio=10 tid=0x00007f3abc2af800 nid=0x1d25 in 
 Object.wait() [0x00007f3aae05d000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c3d371758> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c3d371758> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-110" daemon prio=10 tid=0x00007f3abc6e7800 nid=0x1d24 in 
 Object.wait() [0x00007f3aae15e000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c15814248> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c15814248> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-109" daemon prio=10 tid=0x00007f3abc6e6000 nid=0x1d23 in 
 Object.wait() [0x00007f3aae25f000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c158151d8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c158151d8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-108" daemon prio=10 tid=0x00007f3abc838800 nid=0x1d22 in 
 Object.wait() [0x00007f3aae360000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c017570f0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c017570f0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-107" daemon prio=10 tid=0x00007f3abc837800 nid=0x1d21 in 
 Object.wait() [0x00007f3aae461000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c157c3c08> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c157c3c08> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-106" daemon prio=10 tid=0x00007f3abc773000 nid=0x1d20 in 
 Object.wait() [0x00007f3aae562000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c01756040> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c01756040> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-105" daemon prio=10 tid=0x00007f3abc0b6800 nid=0x1d1f in 
 Object.wait() [0x00007f3aae663000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c01755b90> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c01755b90> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-104" daemon prio=10 tid=0x00007f3abc47a000 nid=0x1d0c in 
 Object.wait() [0x00007f3aae764000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfe8cc508> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfe8cc508> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-103" daemon prio=10 tid=0x00007f3abc478800 nid=0x1d0b in 
 Object.wait() [0x00007f3aae865000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c002ed730> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c002ed730> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-102" daemon prio=10 tid=0x00007f3abc477000 nid=0x1d0a in 
 Object.wait() [0x00007f3aae966000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c002ec7a0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c002ec7a0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-101" daemon prio=10 tid=0x00007f3abc475000 nid=0x1d09 in 
 Object.wait() [0x00007f3aaea67000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c0024ae40> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c0024ae40> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-100" daemon prio=10 tid=0x00007f3abc4d4000 nid=0x1d08 in 
 Object.wait() [0x00007f3aaeb68000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c00238f00> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c00238f00> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-99" daemon prio=10 tid=0x00007f3abc4d2800 nid=0x1d07 in 
 Object.wait() [0x00007f3aaec69000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c00237f10> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c00237f10> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-98" daemon prio=10 tid=0x00007f3abc2b3800 nid=0x1d06 in 
 Object.wait() [0x00007f3aaed6a000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c00121248> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c00121248> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-97" daemon prio=10 tid=0x00007f3abc020000 nid=0x1d05 in 
 Object.wait() [0x00007f3aaee6b000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfe8cc388> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfe8cc388> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-96" daemon prio=10 tid=0x00007f3abc3b9800 nid=0x1d03 in 
 Object.wait() [0x00007f3aaef6c000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfeb088a8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfeb088a8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-95" daemon prio=10 tid=0x00007f3abc01d800 nid=0x1cf1 in 
 Object.wait() [0x00007f3aaf06d000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfee55110> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfee55110> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-94" daemon prio=10 tid=0x00007f3abc2bd000 nid=0x1cf0 in 
 Object.wait() [0x00007f3aaf16e000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfeb07df8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfeb07df8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-93" daemon prio=10 tid=0x00007f3abc2bb000 nid=0x1cef in 
 Object.wait() [0x00007f3aaf26f000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfeb06d18> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfeb06d18> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-92" daemon prio=10 tid=0x00007f3abc16f000 nid=0x1cee in 
 Object.wait() [0x00007f3aaf370000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c013f4820> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c013f4820> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-91" daemon prio=10 tid=0x00007f3abc16d800 nid=0x1ced in 
 Object.wait() [0x00007f3aaf471000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c013f4eb0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c013f4eb0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-90" daemon prio=10 tid=0x00007f3abc16c000 nid=0x1cec in 
 Object.wait() [0x00007f3aaf572000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfeb06a18> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfeb06a18> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-89" daemon prio=10 tid=0x00007f3abc2b8800 nid=0x1ceb in 
 Object.wait() [0x00007f3aaf673000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c00a59cc8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c00a59cc8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-88" daemon prio=10 tid=0x00007f3abc2b6800 nid=0x1cea in 
 Object.wait() [0x00007f3aaf774000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c013f5e40> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c013f5e40> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-87" daemon prio=10 tid=0x00007f3abc594800 nid=0x1ce9 in 
 Object.wait() [0x00007f3aaf875000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfeb05728> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfeb05728> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-86" daemon prio=10 tid=0x00007f3abc592000 nid=0x1ce8 in 
 Object.wait() [0x00007f3aaf976000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c013f6e60> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c013f6e60> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-85" daemon prio=10 tid=0x00007f3abc162000 nid=0x1ce7 in 
 Object.wait() [0x00007f3aafa77000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c013f74c0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c013f74c0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-84" daemon prio=10 tid=0x00007f3abc160000 nid=0x1ce6 in 
 Object.wait() [0x00007f3aafb78000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c0155c4f0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c0155c4f0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-83" daemon prio=10 tid=0x00007f3abc043800 nid=0x1ce5 in 
 Object.wait() [0x00007f3aafc79000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c0155cb50> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c0155cb50> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-82" daemon prio=10 tid=0x00007f3abc12e800 nid=0x1ce4 in 
 Object.wait() [0x00007f3aafd7a000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c0155d150> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c0155d150> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-81" daemon prio=10 tid=0x00007f3abc4bf800 nid=0x1ce3 in 
 Object.wait() [0x00007f3aafe7b000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c0155d750> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c0155d750> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-80" daemon prio=10 tid=0x00007f3abc041800 nid=0x1ce2 in 
 Object.wait() [0x00007f3aaff7c000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c013f75e0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c013f75e0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-79" daemon prio=10 tid=0x00007f3abc03f800 nid=0x1ce1 in 
 Object.wait() [0x00007f3ab007d000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c0155e830> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c0155e830> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-78" daemon prio=10 tid=0x00007f3abc61c000 nid=0x1ce0 in 
 Object.wait() [0x00007f3ab017e000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c00237d00> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c00237d00> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-77" daemon prio=10 tid=0x00007f3abc61a800 nid=0x1cdf in 
 Object.wait() [0x00007f3ab027f000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c013f86c0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c013f86c0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-76" daemon prio=10 tid=0x00007f3abc619000 nid=0x1cde in 
 Object.wait() [0x00007f3ab0380000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c0155f7c0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c0155f7c0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-75" daemon prio=10 tid=0x00007f3abc617000 nid=0x1cdd in 
 Object.wait() [0x00007f3ab0481000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c013f9650> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c013f9650> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-74" daemon prio=10 tid=0x00007f3abc0c7000 nid=0x1cdc in 
 Object.wait() [0x00007f3ab0582000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c016f9d70> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c016f9d70> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-73" daemon prio=10 tid=0x00007f3abc0c5000 nid=0x1cdb in 
 Object.wait() [0x00007f3ab0683000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c01560900> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c01560900> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-72" daemon prio=10 tid=0x00007f3abc0c3000 nid=0x1cda in 
 Object.wait() [0x00007f3ab0784000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c016fa3d0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c016fa3d0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-71" daemon prio=10 tid=0x00007f3abc28b800 nid=0x1cd9 in 
 Object.wait() [0x00007f3ab0885000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c01560f90> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c01560f90> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-70" daemon prio=10 tid=0x00007f3abc289800 nid=0x1cd8 in 
 Object.wait() [0x00007f3ab0986000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c016fb360> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c016fb360> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-69" daemon prio=10 tid=0x00007f3abc288800 nid=0x1cd7 in 
 Object.wait() [0x00007f3ab0a87000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3c016fb3c0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3c016fb3c0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-68" daemon prio=10 tid=0x00007f3abc286000 nid=0x1cae in 
 Object.wait() [0x00007f3ab0b88000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfe793850> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfe793850> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-67" daemon prio=10 tid=0x00007f3abc284800 nid=0x1cad in 
 Object.wait() [0x00007f3ab0c89000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfe8f7c98> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfe8f7c98> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-66" daemon prio=10 tid=0x00007f3abc462800 nid=0x1cac in 
 Object.wait() [0x00007f3ab0d8a000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfe90e1a8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfe90e1a8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-65" daemon prio=10 tid=0x00007f3abc461000 nid=0x1cab in 
 Object.wait() [0x00007f3ab0e8b000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfe962288> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfe962288> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-64" daemon prio=10 tid=0x00007f3abc45f000 nid=0x1caa in 
 Object.wait() [0x00007f3ab0f8c000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfe99d7f8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfe99d7f8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-63" daemon prio=10 tid=0x00007f3abc45d800 nid=0x1ca9 in 
 Object.wait() [0x00007f3ab108d000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfe9d2668> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfe9d2668> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-62" daemon prio=10 tid=0x00007f3abc4af000 nid=0x1ca7 in 
 Object.wait() [0x00007f3ab118e000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfea07fd0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfea07fd0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-61" daemon prio=10 tid=0x00007f3abc4ad800 nid=0x1ca6 in 
 Object.wait() [0x00007f3ab128f000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfe8bcee8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfe8bcee8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-60" daemon prio=10 tid=0x00007f3abc4aa000 nid=0x1ca5 in 
 Object.wait() [0x00007f3ab1390000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfea4f2e0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfea4f2e0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-59" daemon prio=10 tid=0x00007f3abc4a8800 nid=0x1ca4 in 
 Object.wait() [0x00007f3ab1491000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfea4f3d0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfea4f3d0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-58" daemon prio=10 tid=0x00007f3abc4a7800 nid=0x1ca3 in 
 Object.wait() [0x00007f3ab1592000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfea76e70> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfea76e70> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-57" daemon prio=10 tid=0x00007f3abc5f8000 nid=0x1ca2 in 
 Object.wait() [0x00007f3ab1693000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfeac8590> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfeac8590> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-56" daemon prio=10 tid=0x00007f3abc141000 nid=0x1ca1 in 
 Object.wait() [0x00007f3ab1794000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfeaff200> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfeaff200> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-55" daemon prio=10 tid=0x00007f3abc5f4800 nid=0x1ca0 in 
 Object.wait() [0x00007f3ab1895000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfeb36428> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfeb36428> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-54" daemon prio=10 tid=0x00007f3abc5f3000 nid=0x1c9f in 
 Object.wait() [0x00007f3ab1996000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfeb73078> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfeb73078> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-53" daemon prio=10 tid=0x00007f3abc77b000 nid=0x1c9e in 
 Object.wait() [0x00007f3ab1a97000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfeb73228> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfeb73228> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-52" daemon prio=10 tid=0x00007f3abc779800 nid=0x1c9d in 
 Object.wait() [0x00007f3ab1b98000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfe8f7cf8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfe8f7cf8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-51" daemon prio=10 tid=0x00007f3abc778000 nid=0x1c9c in 
 Object.wait() [0x00007f3ab1c99000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfebc1c68> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfebc1c68> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-50" daemon prio=10 tid=0x00007f3abc777000 nid=0x1c9b in 
 Object.wait() [0x00007f3ab1d9a000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfebf6ec8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfebf6ec8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-49" daemon prio=10 tid=0x00007f3abc774800 nid=0x1c88 in 
 Object.wait() [0x00007f3ab1e9b000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfec35918> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfec35918> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-48" daemon prio=10 tid=0x00007f3abc354800 nid=0x1c87 in 
 Object.wait() [0x00007f3ab1f9c000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfec3e3a8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfec3e3a8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-47" daemon prio=10 tid=0x00007f3abc353000 nid=0x1c86 in 
 Object.wait() [0x00007f3ab209d000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfe811fe0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfe811fe0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-46" daemon prio=10 tid=0x00007f3abc351800 nid=0x1c85 in 
 Object.wait() [0x00007f3ab219e000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfec928e0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfec928e0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-45" daemon prio=10 tid=0x00007f3abc34f000 nid=0x1c84 in 
 Object.wait() [0x00007f3ab229f000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfecdb098> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfecdb098> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-44" daemon prio=10 tid=0x00007f3abc049800 nid=0x1c83 in 
 Object.wait() [0x00007f3ab23a0000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfecedfc0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfecedfc0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-43" daemon prio=10 tid=0x00007f3abc047800 nid=0x1c82 in 
 Object.wait() [0x00007f3ab24a1000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfed67f78> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfed67f78> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-42" daemon prio=10 tid=0x00007f3abc046000 nid=0x1c81 in 
 Object.wait() [0x00007f3ab25a2000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfed6e1f8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfed6e1f8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-41" daemon prio=10 tid=0x00007f3abc6d4000 nid=0x1c80 in 
 Object.wait() [0x00007f3ab26a3000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfedd4ed8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfedd4ed8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-40" daemon prio=10 tid=0x00007f3abc6d2800 nid=0x1c7f in 
 Object.wait() [0x00007f3ab27a4000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfede6060> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfede6060> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-39" daemon prio=10 tid=0x00007f3abc6d1000 nid=0x1c7e in 
 Object.wait() [0x00007f3ab28a5000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfee07c70> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfee07c70> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-38" daemon prio=10 tid=0x00007f3abc0b9800 nid=0x1c7d in 
 Object.wait() [0x00007f3ab29a6000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfee537c0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfee537c0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-37" daemon prio=10 tid=0x00007f3abc0b8000 nid=0x1c7c in 
 Object.wait() [0x00007f3ab2aa7000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfee53820> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfee53820> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-36" daemon prio=10 tid=0x00007f3abc062000 nid=0x1c7b in 
 Object.wait() [0x00007f3ab2ba8000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfe812190> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfe812190> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-35" daemon prio=10 tid=0x00007f3abc0b4000 nid=0x1c7a in 
 Object.wait() [0x00007f3ab2ca9000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfeea6170> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfeea6170> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-34" daemon prio=10 tid=0x00007f3abc0b2800 nid=0x1c79 in 
 Object.wait() [0x00007f3ab2daa000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfeecde40> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfeecde40> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-33" daemon prio=10 tid=0x00007f3abc1bb000 nid=0x1c78 in 
 Object.wait() [0x00007f3ab2eab000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfef050f8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfef050f8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-32" daemon prio=10 tid=0x00007f3abc1b9800 nid=0x1c77 in 
 Object.wait() [0x00007f3ab2fac000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfef3bdb0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfef3bdb0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-31" daemon prio=10 tid=0x00007f3abc146000 nid=0x1c76 in 
 Object.wait() [0x00007f3ab30ad000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfef3d690> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfef3d690> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-30" daemon prio=10 tid=0x00007f3abc144800 nid=0x1c75 in 
 Object.wait() [0x00007f3ab31ae000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfef75670> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfef75670> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-29" daemon prio=10 tid=0x00007f3abc143000 nid=0x1c74 in 
 Object.wait() [0x00007f3ab32af000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfefab550> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfefab550> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-28" daemon prio=10 tid=0x00007f3abc141800 nid=0x1c73 runnable 
 [0x00007f3ab33af000]
    java.lang.Thread.State: RUNNABLE
 	at sun.nio.ch.FileDispatcher.pread0(Native Method)
 	at sun.nio.ch.FileDispatcher.pread(FileDispatcher.java:31)
 	at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:234)
 	at sun.nio.ch.IOUtil.read(IOUtil.java:210)
 	at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:622)
 	at 
 org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(NIOFSDirectory.java:161)
 	at 
 org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:222)
 	at 
 org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:39)
 	at org.apache.lucene.store.DataInput.readVInt(DataInput.java:94)
 	at 
 org.apache.lucene.store.BufferedIndexInput.readVInt(BufferedIndexInput.java:190)
 	at 
 org.apache.lucene.index.codecs.BlockTermsReader$FieldReader$SegmentTermsEnum.nextBlock(BlockTermsReader.java:805)
 	at 
 org.apache.lucene.index.codecs.BlockTermsReader$FieldReader$SegmentTermsEnum.seek(BlockTermsReader.java:425)
 	at 
 org.apache.lucene.search.FilteredTermsEnum.next(FilteredTermsEnum.java:192)
 	at 
 org.apache.lucene.search.MultiTermQueryWrapperFilter.getDocIdSet(MultiTermQueryWrapperFilter.java:124)
 	at 
 org.apache.lucene.search.ConstantScoreQuery$ConstantWeight.scorer(ConstantScoreQuery.java:138)
 	at 
 org.apache.lucene.search.BooleanQuery$BooleanWeight.scorer(BooleanQuery.java:302)
 	at 
 org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:536)
 	at 
 org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:303)
 	at 
 org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1202)
 	at 
 org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1090)
 	at 
 org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:337)
 	at 
 org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:440)
 	at 
 org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:231)
 	at 
 org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
 	at org.apache.solr.core.SolrCore.execute(SolrCore.java:1290)
 	at 
 org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:353)
 	at 
 org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:248)
 	at 
 org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
 	at 
 org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
 	at 
 org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
 	at 
 org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
 	at 
 org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
 	at 
 org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
 	at 
 org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:567)
 	at 
 org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
 	at 
 org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
 	at 
 org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:849)
 	at 
 org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:454)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-27" daemon prio=10 tid=0x00007f3abc4b2000 nid=0x1c72 in 
 Object.wait() [0x00007f3ab34b1000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bff01d948> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bff01d948> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-26" daemon prio=10 tid=0x00007f3abc4b0800 nid=0x1c71 in 
 Object.wait() [0x00007f3ab35b2000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bff057638> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bff057638> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-25" daemon prio=10 tid=0x00007f3abc4af800 nid=0x1c70 in 
 Object.wait() [0x00007f3ab36b3000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bff08d958> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bff08d958> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-24" daemon prio=10 tid=0x00007f3abc699800 nid=0x1c6f in 
 Object.wait() [0x00007f3ab37b4000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bff08d9b8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bff08d9b8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-23" daemon prio=10 tid=0x00007f3abc214000 nid=0x1c6e in 
 Object.wait() [0x00007f3ab38b5000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bfede60c0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bfede60c0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-22" daemon prio=10 tid=0x00007f3abc212800 nid=0x1c6d in 
 Object.wait() [0x00007f3ab39b6000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bff0ae788> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bff0ae788> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-21" daemon prio=10 tid=0x00007f3abc211800 nid=0x1c6c in 
 Object.wait() [0x00007f3ab3ab7000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bff1008b0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bff1008b0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-20" daemon prio=10 tid=0x00007f3abc370000 nid=0x1c6b in 
 Object.wait() [0x00007f3ab3bb8000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bff138190> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bff138190> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-19" daemon prio=10 tid=0x00007f3abc356800 nid=0x1c6a in 
 Object.wait() [0x00007f3ab3cb9000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bff199848> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bff199848> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-18" daemon prio=10 tid=0x00007f3abc20f800 nid=0x1c69 in 
 Object.wait() [0x00007f3ab8ce9000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bff19da58> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bff19da58> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-17" daemon prio=10 tid=0x00007f3abc4dc800 nid=0x1c68 in 
 Object.wait() [0x00007f3ab8dea000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bff19dab8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bff19dab8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-16" daemon prio=10 tid=0x00007f3abc4db800 nid=0x1c67 in 
 Object.wait() [0x00007f3ab8eeb000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bff1ab420> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bff1ab420> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-15" daemon prio=10 tid=0x00007f3abc4de000 nid=0x1c66 in 
 Object.wait() [0x00007f3ab8fec000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bff1e3da8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bff1e3da8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-14" daemon prio=10 tid=0x00007f3abc0bc000 nid=0x1c65 in 
 Object.wait() [0x00007f3ab90ed000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bff20cbe0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bff20cbe0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-13" daemon prio=10 tid=0x00007f3abc0ba800 nid=0x1c64 in 
 Object.wait() [0x00007f3ab91ee000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bff24ad78> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bff24ad78> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-12" daemon prio=10 tid=0x00007f3abc0bf000 nid=0x1c63 in 
 Object.wait() [0x00007f3ab92ef000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bff2a3d48> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bff2a3d48> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-11" daemon prio=10 tid=0x00007f3abc0bd000 nid=0x1c62 in 
 Object.wait() [0x00007f3ab93f0000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bff2d8350> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bff2d8350> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-10" daemon prio=10 tid=0x00007f3abc3be800 nid=0x1c61 in 
 Object.wait() [0x00007f3ab94f1000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bff3152c8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bff3152c8> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-9" daemon prio=10 tid=0x00007f3abc0d4800 nid=0x1c60 in 
 Object.wait() [0x00007f3ab95f2000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bff220380> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bff220380> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-8" daemon prio=10 tid=0x00007f3abc0d3800 nid=0x1c5f in 
 Object.wait() [0x00007f3ab96f3000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bff351aa0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bff351aa0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-7" daemon prio=10 tid=0x00007f3abc5fc000 nid=0x1c5e in 
 Object.wait() [0x00007f3ab97f4000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bff38f590> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bff38f590> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-6" daemon prio=10 tid=0x00007f3abc53d800 nid=0x1c5d in 
 Object.wait() [0x00007f3ab98f5000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bff38f5f0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bff38f5f0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-5" daemon prio=10 tid=0x00007f3abc1e5800 nid=0x1c5c in 
 Object.wait() [0x00007f3ab99f6000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bff3d16c0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bff3d16c0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-4" daemon prio=10 tid=0x00007f3abc540800 nid=0x1c5b in 
 Object.wait() [0x00007f3ab9af7000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bff432b08> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bff432b08> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-3" daemon prio=10 tid=0x00007f3abc48e800 nid=0x1c5a in 
 Object.wait() [0x00007f3ab9bf8000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bff451498> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bff451498> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "TP-Monitor" daemon prio=10 tid=0x00007f3abc511800 nid=0x1c47 in 
 Object.wait() [0x00007f3ab9cf9000]
    java.lang.Thread.State: TIMED_WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bb2933a30> (a 
 org.apache.tomcat.util.threads.ThreadPool$MonitorRunnable)
 	at 
 org.apache.tomcat.util.threads.ThreadPool$MonitorRunnable.run(ThreadPool.java:565)
 	- locked <0x00007f3bb2933a30> (a 
 org.apache.tomcat.util.threads.ThreadPool$MonitorRunnable)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "TP-Processor4" daemon prio=10 tid=0x00007f3abc1ce000 nid=0x1c46 
 runnable [0x00007f3aba114000]
    java.lang.Thread.State: RUNNABLE
 	at java.net.PlainSocketImpl.socketAccept(Native Method)
 	at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
 	- locked <0x00007f3bb292e590> (a java.net.SocksSocketImpl)
 	at java.net.ServerSocket.implAccept(ServerSocket.java:453)
 	at java.net.ServerSocket.accept(ServerSocket.java:421)
 	at org.apache.jk.common.ChannelSocket.accept(ChannelSocket.java:307)
 	at 
 org.apache.jk.common.ChannelSocket.acceptConnections(ChannelSocket.java:661)
 	at 
 org.apache.jk.common.ChannelSocket$SocketAcceptor.runIt(ChannelSocket.java:872)
 	at 
 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "TP-Processor3" daemon prio=10 tid=0x00007f3abc395800 nid=0x1c45 in 
 Object.wait() [0x00007f3aba215000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bb2933138> (a 
 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:662)
 	- locked <0x00007f3bb2933138> (a 
 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "TP-Processor2" daemon prio=10 tid=0x00007f3abc225000 nid=0x1c44 in 
 Object.wait() [0x00007f3aba316000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bb2932d08> (a 
 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:662)
 	- locked <0x00007f3bb2932d08> (a 
 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "TP-Processor1" daemon prio=10 tid=0x00007f3abc3d6800 nid=0x1c43 in 
 Object.wait() [0x00007f3aba417000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bb29328d8> (a 
 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:662)
 	- locked <0x00007f3bb29328d8> (a 
 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-2" daemon prio=10 tid=0x00007f3abc10c000 nid=0x1c42 in 
 Object.wait() [0x00007f3ac07fe000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bb29360d0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bb29360d0> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-1" daemon prio=10 tid=0x00007f3abc2a0800 nid=0x1c41 in 
 Object.wait() [0x00007f3ac08ff000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bb2935d10> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at java.lang.Object.wait(Object.java:485)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
 	- locked <0x00007f3bb2935d10> (a 
 org.apache.tomcat.util.net.JIoEndpoint$Worker)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "http-80-Acceptor-0" daemon prio=10 tid=0x00007f3abc2c4000 nid=0x1c40 
 runnable [0x00007f3ac0a00000]
    java.lang.Thread.State: RUNNABLE
 	at java.net.PlainSocketImpl.socketAccept(Native Method)
 	at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
 	- locked <0x00007f3bac53fdb8> (a java.net.SocksSocketImpl)
 	at java.net.ServerSocket.implAccept(ServerSocket.java:453)
 	at java.net.ServerSocket.accept(ServerSocket.java:421)
 	at 
 org.apache.tomcat.util.net.DefaultServerSocketFactory.acceptSocket(DefaultServerSocketFactory.java:61)
 	at 
 org.apache.tomcat.util.net.JIoEndpoint$Acceptor.run(JIoEndpoint.java:317)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "ContainerBackgroundProcessor[StandardEngine[Catalina]]" daemon prio=10 
 tid=0x00007f3abc586000 nid=0x1c3f sleeping[0x00007f3ac0c2c000]
    java.lang.Thread.State: TIMED_WAITING (sleeping)
 	at java.lang.Thread.sleep(Native Method)
 	at 
 org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1579)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "pool-1-thread-1" prio=10 tid=0x00007f3abc3c7000 nid=0x1c1e waiting on 
 condition [0x00007f3ac0d2d000]
    java.lang.Thread.State: WAITING (parking)
 	at sun.misc.Unsafe.park(Native Method)
 	- parking to wait for  <0x00007f3bac781c98> (a 
 java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
 	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
 	at 
 java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
 	at 
 java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
 	at 
 java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
 	at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
 	at java.lang.Thread.run(Thread.java:662)

    Locked ownable synchronizers:
 	- None

 "Low Memory Detector" daemon prio=10 tid=0x0000000040786800 nid=0x1bf8 
 runnable [0x0000000000000000]
    java.lang.Thread.State: RUNNABLE

    Locked ownable synchronizers:
 	- None

 "CompilerThread1" daemon prio=10 tid=0x0000000040784000 nid=0x1bf7 
 waiting on condition [0x0000000000000000]
    java.lang.Thread.State: RUNNABLE

    Locked ownable synchronizers:
 	- None

 "CompilerThread0" daemon prio=10 tid=0x0000000040781800 nid=0x1bf6 
 waiting on condition [0x0000000000000000]
    java.lang.Thread.State: RUNNABLE

    Locked ownable synchronizers:
 	- None

 "Signal Dispatcher" daemon prio=10 tid=0x000000004077f000 nid=0x1bf5 
 runnable [0x0000000000000000]
    java.lang.Thread.State: RUNNABLE

    Locked ownable synchronizers:
 	- None

 "Surrogate Locker Thread (CMS)" daemon prio=10 tid=0x000000004077d000 
 nid=0x1bf4 waiting on condition [0x0000000000000000]
    java.lang.Thread.State: RUNNABLE

    Locked ownable synchronizers:
 	- None

 "Finalizer" daemon prio=10 tid=0x000000004075d000 nid=0x1bf3 in 
 Object.wait() [0x00007f3ac1686000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bac391138> (a 
 java.lang.ref.ReferenceQueue$Lock)
 	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
 	- locked <0x00007f3bac391138> (a java.lang.ref.ReferenceQueue$Lock)
 	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
 	at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)

    Locked ownable synchronizers:
 	- None

 "Reference Handler" daemon prio=10 tid=0x0000000040756000 nid=0x1bf2 in 
 Object.wait() [0x00007f3ac1787000]
    java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00007f3bac390f88> (a java.lang.ref.Reference$Lock)
 	at java.lang.Object.wait(Object.java:485)
 	at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
 	- locked <0x00007f3bac390f88> (a java.lang.ref.Reference$Lock)

    Locked ownable synchronizers:
 	- None

 "main" prio=10 tid=0x0000000040669800 nid=0x1beb runnable 
 [0x00007f41ee977000]
    java.lang.Thread.State: RUNNABLE
 	at java.net.PlainSocketImpl.socketAccept(Native Method)
 	at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
 	- locked <0x00007f3bb2935880> (a java.net.SocksSocketImpl)
 	at java.net.ServerSocket.implAccept(ServerSocket.java:453)
 	at java.net.ServerSocket.accept(ServerSocket.java:421)
 	at 
 org.apache.catalina.core.StandardServer.await(StandardServer.java:389)
 	at org.apache.catalina.startup.Catalina.await(Catalina.java:647)
 	at org.apache.catalina.startup.Catalina.start(Catalina.java:607)
 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 	at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
 	at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
 	at java.lang.reflect.Method.invoke(Method.java:597)
 	at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:288)
 	at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:413)

    Locked ownable synchronizers:
 	- None

 "VM Thread" prio=10 tid=0x0000000040751800 nid=0x1bf1 runnable

 "Gang worker#0 (Parallel GC Threads)" prio=10 tid=0x0000000040678000 
 nid=0x1bec runnable

 "Gang worker#1 (Parallel GC Threads)" prio=10 tid=0x000000004067a000 
 nid=0x1bed runnable

 "Gang worker#2 (Parallel GC Threads)" prio=10 tid=0x000000004067c000 
 nid=0x1bee runnable

 "Gang worker#3 (Parallel GC Threads)" prio=10 tid=0x000000004067d800 
 nid=0x1bef runnable

 "Concurrent Mark-Sweep GC Thread" prio=10 tid=0x000000004070c800 
 nid=0x1bf0 runnable
 "VM Periodic Task Thread" prio=10 tid=0x0000000040791800 nid=0x1bf9 
 waiting on condition

 JNI global references: 1600
 
 ========= jvm.log =========
 81202.191: [GC 81202.191: [ParNew: 2698561K->83440K(2936000K), 
 0.1483670 secs] 20262492K->17662145K(29033920K), 0.1485510 secs] [Times: 
 user=0.47 sys=0.00, real=0.14 secs]
 81209.983: [GC 81209.983: [ParNew: 2685739K->81895K(2936000K), 
 0.1276430 secs] 20264444K->17677113K(29033920K), 0.1278350 secs] [Times: 
 user=0.47 sys=0.00, real=0.13 secs]
 81219.767: [GC 81219.767: [ParNew: 2689632K->73489K(2936000K), 
 0.1268120 secs] 20284850K->17685116K(29033920K), 0.1270010 secs] [Times: 
 user=0.48 sys=0.00, real=0.12 secs]
 81228.011: [GC 81228.011: [ParNew: 2674978K->85620K(2936000K), 
 0.1168250 secs] 20286605K->17710289K(29033920K), 0.1170100 secs] [Times: 
 user=0.42 sys=0.01, real=0.11 secs]
 81237.381: [GC 81237.381: [ParNew: 2690900K->89940K(2936000K), 
 0.1204850 secs] 20315569K->17729524K(29033920K), 0.1206800 secs] [Times: 
 user=0.44 sys=0.00, real=0.12 secs]
 81246.158: [GC 81246.159: [ParNew: 2699732K->96933K(2936000K), 
 0.1211310 secs] 20339316K->17749860K(29033920K), 0.1213190 secs] [Times: 
 user=0.45 sys=0.00, real=0.12 secs]
 81254.992: [GC 81254.992: [ParNew: 2702282K->103172K(2936000K), 
 0.1784240 secs] 20355209K->17771111K(29033920K), 0.1786080 secs] [Times: 
 user=0.50 sys=0.01, real=0.18 secs]
 81263.969: [GC 81263.969: [ParNew: 2712964K->85477K(2936000K), 
 0.1631650 secs] 20380903K->17766365K(29033920K), 0.1633550 secs] [Times: 
 user=0.44 sys=0.00, real=0.16 secs]
 81272.393: [GC 81272.393: [ParNew: 2685528K->91005K(2936000K), 
 0.1739990 secs] 20366416K->17785469K(29033920K), 0.1741920 secs] [Times: 
 user=0.48 sys=0.00, real=0.17 secs]
 81280.461: [GC 81280.461: [ParNew: 2699754K->77821K(2936000K), 
 0.1252730 secs] 20394219K->17790718K(29033920K), 0.1254820 secs] [Times: 
 user=0.45 sys=0.00, real=0.12 secs]
 81290.117: [GC 81290.117: [ParNew: 2684853K->118425K(2936000K), 
 0.1183920 secs] 20397750K->17842157K(29033920K), 0.1185750 secs] [Times: 
 user=0.46 sys=0.00, real=0.12 secs]
 81300.417: [GC 81300.417: [ParNew: 2727522K->118707K(2936000K), 
 0.1229710 secs] 20451254K->17853982K(29033920K), 0.1231570 secs] [Times: 
 user=0.47 sys=0.00, real=0.12 secs]
 81308.669: [GC 81308.669: [ParNew: 2724967K->88444K(2936000K), 
 0.1271680 secs] 20460243K->17838767K(29033920K), 0.1273640 secs] [Times: 
 user=0.45 sys=0.00, real=0.12 secs]
 81317.028: [GC 81317.028: [ParNew: 2691130K->116516K(2936000K), 
 0.1234850 secs] 20441453K->17880748K(29033920K), 0.1236940 secs] [Times: 
 user=0.48 sys=0.00, real=0.12 secs]
 81332.592: [GC 81332.592: [ParNew: 2723734K->76530K(2936000K), 
 0.1260930 secs] 20487966K->17855631K(29033920K), 0.1262780 secs] [Times: 
 user=0.47 sys=0.00, real=0.12 secs]
 81341.962: [GC 81341.962: [ParNew: 2685705K->118213K(2936000K), 
 0.1254730 secs] 20464805K->17911736K(29033920K), 0.1256560 secs] [Times: 
 user=0.46 sys=0.00, real=0.12 secs]
 81349.116: [GC 81349.116: [ParNew: 2718360K->100642K(2936000K), 
 0.1308310 secs] 20511882K->17909005K(29033920K), 0.1310120 secs] [Times: 
 user=0.48 sys=0.01, real=0.13 secs]
 81359.412: [GC 81359.412: [ParNew: 2707712K->113602K(2936000K), 
 0.1544130 secs] 20516075K->17940871K(29033920K), 0.1546180 secs] [Times: 
 user=0.51 sys=0.00, real=0.15 secs]
 81368.010: [GC 81368.010: [ParNew: 2714635K->95117K(2936000K), 
 0.1345420 secs] 20541904K->17940145K(29033920K), 0.1347290 secs] [Times: 
 user=0.49 sys=0.00, real=0.13 secs]
 81380.423: [GC 81380.423: [ParNew: 2694955K->102116K(2936000K), 
 0.1255890 secs] 20539983K->17963318K(29033920K), 0.1257680 secs] [Times: 
 user=0.46 sys=0.01, real=0.12 secs]
 81391.937: [GC 81391.937: [ParNew: 2702673K->114140K(2936000K), 
 0.1516800 secs] 20563875K->17995210K(29033920K), 0.1518720 secs] [Times: 
 user=0.51 sys=0.01, real=0.15 secs]
 81400.412: [GC 81400.412: [ParNew: 2715331K->85776K(2936000K), 
 0.1258830 secs] 20596400K->17981754K(29033920K), 0.1260690 secs] [Times: 
 user=0.45 sys=0.01, real=0.12 secs]
 81409.256: [GC 81409.256: [ParNew: 2689823K->92904K(2936000K), 
 0.1244370 secs] 20585801K->18002286K(29033920K), 0.1246350 secs] [Times: 
 user=0.49 sys=0.00, real=0.13 secs]
 81421.273: [GC 81421.273: [ParNew: 2699707K->69457K(2936000K), 
 0.1215530 secs] 20609088K->17991782K(29033920K), 0.1217620 secs] [Times: 
 user=0.45 sys=0.00, real=0.12 secs]
 81431.384: [GC 81431.384: [ParNew: 2676120K->85461K(2936000K), 
 0.1741880 secs] 20598445K->18025550K(29033920K), 0.1743920 secs] [Times: 
 user=0.51 sys=0.00, real=0.18 secs]
 81443.616: [GC 81443.616: [ParNew: 2694591K->131645K(2936000K), 
 0.1254410 secs] 20634681K->18083309K(29033920K), 0.1256310 secs] [Times: 
 user=0.49 sys=0.00, real=0.13 secs]
 81454.468: [GC 81454.468: [ParNew: 2737855K->70912K(2936000K), 
 0.1452630 secs] 20689519K->18035798K(29033920K), 0.1454620 secs] [Times: 
 user=0.55 sys=0.00, real=0.14 secs]
 81466.009: [GC 81466.009: [ParNew: 2679468K->125762K(2936000K), 
 0.1668410 secs] 20644354K->18105279K(29033920K), 0.1670350 secs] [Times: 
 user=0.54 sys=0.01, real=0.16 secs]
 81477.739: [GC 81477.739: [ParNew: 2731775K->111236K(2936000K), 
 0.1893910 secs] 20711291K->18109805K(29033920K), 0.1895870 secs] [Times: 
 user=0.63 sys=0.01, real=0.19 secs]
 81494.175: [GC 81494.175: [ParNew: 2720174K->204493K(2936000K), 
 0.2008620 secs] 20718743K->18215814K(29033920K), 0.2010680 secs]
 


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: pauses on simple .html pages

Posted by Peter Crowther <pe...@melandra.com>.
On 9 June 2011 21:25, <to...@r322.com> wrote:
> I have a script setup does a wget hit to docs/config/valve.html
> every 1 second. If the wget call takes longer than 1 second it
> grabs a bunch of stats -- iostat, vmstat, top, jstack, last 30
> lines of jvm.log.

Nice setup - I wish more people did that!  Can I just ask a Really
Dumb Question: *exactly* when does this grab the stats?  Does it wait
for the response, or does it start gathering as soon as it's aware of
a problem (say, at 1 second)?  At the moment, for example, jstack
might not be showing anything because you're gathering data after the
call has completed.

- Peter

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: pauses on simple .html pages

Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

To whom it may concern,

On 6/9/2011 4:25 PM, tomcat@r322.com wrote:
> I have a script setup does a wget hit to docs/config/valve.html every 1
> second. If the wget call takes longer than 1 second it grabs a bunch of
> stats -- iostat, vmstat, top, jstack, last 30 lines of jvm.log.
> 
> A lot of these are caused by GC. However, there are a lot that aren't
> and I am having a hardtime finding a correlation between the stats and
> these timeouts. Looking at the stats I have, does anyone see anything?

Obviously, most of your threads are idle: just three concurrent SOLR
requests if I counted correctly, which should be no problem for
SOLR/Lucene and any hardware made during this century.

My first though after looking at the (lack of) CPU load was that you
might have a bad disk. The iowait times do not support that theory,
though: I think iowait includes all wait time, including weird retries
and stuff like that so it should be much higher than that.

Are you in a position where you can test your disk? How about looking at
syslog to see if the I/O system is flagging read errors or anything like
that. Same with any RAID logs you may have.

Is the disk local or a network share?

- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk3ygZsACgkQ9CaO5/Lv0PCdyQCePsK/BzemKyYkxjYdTzSPbu3p
Ah0Anim/0iRDzyCQR99ZIEadZ7tlz6sh
=s0n8
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org