You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@manifoldcf.apache.org by Mike Caceres <mi...@hotmail.com> on 2015/09/20 12:39:51 UTC
Ingestion Job in Starting Up state for over 16 hours
Hi,
I am running a job to ingest documents from Documentum. Things are generally working, but lately the jobs seems to hang for hours in the Starting Up status.
The version of ManifoldCF is 2.1.
Here is what I see in Postgres and a bit of the log file, which seems to show a cycle over and over trying to pick up jobs, but not getting the ones with status B.
Is there anything I can do to make the job change state to Active?
Thank you!Mike
dbname=# select id, description, status, startmethod from jobs; id | description | status | startmethod---------------+--------------------------+--------+------------- 1442492810054 | full-crawl-production-06 | B | D
dbname=# select j.description, j.status, jq.status, jq.seedingprocessid, count (*) from jobqueue jq right join jobs j on j.id = jq.jobid group by j.description, j.status, jq.status, jq.seedingprocessid order by j.description, j.status, jq.status; description | status | status | seedingprocessid | count--------------------------+--------+--------+------------------+-------- full-crawl-production-06 | B | P | A | 12795
DEBUG 2015-09-20 10:07:07,780 (Delete startup thread) - org.apache.manifoldcf.db - Requested query: [SELECT id FROM jobs WHERE status=? FOR UPDATE]DEBUG 2015-09-20 10:07:07,780 (Thread-31855) - org.apache.manifoldcf.db - Actual query: [START TRANSACTION]DEBUG 2015-09-20 10:07:07,780 (Thread-31855) - org.apache.manifoldcf.db - Done actual query (0ms): [START TRANSACTION]DEBUG 2015-09-20 10:07:07,781 (Thread-31856) - org.apache.manifoldcf.db - Actual query: [SELECT id FROM jobs WHERE status=? FOR UPDATE]DEBUG 2015-09-20 10:07:07,781 (Thread-31856) - org.apache.manifoldcf.db - Parameter 0: 'E'DEBUG 2015-09-20 10:07:07,781 (Thread-31856) - org.apache.manifoldcf.db - Done actual query (0ms): [SELECT id FROM jobs WHERE status=? FOR UPDATE]DEBUG 2015-09-20 10:07:07,781 (Delete startup thread) - org.apache.manifoldcf.db - Committing transaction!DEBUG 2015-09-20 10:07:07,781 (Thread-31857) - org.apache.manifoldcf.db - Actual query: [COMMIT]DEBUG 2015-09-20 10:07:07,781 (Thread-31857) - org.apache.manifoldcf.db - Done actual query (0ms): [COMMIT]DEBUG 2015-09-20 10:07:07,782 (Delete startup thread) - org.apache.manifoldcf.db - Ending transactionDEBUG 2015-09-20 10:07:07,860 (Job reset thread) - org.apache.manifoldcf.db - Requested query: [SELECT id FROM jobs WHERE (status=? OR status=? OR status=? OR status=? OR status=? OR status=? OR status=? OR status=? OR status=? OR status=?)]DEBUG 2015-09-20 10:07:07,860 (Thread-31858) - org.apache.manifoldcf.db - Actual query: [SELECT id FROM jobs WHERE (status=? OR status=? OR status=? OR status=? OR status=? OR status=? OR status=? OR status=? OR status=? OR status=?)]DEBUG 2015-09-20 10:07:07,860 (Thread-31858) - org.apache.manifoldcf.db - Parameter 0: 'X'DEBUG 2015-09-20 10:07:07,860 (Thread-31858) - org.apache.manifoldcf.db - Parameter 1: 'Y'DEBUG 2015-09-20 10:07:07,860 (Thread-31858) - org.apache.manifoldcf.db - Parameter 2: 'M'DEBUG 2015-09-20 10:07:07,860 (Thread-31858) - org.apache.manifoldcf.db - Parameter 3: 'v'DEBUG 2015-09-20 10:07:07,860 (Thread-31858) - org.apache.manifoldcf.db - Parameter 4: 'F'DEBUG 2015-09-20 10:07:07,860 (Thread-31858) - org.apache.manifoldcf.db - Parameter 5: 'f'DEBUG 2015-09-20 10:07:07,860 (Thread-31858) - org.apache.manifoldcf.db - Parameter 6: 'H'DEBUG 2015-09-20 10:07:07,860 (Thread-31858) - org.apache.manifoldcf.db - Parameter 7: 'h'DEBUG 2015-09-20 10:07:07,860 (Thread-31858) - org.apache.manifoldcf.db - Parameter 8: 'G'DEBUG 2015-09-20 10:07:07,860 (Thread-31858) - org.apache.manifoldcf.db - Parameter 9: 'g'DEBUG 2015-09-20 10:07:07,860 (Thread-31858) - org.apache.manifoldcf.db - Done actual query (0ms): [SELECT id FROM jobs WHERE (status=? OR status=? OR status=? OR status=? OR status=? OR status=? OR status=? OR status=? OR status=? OR status=?)]DEBUG 2015-09-20 10:07:07,861 (Job reset thread) - org.apache.manifoldcf.db - Requested query: [SELECT id FROM jobs WHERE (status=? OR status=?)]DEBUG 2015-09-20 10:07:07,861 (Thread-31859) - org.apache.manifoldcf.db - Actual query: [SELECT id FROM jobs WHERE (status=? OR status=?)]DEBUG 2015-09-20 10:07:07,861 (Thread-31859) - org.apache.manifoldcf.db - Parameter 0: 'I'DEBUG 2015-09-20 10:07:07,861 (Thread-31859) - org.apache.manifoldcf.db - Parameter 1: 'i'DEBUG 2015-09-20 10:07:07,861 (Thread-31859) - org.apache.manifoldcf.db - Done actual query (0ms): [SELECT id FROM jobs WHERE (status=? OR status=?)]DEBUG 2015-09-20 10:07:07,861 (Job reset thread) - org.apache.manifoldcf.db - Requested query: [SELECT id FROM jobs WHERE status=?]DEBUG 2015-09-20 10:07:07,862 (Thread-31860) - org.apache.manifoldcf.db - Actual query: [SELECT id FROM jobs WHERE status=?]DEBUG 2015-09-20 10:07:07,862 (Thread-31860) - org.apache.manifoldcf.db - Parameter 0: 'S'DEBUG 2015-09-20 10:07:07,862 (Thread-31860) - org.apache.manifoldcf.db - Done actual query (0ms): [SELECT id FROM jobs WHERE status=?]DEBUG 2015-09-20 10:07:07,979 (Job notification thread) - org.apache.manifoldcf.db - Beginning transaction of type 1DEBUG 2015-09-20 10:07:07,979 (Job notification thread) - org.apache.manifoldcf.db - Requested query: [SELECT id,failtime,failcount FROM jobs WHERE status=? FOR UPDATE]DEBUG 2015-09-20 10:07:07,979 (Thread-31861) - org.apache.manifoldcf.db - Actual query: [START TRANSACTION]DEBUG 2015-09-20 10:07:07,980 (Thread-31861) - org.apache.manifoldcf.db - Done actual query (1ms): [START TRANSACTION]DEBUG 2015-09-20 10:07:07,980 (Thread-31862) - org.apache.manifoldcf.db - Actual query: [SELECT id,failtime,failcount FROM jobs WHERE status=? FOR UPDATE]DEBUG 2015-09-20 10:07:07,980 (Thread-31862) - org.apache.manifoldcf.db - Parameter 0: 's'DEBUG 2015-09-20 10:07:07,980 (Thread-31862) - org.apache.manifoldcf.db - Done actual query (0ms): [SELECT id,failtime,failcount FROM jobs WHERE status=? FOR UPDATE]DEBUG 2015-09-20 10:07:07,980 (Job notification thread) - org.apache.manifoldcf.db - Committing transaction!DEBUG 2015-09-20 10:07:07,981 (Thread-31863) - org.apache.manifoldcf.db - Actual query: [COMMIT]DEBUG 2015-09-20 10:07:07,981 (Thread-31863) - org.apache.manifoldcf.db - Done actual query (0ms): [COMMIT]DEBUG 2015-09-20 10:07:07,981 (Job notification thread) - org.apache.manifoldcf.db - Ending transactionDEBUG 2015-09-20 10:07:07,981 (Job notification thread) - org.apache.manifoldcf.db - Beginning transaction of type 1DEBUG 2015-09-20 10:07:07,981 (Job notification thread) - org.apache.manifoldcf.db - Requested query: [SELECT id,failtime,failcount FROM jobs WHERE status=? FOR UPDATE]DEBUG 2015-09-20 10:07:07,981 (Thread-31864) - org.apache.manifoldcf.db - Actual query: [START TRANSACTION]DEBUG 2015-09-20 10:07:07,981 (Thread-31864) - org.apache.manifoldcf.db - Done actual query (0ms): [START TRANSACTION]DEBUG 2015-09-20 10:07:07,982 (Thread-31865) - org.apache.manifoldcf.db - Actual query: [SELECT id,failtime,failcount FROM jobs WHERE status=? FOR UPDATE]DEBUG 2015-09-20 10:07:07,982 (Thread-31865) - org.apache.manifoldcf.db - Parameter 0: 'd'DEBUG 2015-09-20 10:07:07,982 (Thread-31865) - org.apache.manifoldcf.db - Done actual query (0ms): [SELECT id,failtime,failcount FROM jobs WHERE status=? FOR UPDATE]DEBUG 2015-09-20 10:07:07,982 (Job notification thread) - org.apache.manifoldcf.db - Committing transaction!DEBUG 2015-09-20 10:07:07,982 (Thread-31866) - org.apache.manifoldcf.db - Actual query: [COMMIT]DEBUG 2015-09-20 10:07:07,982 (Thread-31866) - org.apache.manifoldcf.db - Done actual query (0ms): [COMMIT]DEBUG 2015-09-20 10:07:07,982 (Job notification thread) - org.apache.manifoldcf.db - Ending transactionDEBUG 2015-09-20 10:07:07,991 (Job start thread) - org.apache.manifoldcf.crawlerthreads - Job start thread - checking for jobs to start at 1442743627991DEBUG 2015-09-20 10:07:07,991 (Job start thread) - org.apache.manifoldcf.db - Beginning transaction of type 1DEBUG 2015-09-20 10:07:07,991 (Job start thread) - org.apache.manifoldcf.db - Requested query: [SELECT id,lasttime,status,startmethod,connectionname FROM jobs WHERE (status=? OR status=? OR status=? OR status=? OR status=?) AND startmethod!=? FOR UPDATE]DEBUG 2015-09-20 10:07:07,991 (Thread-31867) - org.apache.manifoldcf.db - Actual query: [START TRANSACTION]DEBUG 2015-09-20 10:07:07,991 (Thread-31867) - org.apache.manifoldcf.db - Done actual query (0ms): [START TRANSACTION]DEBUG 2015-09-20 10:07:07,992 (Thread-31868) - org.apache.manifoldcf.db - Actual query: [SELECT id,lasttime,status,startmethod,connectionname FROM jobs WHERE (status=? OR status=? OR status=? OR status=? OR status=?) AND startmethod!=? FOR UPDATE]DEBUG 2015-09-20 10:07:07,992 (Thread-31868) - org.apache.manifoldcf.db - Parameter 0: 'N'DEBUG 2015-09-20 10:07:07,992 (Thread-31868) - org.apache.manifoldcf.db - Parameter 1: 'W'DEBUG 2015-09-20 10:07:07,992 (Thread-31868) - org.apache.manifoldcf.db - Parameter 2: 'w'DEBUG 2015-09-20 10:07:07,992 (Thread-31868) - org.apache.manifoldcf.db - Parameter 3: 'Z'DEBUG 2015-09-20 10:07:07,992 (Thread-31868) - org.apache.manifoldcf.db - Parameter 4: 'z'DEBUG 2015-09-20 10:07:07,992 (Thread-31868) - org.apache.manifoldcf.db - Parameter 5: 'D'DEBUG 2015-09-20 10:07:07,992 (Thread-31868) - org.apache.manifoldcf.db - Done actual query (0ms): [SELECT id,lasttime,status,startmethod,connectionname FROM jobs WHERE (status=? OR status=? OR status=? OR status=? OR status=?) AND startmethod!=? FOR UPDATE]DEBUG 2015-09-20 10:07:07,992 (Job start thread) - org.apache.manifoldcf.db - Beginning transaction of type 1DEBUG 2015-09-20 10:07:07,992 (Job start thread) - org.apache.manifoldcf.db - Ending transactionDEBUG 2015-09-20 10:07:07,992 (Job start thread) - org.apache.manifoldcf.db - Committing transaction!DEBUG 2015-09-20 10:07:07,992 (Thread-31869) - org.apache.manifoldcf.db - Actual query: [COMMIT]DEBUG 2015-09-20 10:07:07,992 (Thread-31869) - org.apache.manifoldcf.db - Done actual query (0ms): [COMMIT]DEBUG 2015-09-20 10:07:07,993 (Job start thread) - org.apache.manifoldcf.db - Ending transactionDEBUG 2015-09-20 10:07:07,993 (Job start thread) - org.apache.manifoldcf.db - Beginning transaction of type 1DEBUG 2015-09-20 10:07:07,993 (Job start thread) - org.apache.manifoldcf.db - Requested query: [SELECT id,status FROM jobs WHERE (status=? OR status=? OR status=? OR status=? OR status=? OR status=?) AND windowend<? FOR UPDATE]DEBUG 2015-09-20 10:07:07,993 (Thread-31870) - org.apache.manifoldcf.db - Actual query: [START TRANSACTION]DEBUG 2015-09-20 10:07:07,993 (Thread-31870) - org.apache.manifoldcf.db - Done actual query (0ms): [START TRANSACTION]DEBUG 2015-09-20 10:07:07,993 (Thread-31871) - org.apache.manifoldcf.db - Actual query: [SELECT id,status FROM jobs WHERE (status=? OR status=? OR status=? OR status=? OR status=? OR status=?) AND windowend<? FOR UPDATE]DEBUG 2015-09-20 10:07:07,993 (Thread-31871) - org.apache.manifoldcf.db - Parameter 0: 'A'DEBUG 2015-09-20 10:07:07,993 (Thread-31871) - org.apache.manifoldcf.db - Parameter 1: 'a'DEBUG 2015-09-20 10:07:07,993 (Thread-31871) - org.apache.manifoldcf.db - Parameter 2: 'R'DEBUG 2015-09-20 10:07:07,993 (Thread-31871) - org.apache.manifoldcf.db - Parameter 3: 'r'DEBUG 2015-09-20 10:07:07,993 (Thread-31871) - org.apache.manifoldcf.db - Parameter 4: 'P'DEBUG 2015-09-20 10:07:07,993 (Thread-31871) - org.apache.manifoldcf.db - Parameter 5: 'p'DEBUG 2015-09-20 10:07:07,993 (Thread-31871) - org.apache.manifoldcf.db - Parameter 6: '1442743627991'DEBUG 2015-09-20 10:07:07,994 (Thread-31871) - org.apache.manifoldcf.db - Done actual query (1ms): [SELECT id,status FROM jobs WHERE (status=? OR status=? OR status=? OR status=? OR status=? OR status=?) AND windowend<? FOR UPDATE]DEBUG 2015-09-20 10:07:07,994 (Job start thread) - org.apache.manifoldcf.db - Ending transactionDEBUG 2015-09-20 10:07:07,994 (Job start thread) - org.apache.manifoldcf.db - Committing transaction!DEBUG 2015-09-20 10:07:07,994 (Thread-31872) - org.apache.manifoldcf.db - Actual query: [COMMIT]DEBUG 2015-09-20 10:07:07,994 (Thread-31872) - org.apache.manifoldcf.db - Done actual query (0ms): [COMMIT]DEBUG 2015-09-20 10:07:08,274 (Stuffer thread) - org.apache.manifoldcf.crawlerthreads - Document stuffer thread woke up
RE: Ingestion Job in Starting Up state for over 16 hours
Posted by Mike Caceres <mi...@hotmail.com>.
sure, this is what I get from AgentRun with -Dorg.apache.manifoldcf.processid=a
2015-09-20 11:24:24Full thread dump OpenJDK 64-Bit Server VM (24.75-b04 mixed mode):
"Attach Listener" daemon prio=10 tid=0x00007fcb58002000 nid=0x28c3 waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE
"Thread-518841" daemon prio=10 tid=0x00007fcb30007000 nid=0x19b7 runnable [0x00007fcbdb7f6000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:152) at java.net.SocketInputStream.read(SocketInputStream.java:122) at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) at java.io.BufferedInputStream.read(BufferedInputStream.java:254) - locked <0x00000006c3347fd8> (a java.io.BufferedInputStream) at java.io.DataInputStream.readByte(DataInputStream.java:265) at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:214) at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161) at java.rmi.server.RemoteObjectInvocationHandler.invokeRemoteMethod(RemoteObjectInvocationHandler.java:194) at java.rmi.server.RemoteObjectInvocationHandler.invoke(RemoteObjectInvocationHandler.java:148) at com.sun.proxy.$Proxy1.performDQLQuery(Unknown Source) at org.apache.manifoldcf.crawler.connectors.DCTM.DCTM$GetDocumentsFromQueryThread.run(DCTM.java:858)
"GC Daemon" daemon prio=10 tid=0x00007fc81c042000 nid=0x1b993 in Object.wait() [0x00007fcbdb0ef000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at sun.misc.GC$Daemon.run(GC.java:117) - locked <0x00000006c3000f78> (a sun.misc.GC$LatencyLock)
"RMI RenewClean-[54.40.222.21:34322,org.apache.manifoldcf.crawler.common.DCTM.RMILocalClientSocketFactory@3f1aa]" daemon prio=10 tid=0x00007fc81c03f000 nid=0x1b992 in Object.wait() [0x00007fcbdb5f4000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135) - locked <0x00000006c30057b8> (a java.lang.ref.ReferenceQueue$Lock) at sun.rmi.transport.DGCClient$EndpointEntry$RenewCleanThread.run(DGCClient.java:535) at java.lang.Thread.run(Thread.java:745)
"RMI Scheduler(0)" daemon prio=10 tid=0x00007fc81c03e000 nid=0x1b991 waiting on condition [0x00007fcbf35f4000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000006c3019080> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745)
"Assessment thread" daemon prio=10 tid=0x00007fcb44285000 nid=0x1b5d5 in Object.wait() [0x00007fcbdbaf9000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.java:1290) - locked <0x00000007bfb82c10> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.AssessmentThread.run(AssessmentThread.java:68)
"Idle cleanup thread" daemon prio=10 tid=0x00007fcb44283000 nid=0x1b5d4 in Object.wait() [0x00007fcbdbbfa000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.java:1290) - locked <0x00000007be661f80> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.IdleCleanupThread.run(IdleCleanupThread.java:74)
"Seeding thread" daemon prio=10 tid=0x00007fcb44281000 nid=0x1b5d3 in Object.wait() [0x00007fcbdbcfb000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.java:1290) - locked <0x00000007b75b8cc8> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.SeedingThread.run(SeedingThread.java:104)
"Job reset thread" daemon prio=10 tid=0x00007fcb4427f000 nid=0x1b5d2 in Object.wait() [0x00007fcbdbdfc000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.java:1290) - locked <0x00000007bf8cad58> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.JobResetThread.run(JobResetThread.java:123)
"Document delete thread '24'" daemon prio=10 tid=0x00007fcb4427d000 nid=0x1b5d1 in Object.wait() [0x00007fcbdbefd000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3027eb0> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocuments(DocumentDeleteQueue.java:100) - locked <0x00000006c3027eb0> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:95)
"Document delete thread '23'" daemon prio=10 tid=0x00007fcb4427a800 nid=0x1b5d0 in Object.wait() [0x00007fcbdbffe000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3027eb0> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocuments(DocumentDeleteQueue.java:100) - locked <0x00000006c3027eb0> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:95)
"Document delete thread '22'" daemon prio=10 tid=0x00007fcb44278800 nid=0x1b5cf in Object.wait() [0x00007fcbe01c0000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3027eb0> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocuments(DocumentDeleteQueue.java:100) - locked <0x00000006c3027eb0> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:95)
"Document delete thread '21'" daemon prio=10 tid=0x00007fcb44276800 nid=0x1b5ce in Object.wait() [0x00007fcbe02c1000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3027eb0> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocuments(DocumentDeleteQueue.java:100) - locked <0x00000006c3027eb0> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:95)
"Document delete thread '20'" daemon prio=10 tid=0x00007fcb44274800 nid=0x1b5cd in Object.wait() [0x00007fcbe03c2000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3027eb0> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocuments(DocumentDeleteQueue.java:100) - locked <0x00000006c3027eb0> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:95)
"Document delete thread '19'" daemon prio=10 tid=0x00007fcb44272800 nid=0x1b5cc in Object.wait() [0x00007fcbe04c3000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3027eb0> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocuments(DocumentDeleteQueue.java:100) - locked <0x00000006c3027eb0> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:95)
"Document delete thread '18'" daemon prio=10 tid=0x00007fcb44270800 nid=0x1b5cb in Object.wait() [0x00007fcbe05c4000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3027eb0> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocuments(DocumentDeleteQueue.java:100) - locked <0x00000006c3027eb0> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:95)
"Document delete thread '17'" daemon prio=10 tid=0x00007fcb4426e800 nid=0x1b5ca in Object.wait() [0x00007fcbe06c5000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3027eb0> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocuments(DocumentDeleteQueue.java:100) - locked <0x00000006c3027eb0> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:95)
"Document delete thread '16'" daemon prio=10 tid=0x00007fcb4426c800 nid=0x1b5c9 in Object.wait() [0x00007fcbe07c6000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3027eb0> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocuments(DocumentDeleteQueue.java:100) - locked <0x00000006c3027eb0> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:95)
"Document delete thread '15'" daemon prio=10 tid=0x00007fcb4426a000 nid=0x1b5c8 in Object.wait() [0x00007fcbe08c7000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3027eb0> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocuments(DocumentDeleteQueue.java:100) - locked <0x00000006c3027eb0> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:95)
"Document delete thread '14'" daemon prio=10 tid=0x00007fcb44268000 nid=0x1b5c7 in Object.wait() [0x00007fcbe09c8000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3027eb0> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocuments(DocumentDeleteQueue.java:100) - locked <0x00000006c3027eb0> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:95)
"Document delete thread '13'" daemon prio=10 tid=0x00007fcb44266000 nid=0x1b5c6 in Object.wait() [0x00007fcbe0ac9000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3027eb0> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocuments(DocumentDeleteQueue.java:100) - locked <0x00000006c3027eb0> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:95)
"Document delete thread '12'" daemon prio=10 tid=0x00007fcb44264000 nid=0x1b5c5 in Object.wait() [0x00007fcbe0bca000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3027eb0> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocuments(DocumentDeleteQueue.java:100) - locked <0x00000006c3027eb0> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:95)
"Document delete thread '11'" daemon prio=10 tid=0x00007fcb44262000 nid=0x1b5c4 in Object.wait() [0x00007fcbe0ccb000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3027eb0> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocuments(DocumentDeleteQueue.java:100) - locked <0x00000006c3027eb0> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:95)
"Document delete thread '10'" daemon prio=10 tid=0x00007fcb44260000 nid=0x1b5c3 in Object.wait() [0x00007fcbe0dcc000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3027eb0> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocuments(DocumentDeleteQueue.java:100) - locked <0x00000006c3027eb0> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:95)
"Document delete thread '9'" daemon prio=10 tid=0x00007fcb4425e000 nid=0x1b5c2 in Object.wait() [0x00007fcbe0ecd000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3027eb0> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocuments(DocumentDeleteQueue.java:100) - locked <0x00000006c3027eb0> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:95)
"Document delete thread '8'" daemon prio=10 tid=0x00007fcb4425b800 nid=0x1b5c1 in Object.wait() [0x00007fcbe0fce000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3027eb0> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocuments(DocumentDeleteQueue.java:100) - locked <0x00000006c3027eb0> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:95)
"Document delete thread '7'" daemon prio=10 tid=0x00007fcb44259800 nid=0x1b5c0 in Object.wait() [0x00007fcbe10cf000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3027eb0> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocuments(DocumentDeleteQueue.java:100) - locked <0x00000006c3027eb0> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:95)
"Document delete thread '6'" daemon prio=10 tid=0x00007fcb44257800 nid=0x1b5bf in Object.wait() [0x00007fcbe11d0000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3027eb0> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocuments(DocumentDeleteQueue.java:100) - locked <0x00000006c3027eb0> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:95)
"Document delete thread '5'" daemon prio=10 tid=0x00007fcb44255800 nid=0x1b5be in Object.wait() [0x00007fcbe12d1000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3027eb0> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocuments(DocumentDeleteQueue.java:100) - locked <0x00000006c3027eb0> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:95)
"Document delete thread '4'" daemon prio=10 tid=0x00007fcb44253800 nid=0x1b5bd in Object.wait() [0x00007fcbe13d2000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3027eb0> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocuments(DocumentDeleteQueue.java:100) - locked <0x00000006c3027eb0> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:95)
"Document delete thread '3'" daemon prio=10 tid=0x00007fcb44251800 nid=0x1b5bc in Object.wait() [0x00007fcbe14d3000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3027eb0> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocuments(DocumentDeleteQueue.java:100) - locked <0x00000006c3027eb0> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:95)
"Document delete thread '2'" daemon prio=10 tid=0x00007fcb4424f800 nid=0x1b5bb in Object.wait() [0x00007fcbe15d4000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3027eb0> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocuments(DocumentDeleteQueue.java:100) - locked <0x00000006c3027eb0> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:95)
"Document delete thread '1'" daemon prio=10 tid=0x00007fcb4424d800 nid=0x1b5ba in Object.wait() [0x00007fcbe16d5000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3027eb0> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocuments(DocumentDeleteQueue.java:100) - locked <0x00000006c3027eb0> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:95)
"Document delete thread '0'" daemon prio=10 tid=0x00007fcb4424b000 nid=0x1b5b9 in Object.wait() [0x00007fcbe17d6000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3027eb0> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentDeleteQueue.getDocuments(DocumentDeleteQueue.java:100) - locked <0x00000006c3027eb0> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentDeleteThread.run(DocumentDeleteThread.java:95)
"Document delete stuffer thread" daemon prio=10 tid=0x00007fcb44249000 nid=0x1b5b8 in Object.wait() [0x00007fcbe18d7000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.java:1290) - locked <0x00000007bf74da60> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.DocumentDeleteStufferThread.run(DocumentDeleteStufferThread.java:115)
"Document cleanup thread '24'" daemon prio=10 tid=0x00007fcb44247000 nid=0x1b5b7 in Object.wait() [0x00007fcbe19d8000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3047d28> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c3047d28> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(DocumentCleanupThread.java:106)
"Document cleanup thread '23'" daemon prio=10 tid=0x00007fcb44245000 nid=0x1b5b6 in Object.wait() [0x00007fcbe1ad9000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3047d28> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c3047d28> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(DocumentCleanupThread.java:106)
"Document cleanup thread '22'" daemon prio=10 tid=0x00007fcb44243000 nid=0x1b5b5 in Object.wait() [0x00007fcbe1bda000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3047d28> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c3047d28> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(DocumentCleanupThread.java:106)
"Document cleanup thread '21'" daemon prio=10 tid=0x00007fcb44241000 nid=0x1b5b4 in Object.wait() [0x00007fcbe1cdb000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3047d28> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c3047d28> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(DocumentCleanupThread.java:106)
"Document cleanup thread '20'" daemon prio=10 tid=0x00007fcb4423f000 nid=0x1b5b3 in Object.wait() [0x00007fcbe1ddc000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3047d28> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c3047d28> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(DocumentCleanupThread.java:106)
"Document cleanup thread '19'" daemon prio=10 tid=0x00007fcb4423c800 nid=0x1b5b2 in Object.wait() [0x00007fcbe1edd000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3047d28> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c3047d28> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(DocumentCleanupThread.java:106)
"Document cleanup thread '18'" daemon prio=10 tid=0x00007fcb4423a800 nid=0x1b5b1 in Object.wait() [0x00007fcbe1fde000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3047d28> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c3047d28> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(DocumentCleanupThread.java:106)
"Document cleanup thread '17'" daemon prio=10 tid=0x00007fcb44238800 nid=0x1b5b0 in Object.wait() [0x00007fcbe20df000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3047d28> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c3047d28> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(DocumentCleanupThread.java:106)
"Document cleanup thread '16'" daemon prio=10 tid=0x00007fcb44236800 nid=0x1b5af in Object.wait() [0x00007fcbe21e0000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3047d28> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c3047d28> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(DocumentCleanupThread.java:106)
"Document cleanup thread '15'" daemon prio=10 tid=0x00007fcb44234800 nid=0x1b5ae in Object.wait() [0x00007fcbe22e1000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3047d28> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c3047d28> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(DocumentCleanupThread.java:106)
"Document cleanup thread '14'" daemon prio=10 tid=0x00007fcb44232800 nid=0x1b5ad in Object.wait() [0x00007fcbe23e2000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3047d28> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c3047d28> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(DocumentCleanupThread.java:106)
"Document cleanup thread '13'" daemon prio=10 tid=0x00007fcb44230800 nid=0x1b5ac in Object.wait() [0x00007fcbe24e3000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3047d28> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c3047d28> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(DocumentCleanupThread.java:106)
"Document cleanup thread '12'" daemon prio=10 tid=0x00007fcb4422e800 nid=0x1b5ab in Object.wait() [0x00007fcbe25e4000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3047d28> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c3047d28> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(DocumentCleanupThread.java:106)
"Document cleanup thread '11'" daemon prio=10 tid=0x00007fcb4422c000 nid=0x1b5aa in Object.wait() [0x00007fcbe26e5000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3047d28> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c3047d28> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(DocumentCleanupThread.java:106)
"Document cleanup thread '10'" daemon prio=10 tid=0x00007fcb4422a000 nid=0x1b5a9 in Object.wait() [0x00007fcbe27e6000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3047d28> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c3047d28> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(DocumentCleanupThread.java:106)
"Document cleanup thread '9'" daemon prio=10 tid=0x00007fcb44228000 nid=0x1b5a8 in Object.wait() [0x00007fcbe28e7000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3047d28> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c3047d28> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(DocumentCleanupThread.java:106)
"Document cleanup thread '8'" daemon prio=10 tid=0x00007fcb44226000 nid=0x1b5a7 in Object.wait() [0x00007fcbe29e8000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3047d28> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c3047d28> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(DocumentCleanupThread.java:106)
"Document cleanup thread '7'" daemon prio=10 tid=0x00007fcb44224000 nid=0x1b5a6 in Object.wait() [0x00007fcbe2ae9000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3047d28> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c3047d28> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(DocumentCleanupThread.java:106)
"Document cleanup thread '6'" daemon prio=10 tid=0x00007fcb44222000 nid=0x1b5a5 in Object.wait() [0x00007fcbe2bea000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3047d28> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c3047d28> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(DocumentCleanupThread.java:106)
"Document cleanup thread '5'" daemon prio=10 tid=0x00007fcb44220000 nid=0x1b5a4 in Object.wait() [0x00007fcbe2ceb000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3047d28> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c3047d28> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(DocumentCleanupThread.java:106)
"Document cleanup thread '4'" daemon prio=10 tid=0x00007fcb4421d800 nid=0x1b5a3 in Object.wait() [0x00007fcbe2dec000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3047d28> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c3047d28> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(DocumentCleanupThread.java:106)
"Document cleanup thread '3'" daemon prio=10 tid=0x00007fcb4421b800 nid=0x1b5a2 in Object.wait() [0x00007fcbe2eed000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3047d28> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c3047d28> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(DocumentCleanupThread.java:106)
"Document cleanup thread '2'" daemon prio=10 tid=0x00007fcb44219800 nid=0x1b5a1 in Object.wait() [0x00007fcbe2fee000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3047d28> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c3047d28> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(DocumentCleanupThread.java:106)
"Document cleanup thread '1'" daemon prio=10 tid=0x00007fcb44217800 nid=0x1b5a0 in Object.wait() [0x00007fcbe30ef000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3047d28> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c3047d28> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(DocumentCleanupThread.java:106)
"Document cleanup thread '0'" daemon prio=10 tid=0x00007fcb44215800 nid=0x1b59f in Object.wait() [0x00007fcbe31f0000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3047d28> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c3047d28> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.DocumentCleanupThread.run(DocumentCleanupThread.java:106)
"Document cleanup stuffer thread" daemon prio=10 tid=0x00007fcb44213800 nid=0x1b59e in Object.wait() [0x00007fcbe32f1000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.java:1290) - locked <0x00000007be8d5070> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.DocumentCleanupStufferThread.run(DocumentCleanupStufferThread.java:117)
"Expiration thread '24'" daemon prio=10 tid=0x00007fcb44211800 nid=0x1b59d in Object.wait() [0x00007fcbe33f2000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c305fd68> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c305fd68> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThread.java:95)
"Expiration thread '23'" daemon prio=10 tid=0x00007fcb4420f800 nid=0x1b59c in Object.wait() [0x00007fcbe34f3000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c305fd68> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c305fd68> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThread.java:95)
"Expiration thread '22'" daemon prio=10 tid=0x00007fcb4420d000 nid=0x1b59b in Object.wait() [0x00007fcbe35f4000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c305fd68> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c305fd68> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThread.java:95)
"Expiration thread '21'" daemon prio=10 tid=0x00007fcb4420b000 nid=0x1b59a in Object.wait() [0x00007fcbe36f5000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c305fd68> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c305fd68> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThread.java:95)
"Expiration thread '20'" daemon prio=10 tid=0x00007fcb44209000 nid=0x1b599 in Object.wait() [0x00007fcbe37f6000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c305fd68> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c305fd68> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThread.java:95)
"Expiration thread '19'" daemon prio=10 tid=0x00007fcb44207000 nid=0x1b598 in Object.wait() [0x00007fcbe38f7000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c305fd68> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c305fd68> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThread.java:95)
"Expiration thread '18'" daemon prio=10 tid=0x00007fcb44205000 nid=0x1b597 in Object.wait() [0x00007fcbe39f8000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c305fd68> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c305fd68> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThread.java:95)
"Expiration thread '17'" daemon prio=10 tid=0x00007fcb44203000 nid=0x1b596 in Object.wait() [0x00007fcbe3af9000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c305fd68> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c305fd68> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThread.java:95)
"Expiration thread '16'" daemon prio=10 tid=0x00007fcb44201000 nid=0x1b595 in Object.wait() [0x00007fcbe3bfa000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c305fd68> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c305fd68> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThread.java:95)
"Expiration thread '15'" daemon prio=10 tid=0x00007fcb441fe800 nid=0x1b594 in Object.wait() [0x00007fcbe3cfb000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c305fd68> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c305fd68> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThread.java:95)
"Expiration thread '14'" daemon prio=10 tid=0x00007fcb441fc800 nid=0x1b593 in Object.wait() [0x00007fcbe3dfc000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c305fd68> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c305fd68> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThread.java:95)
"Expiration thread '13'" daemon prio=10 tid=0x00007fcb441fa800 nid=0x1b592 in Object.wait() [0x00007fcbe3efd000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c305fd68> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c305fd68> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThread.java:95)
"Expiration thread '12'" daemon prio=10 tid=0x00007fcb441f8800 nid=0x1b591 in Object.wait() [0x00007fcbe3ffe000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c305fd68> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c305fd68> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThread.java:95)
"Expiration thread '11'" daemon prio=10 tid=0x00007fcb441f6800 nid=0x1b590 in Object.wait() [0x00007fcbe81c0000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c305fd68> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c305fd68> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThread.java:95)
"Expiration thread '10'" daemon prio=10 tid=0x00007fcb441f4800 nid=0x1b58f in Object.wait() [0x00007fcbe82c1000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c305fd68> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c305fd68> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThread.java:95)
"Expiration thread '9'" daemon prio=10 tid=0x00007fcb441f2800 nid=0x1b58e in Object.wait() [0x00007fcbe83c2000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c305fd68> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c305fd68> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThread.java:95)
"Expiration thread '8'" daemon prio=10 tid=0x00007fcb441f0800 nid=0x1b58d in Object.wait() [0x00007fcbe84c3000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c305fd68> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c305fd68> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThread.java:95)
"Expiration thread '7'" daemon prio=10 tid=0x00007fcb441ee000 nid=0x1b58c in Object.wait() [0x00007fcbe85c4000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c305fd68> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c305fd68> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThread.java:95)
"Expiration thread '6'" daemon prio=10 tid=0x00007fcb441ec000 nid=0x1b58b in Object.wait() [0x00007fcbe86c5000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c305fd68> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c305fd68> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThread.java:95)
"Expiration thread '5'" daemon prio=10 tid=0x00007fcb441ea000 nid=0x1b58a in Object.wait() [0x00007fcbe87c6000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c305fd68> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c305fd68> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThread.java:95)
"Expiration thread '4'" daemon prio=10 tid=0x00007fcb441e8000 nid=0x1b589 in Object.wait() [0x00007fcbe88c7000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c305fd68> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c305fd68> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThread.java:95)
"Expiration thread '3'" daemon prio=10 tid=0x00007fcb441e6000 nid=0x1b588 in Object.wait() [0x00007fcbe89c8000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c305fd68> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c305fd68> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThread.java:95)
"Expiration thread '2'" daemon prio=10 tid=0x00007fcb441e4000 nid=0x1b587 in Object.wait() [0x00007fcbe8ac9000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c305fd68> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c305fd68> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThread.java:95)
"Expiration thread '1'" daemon prio=10 tid=0x00007fcb441e2000 nid=0x1b586 in Object.wait() [0x00007fcbe8bca000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c305fd68> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c305fd68> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThread.java:95)
"Expiration thread '0'" daemon prio=10 tid=0x00007fcb441e0000 nid=0x1b585 in Object.wait() [0x00007fcbe8ccb000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c305fd68> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentCleanupQueue.getDocuments(DocumentCleanupQueue.java:107) - locked <0x00000006c305fd68> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.ExpireThread.run(ExpireThread.java:95)
"Worker thread '99'" daemon prio=10 tid=0x00007fcb441dd800 nid=0x1b584 in Object.wait() [0x00007fcbe8dcc000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '98'" daemon prio=10 tid=0x00007fcb441db800 nid=0x1b583 in Object.wait() [0x00007fcbe8ecd000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '97'" daemon prio=10 tid=0x00007fcb441d9800 nid=0x1b582 in Object.wait() [0x00007fcbe8fce000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '96'" daemon prio=10 tid=0x00007fcb441d7800 nid=0x1b581 in Object.wait() [0x00007fcbe90cf000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '95'" daemon prio=10 tid=0x00007fcb441d5800 nid=0x1b580 in Object.wait() [0x00007fcbe91d0000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '94'" daemon prio=10 tid=0x00007fcb441d3800 nid=0x1b57f in Object.wait() [0x00007fcbe92d1000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '93'" daemon prio=10 tid=0x00007fcb441d1800 nid=0x1b57e in Object.wait() [0x00007fcbe93d2000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '92'" daemon prio=10 tid=0x00007fcb441cf000 nid=0x1b57d in Object.wait() [0x00007fcbe94d3000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '91'" daemon prio=10 tid=0x00007fcb441cd000 nid=0x1b57c in Object.wait() [0x00007fcbe95d4000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '90'" daemon prio=10 tid=0x00007fcb441cb000 nid=0x1b57b in Object.wait() [0x00007fcbe96d5000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '89'" daemon prio=10 tid=0x00007fcb441c9000 nid=0x1b57a in Object.wait() [0x00007fcbe97d6000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '88'" daemon prio=10 tid=0x00007fcb441c7000 nid=0x1b579 in Object.wait() [0x00007fcbe98d7000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '87'" daemon prio=10 tid=0x00007fcb441c5000 nid=0x1b578 in Object.wait() [0x00007fcbe99d8000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '86'" daemon prio=10 tid=0x00007fcb441c3000 nid=0x1b577 in Object.wait() [0x00007fcbe9ad9000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '85'" daemon prio=10 tid=0x00007fcb441c1000 nid=0x1b576 in Object.wait() [0x00007fcbe9bda000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '84'" daemon prio=10 tid=0x00007fcb441be800 nid=0x1b575 in Object.wait() [0x00007fcbe9cdb000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '83'" daemon prio=10 tid=0x00007fcb441bc800 nid=0x1b574 in Object.wait() [0x00007fcbe9ddc000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '82'" daemon prio=10 tid=0x00007fcb441ba800 nid=0x1b573 in Object.wait() [0x00007fcbe9edd000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '81'" daemon prio=10 tid=0x00007fcb441b8800 nid=0x1b572 in Object.wait() [0x00007fcbe9fde000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '80'" daemon prio=10 tid=0x00007fcb441b6800 nid=0x1b571 in Object.wait() [0x00007fcbea0df000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '79'" daemon prio=10 tid=0x00007fcb441b4800 nid=0x1b570 in Object.wait() [0x00007fcbea1e0000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '78'" daemon prio=10 tid=0x00007fcb441b2800 nid=0x1b56f in Object.wait() [0x00007fcbea2e1000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '77'" daemon prio=10 tid=0x00007fcb441b0000 nid=0x1b56e in Object.wait() [0x00007fcbea3e2000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '76'" daemon prio=10 tid=0x00007fcb441ae000 nid=0x1b56d in Object.wait() [0x00007fcbea4e3000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '75'" daemon prio=10 tid=0x00007fcb441ac000 nid=0x1b56c in Object.wait() [0x00007fcbea5e4000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '74'" daemon prio=10 tid=0x00007fcb441aa000 nid=0x1b56b in Object.wait() [0x00007fcbea6e5000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '73'" daemon prio=10 tid=0x00007fcb441a8000 nid=0x1b56a in Object.wait() [0x00007fcbea7e6000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '72'" daemon prio=10 tid=0x00007fcb441a6000 nid=0x1b569 in Object.wait() [0x00007fcbea8e7000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '71'" daemon prio=10 tid=0x00007fcb441a4000 nid=0x1b568 in Object.wait() [0x00007fcbea9e8000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '70'" daemon prio=10 tid=0x00007fcb441a2000 nid=0x1b567 in Object.wait() [0x00007fcbeaae9000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '69'" daemon prio=10 tid=0x00007fcb4419f800 nid=0x1b566 in Object.wait() [0x00007fcbeabea000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '68'" daemon prio=10 tid=0x00007fcb4419d800 nid=0x1b565 in Object.wait() [0x00007fcbeaceb000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '67'" daemon prio=10 tid=0x00007fcb4419b800 nid=0x1b564 in Object.wait() [0x00007fcbeadec000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '66'" daemon prio=10 tid=0x00007fcb44199800 nid=0x1b563 in Object.wait() [0x00007fcbeaeed000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '65'" daemon prio=10 tid=0x00007fcb44197800 nid=0x1b562 in Object.wait() [0x00007fcbeafee000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '64'" daemon prio=10 tid=0x00007fcb44195800 nid=0x1b561 in Object.wait() [0x00007fcbeb0ef000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '63'" daemon prio=10 tid=0x00007fcb44193800 nid=0x1b560 in Object.wait() [0x00007fcbeb1f0000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '62'" daemon prio=10 tid=0x00007fcb44191000 nid=0x1b55f in Object.wait() [0x00007fcbeb2f1000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '61'" daemon prio=10 tid=0x00007fcb4418f000 nid=0x1b55e in Object.wait() [0x00007fcbeb3f2000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '60'" daemon prio=10 tid=0x00007fcb4418d000 nid=0x1b55d in Object.wait() [0x00007fcbeb4f3000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '59'" daemon prio=10 tid=0x00007fcb4418b000 nid=0x1b55c in Object.wait() [0x00007fcbeb5f4000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '58'" daemon prio=10 tid=0x00007fcb44189000 nid=0x1b55b in Object.wait() [0x00007fcbeb6f5000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '57'" daemon prio=10 tid=0x00007fcb44187000 nid=0x1b55a in Object.wait() [0x00007fcbeb7f6000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '56'" daemon prio=10 tid=0x00007fcb44185000 nid=0x1b559 in Object.wait() [0x00007fcbeb8f7000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '55'" daemon prio=10 tid=0x00007fcb44183000 nid=0x1b558 in Object.wait() [0x00007fcbeb9f8000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '54'" daemon prio=10 tid=0x00007fcb44180800 nid=0x1b557 in Object.wait() [0x00007fcbebaf9000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '53'" daemon prio=10 tid=0x00007fcb4417e800 nid=0x1b556 in Object.wait() [0x00007fcbebbfa000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '52'" daemon prio=10 tid=0x00007fcb4417c800 nid=0x1b555 in Object.wait() [0x00007fcbebcfb000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '51'" daemon prio=10 tid=0x00007fcb4417a800 nid=0x1b554 in Object.wait() [0x00007fcbebdfc000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '50'" daemon prio=10 tid=0x00007fcb44178800 nid=0x1b553 in Object.wait() [0x00007fcbebefd000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '49'" daemon prio=10 tid=0x00007fcb44176800 nid=0x1b552 in Object.wait() [0x00007fcbebffe000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '48'" daemon prio=10 tid=0x00007fcb44174800 nid=0x1b551 in Object.wait() [0x00007fcbf01c0000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '47'" daemon prio=10 tid=0x00007fcb44172000 nid=0x1b550 in Object.wait() [0x00007fcbf02c1000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '46'" daemon prio=10 tid=0x00007fcb44170000 nid=0x1b54f in Object.wait() [0x00007fcbf03c2000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '45'" daemon prio=10 tid=0x00007fcb4416e000 nid=0x1b54e in Object.wait() [0x00007fcbf04c3000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '44'" daemon prio=10 tid=0x00007fcb4416c000 nid=0x1b54d in Object.wait() [0x00007fcbf05c4000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '43'" daemon prio=10 tid=0x00007fcb4416a000 nid=0x1b54c in Object.wait() [0x00007fcbf06c5000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '42'" daemon prio=10 tid=0x00007fcb44168000 nid=0x1b54b in Object.wait() [0x00007fcbf07c6000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '41'" daemon prio=10 tid=0x00007fcb44166000 nid=0x1b54a in Object.wait() [0x00007fcbf08c7000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '40'" daemon prio=10 tid=0x00007fcb44164000 nid=0x1b549 in Object.wait() [0x00007fcbf09c8000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '39'" daemon prio=10 tid=0x00007fcb44161800 nid=0x1b548 in Object.wait() [0x00007fcbf0ac9000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '38'" daemon prio=10 tid=0x00007fcb4415f800 nid=0x1b547 in Object.wait() [0x00007fcbf0bca000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '37'" daemon prio=10 tid=0x00007fcb4415d800 nid=0x1b546 in Object.wait() [0x00007fcbf0ccb000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '36'" daemon prio=10 tid=0x00007fcb4415b800 nid=0x1b545 in Object.wait() [0x00007fcbf0dcc000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '35'" daemon prio=10 tid=0x00007fcb44159800 nid=0x1b544 in Object.wait() [0x00007fcbf0ecd000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '34'" daemon prio=10 tid=0x00007fcb44157800 nid=0x1b543 in Object.wait() [0x00007fcbf0fce000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '33'" daemon prio=10 tid=0x00007fcb44155800 nid=0x1b542 in Object.wait() [0x00007fcbf10cf000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '32'" daemon prio=10 tid=0x00007fcb44153000 nid=0x1b541 in Object.wait() [0x00007fcbf11d0000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '31'" daemon prio=10 tid=0x00007fcb44151000 nid=0x1b540 in Object.wait() [0x00007fcbf12d1000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '30'" daemon prio=10 tid=0x00007fcb4414f000 nid=0x1b53f in Object.wait() [0x00007fcbf13d2000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '29'" daemon prio=10 tid=0x00007fcb4414d000 nid=0x1b53e in Object.wait() [0x00007fcbf14d3000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '28'" daemon prio=10 tid=0x00007fcb4414b000 nid=0x1b53d in Object.wait() [0x00007fcbf15d4000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '27'" daemon prio=10 tid=0x00007fcb44149000 nid=0x1b53c in Object.wait() [0x00007fcbf16d5000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '26'" daemon prio=10 tid=0x00007fcb44147000 nid=0x1b53b in Object.wait() [0x00007fcbf17d6000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '25'" daemon prio=10 tid=0x00007fcb44145000 nid=0x1b53a in Object.wait() [0x00007fcbf18d7000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '24'" daemon prio=10 tid=0x00007fcb44142800 nid=0x1b539 in Object.wait() [0x00007fcbf19d8000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '23'" daemon prio=10 tid=0x00007fcb44140800 nid=0x1b538 in Object.wait() [0x00007fcbf1ad9000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '22'" daemon prio=10 tid=0x00007fcb4413e800 nid=0x1b537 in Object.wait() [0x00007fcbf1bda000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '21'" daemon prio=10 tid=0x00007fcb4413c800 nid=0x1b536 in Object.wait() [0x00007fcbf1cdb000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '20'" daemon prio=10 tid=0x00007fcb4413a800 nid=0x1b535 in Object.wait() [0x00007fcbf1ddc000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '19'" daemon prio=10 tid=0x00007fcb44138800 nid=0x1b534 in Object.wait() [0x00007fcbf1edd000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '18'" daemon prio=10 tid=0x00007fcb44136800 nid=0x1b533 in Object.wait() [0x00007fcbf1fde000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '17'" daemon prio=10 tid=0x00007fcb44134000 nid=0x1b531 in Object.wait() [0x00007fcbf20df000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '16'" daemon prio=10 tid=0x00007fcb44132000 nid=0x1b530 in Object.wait() [0x00007fcbf21e0000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '15'" daemon prio=10 tid=0x00007fcb44130000 nid=0x1b52f in Object.wait() [0x00007fcbf22e1000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '14'" daemon prio=10 tid=0x00007fcb4412e000 nid=0x1b52e in Object.wait() [0x00007fcbf23e2000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '13'" daemon prio=10 tid=0x00007fcb4412c000 nid=0x1b52d in Object.wait() [0x00007fcbf24e3000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '12'" daemon prio=10 tid=0x00007fcb4412a000 nid=0x1b52c in Object.wait() [0x00007fcbf25e4000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '11'" daemon prio=10 tid=0x00007fcb44128000 nid=0x1b52b in Object.wait() [0x00007fcbf26e5000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '10'" daemon prio=10 tid=0x00007fcb44126000 nid=0x1b52a in Object.wait() [0x00007fcbf27e6000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '9'" daemon prio=10 tid=0x00007fcb44124000 nid=0x1b529 in Object.wait() [0x00007fcbf28e7000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '8'" daemon prio=10 tid=0x00007fcb44122800 nid=0x1b528 in Object.wait() [0x00007fcbf29e8000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '7'" daemon prio=10 tid=0x00007fcb44121000 nid=0x1b527 in Object.wait() [0x00007fcbf2ae9000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '6'" daemon prio=10 tid=0x00007fcb4411f000 nid=0x1b526 in Object.wait() [0x00007fcbf2bea000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '5'" daemon prio=10 tid=0x00007fcb4411c800 nid=0x1b525 in Object.wait() [0x00007fcbf2ceb000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '4'" daemon prio=10 tid=0x00007fcb4411a800 nid=0x1b524 in Object.wait() [0x00007fcbf2dec000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '3'" daemon prio=10 tid=0x00007fcb44118800 nid=0x1b523 in Object.wait() [0x00007fcbf2eed000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '2'" daemon prio=10 tid=0x00007fcb44116800 nid=0x1b522 in Object.wait() [0x00007fcbf2fee000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '1'" daemon prio=10 tid=0x00007fcb44114800 nid=0x1b521 in Object.wait() [0x00007fcbf30ef000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"Worker thread '0'" daemon prio=10 tid=0x00007fcb44112800 nid=0x1b520 in Object.wait() [0x00007fcbf31f0000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000006c3045888> (a java.util.ArrayList) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.system.DocumentQueue.getDocument(DocumentQueue.java:110) - locked <0x00000006c3045888> (a java.util.ArrayList) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:121)
"History cleanup thread" daemon prio=10 tid=0x00007fcb44110800 nid=0x1b51f in Object.wait() [0x00007fcbf32f1000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.java:1290) - locked <0x00000006c342ad90> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.HistoryCleanupThread.run(HistoryCleanupThread.java:78)
"Set priority thread" daemon prio=10 tid=0x00007fcb4410e800 nid=0x1b51e in Object.wait() [0x00007fcbf33f2000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.java:1290) - locked <0x00000007be6c1590> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.SetPriorityThread.run(SetPriorityThread.java:148)
"Expire stuffer thread" daemon prio=10 tid=0x00007fcb4410c000 nid=0x1b51d in Object.wait() [0x00007fcbf34f3000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.java:1290) - locked <0x00000007be55b3d8> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.ExpireStufferThread.run(ExpireStufferThread.java:137)
"Stuffer thread" daemon prio=10 tid=0x00007fcb4410a000 nid=0x1b518 in Object.wait() [0x00007fcbf36f5000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.java:1290) - locked <0x00000007bef15600> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.StufferThread.run(StufferThread.java:206)
"Job delete thread" daemon prio=10 tid=0x00007fcb44108800 nid=0x1b516 in Object.wait() [0x00007fcbf37f6000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.java:1290) - locked <0x00000007bed9f320> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.JobDeleteThread.run(JobDeleteThread.java:78)
"Job notification thread" daemon prio=10 tid=0x00007fcb44107000 nid=0x1b515 in Object.wait() [0x00007fcbf38f7000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.java:1290) - locked <0x00000007bf27fe48> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.JobNotificationThread.run(JobNotificationThread.java:431)
"Finisher thread" daemon prio=10 tid=0x00007fcb44105800 nid=0x1b514 in Object.wait() [0x00007fcbf39f8000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.java:1290) - locked <0x00000007beefa9f0> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.FinisherThread.run(FinisherThread.java:69)
"Delete startup thread" daemon prio=10 tid=0x00007fcb44104000 nid=0x1b513 in Object.wait() [0x00007fcbf3af9000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.java:1290) - locked <0x00000007bed9f288> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.StartDeleteThread.run(StartDeleteThread.java:93)
"Startup thread" daemon prio=10 tid=0x00007fcb440fc800 nid=0x1b512 in Object.wait() [0x00007fcbf3bfa000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) at org.apache.manifoldcf.crawler.connectors.DCTM.DCTM$StringQueue.getNext(DCTM.java:794) - locked <0x00000007c0d50088> (a org.apache.manifoldcf.crawler.connectors.DCTM.DCTM$StringQueue) at org.apache.manifoldcf.crawler.connectors.DCTM.DCTM.addSeedDocuments(DCTM.java:1284) at org.apache.manifoldcf.crawler.system.StartupThread.run(StartupThread.java:154)
"Job start thread" daemon prio=10 tid=0x00007fcb440fb800 nid=0x1b511 in Object.wait() [0x00007fcbf3cfb000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.java:1290) - locked <0x00000007bf697a90> (a java.lang.Integer) at org.apache.manifoldcf.crawler.system.JobStartThread.run(JobStartThread.java:96)
"Connection pool reaper" daemon prio=10 tid=0x00007fcb44024800 nid=0x1b4b0 waiting on condition [0x00007fcbf3dfc000] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.apache.manifoldcf.core.jdbcpool.ConnectionPoolManager$ConnectionCloserThread.run(ConnectionPoolManager.java:138)
"Agents thread" daemon prio=10 tid=0x00007fcc1c5d3800 nid=0x1b4af in Object.wait() [0x00007fcbf3efd000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.java:1290) - locked <0x00000007be61a6f0> (a java.lang.Integer) at org.apache.manifoldcf.agents.system.AgentsDaemon$AgentsThread.run(AgentsDaemon.java:210)
"Idle cleanup thread" daemon prio=10 tid=0x00007fcc1c5e4800 nid=0x1b4ae in Object.wait() [0x00007fcbf3ffe000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.java:1290) - locked <0x00000007b77de438> (a java.lang.Integer) at org.apache.manifoldcf.agents.system.IdleCleanupThread.run(IdleCleanupThread.java:96)
"Service Thread" daemon prio=10 tid=0x00007fcc1c0d6000 nid=0x1b4ac runnable [0x0000000000000000] java.lang.Thread.State: RUNNABLE
"C2 CompilerThread1" daemon prio=10 tid=0x00007fcc1c0d3800 nid=0x1b4ab waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE
"C2 CompilerThread0" daemon prio=10 tid=0x00007fcc1c0d0800 nid=0x1b4aa waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE
"Signal Dispatcher" daemon prio=10 tid=0x00007fcc1c0ce800 nid=0x1b4a9 runnable [0x0000000000000000] java.lang.Thread.State: RUNNABLE
"Finalizer" daemon prio=10 tid=0x00007fcc1c0a3800 nid=0x1b4a3 in Object.wait() [0x00007fcbfbbfa000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135) - locked <0x00000006c3027ae0> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151) at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)
"Reference Handler" daemon prio=10 tid=0x00007fcc1c0a1800 nid=0x1b4a2 in Object.wait() [0x00007fcb71c4e000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133) - locked <0x00000006c3027800> (a java.lang.ref.Reference$Lock)
"main" prio=10 tid=0x00007fcc1c008800 nid=0x1b46f in Object.wait() [0x00007fcc23805000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.manifoldcf.core.system.ManifoldCF.sleep(ManifoldCF.java:1290) - locked <0x00000007be8ca178> (a java.lang.Integer) at org.apache.manifoldcf.agents.system.AgentsDaemon.runAgents(AgentsDaemon.java:115) at org.apache.manifoldcf.agents.AgentRun.doExecute(AgentRun.java:64) at org.apache.manifoldcf.agents.BaseAgentsInitializationCommand.execute(BaseAgentsInitializationCommand.java:37) at org.apache.manifoldcf.agents.AgentRun.main(AgentRun.java:93)
"VM Thread" prio=10 tid=0x00007fcc1c09d000 nid=0x1b4a1 runnable
"GC task thread#0 (ParallelGC)" prio=10 tid=0x00007fcc1c01e800 nid=0x1b487 runnable
"GC task thread#1 (ParallelGC)" prio=10 tid=0x00007fcc1c020000 nid=0x1b488 runnable
"GC task thread#2 (ParallelGC)" prio=10 tid=0x00007fcc1c022000 nid=0x1b489 runnable
"GC task thread#3 (ParallelGC)" prio=10 tid=0x00007fcc1c024000 nid=0x1b48a runnable
"GC task thread#4 (ParallelGC)" prio=10 tid=0x00007fcc1c025800 nid=0x1b48b runnable
"GC task thread#5 (ParallelGC)" prio=10 tid=0x00007fcc1c027800 nid=0x1b48c runnable
"GC task thread#6 (ParallelGC)" prio=10 tid=0x00007fcc1c029800 nid=0x1b48d runnable
"GC task thread#7 (ParallelGC)" prio=10 tid=0x00007fcc1c02b000 nid=0x1b48e runnable
"GC task thread#8 (ParallelGC)" prio=10 tid=0x00007fcc1c02d000 nid=0x1b48f runnable
"GC task thread#9 (ParallelGC)" prio=10 tid=0x00007fcc1c02f000 nid=0x1b490 runnable
"GC task thread#10 (ParallelGC)" prio=10 tid=0x00007fcc1c031000 nid=0x1b491 runnable
"GC task thread#11 (ParallelGC)" prio=10 tid=0x00007fcc1c032800 nid=0x1b492 runnable
"GC task thread#12 (ParallelGC)" prio=10 tid=0x00007fcc1c034800 nid=0x1b493 runnable
"GC task thread#13 (ParallelGC)" prio=10 tid=0x00007fcc1c036800 nid=0x1b494 runnable
"GC task thread#14 (ParallelGC)" prio=10 tid=0x00007fcc1c038000 nid=0x1b495 runnable
"GC task thread#15 (ParallelGC)" prio=10 tid=0x00007fcc1c03a000 nid=0x1b496 runnable
"GC task thread#16 (ParallelGC)" prio=10 tid=0x00007fcc1c03c000 nid=0x1b497 runnable
"GC task thread#17 (ParallelGC)" prio=10 tid=0x00007fcc1c03e000 nid=0x1b498 runnable
"GC task thread#18 (ParallelGC)" prio=10 tid=0x00007fcc1c03f800 nid=0x1b499 runnable
"GC task thread#19 (ParallelGC)" prio=10 tid=0x00007fcc1c041800 nid=0x1b49a runnable
"GC task thread#20 (ParallelGC)" prio=10 tid=0x00007fcc1c043800 nid=0x1b49b runnable
"GC task thread#21 (ParallelGC)" prio=10 tid=0x00007fcc1c045000 nid=0x1b49c runnable
"GC task thread#22 (ParallelGC)" prio=10 tid=0x00007fcc1c047000 nid=0x1b49d runnable
"VM Periodic Task Thread" prio=10 tid=0x00007fcc1c0e0800 nid=0x1b4ad waiting on condition
JNI global references: 169
Date: Sun, 20 Sep 2015 06:54:36 -0400
Subject: Re: Ingestion Job in Starting Up state for over 16 hours
From: daddywri@gmail.com
To: user@manifoldcf.apache.org
The "starting up" state means that MCF is resetting all documents to be PendingPurgatory, and setting other document states and statuses. I have seen situations where this takes longer than some configured Postgresql timeout and then the initialization does not complete, and is rolled back. I didn't think Postgresql could fail silently in this way, but if it does you should see indication of an error in the postgresql logs.
The best way of figuring out what MCF is up to is getting a thread dump of the agents process. If you could get that and send it along, I can tell you more.
Karl
On Sun, Sep 20, 2015 at 6:39 AM, Mike Caceres <mi...@hotmail.com> wrote:
Hi,
I am running a job to ingest documents from Documentum. Things are generally working, but lately the jobs seems to hang for hours in the Starting Up status.
The version of ManifoldCF is 2.1.
Here is what I see in Postgres and a bit of the log file, which seems to show a cycle over and over trying to pick up jobs, but not getting the ones with status B.
Is there anything I can do to make the job change state to Active?
Thank you!Mike
dbname=# select id, description, status, startmethod from jobs; id | description | status | startmethod---------------+--------------------------+--------+------------- 1442492810054 | full-crawl-production-06 | B | D
dbname=# select j.description, j.status, jq.status, jq.seedingprocessid, count (*) from jobqueue jq right join jobs j on j.id = jq.jobid group by j.description, j.status, jq.status, jq.seedingprocessid order by j.description, j.status, jq.status; description | status | status | seedingprocessid | count--------------------------+--------+--------+------------------+-------- full-crawl-production-06 | B | P | A | 12795
DEBUG 2015-09-20 10:07:07,780 (Delete startup thread) - org.apache.manifoldcf.db - Requested query: [SELECT id FROM jobs WHERE status=? FOR UPDATE]DEBUG 2015-09-20 10:07:07,780 (Thread-31855) - org.apache.manifoldcf.db - Actual query: [START TRANSACTION]DEBUG 2015-09-20 10:07:07,780 (Thread-31855) - org.apache.manifoldcf.db - Done actual query (0ms): [START TRANSACTION]DEBUG 2015-09-20 10:07:07,781 (Thread-31856) - org.apache.manifoldcf.db - Actual query: [SELECT id FROM jobs WHERE status=? FOR UPDATE]DEBUG 2015-09-20 10:07:07,781 (Thread-31856) - org.apache.manifoldcf.db - Parameter 0: 'E'DEBUG 2015-09-20 10:07:07,781 (Thread-31856) - org.apache.manifoldcf.db - Done actual query (0ms): [SELECT id FROM jobs WHERE status=? FOR UPDATE]DEBUG 2015-09-20 10:07:07,781 (Delete startup thread) - org.apache.manifoldcf.db - Committing transaction!DEBUG 2015-09-20 10:07:07,781 (Thread-31857) - org.apache.manifoldcf.db - Actual query: [COMMIT]DEBUG 2015-09-20 10:07:07,781 (Thread-31857) - org.apache.manifoldcf.db - Done actual query (0ms): [COMMIT]DEBUG 2015-09-20 10:07:07,782 (Delete startup thread) - org.apache.manifoldcf.db - Ending transactionDEBUG 2015-09-20 10:07:07,860 (Job reset thread) - org.apache.manifoldcf.db - Requested query: [SELECT id FROM jobs WHERE (status=? OR status=? OR status=? OR status=? OR status=? OR status=? OR status=? OR status=? OR status=? OR status=?)]DEBUG 2015-09-20 10:07:07,860 (Thread-31858) - org.apache.manifoldcf.db - Actual query: [SELECT id FROM jobs WHERE (status=? OR status=? OR status=? OR status=? OR status=? OR status=? OR status=? OR status=? OR status=? OR status=?)]DEBUG 2015-09-20 10:07:07,860 (Thread-31858) - org.apache.manifoldcf.db - Parameter 0: 'X'DEBUG 2015-09-20 10:07:07,860 (Thread-31858) - org.apache.manifoldcf.db - Parameter 1: 'Y'DEBUG 2015-09-20 10:07:07,860 (Thread-31858) - org.apache.manifoldcf.db - Parameter 2: 'M'DEBUG 2015-09-20 10:07:07,860 (Thread-31858) - org.apache.manifoldcf.db - Parameter 3: 'v'DEBUG 2015-09-20 10:07:07,860 (Thread-31858) - org.apache.manifoldcf.db - Parameter 4: 'F'DEBUG 2015-09-20 10:07:07,860 (Thread-31858) - org.apache.manifoldcf.db - Parameter 5: 'f'DEBUG 2015-09-20 10:07:07,860 (Thread-31858) - org.apache.manifoldcf.db - Parameter 6: 'H'DEBUG 2015-09-20 10:07:07,860 (Thread-31858) - org.apache.manifoldcf.db - Parameter 7: 'h'DEBUG 2015-09-20 10:07:07,860 (Thread-31858) - org.apache.manifoldcf.db - Parameter 8: 'G'DEBUG 2015-09-20 10:07:07,860 (Thread-31858) - org.apache.manifoldcf.db - Parameter 9: 'g'DEBUG 2015-09-20 10:07:07,860 (Thread-31858) - org.apache.manifoldcf.db - Done actual query (0ms): [SELECT id FROM jobs WHERE (status=? OR status=? OR status=? OR status=? OR status=? OR status=? OR status=? OR status=? OR status=? OR status=?)]DEBUG 2015-09-20 10:07:07,861 (Job reset thread) - org.apache.manifoldcf.db - Requested query: [SELECT id FROM jobs WHERE (status=? OR status=?)]DEBUG 2015-09-20 10:07:07,861 (Thread-31859) - org.apache.manifoldcf.db - Actual query: [SELECT id FROM jobs WHERE (status=? OR status=?)]DEBUG 2015-09-20 10:07:07,861 (Thread-31859) - org.apache.manifoldcf.db - Parameter 0: 'I'DEBUG 2015-09-20 10:07:07,861 (Thread-31859) - org.apache.manifoldcf.db - Parameter 1: 'i'DEBUG 2015-09-20 10:07:07,861 (Thread-31859) - org.apache.manifoldcf.db - Done actual query (0ms): [SELECT id FROM jobs WHERE (status=? OR status=?)]DEBUG 2015-09-20 10:07:07,861 (Job reset thread) - org.apache.manifoldcf.db - Requested query: [SELECT id FROM jobs WHERE status=?]DEBUG 2015-09-20 10:07:07,862 (Thread-31860) - org.apache.manifoldcf.db - Actual query: [SELECT id FROM jobs WHERE status=?]DEBUG 2015-09-20 10:07:07,862 (Thread-31860) - org.apache.manifoldcf.db - Parameter 0: 'S'DEBUG 2015-09-20 10:07:07,862 (Thread-31860) - org.apache.manifoldcf.db - Done actual query (0ms): [SELECT id FROM jobs WHERE status=?]DEBUG 2015-09-20 10:07:07,979 (Job notification thread) - org.apache.manifoldcf.db - Beginning transaction of type 1DEBUG 2015-09-20 10:07:07,979 (Job notification thread) - org.apache.manifoldcf.db - Requested query: [SELECT id,failtime,failcount FROM jobs WHERE status=? FOR UPDATE]DEBUG 2015-09-20 10:07:07,979 (Thread-31861) - org.apache.manifoldcf.db - Actual query: [START TRANSACTION]DEBUG 2015-09-20 10:07:07,980 (Thread-31861) - org.apache.manifoldcf.db - Done actual query (1ms): [START TRANSACTION]DEBUG 2015-09-20 10:07:07,980 (Thread-31862) - org.apache.manifoldcf.db - Actual query: [SELECT id,failtime,failcount FROM jobs WHERE status=? FOR UPDATE]DEBUG 2015-09-20 10:07:07,980 (Thread-31862) - org.apache.manifoldcf.db - Parameter 0: 's'DEBUG 2015-09-20 10:07:07,980 (Thread-31862) - org.apache.manifoldcf.db - Done actual query (0ms): [SELECT id,failtime,failcount FROM jobs WHERE status=? FOR UPDATE]DEBUG 2015-09-20 10:07:07,980 (Job notification thread) - org.apache.manifoldcf.db - Committing transaction!DEBUG 2015-09-20 10:07:07,981 (Thread-31863) - org.apache.manifoldcf.db - Actual query: [COMMIT]DEBUG 2015-09-20 10:07:07,981 (Thread-31863) - org.apache.manifoldcf.db - Done actual query (0ms): [COMMIT]DEBUG 2015-09-20 10:07:07,981 (Job notification thread) - org.apache.manifoldcf.db - Ending transactionDEBUG 2015-09-20 10:07:07,981 (Job notification thread) - org.apache.manifoldcf.db - Beginning transaction of type 1DEBUG 2015-09-20 10:07:07,981 (Job notification thread) - org.apache.manifoldcf.db - Requested query: [SELECT id,failtime,failcount FROM jobs WHERE status=? FOR UPDATE]DEBUG 2015-09-20 10:07:07,981 (Thread-31864) - org.apache.manifoldcf.db - Actual query: [START TRANSACTION]DEBUG 2015-09-20 10:07:07,981 (Thread-31864) - org.apache.manifoldcf.db - Done actual query (0ms): [START TRANSACTION]DEBUG 2015-09-20 10:07:07,982 (Thread-31865) - org.apache.manifoldcf.db - Actual query: [SELECT id,failtime,failcount FROM jobs WHERE status=? FOR UPDATE]DEBUG 2015-09-20 10:07:07,982 (Thread-31865) - org.apache.manifoldcf.db - Parameter 0: 'd'DEBUG 2015-09-20 10:07:07,982 (Thread-31865) - org.apache.manifoldcf.db - Done actual query (0ms): [SELECT id,failtime,failcount FROM jobs WHERE status=? FOR UPDATE]DEBUG 2015-09-20 10:07:07,982 (Job notification thread) - org.apache.manifoldcf.db - Committing transaction!DEBUG 2015-09-20 10:07:07,982 (Thread-31866) - org.apache.manifoldcf.db - Actual query: [COMMIT]DEBUG 2015-09-20 10:07:07,982 (Thread-31866) - org.apache.manifoldcf.db - Done actual query (0ms): [COMMIT]DEBUG 2015-09-20 10:07:07,982 (Job notification thread) - org.apache.manifoldcf.db - Ending transactionDEBUG 2015-09-20 10:07:07,991 (Job start thread) - org.apache.manifoldcf.crawlerthreads - Job start thread - checking for jobs to start at 1442743627991DEBUG 2015-09-20 10:07:07,991 (Job start thread) - org.apache.manifoldcf.db - Beginning transaction of type 1DEBUG 2015-09-20 10:07:07,991 (Job start thread) - org.apache.manifoldcf.db - Requested query: [SELECT id,lasttime,status,startmethod,connectionname FROM jobs WHERE (status=? OR status=? OR status=? OR status=? OR status=?) AND startmethod!=? FOR UPDATE]DEBUG 2015-09-20 10:07:07,991 (Thread-31867) - org.apache.manifoldcf.db - Actual query: [START TRANSACTION]DEBUG 2015-09-20 10:07:07,991 (Thread-31867) - org.apache.manifoldcf.db - Done actual query (0ms): [START TRANSACTION]DEBUG 2015-09-20 10:07:07,992 (Thread-31868) - org.apache.manifoldcf.db - Actual query: [SELECT id,lasttime,status,startmethod,connectionname FROM jobs WHERE (status=? OR status=? OR status=? OR status=? OR status=?) AND startmethod!=? FOR UPDATE]DEBUG 2015-09-20 10:07:07,992 (Thread-31868) - org.apache.manifoldcf.db - Parameter 0: 'N'DEBUG 2015-09-20 10:07:07,992 (Thread-31868) - org.apache.manifoldcf.db - Parameter 1: 'W'DEBUG 2015-09-20 10:07:07,992 (Thread-31868) - org.apache.manifoldcf.db - Parameter 2: 'w'DEBUG 2015-09-20 10:07:07,992 (Thread-31868) - org.apache.manifoldcf.db - Parameter 3: 'Z'DEBUG 2015-09-20 10:07:07,992 (Thread-31868) - org.apache.manifoldcf.db - Parameter 4: 'z'DEBUG 2015-09-20 10:07:07,992 (Thread-31868) - org.apache.manifoldcf.db - Parameter 5: 'D'DEBUG 2015-09-20 10:07:07,992 (Thread-31868) - org.apache.manifoldcf.db - Done actual query (0ms): [SELECT id,lasttime,status,startmethod,connectionname FROM jobs WHERE (status=? OR status=? OR status=? OR status=? OR status=?) AND startmethod!=? FOR UPDATE]DEBUG 2015-09-20 10:07:07,992 (Job start thread) - org.apache.manifoldcf.db - Beginning transaction of type 1DEBUG 2015-09-20 10:07:07,992 (Job start thread) - org.apache.manifoldcf.db - Ending transactionDEBUG 2015-09-20 10:07:07,992 (Job start thread) - org.apache.manifoldcf.db - Committing transaction!DEBUG 2015-09-20 10:07:07,992 (Thread-31869) - org.apache.manifoldcf.db - Actual query: [COMMIT]DEBUG 2015-09-20 10:07:07,992 (Thread-31869) - org.apache.manifoldcf.db - Done actual query (0ms): [COMMIT]DEBUG 2015-09-20 10:07:07,993 (Job start thread) - org.apache.manifoldcf.db - Ending transactionDEBUG 2015-09-20 10:07:07,993 (Job start thread) - org.apache.manifoldcf.db - Beginning transaction of type 1DEBUG 2015-09-20 10:07:07,993 (Job start thread) - org.apache.manifoldcf.db - Requested query: [SELECT id,status FROM jobs WHERE (status=? OR status=? OR status=? OR status=? OR status=? OR status=?) AND windowend<? FOR UPDATE]DEBUG 2015-09-20 10:07:07,993 (Thread-31870) - org.apache.manifoldcf.db - Actual query: [START TRANSACTION]DEBUG 2015-09-20 10:07:07,993 (Thread-31870) - org.apache.manifoldcf.db - Done actual query (0ms): [START TRANSACTION]DEBUG 2015-09-20 10:07:07,993 (Thread-31871) - org.apache.manifoldcf.db - Actual query: [SELECT id,status FROM jobs WHERE (status=? OR status=? OR status=? OR status=? OR status=? OR status=?) AND windowend<? FOR UPDATE]DEBUG 2015-09-20 10:07:07,993 (Thread-31871) - org.apache.manifoldcf.db - Parameter 0: 'A'DEBUG 2015-09-20 10:07:07,993 (Thread-31871) - org.apache.manifoldcf.db - Parameter 1: 'a'DEBUG 2015-09-20 10:07:07,993 (Thread-31871) - org.apache.manifoldcf.db - Parameter 2: 'R'DEBUG 2015-09-20 10:07:07,993 (Thread-31871) - org.apache.manifoldcf.db - Parameter 3: 'r'DEBUG 2015-09-20 10:07:07,993 (Thread-31871) - org.apache.manifoldcf.db - Parameter 4: 'P'DEBUG 2015-09-20 10:07:07,993 (Thread-31871) - org.apache.manifoldcf.db - Parameter 5: 'p'DEBUG 2015-09-20 10:07:07,993 (Thread-31871) - org.apache.manifoldcf.db - Parameter 6: '1442743627991'DEBUG 2015-09-20 10:07:07,994 (Thread-31871) - org.apache.manifoldcf.db - Done actual query (1ms): [SELECT id,status FROM jobs WHERE (status=? OR status=? OR status=? OR status=? OR status=? OR status=?) AND windowend<? FOR UPDATE]DEBUG 2015-09-20 10:07:07,994 (Job start thread) - org.apache.manifoldcf.db - Ending transactionDEBUG 2015-09-20 10:07:07,994 (Job start thread) - org.apache.manifoldcf.db - Committing transaction!DEBUG 2015-09-20 10:07:07,994 (Thread-31872) - org.apache.manifoldcf.db - Actual query: [COMMIT]DEBUG 2015-09-20 10:07:07,994 (Thread-31872) - org.apache.manifoldcf.db - Done actual query (0ms): [COMMIT]DEBUG 2015-09-20 10:07:08,274 (Stuffer thread) - org.apache.manifoldcf.crawlerthreads - Document stuffer thread woke up
Re: Ingestion Job in Starting Up state for over 16 hours
Posted by Karl Wright <da...@gmail.com>.
The "starting up" state means that MCF is resetting all documents to be
PendingPurgatory, and setting other document states and statuses. I have
seen situations where this takes longer than some configured Postgresql
timeout and then the initialization does not complete, and is rolled back.
I didn't think Postgresql could fail silently in this way, but if it does
you should see indication of an error in the postgresql logs.
The best way of figuring out what MCF is up to is getting a thread dump of
the agents process. If you could get that and send it along, I can tell
you more.
Karl
On Sun, Sep 20, 2015 at 6:39 AM, Mike Caceres <mi...@hotmail.com> wrote:
> Hi,
>
> I am running a job to ingest documents from Documentum. Things are
> generally working, but lately the jobs seems to hang for hours in the
> Starting Up status.
>
> The version of ManifoldCF is 2.1.
>
> Here is what I see in Postgres and a bit of the log file, which seems to
> show a cycle over and over trying to pick up jobs, but not getting the ones
> with status B.
>
> Is there anything I can do to make the job change state to Active?
>
> Thank you!
> Mike
>
>
> dbname=# select id, description, status, startmethod from jobs;
> id | description | status | startmethod
> ---------------+--------------------------+--------+-------------
> 1442492810054 | full-crawl-production-06 | B | D
>
> dbname=# select j.description, j.status, jq.status, jq.seedingprocessid,
> count (*) from jobqueue jq right join jobs j on j.id = jq.jobid group by
> j.description, j.status, jq.status, jq.seedingprocessid order by
> j.description, j.status, jq.status;
> description | status | status | seedingprocessid | count
> --------------------------+--------+--------+------------------+--------
> full-crawl-production-06 | B | P | A | 12795
>
>
> DEBUG 2015-09-20 10:07:07,780 (Delete startup thread) -
> org.apache.manifoldcf.db - Requested query: [SELECT id FROM jobs WHERE
> status=? FOR UPDATE]
> DEBUG 2015-09-20 10:07:07,780 (Thread-31855) - org.apache.manifoldcf.db -
> Actual query: [START TRANSACTION]
> DEBUG 2015-09-20 10:07:07,780 (Thread-31855) - org.apache.manifoldcf.db -
> Done actual query (0ms): [START TRANSACTION]
> DEBUG 2015-09-20 10:07:07,781 (Thread-31856) - org.apache.manifoldcf.db -
> Actual query: [SELECT id FROM jobs WHERE status=? FOR UPDATE]
> DEBUG 2015-09-20 10:07:07,781 (Thread-31856) - org.apache.manifoldcf.db -
> Parameter 0: 'E'
> DEBUG 2015-09-20 10:07:07,781 (Thread-31856) - org.apache.manifoldcf.db -
> Done actual query (0ms): [SELECT id FROM jobs WHERE status=? FOR UPDATE]
> DEBUG 2015-09-20 10:07:07,781 (Delete startup thread) -
> org.apache.manifoldcf.db - Committing transaction!
> DEBUG 2015-09-20 10:07:07,781 (Thread-31857) - org.apache.manifoldcf.db -
> Actual query: [COMMIT]
> DEBUG 2015-09-20 10:07:07,781 (Thread-31857) - org.apache.manifoldcf.db -
> Done actual query (0ms): [COMMIT]
> DEBUG 2015-09-20 10:07:07,782 (Delete startup thread) -
> org.apache.manifoldcf.db - Ending transaction
> DEBUG 2015-09-20 10:07:07,860 (Job reset thread) -
> org.apache.manifoldcf.db - Requested query: [SELECT id FROM jobs WHERE
> (status=? OR status=? OR status=? OR status=? OR status=? OR status=? OR
> status=? OR status=? OR status=? OR status=?)]
> DEBUG 2015-09-20 10:07:07,860 (Thread-31858) - org.apache.manifoldcf.db -
> Actual query: [SELECT id FROM jobs WHERE (status=? OR status=? OR status=?
> OR status=? OR status=? OR status=? OR status=? OR status=? OR status=? OR
> status=?)]
> DEBUG 2015-09-20 10:07:07,860 (Thread-31858) - org.apache.manifoldcf.db -
> Parameter 0: 'X'
> DEBUG 2015-09-20 10:07:07,860 (Thread-31858) - org.apache.manifoldcf.db -
> Parameter 1: 'Y'
> DEBUG 2015-09-20 10:07:07,860 (Thread-31858) - org.apache.manifoldcf.db -
> Parameter 2: 'M'
> DEBUG 2015-09-20 10:07:07,860 (Thread-31858) - org.apache.manifoldcf.db -
> Parameter 3: 'v'
> DEBUG 2015-09-20 10:07:07,860 (Thread-31858) - org.apache.manifoldcf.db -
> Parameter 4: 'F'
> DEBUG 2015-09-20 10:07:07,860 (Thread-31858) - org.apache.manifoldcf.db -
> Parameter 5: 'f'
> DEBUG 2015-09-20 10:07:07,860 (Thread-31858) - org.apache.manifoldcf.db -
> Parameter 6: 'H'
> DEBUG 2015-09-20 10:07:07,860 (Thread-31858) - org.apache.manifoldcf.db -
> Parameter 7: 'h'
> DEBUG 2015-09-20 10:07:07,860 (Thread-31858) - org.apache.manifoldcf.db -
> Parameter 8: 'G'
> DEBUG 2015-09-20 10:07:07,860 (Thread-31858) - org.apache.manifoldcf.db -
> Parameter 9: 'g'
> DEBUG 2015-09-20 10:07:07,860 (Thread-31858) - org.apache.manifoldcf.db -
> Done actual query (0ms): [SELECT id FROM jobs WHERE (status=? OR status=?
> OR status=? OR status=? OR status=? OR status=? OR status=? OR status=? OR
> status=? OR status=?)]
> DEBUG 2015-09-20 10:07:07,861 (Job reset thread) -
> org.apache.manifoldcf.db - Requested query: [SELECT id FROM jobs WHERE
> (status=? OR status=?)]
> DEBUG 2015-09-20 10:07:07,861 (Thread-31859) - org.apache.manifoldcf.db -
> Actual query: [SELECT id FROM jobs WHERE (status=? OR status=?)]
> DEBUG 2015-09-20 10:07:07,861 (Thread-31859) - org.apache.manifoldcf.db -
> Parameter 0: 'I'
> DEBUG 2015-09-20 10:07:07,861 (Thread-31859) - org.apache.manifoldcf.db -
> Parameter 1: 'i'
> DEBUG 2015-09-20 10:07:07,861 (Thread-31859) - org.apache.manifoldcf.db -
> Done actual query (0ms): [SELECT id FROM jobs WHERE (status=? OR status=?)]
> DEBUG 2015-09-20 10:07:07,861 (Job reset thread) -
> org.apache.manifoldcf.db - Requested query: [SELECT id FROM jobs WHERE
> status=?]
> DEBUG 2015-09-20 10:07:07,862 (Thread-31860) - org.apache.manifoldcf.db -
> Actual query: [SELECT id FROM jobs WHERE status=?]
> DEBUG 2015-09-20 10:07:07,862 (Thread-31860) - org.apache.manifoldcf.db -
> Parameter 0: 'S'
> DEBUG 2015-09-20 10:07:07,862 (Thread-31860) - org.apache.manifoldcf.db -
> Done actual query (0ms): [SELECT id FROM jobs WHERE status=?]
> DEBUG 2015-09-20 10:07:07,979 (Job notification thread) -
> org.apache.manifoldcf.db - Beginning transaction of type 1
> DEBUG 2015-09-20 10:07:07,979 (Job notification thread) -
> org.apache.manifoldcf.db - Requested query: [SELECT id,failtime,failcount
> FROM jobs WHERE status=? FOR UPDATE]
> DEBUG 2015-09-20 10:07:07,979 (Thread-31861) - org.apache.manifoldcf.db -
> Actual query: [START TRANSACTION]
> DEBUG 2015-09-20 10:07:07,980 (Thread-31861) - org.apache.manifoldcf.db -
> Done actual query (1ms): [START TRANSACTION]
> DEBUG 2015-09-20 10:07:07,980 (Thread-31862) - org.apache.manifoldcf.db -
> Actual query: [SELECT id,failtime,failcount FROM jobs WHERE status=? FOR
> UPDATE]
> DEBUG 2015-09-20 10:07:07,980 (Thread-31862) - org.apache.manifoldcf.db -
> Parameter 0: 's'
> DEBUG 2015-09-20 10:07:07,980 (Thread-31862) - org.apache.manifoldcf.db -
> Done actual query (0ms): [SELECT id,failtime,failcount FROM jobs WHERE
> status=? FOR UPDATE]
> DEBUG 2015-09-20 10:07:07,980 (Job notification thread) -
> org.apache.manifoldcf.db - Committing transaction!
> DEBUG 2015-09-20 10:07:07,981 (Thread-31863) - org.apache.manifoldcf.db -
> Actual query: [COMMIT]
> DEBUG 2015-09-20 10:07:07,981 (Thread-31863) - org.apache.manifoldcf.db -
> Done actual query (0ms): [COMMIT]
> DEBUG 2015-09-20 10:07:07,981 (Job notification thread) -
> org.apache.manifoldcf.db - Ending transaction
> DEBUG 2015-09-20 10:07:07,981 (Job notification thread) -
> org.apache.manifoldcf.db - Beginning transaction of type 1
> DEBUG 2015-09-20 10:07:07,981 (Job notification thread) -
> org.apache.manifoldcf.db - Requested query: [SELECT id,failtime,failcount
> FROM jobs WHERE status=? FOR UPDATE]
> DEBUG 2015-09-20 10:07:07,981 (Thread-31864) - org.apache.manifoldcf.db -
> Actual query: [START TRANSACTION]
> DEBUG 2015-09-20 10:07:07,981 (Thread-31864) - org.apache.manifoldcf.db -
> Done actual query (0ms): [START TRANSACTION]
> DEBUG 2015-09-20 10:07:07,982 (Thread-31865) - org.apache.manifoldcf.db -
> Actual query: [SELECT id,failtime,failcount FROM jobs WHERE status=? FOR
> UPDATE]
> DEBUG 2015-09-20 10:07:07,982 (Thread-31865) - org.apache.manifoldcf.db -
> Parameter 0: 'd'
> DEBUG 2015-09-20 10:07:07,982 (Thread-31865) - org.apache.manifoldcf.db -
> Done actual query (0ms): [SELECT id,failtime,failcount FROM jobs WHERE
> status=? FOR UPDATE]
> DEBUG 2015-09-20 10:07:07,982 (Job notification thread) -
> org.apache.manifoldcf.db - Committing transaction!
> DEBUG 2015-09-20 10:07:07,982 (Thread-31866) - org.apache.manifoldcf.db -
> Actual query: [COMMIT]
> DEBUG 2015-09-20 10:07:07,982 (Thread-31866) - org.apache.manifoldcf.db -
> Done actual query (0ms): [COMMIT]
> DEBUG 2015-09-20 10:07:07,982 (Job notification thread) -
> org.apache.manifoldcf.db - Ending transaction
> DEBUG 2015-09-20 10:07:07,991 (Job start thread) -
> org.apache.manifoldcf.crawlerthreads - Job start thread - checking for jobs
> to start at 1442743627991
> DEBUG 2015-09-20 10:07:07,991 (Job start thread) -
> org.apache.manifoldcf.db - Beginning transaction of type 1
> DEBUG 2015-09-20 10:07:07,991 (Job start thread) -
> org.apache.manifoldcf.db - Requested query: [SELECT
> id,lasttime,status,startmethod,connectionname FROM jobs WHERE (status=? OR
> status=? OR status=? OR status=? OR status=?) AND startmethod!=? FOR UPDATE]
> DEBUG 2015-09-20 10:07:07,991 (Thread-31867) - org.apache.manifoldcf.db -
> Actual query: [START TRANSACTION]
> DEBUG 2015-09-20 10:07:07,991 (Thread-31867) - org.apache.manifoldcf.db -
> Done actual query (0ms): [START TRANSACTION]
> DEBUG 2015-09-20 10:07:07,992 (Thread-31868) - org.apache.manifoldcf.db -
> Actual query: [SELECT id,lasttime,status,startmethod,connectionname FROM
> jobs WHERE (status=? OR status=? OR status=? OR status=? OR status=?) AND
> startmethod!=? FOR UPDATE]
> DEBUG 2015-09-20 10:07:07,992 (Thread-31868) - org.apache.manifoldcf.db -
> Parameter 0: 'N'
> DEBUG 2015-09-20 10:07:07,992 (Thread-31868) - org.apache.manifoldcf.db -
> Parameter 1: 'W'
> DEBUG 2015-09-20 10:07:07,992 (Thread-31868) - org.apache.manifoldcf.db -
> Parameter 2: 'w'
> DEBUG 2015-09-20 10:07:07,992 (Thread-31868) - org.apache.manifoldcf.db -
> Parameter 3: 'Z'
> DEBUG 2015-09-20 10:07:07,992 (Thread-31868) - org.apache.manifoldcf.db -
> Parameter 4: 'z'
> DEBUG 2015-09-20 10:07:07,992 (Thread-31868) - org.apache.manifoldcf.db -
> Parameter 5: 'D'
> DEBUG 2015-09-20 10:07:07,992 (Thread-31868) - org.apache.manifoldcf.db -
> Done actual query (0ms): [SELECT
> id,lasttime,status,startmethod,connectionname FROM jobs WHERE (status=? OR
> status=? OR status=? OR status=? OR status=?) AND startmethod!=? FOR UPDATE]
> DEBUG 2015-09-20 10:07:07,992 (Job start thread) -
> org.apache.manifoldcf.db - Beginning transaction of type 1
> DEBUG 2015-09-20 10:07:07,992 (Job start thread) -
> org.apache.manifoldcf.db - Ending transaction
> DEBUG 2015-09-20 10:07:07,992 (Job start thread) -
> org.apache.manifoldcf.db - Committing transaction!
> DEBUG 2015-09-20 10:07:07,992 (Thread-31869) - org.apache.manifoldcf.db -
> Actual query: [COMMIT]
> DEBUG 2015-09-20 10:07:07,992 (Thread-31869) - org.apache.manifoldcf.db -
> Done actual query (0ms): [COMMIT]
> DEBUG 2015-09-20 10:07:07,993 (Job start thread) -
> org.apache.manifoldcf.db - Ending transaction
> DEBUG 2015-09-20 10:07:07,993 (Job start thread) -
> org.apache.manifoldcf.db - Beginning transaction of type 1
> DEBUG 2015-09-20 10:07:07,993 (Job start thread) -
> org.apache.manifoldcf.db - Requested query: [SELECT id,status FROM jobs
> WHERE (status=? OR status=? OR status=? OR status=? OR status=? OR
> status=?) AND windowend<? FOR UPDATE]
> DEBUG 2015-09-20 10:07:07,993 (Thread-31870) - org.apache.manifoldcf.db -
> Actual query: [START TRANSACTION]
> DEBUG 2015-09-20 10:07:07,993 (Thread-31870) - org.apache.manifoldcf.db -
> Done actual query (0ms): [START TRANSACTION]
> DEBUG 2015-09-20 10:07:07,993 (Thread-31871) - org.apache.manifoldcf.db -
> Actual query: [SELECT id,status FROM jobs WHERE (status=? OR status=? OR
> status=? OR status=? OR status=? OR status=?) AND windowend<? FOR UPDATE]
> DEBUG 2015-09-20 10:07:07,993 (Thread-31871) - org.apache.manifoldcf.db -
> Parameter 0: 'A'
> DEBUG 2015-09-20 10:07:07,993 (Thread-31871) - org.apache.manifoldcf.db -
> Parameter 1: 'a'
> DEBUG 2015-09-20 10:07:07,993 (Thread-31871) - org.apache.manifoldcf.db -
> Parameter 2: 'R'
> DEBUG 2015-09-20 10:07:07,993 (Thread-31871) - org.apache.manifoldcf.db -
> Parameter 3: 'r'
> DEBUG 2015-09-20 10:07:07,993 (Thread-31871) - org.apache.manifoldcf.db -
> Parameter 4: 'P'
> DEBUG 2015-09-20 10:07:07,993 (Thread-31871) - org.apache.manifoldcf.db -
> Parameter 5: 'p'
> DEBUG 2015-09-20 10:07:07,993 (Thread-31871) - org.apache.manifoldcf.db -
> Parameter 6: '1442743627991'
> DEBUG 2015-09-20 10:07:07,994 (Thread-31871) - org.apache.manifoldcf.db -
> Done actual query (1ms): [SELECT id,status FROM jobs WHERE (status=? OR
> status=? OR status=? OR status=? OR status=? OR status=?) AND windowend<?
> FOR UPDATE]
> DEBUG 2015-09-20 10:07:07,994 (Job start thread) -
> org.apache.manifoldcf.db - Ending transaction
> DEBUG 2015-09-20 10:07:07,994 (Job start thread) -
> org.apache.manifoldcf.db - Committing transaction!
> DEBUG 2015-09-20 10:07:07,994 (Thread-31872) - org.apache.manifoldcf.db -
> Actual query: [COMMIT]
> DEBUG 2015-09-20 10:07:07,994 (Thread-31872) - org.apache.manifoldcf.db -
> Done actual query (0ms): [COMMIT]
> DEBUG 2015-09-20 10:07:08,274 (Stuffer thread) -
> org.apache.manifoldcf.crawlerthreads - Document stuffer thread woke up
>