You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@uima.apache.org by Yi-Wen Liu <yi...@usc.edu> on 2015/11/11 07:47:45 UTC
run fails
Hello,
When I ran the project on DUCC, it finally reached "running" but then it
showed "MonitorWarning", and canceled by driver.
Here's the process info:
And log:
10 Nov 2015 22:30:49,643 INFO HttpWorkerThread - T[33]
HttpWorkerThread.run() Begin Processing Work Items - Thread Id:33
10 Nov 2015 22:30:49,644 INFO HttpWorkerThread - T[34]
HttpWorkerThread.run() Begin Processing Work Items - Thread Id:34
10 Nov 2015 22:31:23,270 ERROR DuccHttpClient - T[34] run
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at java.net.Socket.connect(Socket.java:528)
at java.net.Socket.<init>(Socket.java:425)
at java.net.Socket.<init>(Socket.java:280)
at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80)
at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:122)
at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361)
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
at org.apache.uima.ducc.transport.configuration.jp.DuccHttpClient.execute(DuccHttpClient.java:217)
at org.apache.uima.ducc.transport.configuration.jp.HttpWorkerThread.run(HttpWorkerThread.java:168)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at org.apache.uima.ducc.transport.configuration.jp.UimaServiceThreadFactory$1.run(UimaServiceThreadFactory.java:85)
at java.lang.Thread.run(Thread.java:745)
10 Nov 2015 22:31:23,276 ERROR DuccHttpClient - T[34] run
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at java.net.Socket.connect(Socket.java:528)
at java.net.Socket.<init>(Socket.java:425)
I have no idea why the connection got refused, since when I submitted other
test job it worked well.
I tried to debug using all_in_one local/remote, but it didn't have error
which I thought it was working, however there's no output...
Could it be any privilege setting problem? Really hope somebody can help me
out, I didn't find other keyword useful in log file, the connection refused
error showed up 12 times in total.
Thanks,
Yi-Wen
Re: run fails
Posted by Lou DeGenaro <lo...@gmail.com>.
Yi-Wen,
The DUCC job during runtime comprises two parts, one Job Driver and one or
more Job Processes. The log file snippit you posted comprises the latter.
Have you examined the Job Driver log file for any errors? The Job Driver
log file is found in the same directory and starts with "JD-".
Also I believe you said in an earlier post that you have successfully run
one of the sample jobs, such as 1.job. Is that correct?
Lou.
On Wed, Nov 11, 2015 at 1:47 AM, Yi-Wen Liu <yi...@usc.edu> wrote:
> Hello,
>
> When I ran the project on DUCC, it finally reached "running" but then it
> showed "MonitorWarning", and canceled by driver.
> Here's the process info:
>
> And log:
>
> 10 Nov 2015 22:30:49,643 INFO HttpWorkerThread - T[33] HttpWorkerThread.run() Begin Processing Work Items - Thread Id:33
> 10 Nov 2015 22:30:49,644 INFO HttpWorkerThread - T[34] HttpWorkerThread.run() Begin Processing Work Items - Thread Id:34
> 10 Nov 2015 22:31:23,270 ERROR DuccHttpClient - T[34] run
> java.net.ConnectException: Connection refused
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
> at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at java.net.Socket.connect(Socket.java:579)
> at java.net.Socket.connect(Socket.java:528)
> at java.net.Socket.<init>(Socket.java:425)
> at java.net.Socket.<init>(Socket.java:280)
> at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80)
> at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:122)
> at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
> at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361)
> at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
> at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
> at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
> at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
> at org.apache.uima.ducc.transport.configuration.jp.DuccHttpClient.execute(DuccHttpClient.java:217)
> at org.apache.uima.ducc.transport.configuration.jp.HttpWorkerThread.run(HttpWorkerThread.java:168)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at org.apache.uima.ducc.transport.configuration.jp.UimaServiceThreadFactory$1.run(UimaServiceThreadFactory.java:85)
> at java.lang.Thread.run(Thread.java:745)
> 10 Nov 2015 22:31:23,276 ERROR DuccHttpClient - T[34] run
> java.net.ConnectException: Connection refused
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
> at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at java.net.Socket.connect(Socket.java:579)
> at java.net.Socket.connect(Socket.java:528)
> at java.net.Socket.<init>(Socket.java:425)
>
> I have no idea why the connection got refused, since when I submitted
> other test job it worked well.
>
> I tried to debug using all_in_one local/remote, but it didn't have error
> which I thought it was working, however there's no output...
>
> Could it be any privilege setting problem? Really hope somebody can help
> me out, I didn't find other keyword useful in log file, the connection
> refused error showed up 12 times in total.
>
> Thanks,
> Yi-Wen
>
>
Re: run fails
Posted by Yi-Wen Liu <yi...@usc.edu>.
Hello,
Thanks for your replies!
Yes I have run 1.job successfully before and I did find something in the
driver log file:
10 Nov 2015 22:30:49,850 INFO JobDriver - T[21] advanceJdState
current=Initializing request=Active result=Active
10 Nov 2015 22:30:49,982 ERROR ProxyLogger - T[18] loggify
org.apache.uima.cas.CASRuntimeException: JCas type
"org.apache.ctakes.typesystem.type.structured.DocumentID" used in Java
code, but was not declared in the XML type descriptor.
at org.apache.uima.jcas.impl.JCasImpl.getType(JCasImpl.java:412)
at org.apache.uima.jcas.cas.TOP.<init>(TOP.java:92)
at
org.apache.ctakes.typesystem.type.structured.DocumentID.<init>(DocumentID.java:52)
at
Thanks for your suggestions :) I will look into it.
Thanks,
Yi-Wen
On Wed, Nov 11, 2015 at 4:59 AM, Eddie Epstein <ea...@gmail.com> wrote:
> Hi Yi-Wen,
>
> On Wed, Nov 11, 2015 at 1:47 AM, Yi-Wen Liu <yi...@usc.edu> wrote:
>
> > Hello,
> >
> > When I ran the project on DUCC, it finally reached "running" but then it
> > showed "MonitorWarning", and canceled by driver.
> >
>
> Cancelled by driver indicates that the Job Driver process failed, probably
> because of a problem in the application CR.
>
>
> >
> > I tried to debug using all_in_one local/remote, but it didn't have error
> > which I thought it was working, however there's no output...
> >
>
> No output is not right if some was expected.
>
>
> >
> > Could it be any privilege setting problem?
> >
>
> Are there any custom environmental parameters required for the application
> code? DUCC does *not* pass on the environment active when submitting work.
> Any parameters must be explicitly specified.
>
> Is DUCC running as user=ducc? If so, is ducc_ling properly configured to
> spawn application processes as the submitting user?
>
>
> > Really hope somebody can help me out, I didn't find other keyword useful
> > in log file, the connection refused error showed up 12 times in total.
> >
>
> This error is because the Job Driver process terminated.
>
> Eddie
>
Re: run fails
Posted by Eddie Epstein <ea...@gmail.com>.
Hi Yi-Wen,
On Wed, Nov 11, 2015 at 1:47 AM, Yi-Wen Liu <yi...@usc.edu> wrote:
> Hello,
>
> When I ran the project on DUCC, it finally reached "running" but then it
> showed "MonitorWarning", and canceled by driver.
>
Cancelled by driver indicates that the Job Driver process failed, probably
because of a problem in the application CR.
>
> I tried to debug using all_in_one local/remote, but it didn't have error
> which I thought it was working, however there's no output...
>
No output is not right if some was expected.
>
> Could it be any privilege setting problem?
>
Are there any custom environmental parameters required for the application
code? DUCC does *not* pass on the environment active when submitting work.
Any parameters must be explicitly specified.
Is DUCC running as user=ducc? If so, is ducc_ling properly configured to
spawn application processes as the submitting user?
> Really hope somebody can help me out, I didn't find other keyword useful
> in log file, the connection refused error showed up 12 times in total.
>
This error is because the Job Driver process terminated.
Eddie