You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@uima.apache.org by Neal R Lewis <nr...@us.ibm.com> on 2013/11/14 01:10:35 UTC
DUCC not leaving Initializing State
Hello All,
I'm not sure if this belongs in dev because DUCC isn't released yet, but
it feels like a user question.
I have modeled a CasReader and CasMultiplier based on the RawTextExample
in the duccbook, and have successfully ran the CR and CM in eclipse with a
minimal wrapper. I am using the same CC from the Sample (DuccCasCC)
I am experiencing a problem where my DUCC job goes through
WaitingForDriver, WaitingForResources, then Initializing, then doesn't
change state. After several minutes I just cancel the job.
The jd-out.log loops the following:
13 Nov 2013 13:47:02,966 11 DEBUG client.CasSource init 90 CR creation...
13 Nov 2013 13:47:03,706 11 DEBUG client.CasSource init 90 CR created.
13 Nov 2013 13:47:03,706 11 DEBUG jd.JobDriver initialize 90 CAS source
initialized
13 Nov 2013 13:47:03,707 30 WARN jd.JobDriver run 90 no work items to
process
13 Nov 2013 13:47:03,707 30 DEBUG jd.JobDriver run 90 thread processing
complete
13 Nov 2013 13:47:09,092 12 DEBUG jd.JobDriverComponent getState N/A
publishing state
13 Nov 2013 13:47:09,121 12 DEBUG jd.JobDriverComponent getState N/A
driverState:Initializing
13 Nov 2013 13:47:09,121 12 DEBUG jd.JobDriverComponent getState 90
state:Initializing threads:0 total:-1 fetched:0 started:0 completed:0
error:0 lost:0 queued:0 in-progress:0 pending:true unassigned:0 retry:0
13 Nov 2013 13:47:12,304 11 DEBUG jd.JobDriverComponent
evaluateDispatchedJobConstraints N/A received:OrchestratorStateEvent
13 Nov 2013 13:47:22,310 11 DEBUG jd.JobDriverComponent
evaluateDispatchedJobConstraints N/A received:OrchestratorStateEvent
13 Nov 2013 13:47:24,091 12 DEBUG jd.JobDriverComponent getState N/A
publishing state
13 Nov 2013 13:47:24,103 12 DEBUG jd.JobDriverComponent getState N/A
driverState:Initializing
13 Nov 2013 13:47:24,103 12 DEBUG jd.JobDriverComponent getState 90
state:Initializing threads:0 total:-1 fetched:0 started:0 completed:0
error:0 lost:0 queued:0 in-progress:0 pending:true unassigned:0 retry:0
13 Nov 2013 13:47:32,304 11 DEBUG jd.JobDriverComponent
evaluateDispatchedJobConstraints N/A received:OrchestratorStateEvent
13 Nov 2013 13:47:39,092 12 DEBUG jd.JobDriverComponent getState N/A
publishing state
13 Nov 2013 13:47:39,106 12 DEBUG jd.JobDriverComponent getState N/A
driverState:Initializing
13 Nov 2013 13:47:39,106 12 DEBUG jd.JobDriverComponent getState 90
state:Initializing threads:0 total:-1 fetched:0 started:0 completed:0
error:0 lost:0 queued:0 in-progress:0 pending:true unassigned:0 retry:0
.... Repeat.
My Cas Reader and Cas Multiplier have initialized, because I get logging
output from those functions. I don't see logging output of the CM's
process() method, or the CR's getNext();
I'm pulling CASes from a web service. So my CR initializes by querying
the service and receiving a list of ids that go into a queue. On
getNext(), I add a Workitem for each id, setting input source to the id.
On the CM, the process() sets a private class varaible to the WorkItem
Iterator, and next() graps the next Workitem, its id, sends a GET to the
service, and builds a CAS from the response and returns it.
Any thoughts on how to trouble shoot?
Thanks!
Neal
Re: DUCC not leaving Initializing State
Posted by Neal R Lewis <nr...@us.ibm.com>.
No, no exceptions. I get a list of all of the log files.
I did realize that I didn't try the entire pipeline with the CC. There
might be a conflict with the Workitems. I'll try a sample pipeline in
uimaFit or a separate PEAR to root out any problems there again.
From: Lou DeGenaro <lo...@gmail.com>
To: user@uima.apache.org
Date: 11/14/2013 05:16 AM
Subject: Re: DUCC not leaving Initializing State
Try this: navigate to the Files tab on the Job Details page and look
inside
the *JD* file. Any exceptions?
Lou.
On Wed, Nov 13, 2013 at 7:10 PM, Neal R Lewis <nr...@us.ibm.com> wrote:
> Hello All,
>
> I'm not sure if this belongs in dev because DUCC isn't released yet, but
> it feels like a user question.
>
>
> I have modeled a CasReader and CasMultiplier based on the RawTextExample
> in the duccbook, and have successfully ran the CR and CM in eclipse with
a
> minimal wrapper. I am using the same CC from the Sample (DuccCasCC)
> I am experiencing a problem where my DUCC job goes through
> WaitingForDriver, WaitingForResources, then Initializing, then doesn't
> change state. After several minutes I just cancel the job.
>
> The jd-out.log loops the following:
>
> 13 Nov 2013 13:47:02,966 11 DEBUG client.CasSource init 90 CR
creation...
> 13 Nov 2013 13:47:03,706 11 DEBUG client.CasSource init 90 CR created.
> 13 Nov 2013 13:47:03,706 11 DEBUG jd.JobDriver initialize 90 CAS source
> initialized
> 13 Nov 2013 13:47:03,707 30 WARN jd.JobDriver run 90 no work items to
> process
> 13 Nov 2013 13:47:03,707 30 DEBUG jd.JobDriver run 90 thread processing
> complete
> 13 Nov 2013 13:47:09,092 12 DEBUG jd.JobDriverComponent getState N/A
> publishing state
> 13 Nov 2013 13:47:09,121 12 DEBUG jd.JobDriverComponent getState N/A
> driverState:Initializing
> 13 Nov 2013 13:47:09,121 12 DEBUG jd.JobDriverComponent getState 90
> state:Initializing threads:0 total:-1 fetched:0 started:0 completed:0
> error:0 lost:0 queued:0 in-progress:0 pending:true unassigned:0 retry:0
> 13 Nov 2013 13:47:12,304 11 DEBUG jd.JobDriverComponent
> evaluateDispatchedJobConstraints N/A received:OrchestratorStateEvent
> 13 Nov 2013 13:47:22,310 11 DEBUG jd.JobDriverComponent
> evaluateDispatchedJobConstraints N/A received:OrchestratorStateEvent
> 13 Nov 2013 13:47:24,091 12 DEBUG jd.JobDriverComponent getState N/A
> publishing state
> 13 Nov 2013 13:47:24,103 12 DEBUG jd.JobDriverComponent getState N/A
> driverState:Initializing
> 13 Nov 2013 13:47:24,103 12 DEBUG jd.JobDriverComponent getState 90
> state:Initializing threads:0 total:-1 fetched:0 started:0 completed:0
> error:0 lost:0 queued:0 in-progress:0 pending:true unassigned:0 retry:0
> 13 Nov 2013 13:47:32,304 11 DEBUG jd.JobDriverComponent
> evaluateDispatchedJobConstraints N/A received:OrchestratorStateEvent
> 13 Nov 2013 13:47:39,092 12 DEBUG jd.JobDriverComponent getState N/A
> publishing state
> 13 Nov 2013 13:47:39,106 12 DEBUG jd.JobDriverComponent getState N/A
> driverState:Initializing
> 13 Nov 2013 13:47:39,106 12 DEBUG jd.JobDriverComponent getState 90
> state:Initializing threads:0 total:-1 fetched:0 started:0 completed:0
> error:0 lost:0 queued:0 in-progress:0 pending:true unassigned:0 retry:0
>
> .... Repeat.
>
> My Cas Reader and Cas Multiplier have initialized, because I get logging
> output from those functions. I don't see logging output of the CM's
> process() method, or the CR's getNext();
>
> I'm pulling CASes from a web service. So my CR initializes by querying
> the service and receiving a list of ids that go into a queue. On
> getNext(), I add a Workitem for each id, setting input source to the id.
>
> On the CM, the process() sets a private class varaible to the WorkItem
> Iterator, and next() graps the next Workitem, its id, sends a GET to the
> service, and builds a CAS from the response and returns it.
>
> Any thoughts on how to trouble shoot?
>
>
> Thanks!
>
> Neal
Re: DUCC not leaving Initializing State
Posted by Lou DeGenaro <lo...@gmail.com>.
Try this: navigate to the Files tab on the Job Details page and look inside
the *JD* file. Any exceptions?
Lou.
On Wed, Nov 13, 2013 at 7:10 PM, Neal R Lewis <nr...@us.ibm.com> wrote:
> Hello All,
>
> I'm not sure if this belongs in dev because DUCC isn't released yet, but
> it feels like a user question.
>
>
> I have modeled a CasReader and CasMultiplier based on the RawTextExample
> in the duccbook, and have successfully ran the CR and CM in eclipse with a
> minimal wrapper. I am using the same CC from the Sample (DuccCasCC)
> I am experiencing a problem where my DUCC job goes through
> WaitingForDriver, WaitingForResources, then Initializing, then doesn't
> change state. After several minutes I just cancel the job.
>
> The jd-out.log loops the following:
>
> 13 Nov 2013 13:47:02,966 11 DEBUG client.CasSource init 90 CR creation...
> 13 Nov 2013 13:47:03,706 11 DEBUG client.CasSource init 90 CR created.
> 13 Nov 2013 13:47:03,706 11 DEBUG jd.JobDriver initialize 90 CAS source
> initialized
> 13 Nov 2013 13:47:03,707 30 WARN jd.JobDriver run 90 no work items to
> process
> 13 Nov 2013 13:47:03,707 30 DEBUG jd.JobDriver run 90 thread processing
> complete
> 13 Nov 2013 13:47:09,092 12 DEBUG jd.JobDriverComponent getState N/A
> publishing state
> 13 Nov 2013 13:47:09,121 12 DEBUG jd.JobDriverComponent getState N/A
> driverState:Initializing
> 13 Nov 2013 13:47:09,121 12 DEBUG jd.JobDriverComponent getState 90
> state:Initializing threads:0 total:-1 fetched:0 started:0 completed:0
> error:0 lost:0 queued:0 in-progress:0 pending:true unassigned:0 retry:0
> 13 Nov 2013 13:47:12,304 11 DEBUG jd.JobDriverComponent
> evaluateDispatchedJobConstraints N/A received:OrchestratorStateEvent
> 13 Nov 2013 13:47:22,310 11 DEBUG jd.JobDriverComponent
> evaluateDispatchedJobConstraints N/A received:OrchestratorStateEvent
> 13 Nov 2013 13:47:24,091 12 DEBUG jd.JobDriverComponent getState N/A
> publishing state
> 13 Nov 2013 13:47:24,103 12 DEBUG jd.JobDriverComponent getState N/A
> driverState:Initializing
> 13 Nov 2013 13:47:24,103 12 DEBUG jd.JobDriverComponent getState 90
> state:Initializing threads:0 total:-1 fetched:0 started:0 completed:0
> error:0 lost:0 queued:0 in-progress:0 pending:true unassigned:0 retry:0
> 13 Nov 2013 13:47:32,304 11 DEBUG jd.JobDriverComponent
> evaluateDispatchedJobConstraints N/A received:OrchestratorStateEvent
> 13 Nov 2013 13:47:39,092 12 DEBUG jd.JobDriverComponent getState N/A
> publishing state
> 13 Nov 2013 13:47:39,106 12 DEBUG jd.JobDriverComponent getState N/A
> driverState:Initializing
> 13 Nov 2013 13:47:39,106 12 DEBUG jd.JobDriverComponent getState 90
> state:Initializing threads:0 total:-1 fetched:0 started:0 completed:0
> error:0 lost:0 queued:0 in-progress:0 pending:true unassigned:0 retry:0
>
> .... Repeat.
>
> My Cas Reader and Cas Multiplier have initialized, because I get logging
> output from those functions. I don't see logging output of the CM's
> process() method, or the CR's getNext();
>
> I'm pulling CASes from a web service. So my CR initializes by querying
> the service and receiving a list of ids that go into a queue. On
> getNext(), I add a Workitem for each id, setting input source to the id.
>
> On the CM, the process() sets a private class varaible to the WorkItem
> Iterator, and next() graps the next Workitem, its id, sends a GET to the
> service, and builds a CAS from the response and returns it.
>
> Any thoughts on how to trouble shoot?
>
>
> Thanks!
>
> Neal
Re: DUCC not leaving Initializing State
Posted by Neal R Lewis <nr...@us.ibm.com>.
I'll update the build and try again.
Below is the function after autofill from eclipse:
@Override
public Progress[] getProgress() {
return null;
}
From: Lou DeGenaro <lo...@gmail.com>
To: user@uima.apache.org
Date: 11/26/2013 08:53 AM
Subject: Re: DUCC not leaving Initializing State
I'm looking for a way to reproduce this error without success. I have
current DUCC svn code (from today). I have a simple CR that has a
getProgress() method like this:
@Override
public Progress[] getProgress() {
debug(msgprefix+"getProgress");
ProgressImpl[] retVal = null;
return retVal;
}
No hang.
If you could post a simple example CR that demonstrates the bad behavior
that would be very helpful!
Lou.
On Wed, Nov 20, 2013 at 1:22 PM, Neal R Lewis <nr...@us.ibm.com> wrote:
> I tried again with my current application. As is, it works fine in All
> In ONe, and in the DUCC service. When I return null from getProgress(),
> AllInONe throws a "No Work Items" exception, and in the DUCC Job, it
hangs
> at the Initialize state.
>
> I can't share my code on the forum, but I recompiled the RawTextCasCR
from
> the sample, returning null from getProgress() reproduces the same
results
> in both AllInOne and the DUCC job.
>
> I'm not sure about the JobDriver, but looking at the CasGenerator points
> to returning a null "total" value if there is no progress.
>
> It's possible that I'm performing some operations out of order, but I'm
> following the guidlines from the sample application. I haven't rebuilt
> DUCC since November 5th, would that be a problem?
>
>
>
>
> From: Lou DeGenaro <lo...@gmail.com>
> To: user@uima.apache.org
> Date: 11/20/2013 08:55 AM
> Subject: Re: DUCC not leaving Initializing State
>
>
>
> The DUCC Job Driver (JD) employs the user supplied CR and handles the
case
> where getProgress() returns null, or so is my claim anyway. The CR's
> getProgress() is not required for the JD to operate.
>
> I'm not clear why this did not work for you? Adding to what I said
above,
> I created my own CR that a) returned null for getProgress() and b)
> produced
> 20 CASes in response to 20 getNext() invocations by the JD. My Job ran
> properly and processed all 20 CASes.
>
> Do you have a simple CR that you could share that demonstrates the
problem
> you are seeing?
>
> Lou.
>
>
>
>
> On Mon, Nov 18, 2013 at 4:42 PM, Neal R Lewis <nr...@us.ibm.com>
wrote:
>
> > getNext was returning a CAS, which was why it worked in my
> SimplePipeline
> > in uimafit.
> >
> > After digging in with debug and AllInOne, I found the exception being
> > thrown was "No Workitem in CAS". It will through an exception if the
> > CasGenerator does not generate any CASes, which it determines through
> the
> > CasGenerator's total variable. This was in the initialization of
> > AllinOne.java and where the exception is thrown.
> >
> > //AllInOne.java
> > private void initialize() throws Exception {
> > String mid = "initialize";
> > mh.frameworkTrace(cid, mid, "enter");
> > // Generator
> > casGenerator = new
> > CasGenerator(jobRequestProperties, mh);
> > casGenerator.initialize();
> > int total = casGenerator.getTotal();
> > if(total > 0) {
> > // Pipeline
> > casPipeline = new
> > CasPipeline(jobRequestProperties, mh);
> > casPipeline.initialize();
> > }
> > else {
> > throw new NoWorkItems();
> > }
> > mh.frameworkTrace(cid, mid, "exit");
> > }
> >
> >
> > the casGenerator.getTotal(), get's the private class variable
> > total, which is initialized from the progress:
> >
> > //CasGenerator.java
> > private void initTotal() {
> > String mid = "initTotal";
> > mh.frameworkTrace(cid, mid, "enter");
> > Progress progress = getProgress();
> > if(progress != null) {
> > total = (int)progress.getTotal();
> > }
> > mh.frameworkTrace(cid, mid, "exit");
> > }
> >
> >
> > So, my progress bar was returning null, so the CasGenerator
just
> > skip it, keep total as null. I don't know how this goes through in
the
> > DUCC Flow Controller, but it looks like outside of AllInOne, it causes
a
> > hang
> >
> >
> >
> >
> > From: Lou DeGenaro <lo...@gmail.com>
> > To: user@uima.apache.org
> > Date: 11/18/2013 01:17 PM
> > Subject: Re: DUCC not leaving Initializing State
> >
> >
> >
> > Neal,
> >
> > Did getNext() get called and did it return a filled-in CAS? I tried a
> > simple test case with my CR returning null for getProgress() and the
Job
> > seemed to run just fine. However, one may be penalized by the
scheduler
> if
> > is had to guess how much work the Job comprises.
> >
> > Lou.
> >
> >
> > On Fri, Nov 15, 2013 at 5:15 PM, Neal R Lewis <nr...@us.ibm.com>
> wrote:
> >
> > > I debugged using the all_in_one setup and found that I was returning
> > null
> > > from getProgress(), which was not creating any workitems. So I
setup
> > the
> > > function and Cases started moving.
> > >
> > > Thanks for all of your help,
> > >
> > > Neal.
> > >
> > >
> > >
> > >
> > > From: Neal Lewis <im...@gmail.com>
> > > To: user@uima.apache.org
> > > Date: 11/15/2013 05:48 AM
> > > Subject: Re: DUCC not leaving Initializing State
> > >
> > >
> > >
> > > Thanks Eddie,
> > >
> > > I missed the all_in_one in the documentation, so I'll try it out and
> get
> > > back to you guys.
> > >
> > > -Neal
> > >
> > > On 11/14/2013 06:02 PM, Eddie Epstein wrote:
> > > > Good job. FYI, it is strongly recommended to use all_in_one as
> > described
> > > > in the documentation on developing a new sample application. Your
> > > > aggregate most likely does not include the built-in DUCC flow
> > controller
> > > > which implements SendToLast.
> > > >
> > > > After all child CASes created for a work itemhave been generated,
if
> > > > the CC needs a signal to clean up, then use SendToLast. The
DucCasCC
> > > > does need that signal to close the output zip package.
> > > >
> > > > Eddie
> > > >
> > > >
> > > >
> > > >
> > > > On Thu, Nov 14, 2013 at 6:38 PM, Neal R Lewis <nr...@us.ibm.com>
> > > wrote:
> > > >
> > > >> So, I found one GLARING problem that I missed completely: When
> > > testing
> > > >> my CR and CM setup, I looped through the CR's getNext(), adding
to
> > the
> > > >> JCas that I just created. But the CR's getNext() didn't loop
> > > internally,
> > > >> it only added IDs one at a time. So, instead of the CR creating a
> CAS
> > > with
> > > >> N Workitems and halting, it was create N CASes with 1 workitem.
> I've
> > > >> fixed that.
> > > >>
> > > >> So I've made an Aggregate AE of my CM and a simple AE, and ran
> them
> > > with
> > > >> my CR in a uimaFit SimplePipeline. The outputs are as I expect
> for
> > > now.
> > > >>
> > > >>
> > > >> But I do have a question. The Workitems have an option of
> > SendToLast,
> > > >> which in the RawText Example is set to "true". The DuccCasCC in
> the
> > > >> sample pulls WorkItem FSes for output stuff. The CR adds the
> > > Workitems,
> > > >> but the CM doesn't reattach them to the new CASes before
returning
> > > next().
> > > >>
> > > >>
> > > >> Should the Workitems stay in the CAS throughout processing (ie.,
> > should
> > > >> the CMs add them to the new CASes) ? OR is SendToLast enough?
> > > >>
> > > >> Thanks!
> > > >>
> > > >> Neal
> > > >>
> > > >>
> > > >>
> > > >> From: Neal R Lewis/Almaden/IBM@IBMUS
> > > >> To: user@uima.apache.org
> > > >> Date: 11/14/2013 01:20 PM
> > > >> Subject: Re: DUCC not leaving Initializing State
> > > >>
> > > >>
> > > >>
> > > >> The minimal wrapper creates a Cas using uimaFit and loop over
the
> > CR,
> > > >> then to run over the iterator from cm.processAndOutputNewCASes,
> then
> > > >> output each cas in the loop like below:
> > > >>
> > > >> CollectionReader cr =
> > > CollectionReaderFactory.createReader
> > > >> ("path.to.my.CasReader");
> > > >> AnalysisEngine cm =
> > > AnalysisEngineFactory.createEngine(
> > > >> "path.to.my.CasMultiplier");
> > > >>
> > > >> JCas jcas = JCasFactory.createJCas("
> > > >> desc.DuccJobFlowControlTS);
> > > >>
> > > >> while (cr.hasNext()){
> > > >> cr.getNext(jcas.getCas());
> > > >> }
> > > >>
> > > >> CasIterator casIterator =
> > > >> cm.processAndOutputNewCASes(jcas.getCas());
> > > >> while (casIterator.hasNext()) {
> > > >> CAS outCas = casIterator.next();
> > > >> System.out.println(outCas.getDocumentText());
> > > >> }
> > > >>
> > > >> I'll need to debug a full standalone PEAR or Pipeline. But at
this
> > > point I
> > > >>
> > > >> was able to at least get the CASes out.
> > > >>
> > > >> Below is the output of the JP file before the stall and after
> loading
> > > JVM
> > > >> stuff. It stays like this until shutdown. As far as I can tell
> the
> > > CM
> > > >> Loads and is waiting.
> > > >>
> > > >>
> > > >> + JVM
> > > >>
> LIB_PATH:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> > > >>
+------------------------------------------------------------------
> > > >> 13 Nov 2013 15:53:44,761 INFO DUCC.ThreadPoolTaskExecutor -
> > > Initializing
> > > >> ExecutorService 'pooling_ducc.jd.queue.91_1'
> > > >> 03:53:44.794 - 1:
> > > >>
> org.apache.uima.adapter.jms.activemq.JmsInputChannel.setEndpointName:
> > > >> INFO: top_level_input_queue_service_1 Service Starting -
Listening
> > for
> > > >> Messages
> > > >> 13 Nov 2013 15:53:44,795 INFO DUCC.ThreadPoolTaskExecutor -
> > > Initializing
> > > >> ExecutorService 'pooling_ducc.jd.queue.91_1'
> > > >> 03:53:44.797 - 30:
> > > >>
> org.apache.uima.aae.UimaAsThreadFactory$1.UimaAsThreadFactory.run():
> > > INFO:
> > > >>
> > > >> Controller: ducc.jd.queue.91 Initializing AE instance on Thread
Id:
> > 30
> > > >> 03:53:44.799 - 1:
> > > >>
> > > >>
> > >
> > >
> >
> >
>
>
org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.waitForServiceNotification:
> > > >>
> > > >> INFO: Uima EE Client Blocking - Awaiting Top Level Controller
> > > >> Initialization Notification
> > > >> 03:53:44.887 - 30:
> > > >>
> > > >>
> > >
> > >
> >
> >
>
>
com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(70):
> > > >>
> > > >> INFO: Initializing Cas Multiplier Parameters
> > > >> 03:53:44.887 - 30:
> > > >>
> > > >>
> > >
> > >
> >
> >
>
>
com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(71):
> > > >>
> > > >> INFO: init HOST ADDRESS:
> > > >> https://thepit.element.almaden.ibm.com/cgi-bin/qm2
> > > >> 03:53:44.887 - 30:
> > > >>
> > > >>
> > >
> > >
> >
> >
>
>
com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(72):
> > > >>
> > > >> INFO: init LENIENT: false
> > > >> 03:53:44.887 - 30:
> > > >>
> > > >>
> > >
> > >
> >
> >
>
>
com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(73):
> > > >>
> > > >> INFO: init PARAM_ENCODING: UTF-8
> > > >> 03:53:44.887 - 30:
> > > >>
> > > >>
> > >
> > >
> >
> >
>
>
com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(74):
> > > >>
> > > >> INFO: init PARAM_LANGUAGE: en
> > > >> 03:53:44.933 - 30:
> > > >> org.apache.uima.ducc.sampleapps.DuccCasCC.initialize(74): INFO:
> > > Outputting
> > > >>
> > > >> CASes in XmiCas format, zip compressed at level=7
> > > >> Service:ducc.jd.queue.91 Initialized. Ready To Process Messages
> From
> > > >> Queue:ducc.jd.queue.91
> > > >> 03:53:45.97 - 30:
> > > >>
> > > >>
> > >
> > >
> >
> >
>
>
org.apache.uima.aae.controller.PrimitiveAnalysisEngineController_impl.postInitialize:
> > > >>
> > > >> INFO: ********* Initialized the Controller. ducc.jd.queue.91
Ready
> To
> > > >> Process. ********
> > > >> 03:53:45.97 - 1:
> > > >>
> > > >>
> > >
> > >
> >
> >
>
>
org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.doStartListeners:
> > > >>
> > > >> INFO: Controller: ducc.jd.queue.91 Starting Listener on Endpoint:
> > > >> queue://ducc.jd.queue.91 Selector: Command=2000 OR Command=2002
> > Broker:
> > > >>
> > >
> >
>
tcp://greenfairy:61617?wireFormat.maxInactivityDuration=0&closeAsync=false
> > > >> 03:53:45.290 - 1:
> > > >>
> > > >>
> > >
> > >
> >
> >
>
>
org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.doStartListeners:
> > > >>
> > > >> INFO: Controller: ducc.jd.queue.91 Starting Listener on Endpoint:
> > > >> queue://ducc.jd.queue.91 Selector: Command=2001 Broker:
> > > >>
> > >
> >
>
tcp://greenfairy:61617?wireFormat.maxInactivityDuration=0&closeAsync=false
> > > >> 13 Nov 2013 15:53:45,299 INFO DUCC.DuccService - boot N/A
...
> > > >> Component started: managedService
> > > >> 13 Nov 2013 15:53:45,300 INFO DUCC.DuccService - boot N/A
> > Starting
> > > >> Camel. Use ctrl + c to terminate the JVM.
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >> From: Eddie Epstein <ea...@gmail.com>
> > > >> To: user@uima.apache.org
> > > >> Date: 11/14/2013 10:37 AM
> > > >> Subject: Re: DUCC not leaving Initializing State
> > > >>
> > > >>
> > > >>
> > > >> On Wed, Nov 13, 2013 at 7:10 PM, Neal R Lewis
<nr...@us.ibm.com>
> > > wrote:
> > > >>
> > > >>> I have modeled a CasReader and CasMultiplier based on the
> > > RawTextExample
> > > >>> in the duccbook, and have successfully ran the CR and CM in
> eclipse
> > > with
> > > >> a
> > > >>> minimal wrapper.
> > > >>
> > > >> Did you debug the job using one of the --all_in_one varieties, or
> > some
> > > >> other
> > > >> minimal wrapper?
> > > >>
> > > >>
> > > >> I am using the same CC from the Sample (DuccCasCC)
> > > >>> I am experiencing a problem where my DUCC job goes through
> > > >>> WaitingForDriver, WaitingForResources, then Initializing, then
> > doesn't
> > > >>> change state. After several minutes I just cancel the job.
> > > >>>
> > > >> Did you look in the JP logfile?
> > > >>
> > > >> Eddie
> > > >>
> > > >>
> > > >>
> > >
> > >
> > >
> >
> >
>
>
Re: DUCC not leaving Initializing State
Posted by Lou DeGenaro <lo...@gmail.com>.
I'm looking for a way to reproduce this error without success. I have
current DUCC svn code (from today). I have a simple CR that has a
getProgress() method like this:
@Override
public Progress[] getProgress() {
debug(msgprefix+"getProgress");
ProgressImpl[] retVal = null;
return retVal;
}
No hang.
If you could post a simple example CR that demonstrates the bad behavior
that would be very helpful!
Lou.
On Wed, Nov 20, 2013 at 1:22 PM, Neal R Lewis <nr...@us.ibm.com> wrote:
> I tried again with my current application. As is, it works fine in All
> In ONe, and in the DUCC service. When I return null from getProgress(),
> AllInONe throws a "No Work Items" exception, and in the DUCC Job, it hangs
> at the Initialize state.
>
> I can't share my code on the forum, but I recompiled the RawTextCasCR from
> the sample, returning null from getProgress() reproduces the same results
> in both AllInOne and the DUCC job.
>
> I'm not sure about the JobDriver, but looking at the CasGenerator points
> to returning a null "total" value if there is no progress.
>
> It's possible that I'm performing some operations out of order, but I'm
> following the guidlines from the sample application. I haven't rebuilt
> DUCC since November 5th, would that be a problem?
>
>
>
>
> From: Lou DeGenaro <lo...@gmail.com>
> To: user@uima.apache.org
> Date: 11/20/2013 08:55 AM
> Subject: Re: DUCC not leaving Initializing State
>
>
>
> The DUCC Job Driver (JD) employs the user supplied CR and handles the case
> where getProgress() returns null, or so is my claim anyway. The CR's
> getProgress() is not required for the JD to operate.
>
> I'm not clear why this did not work for you? Adding to what I said above,
> I created my own CR that a) returned null for getProgress() and b)
> produced
> 20 CASes in response to 20 getNext() invocations by the JD. My Job ran
> properly and processed all 20 CASes.
>
> Do you have a simple CR that you could share that demonstrates the problem
> you are seeing?
>
> Lou.
>
>
>
>
> On Mon, Nov 18, 2013 at 4:42 PM, Neal R Lewis <nr...@us.ibm.com> wrote:
>
> > getNext was returning a CAS, which was why it worked in my
> SimplePipeline
> > in uimafit.
> >
> > After digging in with debug and AllInOne, I found the exception being
> > thrown was "No Workitem in CAS". It will through an exception if the
> > CasGenerator does not generate any CASes, which it determines through
> the
> > CasGenerator's total variable. This was in the initialization of
> > AllinOne.java and where the exception is thrown.
> >
> > //AllInOne.java
> > private void initialize() throws Exception {
> > String mid = "initialize";
> > mh.frameworkTrace(cid, mid, "enter");
> > // Generator
> > casGenerator = new
> > CasGenerator(jobRequestProperties, mh);
> > casGenerator.initialize();
> > int total = casGenerator.getTotal();
> > if(total > 0) {
> > // Pipeline
> > casPipeline = new
> > CasPipeline(jobRequestProperties, mh);
> > casPipeline.initialize();
> > }
> > else {
> > throw new NoWorkItems();
> > }
> > mh.frameworkTrace(cid, mid, "exit");
> > }
> >
> >
> > the casGenerator.getTotal(), get's the private class variable
> > total, which is initialized from the progress:
> >
> > //CasGenerator.java
> > private void initTotal() {
> > String mid = "initTotal";
> > mh.frameworkTrace(cid, mid, "enter");
> > Progress progress = getProgress();
> > if(progress != null) {
> > total = (int)progress.getTotal();
> > }
> > mh.frameworkTrace(cid, mid, "exit");
> > }
> >
> >
> > So, my progress bar was returning null, so the CasGenerator just
> > skip it, keep total as null. I don't know how this goes through in the
> > DUCC Flow Controller, but it looks like outside of AllInOne, it causes a
> > hang
> >
> >
> >
> >
> > From: Lou DeGenaro <lo...@gmail.com>
> > To: user@uima.apache.org
> > Date: 11/18/2013 01:17 PM
> > Subject: Re: DUCC not leaving Initializing State
> >
> >
> >
> > Neal,
> >
> > Did getNext() get called and did it return a filled-in CAS? I tried a
> > simple test case with my CR returning null for getProgress() and the Job
> > seemed to run just fine. However, one may be penalized by the scheduler
> if
> > is had to guess how much work the Job comprises.
> >
> > Lou.
> >
> >
> > On Fri, Nov 15, 2013 at 5:15 PM, Neal R Lewis <nr...@us.ibm.com>
> wrote:
> >
> > > I debugged using the all_in_one setup and found that I was returning
> > null
> > > from getProgress(), which was not creating any workitems. So I setup
> > the
> > > function and Cases started moving.
> > >
> > > Thanks for all of your help,
> > >
> > > Neal.
> > >
> > >
> > >
> > >
> > > From: Neal Lewis <im...@gmail.com>
> > > To: user@uima.apache.org
> > > Date: 11/15/2013 05:48 AM
> > > Subject: Re: DUCC not leaving Initializing State
> > >
> > >
> > >
> > > Thanks Eddie,
> > >
> > > I missed the all_in_one in the documentation, so I'll try it out and
> get
> > > back to you guys.
> > >
> > > -Neal
> > >
> > > On 11/14/2013 06:02 PM, Eddie Epstein wrote:
> > > > Good job. FYI, it is strongly recommended to use all_in_one as
> > described
> > > > in the documentation on developing a new sample application. Your
> > > > aggregate most likely does not include the built-in DUCC flow
> > controller
> > > > which implements SendToLast.
> > > >
> > > > After all child CASes created for a work itemhave been generated, if
> > > > the CC needs a signal to clean up, then use SendToLast. The DucCasCC
> > > > does need that signal to close the output zip package.
> > > >
> > > > Eddie
> > > >
> > > >
> > > >
> > > >
> > > > On Thu, Nov 14, 2013 at 6:38 PM, Neal R Lewis <nr...@us.ibm.com>
> > > wrote:
> > > >
> > > >> So, I found one GLARING problem that I missed completely: When
> > > testing
> > > >> my CR and CM setup, I looped through the CR's getNext(), adding to
> > the
> > > >> JCas that I just created. But the CR's getNext() didn't loop
> > > internally,
> > > >> it only added IDs one at a time. So, instead of the CR creating a
> CAS
> > > with
> > > >> N Workitems and halting, it was create N CASes with 1 workitem.
> I've
> > > >> fixed that.
> > > >>
> > > >> So I've made an Aggregate AE of my CM and a simple AE, and ran
> them
> > > with
> > > >> my CR in a uimaFit SimplePipeline. The outputs are as I expect
> for
> > > now.
> > > >>
> > > >>
> > > >> But I do have a question. The Workitems have an option of
> > SendToLast,
> > > >> which in the RawText Example is set to "true". The DuccCasCC in
> the
> > > >> sample pulls WorkItem FSes for output stuff. The CR adds the
> > > Workitems,
> > > >> but the CM doesn't reattach them to the new CASes before returning
> > > next().
> > > >>
> > > >>
> > > >> Should the Workitems stay in the CAS throughout processing (ie.,
> > should
> > > >> the CMs add them to the new CASes) ? OR is SendToLast enough?
> > > >>
> > > >> Thanks!
> > > >>
> > > >> Neal
> > > >>
> > > >>
> > > >>
> > > >> From: Neal R Lewis/Almaden/IBM@IBMUS
> > > >> To: user@uima.apache.org
> > > >> Date: 11/14/2013 01:20 PM
> > > >> Subject: Re: DUCC not leaving Initializing State
> > > >>
> > > >>
> > > >>
> > > >> The minimal wrapper creates a Cas using uimaFit and loop over the
> > CR,
> > > >> then to run over the iterator from cm.processAndOutputNewCASes,
> then
> > > >> output each cas in the loop like below:
> > > >>
> > > >> CollectionReader cr =
> > > CollectionReaderFactory.createReader
> > > >> ("path.to.my.CasReader");
> > > >> AnalysisEngine cm =
> > > AnalysisEngineFactory.createEngine(
> > > >> "path.to.my.CasMultiplier");
> > > >>
> > > >> JCas jcas = JCasFactory.createJCas("
> > > >> desc.DuccJobFlowControlTS);
> > > >>
> > > >> while (cr.hasNext()){
> > > >> cr.getNext(jcas.getCas());
> > > >> }
> > > >>
> > > >> CasIterator casIterator =
> > > >> cm.processAndOutputNewCASes(jcas.getCas());
> > > >> while (casIterator.hasNext()) {
> > > >> CAS outCas = casIterator.next();
> > > >> System.out.println(outCas.getDocumentText());
> > > >> }
> > > >>
> > > >> I'll need to debug a full standalone PEAR or Pipeline. But at this
> > > point I
> > > >>
> > > >> was able to at least get the CASes out.
> > > >>
> > > >> Below is the output of the JP file before the stall and after
> loading
> > > JVM
> > > >> stuff. It stays like this until shutdown. As far as I can tell
> the
> > > CM
> > > >> Loads and is waiting.
> > > >>
> > > >>
> > > >> + JVM
> > > >>
> LIB_PATH:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> > > >> +------------------------------------------------------------------
> > > >> 13 Nov 2013 15:53:44,761 INFO DUCC.ThreadPoolTaskExecutor -
> > > Initializing
> > > >> ExecutorService 'pooling_ducc.jd.queue.91_1'
> > > >> 03:53:44.794 - 1:
> > > >>
> org.apache.uima.adapter.jms.activemq.JmsInputChannel.setEndpointName:
> > > >> INFO: top_level_input_queue_service_1 Service Starting - Listening
> > for
> > > >> Messages
> > > >> 13 Nov 2013 15:53:44,795 INFO DUCC.ThreadPoolTaskExecutor -
> > > Initializing
> > > >> ExecutorService 'pooling_ducc.jd.queue.91_1'
> > > >> 03:53:44.797 - 30:
> > > >>
> org.apache.uima.aae.UimaAsThreadFactory$1.UimaAsThreadFactory.run():
> > > INFO:
> > > >>
> > > >> Controller: ducc.jd.queue.91 Initializing AE instance on Thread Id:
> > 30
> > > >> 03:53:44.799 - 1:
> > > >>
> > > >>
> > >
> > >
> >
> >
>
> org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.waitForServiceNotification:
> > > >>
> > > >> INFO: Uima EE Client Blocking - Awaiting Top Level Controller
> > > >> Initialization Notification
> > > >> 03:53:44.887 - 30:
> > > >>
> > > >>
> > >
> > >
> >
> >
>
> com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(70):
> > > >>
> > > >> INFO: Initializing Cas Multiplier Parameters
> > > >> 03:53:44.887 - 30:
> > > >>
> > > >>
> > >
> > >
> >
> >
>
> com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(71):
> > > >>
> > > >> INFO: init HOST ADDRESS:
> > > >> https://thepit.element.almaden.ibm.com/cgi-bin/qm2
> > > >> 03:53:44.887 - 30:
> > > >>
> > > >>
> > >
> > >
> >
> >
>
> com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(72):
> > > >>
> > > >> INFO: init LENIENT: false
> > > >> 03:53:44.887 - 30:
> > > >>
> > > >>
> > >
> > >
> >
> >
>
> com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(73):
> > > >>
> > > >> INFO: init PARAM_ENCODING: UTF-8
> > > >> 03:53:44.887 - 30:
> > > >>
> > > >>
> > >
> > >
> >
> >
>
> com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(74):
> > > >>
> > > >> INFO: init PARAM_LANGUAGE: en
> > > >> 03:53:44.933 - 30:
> > > >> org.apache.uima.ducc.sampleapps.DuccCasCC.initialize(74): INFO:
> > > Outputting
> > > >>
> > > >> CASes in XmiCas format, zip compressed at level=7
> > > >> Service:ducc.jd.queue.91 Initialized. Ready To Process Messages
> From
> > > >> Queue:ducc.jd.queue.91
> > > >> 03:53:45.97 - 30:
> > > >>
> > > >>
> > >
> > >
> >
> >
>
> org.apache.uima.aae.controller.PrimitiveAnalysisEngineController_impl.postInitialize:
> > > >>
> > > >> INFO: ********* Initialized the Controller. ducc.jd.queue.91 Ready
> To
> > > >> Process. ********
> > > >> 03:53:45.97 - 1:
> > > >>
> > > >>
> > >
> > >
> >
> >
>
> org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.doStartListeners:
> > > >>
> > > >> INFO: Controller: ducc.jd.queue.91 Starting Listener on Endpoint:
> > > >> queue://ducc.jd.queue.91 Selector: Command=2000 OR Command=2002
> > Broker:
> > > >>
> > >
> >
> tcp://greenfairy:61617?wireFormat.maxInactivityDuration=0&closeAsync=false
> > > >> 03:53:45.290 - 1:
> > > >>
> > > >>
> > >
> > >
> >
> >
>
> org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.doStartListeners:
> > > >>
> > > >> INFO: Controller: ducc.jd.queue.91 Starting Listener on Endpoint:
> > > >> queue://ducc.jd.queue.91 Selector: Command=2001 Broker:
> > > >>
> > >
> >
> tcp://greenfairy:61617?wireFormat.maxInactivityDuration=0&closeAsync=false
> > > >> 13 Nov 2013 15:53:45,299 INFO DUCC.DuccService - boot N/A ...
> > > >> Component started: managedService
> > > >> 13 Nov 2013 15:53:45,300 INFO DUCC.DuccService - boot N/A
> > Starting
> > > >> Camel. Use ctrl + c to terminate the JVM.
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >> From: Eddie Epstein <ea...@gmail.com>
> > > >> To: user@uima.apache.org
> > > >> Date: 11/14/2013 10:37 AM
> > > >> Subject: Re: DUCC not leaving Initializing State
> > > >>
> > > >>
> > > >>
> > > >> On Wed, Nov 13, 2013 at 7:10 PM, Neal R Lewis <nr...@us.ibm.com>
> > > wrote:
> > > >>
> > > >>> I have modeled a CasReader and CasMultiplier based on the
> > > RawTextExample
> > > >>> in the duccbook, and have successfully ran the CR and CM in
> eclipse
> > > with
> > > >> a
> > > >>> minimal wrapper.
> > > >>
> > > >> Did you debug the job using one of the --all_in_one varieties, or
> > some
> > > >> other
> > > >> minimal wrapper?
> > > >>
> > > >>
> > > >> I am using the same CC from the Sample (DuccCasCC)
> > > >>> I am experiencing a problem where my DUCC job goes through
> > > >>> WaitingForDriver, WaitingForResources, then Initializing, then
> > doesn't
> > > >>> change state. After several minutes I just cancel the job.
> > > >>>
> > > >> Did you look in the JP logfile?
> > > >>
> > > >> Eddie
> > > >>
> > > >>
> > > >>
> > >
> > >
> > >
> >
> >
>
>
Re: DUCC not leaving Initializing State
Posted by Neal R Lewis <nr...@us.ibm.com>.
I tried again with my current application. As is, it works fine in All
In ONe, and in the DUCC service. When I return null from getProgress(),
AllInONe throws a "No Work Items" exception, and in the DUCC Job, it hangs
at the Initialize state.
I can't share my code on the forum, but I recompiled the RawTextCasCR from
the sample, returning null from getProgress() reproduces the same results
in both AllInOne and the DUCC job.
I'm not sure about the JobDriver, but looking at the CasGenerator points
to returning a null "total" value if there is no progress.
It's possible that I'm performing some operations out of order, but I'm
following the guidlines from the sample application. I haven't rebuilt
DUCC since November 5th, would that be a problem?
From: Lou DeGenaro <lo...@gmail.com>
To: user@uima.apache.org
Date: 11/20/2013 08:55 AM
Subject: Re: DUCC not leaving Initializing State
The DUCC Job Driver (JD) employs the user supplied CR and handles the case
where getProgress() returns null, or so is my claim anyway. The CR's
getProgress() is not required for the JD to operate.
I'm not clear why this did not work for you? Adding to what I said above,
I created my own CR that a) returned null for getProgress() and b)
produced
20 CASes in response to 20 getNext() invocations by the JD. My Job ran
properly and processed all 20 CASes.
Do you have a simple CR that you could share that demonstrates the problem
you are seeing?
Lou.
On Mon, Nov 18, 2013 at 4:42 PM, Neal R Lewis <nr...@us.ibm.com> wrote:
> getNext was returning a CAS, which was why it worked in my
SimplePipeline
> in uimafit.
>
> After digging in with debug and AllInOne, I found the exception being
> thrown was "No Workitem in CAS". It will through an exception if the
> CasGenerator does not generate any CASes, which it determines through
the
> CasGenerator's total variable. This was in the initialization of
> AllinOne.java and where the exception is thrown.
>
> //AllInOne.java
> private void initialize() throws Exception {
> String mid = "initialize";
> mh.frameworkTrace(cid, mid, "enter");
> // Generator
> casGenerator = new
> CasGenerator(jobRequestProperties, mh);
> casGenerator.initialize();
> int total = casGenerator.getTotal();
> if(total > 0) {
> // Pipeline
> casPipeline = new
> CasPipeline(jobRequestProperties, mh);
> casPipeline.initialize();
> }
> else {
> throw new NoWorkItems();
> }
> mh.frameworkTrace(cid, mid, "exit");
> }
>
>
> the casGenerator.getTotal(), get's the private class variable
> total, which is initialized from the progress:
>
> //CasGenerator.java
> private void initTotal() {
> String mid = "initTotal";
> mh.frameworkTrace(cid, mid, "enter");
> Progress progress = getProgress();
> if(progress != null) {
> total = (int)progress.getTotal();
> }
> mh.frameworkTrace(cid, mid, "exit");
> }
>
>
> So, my progress bar was returning null, so the CasGenerator just
> skip it, keep total as null. I don't know how this goes through in the
> DUCC Flow Controller, but it looks like outside of AllInOne, it causes a
> hang
>
>
>
>
> From: Lou DeGenaro <lo...@gmail.com>
> To: user@uima.apache.org
> Date: 11/18/2013 01:17 PM
> Subject: Re: DUCC not leaving Initializing State
>
>
>
> Neal,
>
> Did getNext() get called and did it return a filled-in CAS? I tried a
> simple test case with my CR returning null for getProgress() and the Job
> seemed to run just fine. However, one may be penalized by the scheduler
if
> is had to guess how much work the Job comprises.
>
> Lou.
>
>
> On Fri, Nov 15, 2013 at 5:15 PM, Neal R Lewis <nr...@us.ibm.com>
wrote:
>
> > I debugged using the all_in_one setup and found that I was returning
> null
> > from getProgress(), which was not creating any workitems. So I setup
> the
> > function and Cases started moving.
> >
> > Thanks for all of your help,
> >
> > Neal.
> >
> >
> >
> >
> > From: Neal Lewis <im...@gmail.com>
> > To: user@uima.apache.org
> > Date: 11/15/2013 05:48 AM
> > Subject: Re: DUCC not leaving Initializing State
> >
> >
> >
> > Thanks Eddie,
> >
> > I missed the all_in_one in the documentation, so I'll try it out and
get
> > back to you guys.
> >
> > -Neal
> >
> > On 11/14/2013 06:02 PM, Eddie Epstein wrote:
> > > Good job. FYI, it is strongly recommended to use all_in_one as
> described
> > > in the documentation on developing a new sample application. Your
> > > aggregate most likely does not include the built-in DUCC flow
> controller
> > > which implements SendToLast.
> > >
> > > After all child CASes created for a work itemhave been generated, if
> > > the CC needs a signal to clean up, then use SendToLast. The DucCasCC
> > > does need that signal to close the output zip package.
> > >
> > > Eddie
> > >
> > >
> > >
> > >
> > > On Thu, Nov 14, 2013 at 6:38 PM, Neal R Lewis <nr...@us.ibm.com>
> > wrote:
> > >
> > >> So, I found one GLARING problem that I missed completely: When
> > testing
> > >> my CR and CM setup, I looped through the CR's getNext(), adding to
> the
> > >> JCas that I just created. But the CR's getNext() didn't loop
> > internally,
> > >> it only added IDs one at a time. So, instead of the CR creating a
CAS
> > with
> > >> N Workitems and halting, it was create N CASes with 1 workitem.
I've
> > >> fixed that.
> > >>
> > >> So I've made an Aggregate AE of my CM and a simple AE, and ran
them
> > with
> > >> my CR in a uimaFit SimplePipeline. The outputs are as I expect
for
> > now.
> > >>
> > >>
> > >> But I do have a question. The Workitems have an option of
> SendToLast,
> > >> which in the RawText Example is set to "true". The DuccCasCC in
the
> > >> sample pulls WorkItem FSes for output stuff. The CR adds the
> > Workitems,
> > >> but the CM doesn't reattach them to the new CASes before returning
> > next().
> > >>
> > >>
> > >> Should the Workitems stay in the CAS throughout processing (ie.,
> should
> > >> the CMs add them to the new CASes) ? OR is SendToLast enough?
> > >>
> > >> Thanks!
> > >>
> > >> Neal
> > >>
> > >>
> > >>
> > >> From: Neal R Lewis/Almaden/IBM@IBMUS
> > >> To: user@uima.apache.org
> > >> Date: 11/14/2013 01:20 PM
> > >> Subject: Re: DUCC not leaving Initializing State
> > >>
> > >>
> > >>
> > >> The minimal wrapper creates a Cas using uimaFit and loop over the
> CR,
> > >> then to run over the iterator from cm.processAndOutputNewCASes,
then
> > >> output each cas in the loop like below:
> > >>
> > >> CollectionReader cr =
> > CollectionReaderFactory.createReader
> > >> ("path.to.my.CasReader");
> > >> AnalysisEngine cm =
> > AnalysisEngineFactory.createEngine(
> > >> "path.to.my.CasMultiplier");
> > >>
> > >> JCas jcas = JCasFactory.createJCas("
> > >> desc.DuccJobFlowControlTS);
> > >>
> > >> while (cr.hasNext()){
> > >> cr.getNext(jcas.getCas());
> > >> }
> > >>
> > >> CasIterator casIterator =
> > >> cm.processAndOutputNewCASes(jcas.getCas());
> > >> while (casIterator.hasNext()) {
> > >> CAS outCas = casIterator.next();
> > >> System.out.println(outCas.getDocumentText());
> > >> }
> > >>
> > >> I'll need to debug a full standalone PEAR or Pipeline. But at this
> > point I
> > >>
> > >> was able to at least get the CASes out.
> > >>
> > >> Below is the output of the JP file before the stall and after
loading
> > JVM
> > >> stuff. It stays like this until shutdown. As far as I can tell
the
> > CM
> > >> Loads and is waiting.
> > >>
> > >>
> > >> + JVM
> > >>
LIB_PATH:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> > >> +------------------------------------------------------------------
> > >> 13 Nov 2013 15:53:44,761 INFO DUCC.ThreadPoolTaskExecutor -
> > Initializing
> > >> ExecutorService 'pooling_ducc.jd.queue.91_1'
> > >> 03:53:44.794 - 1:
> > >>
org.apache.uima.adapter.jms.activemq.JmsInputChannel.setEndpointName:
> > >> INFO: top_level_input_queue_service_1 Service Starting - Listening
> for
> > >> Messages
> > >> 13 Nov 2013 15:53:44,795 INFO DUCC.ThreadPoolTaskExecutor -
> > Initializing
> > >> ExecutorService 'pooling_ducc.jd.queue.91_1'
> > >> 03:53:44.797 - 30:
> > >>
org.apache.uima.aae.UimaAsThreadFactory$1.UimaAsThreadFactory.run():
> > INFO:
> > >>
> > >> Controller: ducc.jd.queue.91 Initializing AE instance on Thread Id:
> 30
> > >> 03:53:44.799 - 1:
> > >>
> > >>
> >
> >
>
>
org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.waitForServiceNotification:
> > >>
> > >> INFO: Uima EE Client Blocking - Awaiting Top Level Controller
> > >> Initialization Notification
> > >> 03:53:44.887 - 30:
> > >>
> > >>
> >
> >
>
>
com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(70):
> > >>
> > >> INFO: Initializing Cas Multiplier Parameters
> > >> 03:53:44.887 - 30:
> > >>
> > >>
> >
> >
>
>
com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(71):
> > >>
> > >> INFO: init HOST ADDRESS:
> > >> https://thepit.element.almaden.ibm.com/cgi-bin/qm2
> > >> 03:53:44.887 - 30:
> > >>
> > >>
> >
> >
>
>
com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(72):
> > >>
> > >> INFO: init LENIENT: false
> > >> 03:53:44.887 - 30:
> > >>
> > >>
> >
> >
>
>
com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(73):
> > >>
> > >> INFO: init PARAM_ENCODING: UTF-8
> > >> 03:53:44.887 - 30:
> > >>
> > >>
> >
> >
>
>
com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(74):
> > >>
> > >> INFO: init PARAM_LANGUAGE: en
> > >> 03:53:44.933 - 30:
> > >> org.apache.uima.ducc.sampleapps.DuccCasCC.initialize(74): INFO:
> > Outputting
> > >>
> > >> CASes in XmiCas format, zip compressed at level=7
> > >> Service:ducc.jd.queue.91 Initialized. Ready To Process Messages
From
> > >> Queue:ducc.jd.queue.91
> > >> 03:53:45.97 - 30:
> > >>
> > >>
> >
> >
>
>
org.apache.uima.aae.controller.PrimitiveAnalysisEngineController_impl.postInitialize:
> > >>
> > >> INFO: ********* Initialized the Controller. ducc.jd.queue.91 Ready
To
> > >> Process. ********
> > >> 03:53:45.97 - 1:
> > >>
> > >>
> >
> >
>
>
org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.doStartListeners:
> > >>
> > >> INFO: Controller: ducc.jd.queue.91 Starting Listener on Endpoint:
> > >> queue://ducc.jd.queue.91 Selector: Command=2000 OR Command=2002
> Broker:
> > >>
> >
>
tcp://greenfairy:61617?wireFormat.maxInactivityDuration=0&closeAsync=false
> > >> 03:53:45.290 - 1:
> > >>
> > >>
> >
> >
>
>
org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.doStartListeners:
> > >>
> > >> INFO: Controller: ducc.jd.queue.91 Starting Listener on Endpoint:
> > >> queue://ducc.jd.queue.91 Selector: Command=2001 Broker:
> > >>
> >
>
tcp://greenfairy:61617?wireFormat.maxInactivityDuration=0&closeAsync=false
> > >> 13 Nov 2013 15:53:45,299 INFO DUCC.DuccService - boot N/A ...
> > >> Component started: managedService
> > >> 13 Nov 2013 15:53:45,300 INFO DUCC.DuccService - boot N/A
> Starting
> > >> Camel. Use ctrl + c to terminate the JVM.
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> From: Eddie Epstein <ea...@gmail.com>
> > >> To: user@uima.apache.org
> > >> Date: 11/14/2013 10:37 AM
> > >> Subject: Re: DUCC not leaving Initializing State
> > >>
> > >>
> > >>
> > >> On Wed, Nov 13, 2013 at 7:10 PM, Neal R Lewis <nr...@us.ibm.com>
> > wrote:
> > >>
> > >>> I have modeled a CasReader and CasMultiplier based on the
> > RawTextExample
> > >>> in the duccbook, and have successfully ran the CR and CM in
eclipse
> > with
> > >> a
> > >>> minimal wrapper.
> > >>
> > >> Did you debug the job using one of the --all_in_one varieties, or
> some
> > >> other
> > >> minimal wrapper?
> > >>
> > >>
> > >> I am using the same CC from the Sample (DuccCasCC)
> > >>> I am experiencing a problem where my DUCC job goes through
> > >>> WaitingForDriver, WaitingForResources, then Initializing, then
> doesn't
> > >>> change state. After several minutes I just cancel the job.
> > >>>
> > >> Did you look in the JP logfile?
> > >>
> > >> Eddie
> > >>
> > >>
> > >>
> >
> >
> >
>
>
Re: DUCC not leaving Initializing State
Posted by Lou DeGenaro <lo...@gmail.com>.
The DUCC Job Driver (JD) employs the user supplied CR and handles the case
where getProgress() returns null, or so is my claim anyway. The CR's
getProgress() is not required for the JD to operate.
I'm not clear why this did not work for you? Adding to what I said above,
I created my own CR that a) returned null for getProgress() and b) produced
20 CASes in response to 20 getNext() invocations by the JD. My Job ran
properly and processed all 20 CASes.
Do you have a simple CR that you could share that demonstrates the problem
you are seeing?
Lou.
On Mon, Nov 18, 2013 at 4:42 PM, Neal R Lewis <nr...@us.ibm.com> wrote:
> getNext was returning a CAS, which was why it worked in my SimplePipeline
> in uimafit.
>
> After digging in with debug and AllInOne, I found the exception being
> thrown was "No Workitem in CAS". It will through an exception if the
> CasGenerator does not generate any CASes, which it determines through the
> CasGenerator's total variable. This was in the initialization of
> AllinOne.java and where the exception is thrown.
>
> //AllInOne.java
> private void initialize() throws Exception {
> String mid = "initialize";
> mh.frameworkTrace(cid, mid, "enter");
> // Generator
> casGenerator = new
> CasGenerator(jobRequestProperties, mh);
> casGenerator.initialize();
> int total = casGenerator.getTotal();
> if(total > 0) {
> // Pipeline
> casPipeline = new
> CasPipeline(jobRequestProperties, mh);
> casPipeline.initialize();
> }
> else {
> throw new NoWorkItems();
> }
> mh.frameworkTrace(cid, mid, "exit");
> }
>
>
> the casGenerator.getTotal(), get's the private class variable
> total, which is initialized from the progress:
>
> //CasGenerator.java
> private void initTotal() {
> String mid = "initTotal";
> mh.frameworkTrace(cid, mid, "enter");
> Progress progress = getProgress();
> if(progress != null) {
> total = (int)progress.getTotal();
> }
> mh.frameworkTrace(cid, mid, "exit");
> }
>
>
> So, my progress bar was returning null, so the CasGenerator just
> skip it, keep total as null. I don't know how this goes through in the
> DUCC Flow Controller, but it looks like outside of AllInOne, it causes a
> hang
>
>
>
>
> From: Lou DeGenaro <lo...@gmail.com>
> To: user@uima.apache.org
> Date: 11/18/2013 01:17 PM
> Subject: Re: DUCC not leaving Initializing State
>
>
>
> Neal,
>
> Did getNext() get called and did it return a filled-in CAS? I tried a
> simple test case with my CR returning null for getProgress() and the Job
> seemed to run just fine. However, one may be penalized by the scheduler if
> is had to guess how much work the Job comprises.
>
> Lou.
>
>
> On Fri, Nov 15, 2013 at 5:15 PM, Neal R Lewis <nr...@us.ibm.com> wrote:
>
> > I debugged using the all_in_one setup and found that I was returning
> null
> > from getProgress(), which was not creating any workitems. So I setup
> the
> > function and Cases started moving.
> >
> > Thanks for all of your help,
> >
> > Neal.
> >
> >
> >
> >
> > From: Neal Lewis <im...@gmail.com>
> > To: user@uima.apache.org
> > Date: 11/15/2013 05:48 AM
> > Subject: Re: DUCC not leaving Initializing State
> >
> >
> >
> > Thanks Eddie,
> >
> > I missed the all_in_one in the documentation, so I'll try it out and get
> > back to you guys.
> >
> > -Neal
> >
> > On 11/14/2013 06:02 PM, Eddie Epstein wrote:
> > > Good job. FYI, it is strongly recommended to use all_in_one as
> described
> > > in the documentation on developing a new sample application. Your
> > > aggregate most likely does not include the built-in DUCC flow
> controller
> > > which implements SendToLast.
> > >
> > > After all child CASes created for a work itemhave been generated, if
> > > the CC needs a signal to clean up, then use SendToLast. The DucCasCC
> > > does need that signal to close the output zip package.
> > >
> > > Eddie
> > >
> > >
> > >
> > >
> > > On Thu, Nov 14, 2013 at 6:38 PM, Neal R Lewis <nr...@us.ibm.com>
> > wrote:
> > >
> > >> So, I found one GLARING problem that I missed completely: When
> > testing
> > >> my CR and CM setup, I looped through the CR's getNext(), adding to
> the
> > >> JCas that I just created. But the CR's getNext() didn't loop
> > internally,
> > >> it only added IDs one at a time. So, instead of the CR creating a CAS
> > with
> > >> N Workitems and halting, it was create N CASes with 1 workitem. I've
> > >> fixed that.
> > >>
> > >> So I've made an Aggregate AE of my CM and a simple AE, and ran them
> > with
> > >> my CR in a uimaFit SimplePipeline. The outputs are as I expect for
> > now.
> > >>
> > >>
> > >> But I do have a question. The Workitems have an option of
> SendToLast,
> > >> which in the RawText Example is set to "true". The DuccCasCC in the
> > >> sample pulls WorkItem FSes for output stuff. The CR adds the
> > Workitems,
> > >> but the CM doesn't reattach them to the new CASes before returning
> > next().
> > >>
> > >>
> > >> Should the Workitems stay in the CAS throughout processing (ie.,
> should
> > >> the CMs add them to the new CASes) ? OR is SendToLast enough?
> > >>
> > >> Thanks!
> > >>
> > >> Neal
> > >>
> > >>
> > >>
> > >> From: Neal R Lewis/Almaden/IBM@IBMUS
> > >> To: user@uima.apache.org
> > >> Date: 11/14/2013 01:20 PM
> > >> Subject: Re: DUCC not leaving Initializing State
> > >>
> > >>
> > >>
> > >> The minimal wrapper creates a Cas using uimaFit and loop over the
> CR,
> > >> then to run over the iterator from cm.processAndOutputNewCASes, then
> > >> output each cas in the loop like below:
> > >>
> > >> CollectionReader cr =
> > CollectionReaderFactory.createReader
> > >> ("path.to.my.CasReader");
> > >> AnalysisEngine cm =
> > AnalysisEngineFactory.createEngine(
> > >> "path.to.my.CasMultiplier");
> > >>
> > >> JCas jcas = JCasFactory.createJCas("
> > >> desc.DuccJobFlowControlTS);
> > >>
> > >> while (cr.hasNext()){
> > >> cr.getNext(jcas.getCas());
> > >> }
> > >>
> > >> CasIterator casIterator =
> > >> cm.processAndOutputNewCASes(jcas.getCas());
> > >> while (casIterator.hasNext()) {
> > >> CAS outCas = casIterator.next();
> > >> System.out.println(outCas.getDocumentText());
> > >> }
> > >>
> > >> I'll need to debug a full standalone PEAR or Pipeline. But at this
> > point I
> > >>
> > >> was able to at least get the CASes out.
> > >>
> > >> Below is the output of the JP file before the stall and after loading
> > JVM
> > >> stuff. It stays like this until shutdown. As far as I can tell the
> > CM
> > >> Loads and is waiting.
> > >>
> > >>
> > >> + JVM
> > >> LIB_PATH:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> > >> +------------------------------------------------------------------
> > >> 13 Nov 2013 15:53:44,761 INFO DUCC.ThreadPoolTaskExecutor -
> > Initializing
> > >> ExecutorService 'pooling_ducc.jd.queue.91_1'
> > >> 03:53:44.794 - 1:
> > >> org.apache.uima.adapter.jms.activemq.JmsInputChannel.setEndpointName:
> > >> INFO: top_level_input_queue_service_1 Service Starting - Listening
> for
> > >> Messages
> > >> 13 Nov 2013 15:53:44,795 INFO DUCC.ThreadPoolTaskExecutor -
> > Initializing
> > >> ExecutorService 'pooling_ducc.jd.queue.91_1'
> > >> 03:53:44.797 - 30:
> > >> org.apache.uima.aae.UimaAsThreadFactory$1.UimaAsThreadFactory.run():
> > INFO:
> > >>
> > >> Controller: ducc.jd.queue.91 Initializing AE instance on Thread Id:
> 30
> > >> 03:53:44.799 - 1:
> > >>
> > >>
> >
> >
>
> org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.waitForServiceNotification:
> > >>
> > >> INFO: Uima EE Client Blocking - Awaiting Top Level Controller
> > >> Initialization Notification
> > >> 03:53:44.887 - 30:
> > >>
> > >>
> >
> >
>
> com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(70):
> > >>
> > >> INFO: Initializing Cas Multiplier Parameters
> > >> 03:53:44.887 - 30:
> > >>
> > >>
> >
> >
>
> com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(71):
> > >>
> > >> INFO: init HOST ADDRESS:
> > >> https://thepit.element.almaden.ibm.com/cgi-bin/qm2
> > >> 03:53:44.887 - 30:
> > >>
> > >>
> >
> >
>
> com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(72):
> > >>
> > >> INFO: init LENIENT: false
> > >> 03:53:44.887 - 30:
> > >>
> > >>
> >
> >
>
> com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(73):
> > >>
> > >> INFO: init PARAM_ENCODING: UTF-8
> > >> 03:53:44.887 - 30:
> > >>
> > >>
> >
> >
>
> com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(74):
> > >>
> > >> INFO: init PARAM_LANGUAGE: en
> > >> 03:53:44.933 - 30:
> > >> org.apache.uima.ducc.sampleapps.DuccCasCC.initialize(74): INFO:
> > Outputting
> > >>
> > >> CASes in XmiCas format, zip compressed at level=7
> > >> Service:ducc.jd.queue.91 Initialized. Ready To Process Messages From
> > >> Queue:ducc.jd.queue.91
> > >> 03:53:45.97 - 30:
> > >>
> > >>
> >
> >
>
> org.apache.uima.aae.controller.PrimitiveAnalysisEngineController_impl.postInitialize:
> > >>
> > >> INFO: ********* Initialized the Controller. ducc.jd.queue.91 Ready To
> > >> Process. ********
> > >> 03:53:45.97 - 1:
> > >>
> > >>
> >
> >
>
> org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.doStartListeners:
> > >>
> > >> INFO: Controller: ducc.jd.queue.91 Starting Listener on Endpoint:
> > >> queue://ducc.jd.queue.91 Selector: Command=2000 OR Command=2002
> Broker:
> > >>
> >
> tcp://greenfairy:61617?wireFormat.maxInactivityDuration=0&closeAsync=false
> > >> 03:53:45.290 - 1:
> > >>
> > >>
> >
> >
>
> org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.doStartListeners:
> > >>
> > >> INFO: Controller: ducc.jd.queue.91 Starting Listener on Endpoint:
> > >> queue://ducc.jd.queue.91 Selector: Command=2001 Broker:
> > >>
> >
> tcp://greenfairy:61617?wireFormat.maxInactivityDuration=0&closeAsync=false
> > >> 13 Nov 2013 15:53:45,299 INFO DUCC.DuccService - boot N/A ...
> > >> Component started: managedService
> > >> 13 Nov 2013 15:53:45,300 INFO DUCC.DuccService - boot N/A
> Starting
> > >> Camel. Use ctrl + c to terminate the JVM.
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> From: Eddie Epstein <ea...@gmail.com>
> > >> To: user@uima.apache.org
> > >> Date: 11/14/2013 10:37 AM
> > >> Subject: Re: DUCC not leaving Initializing State
> > >>
> > >>
> > >>
> > >> On Wed, Nov 13, 2013 at 7:10 PM, Neal R Lewis <nr...@us.ibm.com>
> > wrote:
> > >>
> > >>> I have modeled a CasReader and CasMultiplier based on the
> > RawTextExample
> > >>> in the duccbook, and have successfully ran the CR and CM in eclipse
> > with
> > >> a
> > >>> minimal wrapper.
> > >>
> > >> Did you debug the job using one of the --all_in_one varieties, or
> some
> > >> other
> > >> minimal wrapper?
> > >>
> > >>
> > >> I am using the same CC from the Sample (DuccCasCC)
> > >>> I am experiencing a problem where my DUCC job goes through
> > >>> WaitingForDriver, WaitingForResources, then Initializing, then
> doesn't
> > >>> change state. After several minutes I just cancel the job.
> > >>>
> > >> Did you look in the JP logfile?
> > >>
> > >> Eddie
> > >>
> > >>
> > >>
> >
> >
> >
>
>
Re: DUCC not leaving Initializing State
Posted by Neal R Lewis <nr...@us.ibm.com>.
getNext was returning a CAS, which was why it worked in my SimplePipeline
in uimafit.
After digging in with debug and AllInOne, I found the exception being
thrown was "No Workitem in CAS". It will through an exception if the
CasGenerator does not generate any CASes, which it determines through the
CasGenerator's total variable. This was in the initialization of
AllinOne.java and where the exception is thrown.
//AllInOne.java
private void initialize() throws Exception {
String mid = "initialize";
mh.frameworkTrace(cid, mid, "enter");
// Generator
casGenerator = new
CasGenerator(jobRequestProperties, mh);
casGenerator.initialize();
int total = casGenerator.getTotal();
if(total > 0) {
// Pipeline
casPipeline = new
CasPipeline(jobRequestProperties, mh);
casPipeline.initialize();
}
else {
throw new NoWorkItems();
}
mh.frameworkTrace(cid, mid, "exit");
}
the casGenerator.getTotal(), get's the private class variable
total, which is initialized from the progress:
//CasGenerator.java
private void initTotal() {
String mid = "initTotal";
mh.frameworkTrace(cid, mid, "enter");
Progress progress = getProgress();
if(progress != null) {
total = (int)progress.getTotal();
}
mh.frameworkTrace(cid, mid, "exit");
}
So, my progress bar was returning null, so the CasGenerator just
skip it, keep total as null. I don't know how this goes through in the
DUCC Flow Controller, but it looks like outside of AllInOne, it causes a
hang
From: Lou DeGenaro <lo...@gmail.com>
To: user@uima.apache.org
Date: 11/18/2013 01:17 PM
Subject: Re: DUCC not leaving Initializing State
Neal,
Did getNext() get called and did it return a filled-in CAS? I tried a
simple test case with my CR returning null for getProgress() and the Job
seemed to run just fine. However, one may be penalized by the scheduler if
is had to guess how much work the Job comprises.
Lou.
On Fri, Nov 15, 2013 at 5:15 PM, Neal R Lewis <nr...@us.ibm.com> wrote:
> I debugged using the all_in_one setup and found that I was returning
null
> from getProgress(), which was not creating any workitems. So I setup
the
> function and Cases started moving.
>
> Thanks for all of your help,
>
> Neal.
>
>
>
>
> From: Neal Lewis <im...@gmail.com>
> To: user@uima.apache.org
> Date: 11/15/2013 05:48 AM
> Subject: Re: DUCC not leaving Initializing State
>
>
>
> Thanks Eddie,
>
> I missed the all_in_one in the documentation, so I'll try it out and get
> back to you guys.
>
> -Neal
>
> On 11/14/2013 06:02 PM, Eddie Epstein wrote:
> > Good job. FYI, it is strongly recommended to use all_in_one as
described
> > in the documentation on developing a new sample application. Your
> > aggregate most likely does not include the built-in DUCC flow
controller
> > which implements SendToLast.
> >
> > After all child CASes created for a work itemhave been generated, if
> > the CC needs a signal to clean up, then use SendToLast. The DucCasCC
> > does need that signal to close the output zip package.
> >
> > Eddie
> >
> >
> >
> >
> > On Thu, Nov 14, 2013 at 6:38 PM, Neal R Lewis <nr...@us.ibm.com>
> wrote:
> >
> >> So, I found one GLARING problem that I missed completely: When
> testing
> >> my CR and CM setup, I looped through the CR's getNext(), adding to
the
> >> JCas that I just created. But the CR's getNext() didn't loop
> internally,
> >> it only added IDs one at a time. So, instead of the CR creating a CAS
> with
> >> N Workitems and halting, it was create N CASes with 1 workitem. I've
> >> fixed that.
> >>
> >> So I've made an Aggregate AE of my CM and a simple AE, and ran them
> with
> >> my CR in a uimaFit SimplePipeline. The outputs are as I expect for
> now.
> >>
> >>
> >> But I do have a question. The Workitems have an option of
SendToLast,
> >> which in the RawText Example is set to "true". The DuccCasCC in the
> >> sample pulls WorkItem FSes for output stuff. The CR adds the
> Workitems,
> >> but the CM doesn't reattach them to the new CASes before returning
> next().
> >>
> >>
> >> Should the Workitems stay in the CAS throughout processing (ie.,
should
> >> the CMs add them to the new CASes) ? OR is SendToLast enough?
> >>
> >> Thanks!
> >>
> >> Neal
> >>
> >>
> >>
> >> From: Neal R Lewis/Almaden/IBM@IBMUS
> >> To: user@uima.apache.org
> >> Date: 11/14/2013 01:20 PM
> >> Subject: Re: DUCC not leaving Initializing State
> >>
> >>
> >>
> >> The minimal wrapper creates a Cas using uimaFit and loop over the
CR,
> >> then to run over the iterator from cm.processAndOutputNewCASes, then
> >> output each cas in the loop like below:
> >>
> >> CollectionReader cr =
> CollectionReaderFactory.createReader
> >> ("path.to.my.CasReader");
> >> AnalysisEngine cm =
> AnalysisEngineFactory.createEngine(
> >> "path.to.my.CasMultiplier");
> >>
> >> JCas jcas = JCasFactory.createJCas("
> >> desc.DuccJobFlowControlTS);
> >>
> >> while (cr.hasNext()){
> >> cr.getNext(jcas.getCas());
> >> }
> >>
> >> CasIterator casIterator =
> >> cm.processAndOutputNewCASes(jcas.getCas());
> >> while (casIterator.hasNext()) {
> >> CAS outCas = casIterator.next();
> >> System.out.println(outCas.getDocumentText());
> >> }
> >>
> >> I'll need to debug a full standalone PEAR or Pipeline. But at this
> point I
> >>
> >> was able to at least get the CASes out.
> >>
> >> Below is the output of the JP file before the stall and after loading
> JVM
> >> stuff. It stays like this until shutdown. As far as I can tell the
> CM
> >> Loads and is waiting.
> >>
> >>
> >> + JVM
> >> LIB_PATH:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> >> +------------------------------------------------------------------
> >> 13 Nov 2013 15:53:44,761 INFO DUCC.ThreadPoolTaskExecutor -
> Initializing
> >> ExecutorService 'pooling_ducc.jd.queue.91_1'
> >> 03:53:44.794 - 1:
> >> org.apache.uima.adapter.jms.activemq.JmsInputChannel.setEndpointName:
> >> INFO: top_level_input_queue_service_1 Service Starting - Listening
for
> >> Messages
> >> 13 Nov 2013 15:53:44,795 INFO DUCC.ThreadPoolTaskExecutor -
> Initializing
> >> ExecutorService 'pooling_ducc.jd.queue.91_1'
> >> 03:53:44.797 - 30:
> >> org.apache.uima.aae.UimaAsThreadFactory$1.UimaAsThreadFactory.run():
> INFO:
> >>
> >> Controller: ducc.jd.queue.91 Initializing AE instance on Thread Id:
30
> >> 03:53:44.799 - 1:
> >>
> >>
>
>
org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.waitForServiceNotification:
> >>
> >> INFO: Uima EE Client Blocking - Awaiting Top Level Controller
> >> Initialization Notification
> >> 03:53:44.887 - 30:
> >>
> >>
>
>
com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(70):
> >>
> >> INFO: Initializing Cas Multiplier Parameters
> >> 03:53:44.887 - 30:
> >>
> >>
>
>
com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(71):
> >>
> >> INFO: init HOST ADDRESS:
> >> https://thepit.element.almaden.ibm.com/cgi-bin/qm2
> >> 03:53:44.887 - 30:
> >>
> >>
>
>
com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(72):
> >>
> >> INFO: init LENIENT: false
> >> 03:53:44.887 - 30:
> >>
> >>
>
>
com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(73):
> >>
> >> INFO: init PARAM_ENCODING: UTF-8
> >> 03:53:44.887 - 30:
> >>
> >>
>
>
com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(74):
> >>
> >> INFO: init PARAM_LANGUAGE: en
> >> 03:53:44.933 - 30:
> >> org.apache.uima.ducc.sampleapps.DuccCasCC.initialize(74): INFO:
> Outputting
> >>
> >> CASes in XmiCas format, zip compressed at level=7
> >> Service:ducc.jd.queue.91 Initialized. Ready To Process Messages From
> >> Queue:ducc.jd.queue.91
> >> 03:53:45.97 - 30:
> >>
> >>
>
>
org.apache.uima.aae.controller.PrimitiveAnalysisEngineController_impl.postInitialize:
> >>
> >> INFO: ********* Initialized the Controller. ducc.jd.queue.91 Ready To
> >> Process. ********
> >> 03:53:45.97 - 1:
> >>
> >>
>
>
org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.doStartListeners:
> >>
> >> INFO: Controller: ducc.jd.queue.91 Starting Listener on Endpoint:
> >> queue://ducc.jd.queue.91 Selector: Command=2000 OR Command=2002
Broker:
> >>
>
tcp://greenfairy:61617?wireFormat.maxInactivityDuration=0&closeAsync=false
> >> 03:53:45.290 - 1:
> >>
> >>
>
>
org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.doStartListeners:
> >>
> >> INFO: Controller: ducc.jd.queue.91 Starting Listener on Endpoint:
> >> queue://ducc.jd.queue.91 Selector: Command=2001 Broker:
> >>
>
tcp://greenfairy:61617?wireFormat.maxInactivityDuration=0&closeAsync=false
> >> 13 Nov 2013 15:53:45,299 INFO DUCC.DuccService - boot N/A ...
> >> Component started: managedService
> >> 13 Nov 2013 15:53:45,300 INFO DUCC.DuccService - boot N/A
Starting
> >> Camel. Use ctrl + c to terminate the JVM.
> >>
> >>
> >>
> >>
> >>
> >>
> >> From: Eddie Epstein <ea...@gmail.com>
> >> To: user@uima.apache.org
> >> Date: 11/14/2013 10:37 AM
> >> Subject: Re: DUCC not leaving Initializing State
> >>
> >>
> >>
> >> On Wed, Nov 13, 2013 at 7:10 PM, Neal R Lewis <nr...@us.ibm.com>
> wrote:
> >>
> >>> I have modeled a CasReader and CasMultiplier based on the
> RawTextExample
> >>> in the duccbook, and have successfully ran the CR and CM in eclipse
> with
> >> a
> >>> minimal wrapper.
> >>
> >> Did you debug the job using one of the --all_in_one varieties, or
some
> >> other
> >> minimal wrapper?
> >>
> >>
> >> I am using the same CC from the Sample (DuccCasCC)
> >>> I am experiencing a problem where my DUCC job goes through
> >>> WaitingForDriver, WaitingForResources, then Initializing, then
doesn't
> >>> change state. After several minutes I just cancel the job.
> >>>
> >> Did you look in the JP logfile?
> >>
> >> Eddie
> >>
> >>
> >>
>
>
>
Re: DUCC not leaving Initializing State
Posted by Lou DeGenaro <lo...@gmail.com>.
Neal,
Did getNext() get called and did it return a filled-in CAS? I tried a
simple test case with my CR returning null for getProgress() and the Job
seemed to run just fine. However, one may be penalized by the scheduler if
is had to guess how much work the Job comprises.
Lou.
On Fri, Nov 15, 2013 at 5:15 PM, Neal R Lewis <nr...@us.ibm.com> wrote:
> I debugged using the all_in_one setup and found that I was returning null
> from getProgress(), which was not creating any workitems. So I setup the
> function and Cases started moving.
>
> Thanks for all of your help,
>
> Neal.
>
>
>
>
> From: Neal Lewis <im...@gmail.com>
> To: user@uima.apache.org
> Date: 11/15/2013 05:48 AM
> Subject: Re: DUCC not leaving Initializing State
>
>
>
> Thanks Eddie,
>
> I missed the all_in_one in the documentation, so I'll try it out and get
> back to you guys.
>
> -Neal
>
> On 11/14/2013 06:02 PM, Eddie Epstein wrote:
> > Good job. FYI, it is strongly recommended to use all_in_one as described
> > in the documentation on developing a new sample application. Your
> > aggregate most likely does not include the built-in DUCC flow controller
> > which implements SendToLast.
> >
> > After all child CASes created for a work itemhave been generated, if
> > the CC needs a signal to clean up, then use SendToLast. The DucCasCC
> > does need that signal to close the output zip package.
> >
> > Eddie
> >
> >
> >
> >
> > On Thu, Nov 14, 2013 at 6:38 PM, Neal R Lewis <nr...@us.ibm.com>
> wrote:
> >
> >> So, I found one GLARING problem that I missed completely: When
> testing
> >> my CR and CM setup, I looped through the CR's getNext(), adding to the
> >> JCas that I just created. But the CR's getNext() didn't loop
> internally,
> >> it only added IDs one at a time. So, instead of the CR creating a CAS
> with
> >> N Workitems and halting, it was create N CASes with 1 workitem. I've
> >> fixed that.
> >>
> >> So I've made an Aggregate AE of my CM and a simple AE, and ran them
> with
> >> my CR in a uimaFit SimplePipeline. The outputs are as I expect for
> now.
> >>
> >>
> >> But I do have a question. The Workitems have an option of SendToLast,
> >> which in the RawText Example is set to "true". The DuccCasCC in the
> >> sample pulls WorkItem FSes for output stuff. The CR adds the
> Workitems,
> >> but the CM doesn't reattach them to the new CASes before returning
> next().
> >>
> >>
> >> Should the Workitems stay in the CAS throughout processing (ie., should
> >> the CMs add them to the new CASes) ? OR is SendToLast enough?
> >>
> >> Thanks!
> >>
> >> Neal
> >>
> >>
> >>
> >> From: Neal R Lewis/Almaden/IBM@IBMUS
> >> To: user@uima.apache.org
> >> Date: 11/14/2013 01:20 PM
> >> Subject: Re: DUCC not leaving Initializing State
> >>
> >>
> >>
> >> The minimal wrapper creates a Cas using uimaFit and loop over the CR,
> >> then to run over the iterator from cm.processAndOutputNewCASes, then
> >> output each cas in the loop like below:
> >>
> >> CollectionReader cr =
> CollectionReaderFactory.createReader
> >> ("path.to.my.CasReader");
> >> AnalysisEngine cm =
> AnalysisEngineFactory.createEngine(
> >> "path.to.my.CasMultiplier");
> >>
> >> JCas jcas = JCasFactory.createJCas("
> >> desc.DuccJobFlowControlTS);
> >>
> >> while (cr.hasNext()){
> >> cr.getNext(jcas.getCas());
> >> }
> >>
> >> CasIterator casIterator =
> >> cm.processAndOutputNewCASes(jcas.getCas());
> >> while (casIterator.hasNext()) {
> >> CAS outCas = casIterator.next();
> >> System.out.println(outCas.getDocumentText());
> >> }
> >>
> >> I'll need to debug a full standalone PEAR or Pipeline. But at this
> point I
> >>
> >> was able to at least get the CASes out.
> >>
> >> Below is the output of the JP file before the stall and after loading
> JVM
> >> stuff. It stays like this until shutdown. As far as I can tell the
> CM
> >> Loads and is waiting.
> >>
> >>
> >> + JVM
> >> LIB_PATH:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> >> +------------------------------------------------------------------
> >> 13 Nov 2013 15:53:44,761 INFO DUCC.ThreadPoolTaskExecutor -
> Initializing
> >> ExecutorService 'pooling_ducc.jd.queue.91_1'
> >> 03:53:44.794 - 1:
> >> org.apache.uima.adapter.jms.activemq.JmsInputChannel.setEndpointName:
> >> INFO: top_level_input_queue_service_1 Service Starting - Listening for
> >> Messages
> >> 13 Nov 2013 15:53:44,795 INFO DUCC.ThreadPoolTaskExecutor -
> Initializing
> >> ExecutorService 'pooling_ducc.jd.queue.91_1'
> >> 03:53:44.797 - 30:
> >> org.apache.uima.aae.UimaAsThreadFactory$1.UimaAsThreadFactory.run():
> INFO:
> >>
> >> Controller: ducc.jd.queue.91 Initializing AE instance on Thread Id: 30
> >> 03:53:44.799 - 1:
> >>
> >>
>
> org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.waitForServiceNotification:
> >>
> >> INFO: Uima EE Client Blocking - Awaiting Top Level Controller
> >> Initialization Notification
> >> 03:53:44.887 - 30:
> >>
> >>
>
> com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(70):
> >>
> >> INFO: Initializing Cas Multiplier Parameters
> >> 03:53:44.887 - 30:
> >>
> >>
>
> com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(71):
> >>
> >> INFO: init HOST ADDRESS:
> >> https://thepit.element.almaden.ibm.com/cgi-bin/qm2
> >> 03:53:44.887 - 30:
> >>
> >>
>
> com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(72):
> >>
> >> INFO: init LENIENT: false
> >> 03:53:44.887 - 30:
> >>
> >>
>
> com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(73):
> >>
> >> INFO: init PARAM_ENCODING: UTF-8
> >> 03:53:44.887 - 30:
> >>
> >>
>
> com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(74):
> >>
> >> INFO: init PARAM_LANGUAGE: en
> >> 03:53:44.933 - 30:
> >> org.apache.uima.ducc.sampleapps.DuccCasCC.initialize(74): INFO:
> Outputting
> >>
> >> CASes in XmiCas format, zip compressed at level=7
> >> Service:ducc.jd.queue.91 Initialized. Ready To Process Messages From
> >> Queue:ducc.jd.queue.91
> >> 03:53:45.97 - 30:
> >>
> >>
>
> org.apache.uima.aae.controller.PrimitiveAnalysisEngineController_impl.postInitialize:
> >>
> >> INFO: ********* Initialized the Controller. ducc.jd.queue.91 Ready To
> >> Process. ********
> >> 03:53:45.97 - 1:
> >>
> >>
>
> org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.doStartListeners:
> >>
> >> INFO: Controller: ducc.jd.queue.91 Starting Listener on Endpoint:
> >> queue://ducc.jd.queue.91 Selector: Command=2000 OR Command=2002 Broker:
> >>
> tcp://greenfairy:61617?wireFormat.maxInactivityDuration=0&closeAsync=false
> >> 03:53:45.290 - 1:
> >>
> >>
>
> org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.doStartListeners:
> >>
> >> INFO: Controller: ducc.jd.queue.91 Starting Listener on Endpoint:
> >> queue://ducc.jd.queue.91 Selector: Command=2001 Broker:
> >>
> tcp://greenfairy:61617?wireFormat.maxInactivityDuration=0&closeAsync=false
> >> 13 Nov 2013 15:53:45,299 INFO DUCC.DuccService - boot N/A ...
> >> Component started: managedService
> >> 13 Nov 2013 15:53:45,300 INFO DUCC.DuccService - boot N/A Starting
> >> Camel. Use ctrl + c to terminate the JVM.
> >>
> >>
> >>
> >>
> >>
> >>
> >> From: Eddie Epstein <ea...@gmail.com>
> >> To: user@uima.apache.org
> >> Date: 11/14/2013 10:37 AM
> >> Subject: Re: DUCC not leaving Initializing State
> >>
> >>
> >>
> >> On Wed, Nov 13, 2013 at 7:10 PM, Neal R Lewis <nr...@us.ibm.com>
> wrote:
> >>
> >>> I have modeled a CasReader and CasMultiplier based on the
> RawTextExample
> >>> in the duccbook, and have successfully ran the CR and CM in eclipse
> with
> >> a
> >>> minimal wrapper.
> >>
> >> Did you debug the job using one of the --all_in_one varieties, or some
> >> other
> >> minimal wrapper?
> >>
> >>
> >> I am using the same CC from the Sample (DuccCasCC)
> >>> I am experiencing a problem where my DUCC job goes through
> >>> WaitingForDriver, WaitingForResources, then Initializing, then doesn't
> >>> change state. After several minutes I just cancel the job.
> >>>
> >> Did you look in the JP logfile?
> >>
> >> Eddie
> >>
> >>
> >>
>
>
>
Re: DUCC not leaving Initializing State
Posted by Neal R Lewis <nr...@us.ibm.com>.
I debugged using the all_in_one setup and found that I was returning null
from getProgress(), which was not creating any workitems. So I setup the
function and Cases started moving.
Thanks for all of your help,
Neal.
From: Neal Lewis <im...@gmail.com>
To: user@uima.apache.org
Date: 11/15/2013 05:48 AM
Subject: Re: DUCC not leaving Initializing State
Thanks Eddie,
I missed the all_in_one in the documentation, so I'll try it out and get
back to you guys.
-Neal
On 11/14/2013 06:02 PM, Eddie Epstein wrote:
> Good job. FYI, it is strongly recommended to use all_in_one as described
> in the documentation on developing a new sample application. Your
> aggregate most likely does not include the built-in DUCC flow controller
> which implements SendToLast.
>
> After all child CASes created for a work itemhave been generated, if
> the CC needs a signal to clean up, then use SendToLast. The DucCasCC
> does need that signal to close the output zip package.
>
> Eddie
>
>
>
>
> On Thu, Nov 14, 2013 at 6:38 PM, Neal R Lewis <nr...@us.ibm.com>
wrote:
>
>> So, I found one GLARING problem that I missed completely: When
testing
>> my CR and CM setup, I looped through the CR's getNext(), adding to the
>> JCas that I just created. But the CR's getNext() didn't loop
internally,
>> it only added IDs one at a time. So, instead of the CR creating a CAS
with
>> N Workitems and halting, it was create N CASes with 1 workitem. I've
>> fixed that.
>>
>> So I've made an Aggregate AE of my CM and a simple AE, and ran them
with
>> my CR in a uimaFit SimplePipeline. The outputs are as I expect for
now.
>>
>>
>> But I do have a question. The Workitems have an option of SendToLast,
>> which in the RawText Example is set to "true". The DuccCasCC in the
>> sample pulls WorkItem FSes for output stuff. The CR adds the
Workitems,
>> but the CM doesn't reattach them to the new CASes before returning
next().
>>
>>
>> Should the Workitems stay in the CAS throughout processing (ie., should
>> the CMs add them to the new CASes) ? OR is SendToLast enough?
>>
>> Thanks!
>>
>> Neal
>>
>>
>>
>> From: Neal R Lewis/Almaden/IBM@IBMUS
>> To: user@uima.apache.org
>> Date: 11/14/2013 01:20 PM
>> Subject: Re: DUCC not leaving Initializing State
>>
>>
>>
>> The minimal wrapper creates a Cas using uimaFit and loop over the CR,
>> then to run over the iterator from cm.processAndOutputNewCASes, then
>> output each cas in the loop like below:
>>
>> CollectionReader cr =
CollectionReaderFactory.createReader
>> ("path.to.my.CasReader");
>> AnalysisEngine cm =
AnalysisEngineFactory.createEngine(
>> "path.to.my.CasMultiplier");
>>
>> JCas jcas = JCasFactory.createJCas("
>> desc.DuccJobFlowControlTS);
>>
>> while (cr.hasNext()){
>> cr.getNext(jcas.getCas());
>> }
>>
>> CasIterator casIterator =
>> cm.processAndOutputNewCASes(jcas.getCas());
>> while (casIterator.hasNext()) {
>> CAS outCas = casIterator.next();
>> System.out.println(outCas.getDocumentText());
>> }
>>
>> I'll need to debug a full standalone PEAR or Pipeline. But at this
point I
>>
>> was able to at least get the CASes out.
>>
>> Below is the output of the JP file before the stall and after loading
JVM
>> stuff. It stays like this until shutdown. As far as I can tell the
CM
>> Loads and is waiting.
>>
>>
>> + JVM
>> LIB_PATH:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
>> +------------------------------------------------------------------
>> 13 Nov 2013 15:53:44,761 INFO DUCC.ThreadPoolTaskExecutor -
Initializing
>> ExecutorService 'pooling_ducc.jd.queue.91_1'
>> 03:53:44.794 - 1:
>> org.apache.uima.adapter.jms.activemq.JmsInputChannel.setEndpointName:
>> INFO: top_level_input_queue_service_1 Service Starting - Listening for
>> Messages
>> 13 Nov 2013 15:53:44,795 INFO DUCC.ThreadPoolTaskExecutor -
Initializing
>> ExecutorService 'pooling_ducc.jd.queue.91_1'
>> 03:53:44.797 - 30:
>> org.apache.uima.aae.UimaAsThreadFactory$1.UimaAsThreadFactory.run():
INFO:
>>
>> Controller: ducc.jd.queue.91 Initializing AE instance on Thread Id: 30
>> 03:53:44.799 - 1:
>>
>>
org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.waitForServiceNotification:
>>
>> INFO: Uima EE Client Blocking - Awaiting Top Level Controller
>> Initialization Notification
>> 03:53:44.887 - 30:
>>
>>
com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(70):
>>
>> INFO: Initializing Cas Multiplier Parameters
>> 03:53:44.887 - 30:
>>
>>
com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(71):
>>
>> INFO: init HOST ADDRESS:
>> https://thepit.element.almaden.ibm.com/cgi-bin/qm2
>> 03:53:44.887 - 30:
>>
>>
com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(72):
>>
>> INFO: init LENIENT: false
>> 03:53:44.887 - 30:
>>
>>
com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(73):
>>
>> INFO: init PARAM_ENCODING: UTF-8
>> 03:53:44.887 - 30:
>>
>>
com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(74):
>>
>> INFO: init PARAM_LANGUAGE: en
>> 03:53:44.933 - 30:
>> org.apache.uima.ducc.sampleapps.DuccCasCC.initialize(74): INFO:
Outputting
>>
>> CASes in XmiCas format, zip compressed at level=7
>> Service:ducc.jd.queue.91 Initialized. Ready To Process Messages From
>> Queue:ducc.jd.queue.91
>> 03:53:45.97 - 30:
>>
>>
org.apache.uima.aae.controller.PrimitiveAnalysisEngineController_impl.postInitialize:
>>
>> INFO: ********* Initialized the Controller. ducc.jd.queue.91 Ready To
>> Process. ********
>> 03:53:45.97 - 1:
>>
>>
org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.doStartListeners:
>>
>> INFO: Controller: ducc.jd.queue.91 Starting Listener on Endpoint:
>> queue://ducc.jd.queue.91 Selector: Command=2000 OR Command=2002 Broker:
>>
tcp://greenfairy:61617?wireFormat.maxInactivityDuration=0&closeAsync=false
>> 03:53:45.290 - 1:
>>
>>
org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.doStartListeners:
>>
>> INFO: Controller: ducc.jd.queue.91 Starting Listener on Endpoint:
>> queue://ducc.jd.queue.91 Selector: Command=2001 Broker:
>>
tcp://greenfairy:61617?wireFormat.maxInactivityDuration=0&closeAsync=false
>> 13 Nov 2013 15:53:45,299 INFO DUCC.DuccService - boot N/A ...
>> Component started: managedService
>> 13 Nov 2013 15:53:45,300 INFO DUCC.DuccService - boot N/A Starting
>> Camel. Use ctrl + c to terminate the JVM.
>>
>>
>>
>>
>>
>>
>> From: Eddie Epstein <ea...@gmail.com>
>> To: user@uima.apache.org
>> Date: 11/14/2013 10:37 AM
>> Subject: Re: DUCC not leaving Initializing State
>>
>>
>>
>> On Wed, Nov 13, 2013 at 7:10 PM, Neal R Lewis <nr...@us.ibm.com>
wrote:
>>
>>> I have modeled a CasReader and CasMultiplier based on the
RawTextExample
>>> in the duccbook, and have successfully ran the CR and CM in eclipse
with
>> a
>>> minimal wrapper.
>>
>> Did you debug the job using one of the --all_in_one varieties, or some
>> other
>> minimal wrapper?
>>
>>
>> I am using the same CC from the Sample (DuccCasCC)
>>> I am experiencing a problem where my DUCC job goes through
>>> WaitingForDriver, WaitingForResources, then Initializing, then doesn't
>>> change state. After several minutes I just cancel the job.
>>>
>> Did you look in the JP logfile?
>>
>> Eddie
>>
>>
>>
Re: DUCC not leaving Initializing State
Posted by Neal Lewis <im...@gmail.com>.
Thanks Eddie,
I missed the all_in_one in the documentation, so I'll try it out and get
back to you guys.
-Neal
On 11/14/2013 06:02 PM, Eddie Epstein wrote:
> Good job. FYI, it is strongly recommended to use all_in_one as described
> in the documentation on developing a new sample application. Your
> aggregate most likely does not include the built-in DUCC flow controller
> which implements SendToLast.
>
> After all child CASes created for a work itemhave been generated, if
> the CC needs a signal to clean up, then use SendToLast. The DucCasCC
> does need that signal to close the output zip package.
>
> Eddie
>
>
>
>
> On Thu, Nov 14, 2013 at 6:38 PM, Neal R Lewis <nr...@us.ibm.com> wrote:
>
>> So, I found one GLARING problem that I missed completely: When testing
>> my CR and CM setup, I looped through the CR's getNext(), adding to the
>> JCas that I just created. But the CR's getNext() didn't loop internally,
>> it only added IDs one at a time. So, instead of the CR creating a CAS with
>> N Workitems and halting, it was create N CASes with 1 workitem. I've
>> fixed that.
>>
>> So I've made an Aggregate AE of my CM and a simple AE, and ran them with
>> my CR in a uimaFit SimplePipeline. The outputs are as I expect for now.
>>
>>
>> But I do have a question. The Workitems have an option of SendToLast,
>> which in the RawText Example is set to "true". The DuccCasCC in the
>> sample pulls WorkItem FSes for output stuff. The CR adds the Workitems,
>> but the CM doesn't reattach them to the new CASes before returning next().
>>
>>
>> Should the Workitems stay in the CAS throughout processing (ie., should
>> the CMs add them to the new CASes) ? OR is SendToLast enough?
>>
>> Thanks!
>>
>> Neal
>>
>>
>>
>> From: Neal R Lewis/Almaden/IBM@IBMUS
>> To: user@uima.apache.org
>> Date: 11/14/2013 01:20 PM
>> Subject: Re: DUCC not leaving Initializing State
>>
>>
>>
>> The minimal wrapper creates a Cas using uimaFit and loop over the CR,
>> then to run over the iterator from cm.processAndOutputNewCASes, then
>> output each cas in the loop like below:
>>
>> CollectionReader cr = CollectionReaderFactory.createReader
>> ("path.to.my.CasReader");
>> AnalysisEngine cm = AnalysisEngineFactory.createEngine(
>> "path.to.my.CasMultiplier");
>>
>> JCas jcas = JCasFactory.createJCas("
>> desc.DuccJobFlowControlTS);
>>
>> while (cr.hasNext()){
>> cr.getNext(jcas.getCas());
>> }
>>
>> CasIterator casIterator =
>> cm.processAndOutputNewCASes(jcas.getCas());
>> while (casIterator.hasNext()) {
>> CAS outCas = casIterator.next();
>> System.out.println(outCas.getDocumentText());
>> }
>>
>> I'll need to debug a full standalone PEAR or Pipeline. But at this point I
>>
>> was able to at least get the CASes out.
>>
>> Below is the output of the JP file before the stall and after loading JVM
>> stuff. It stays like this until shutdown. As far as I can tell the CM
>> Loads and is waiting.
>>
>>
>> + JVM
>> LIB_PATH:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
>> +------------------------------------------------------------------
>> 13 Nov 2013 15:53:44,761 INFO DUCC.ThreadPoolTaskExecutor - Initializing
>> ExecutorService 'pooling_ducc.jd.queue.91_1'
>> 03:53:44.794 - 1:
>> org.apache.uima.adapter.jms.activemq.JmsInputChannel.setEndpointName:
>> INFO: top_level_input_queue_service_1 Service Starting - Listening for
>> Messages
>> 13 Nov 2013 15:53:44,795 INFO DUCC.ThreadPoolTaskExecutor - Initializing
>> ExecutorService 'pooling_ducc.jd.queue.91_1'
>> 03:53:44.797 - 30:
>> org.apache.uima.aae.UimaAsThreadFactory$1.UimaAsThreadFactory.run(): INFO:
>>
>> Controller: ducc.jd.queue.91 Initializing AE instance on Thread Id: 30
>> 03:53:44.799 - 1:
>>
>> org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.waitForServiceNotification:
>>
>> INFO: Uima EE Client Blocking - Awaiting Top Level Controller
>> Initialization Notification
>> 03:53:44.887 - 30:
>>
>> com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(70):
>>
>> INFO: Initializing Cas Multiplier Parameters
>> 03:53:44.887 - 30:
>>
>> com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(71):
>>
>> INFO: init HOST ADDRESS:
>> https://thepit.element.almaden.ibm.com/cgi-bin/qm2
>> 03:53:44.887 - 30:
>>
>> com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(72):
>>
>> INFO: init LENIENT: false
>> 03:53:44.887 - 30:
>>
>> com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(73):
>>
>> INFO: init PARAM_ENCODING: UTF-8
>> 03:53:44.887 - 30:
>>
>> com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(74):
>>
>> INFO: init PARAM_LANGUAGE: en
>> 03:53:44.933 - 30:
>> org.apache.uima.ducc.sampleapps.DuccCasCC.initialize(74): INFO: Outputting
>>
>> CASes in XmiCas format, zip compressed at level=7
>> Service:ducc.jd.queue.91 Initialized. Ready To Process Messages From
>> Queue:ducc.jd.queue.91
>> 03:53:45.97 - 30:
>>
>> org.apache.uima.aae.controller.PrimitiveAnalysisEngineController_impl.postInitialize:
>>
>> INFO: ********* Initialized the Controller. ducc.jd.queue.91 Ready To
>> Process. ********
>> 03:53:45.97 - 1:
>>
>> org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.doStartListeners:
>>
>> INFO: Controller: ducc.jd.queue.91 Starting Listener on Endpoint:
>> queue://ducc.jd.queue.91 Selector: Command=2000 OR Command=2002 Broker:
>> tcp://greenfairy:61617?wireFormat.maxInactivityDuration=0&closeAsync=false
>> 03:53:45.290 - 1:
>>
>> org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.doStartListeners:
>>
>> INFO: Controller: ducc.jd.queue.91 Starting Listener on Endpoint:
>> queue://ducc.jd.queue.91 Selector: Command=2001 Broker:
>> tcp://greenfairy:61617?wireFormat.maxInactivityDuration=0&closeAsync=false
>> 13 Nov 2013 15:53:45,299 INFO DUCC.DuccService - boot N/A ...
>> Component started: managedService
>> 13 Nov 2013 15:53:45,300 INFO DUCC.DuccService - boot N/A Starting
>> Camel. Use ctrl + c to terminate the JVM.
>>
>>
>>
>>
>>
>>
>> From: Eddie Epstein <ea...@gmail.com>
>> To: user@uima.apache.org
>> Date: 11/14/2013 10:37 AM
>> Subject: Re: DUCC not leaving Initializing State
>>
>>
>>
>> On Wed, Nov 13, 2013 at 7:10 PM, Neal R Lewis <nr...@us.ibm.com> wrote:
>>
>>> I have modeled a CasReader and CasMultiplier based on the RawTextExample
>>> in the duccbook, and have successfully ran the CR and CM in eclipse with
>> a
>>> minimal wrapper.
>>
>> Did you debug the job using one of the --all_in_one varieties, or some
>> other
>> minimal wrapper?
>>
>>
>> I am using the same CC from the Sample (DuccCasCC)
>>> I am experiencing a problem where my DUCC job goes through
>>> WaitingForDriver, WaitingForResources, then Initializing, then doesn't
>>> change state. After several minutes I just cancel the job.
>>>
>> Did you look in the JP logfile?
>>
>> Eddie
>>
>>
>>
Re: DUCC not leaving Initializing State
Posted by Eddie Epstein <ea...@gmail.com>.
Good job. FYI, it is strongly recommended to use all_in_one as described
in the documentation on developing a new sample application. Your
aggregate most likely does not include the built-in DUCC flow controller
which implements SendToLast.
After all child CASes created for a work itemhave been generated, if
the CC needs a signal to clean up, then use SendToLast. The DucCasCC
does need that signal to close the output zip package.
Eddie
On Thu, Nov 14, 2013 at 6:38 PM, Neal R Lewis <nr...@us.ibm.com> wrote:
> So, I found one GLARING problem that I missed completely: When testing
> my CR and CM setup, I looped through the CR's getNext(), adding to the
> JCas that I just created. But the CR's getNext() didn't loop internally,
> it only added IDs one at a time. So, instead of the CR creating a CAS with
> N Workitems and halting, it was create N CASes with 1 workitem. I've
> fixed that.
>
> So I've made an Aggregate AE of my CM and a simple AE, and ran them with
> my CR in a uimaFit SimplePipeline. The outputs are as I expect for now.
>
>
> But I do have a question. The Workitems have an option of SendToLast,
> which in the RawText Example is set to "true". The DuccCasCC in the
> sample pulls WorkItem FSes for output stuff. The CR adds the Workitems,
> but the CM doesn't reattach them to the new CASes before returning next().
>
>
> Should the Workitems stay in the CAS throughout processing (ie., should
> the CMs add them to the new CASes) ? OR is SendToLast enough?
>
> Thanks!
>
> Neal
>
>
>
> From: Neal R Lewis/Almaden/IBM@IBMUS
> To: user@uima.apache.org
> Date: 11/14/2013 01:20 PM
> Subject: Re: DUCC not leaving Initializing State
>
>
>
> The minimal wrapper creates a Cas using uimaFit and loop over the CR,
> then to run over the iterator from cm.processAndOutputNewCASes, then
> output each cas in the loop like below:
>
> CollectionReader cr = CollectionReaderFactory.createReader
> ("path.to.my.CasReader");
> AnalysisEngine cm = AnalysisEngineFactory.createEngine(
> "path.to.my.CasMultiplier");
>
> JCas jcas = JCasFactory.createJCas("
> desc.DuccJobFlowControlTS);
>
> while (cr.hasNext()){
> cr.getNext(jcas.getCas());
> }
>
> CasIterator casIterator =
> cm.processAndOutputNewCASes(jcas.getCas());
> while (casIterator.hasNext()) {
> CAS outCas = casIterator.next();
> System.out.println(outCas.getDocumentText());
> }
>
> I'll need to debug a full standalone PEAR or Pipeline. But at this point I
>
> was able to at least get the CASes out.
>
> Below is the output of the JP file before the stall and after loading JVM
> stuff. It stays like this until shutdown. As far as I can tell the CM
> Loads and is waiting.
>
>
> + JVM
> LIB_PATH:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> +------------------------------------------------------------------
> 13 Nov 2013 15:53:44,761 INFO DUCC.ThreadPoolTaskExecutor - Initializing
> ExecutorService 'pooling_ducc.jd.queue.91_1'
> 03:53:44.794 - 1:
> org.apache.uima.adapter.jms.activemq.JmsInputChannel.setEndpointName:
> INFO: top_level_input_queue_service_1 Service Starting - Listening for
> Messages
> 13 Nov 2013 15:53:44,795 INFO DUCC.ThreadPoolTaskExecutor - Initializing
> ExecutorService 'pooling_ducc.jd.queue.91_1'
> 03:53:44.797 - 30:
> org.apache.uima.aae.UimaAsThreadFactory$1.UimaAsThreadFactory.run(): INFO:
>
> Controller: ducc.jd.queue.91 Initializing AE instance on Thread Id: 30
> 03:53:44.799 - 1:
>
> org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.waitForServiceNotification:
>
> INFO: Uima EE Client Blocking - Awaiting Top Level Controller
> Initialization Notification
> 03:53:44.887 - 30:
>
> com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(70):
>
> INFO: Initializing Cas Multiplier Parameters
> 03:53:44.887 - 30:
>
> com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(71):
>
> INFO: init HOST ADDRESS:
> https://thepit.element.almaden.ibm.com/cgi-bin/qm2
> 03:53:44.887 - 30:
>
> com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(72):
>
> INFO: init LENIENT: false
> 03:53:44.887 - 30:
>
> com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(73):
>
> INFO: init PARAM_ENCODING: UTF-8
> 03:53:44.887 - 30:
>
> com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(74):
>
> INFO: init PARAM_LANGUAGE: en
> 03:53:44.933 - 30:
> org.apache.uima.ducc.sampleapps.DuccCasCC.initialize(74): INFO: Outputting
>
> CASes in XmiCas format, zip compressed at level=7
> Service:ducc.jd.queue.91 Initialized. Ready To Process Messages From
> Queue:ducc.jd.queue.91
> 03:53:45.97 - 30:
>
> org.apache.uima.aae.controller.PrimitiveAnalysisEngineController_impl.postInitialize:
>
> INFO: ********* Initialized the Controller. ducc.jd.queue.91 Ready To
> Process. ********
> 03:53:45.97 - 1:
>
> org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.doStartListeners:
>
> INFO: Controller: ducc.jd.queue.91 Starting Listener on Endpoint:
> queue://ducc.jd.queue.91 Selector: Command=2000 OR Command=2002 Broker:
> tcp://greenfairy:61617?wireFormat.maxInactivityDuration=0&closeAsync=false
> 03:53:45.290 - 1:
>
> org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.doStartListeners:
>
> INFO: Controller: ducc.jd.queue.91 Starting Listener on Endpoint:
> queue://ducc.jd.queue.91 Selector: Command=2001 Broker:
> tcp://greenfairy:61617?wireFormat.maxInactivityDuration=0&closeAsync=false
> 13 Nov 2013 15:53:45,299 INFO DUCC.DuccService - boot N/A ...
> Component started: managedService
> 13 Nov 2013 15:53:45,300 INFO DUCC.DuccService - boot N/A Starting
> Camel. Use ctrl + c to terminate the JVM.
>
>
>
>
>
>
> From: Eddie Epstein <ea...@gmail.com>
> To: user@uima.apache.org
> Date: 11/14/2013 10:37 AM
> Subject: Re: DUCC not leaving Initializing State
>
>
>
> On Wed, Nov 13, 2013 at 7:10 PM, Neal R Lewis <nr...@us.ibm.com> wrote:
>
> > I have modeled a CasReader and CasMultiplier based on the RawTextExample
> > in the duccbook, and have successfully ran the CR and CM in eclipse with
>
> a
> > minimal wrapper.
>
>
> Did you debug the job using one of the --all_in_one varieties, or some
> other
> minimal wrapper?
>
>
> I am using the same CC from the Sample (DuccCasCC)
> > I am experiencing a problem where my DUCC job goes through
> > WaitingForDriver, WaitingForResources, then Initializing, then doesn't
> > change state. After several minutes I just cancel the job.
> >
>
> Did you look in the JP logfile?
>
> Eddie
>
>
>
Re: DUCC not leaving Initializing State
Posted by Neal R Lewis <nr...@us.ibm.com>.
So, I found one GLARING problem that I missed completely: When testing
my CR and CM setup, I looped through the CR's getNext(), adding to the
JCas that I just created. But the CR's getNext() didn't loop internally,
it only added IDs one at a time. So, instead of the CR creating a CAS with
N Workitems and halting, it was create N CASes with 1 workitem. I've
fixed that.
So I've made an Aggregate AE of my CM and a simple AE, and ran them with
my CR in a uimaFit SimplePipeline. The outputs are as I expect for now.
But I do have a question. The Workitems have an option of SendToLast,
which in the RawText Example is set to "true". The DuccCasCC in the
sample pulls WorkItem FSes for output stuff. The CR adds the Workitems,
but the CM doesn't reattach them to the new CASes before returning next().
Should the Workitems stay in the CAS throughout processing (ie., should
the CMs add them to the new CASes) ? OR is SendToLast enough?
Thanks!
Neal
From: Neal R Lewis/Almaden/IBM@IBMUS
To: user@uima.apache.org
Date: 11/14/2013 01:20 PM
Subject: Re: DUCC not leaving Initializing State
The minimal wrapper creates a Cas using uimaFit and loop over the CR,
then to run over the iterator from cm.processAndOutputNewCASes, then
output each cas in the loop like below:
CollectionReader cr = CollectionReaderFactory.createReader
("path.to.my.CasReader");
AnalysisEngine cm = AnalysisEngineFactory.createEngine(
"path.to.my.CasMultiplier");
JCas jcas = JCasFactory.createJCas("
desc.DuccJobFlowControlTS);
while (cr.hasNext()){
cr.getNext(jcas.getCas());
}
CasIterator casIterator =
cm.processAndOutputNewCASes(jcas.getCas());
while (casIterator.hasNext()) {
CAS outCas = casIterator.next();
System.out.println(outCas.getDocumentText());
}
I'll need to debug a full standalone PEAR or Pipeline. But at this point I
was able to at least get the CASes out.
Below is the output of the JP file before the stall and after loading JVM
stuff. It stays like this until shutdown. As far as I can tell the CM
Loads and is waiting.
+ JVM
LIB_PATH:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
+------------------------------------------------------------------
13 Nov 2013 15:53:44,761 INFO DUCC.ThreadPoolTaskExecutor - Initializing
ExecutorService 'pooling_ducc.jd.queue.91_1'
03:53:44.794 - 1:
org.apache.uima.adapter.jms.activemq.JmsInputChannel.setEndpointName:
INFO: top_level_input_queue_service_1 Service Starting - Listening for
Messages
13 Nov 2013 15:53:44,795 INFO DUCC.ThreadPoolTaskExecutor - Initializing
ExecutorService 'pooling_ducc.jd.queue.91_1'
03:53:44.797 - 30:
org.apache.uima.aae.UimaAsThreadFactory$1.UimaAsThreadFactory.run(): INFO:
Controller: ducc.jd.queue.91 Initializing AE instance on Thread Id: 30
03:53:44.799 - 1:
org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.waitForServiceNotification:
INFO: Uima EE Client Blocking - Awaiting Top Level Controller
Initialization Notification
03:53:44.887 - 30:
com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(70):
INFO: Initializing Cas Multiplier Parameters
03:53:44.887 - 30:
com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(71):
INFO: init HOST ADDRESS:
https://thepit.element.almaden.ibm.com/cgi-bin/qm2
03:53:44.887 - 30:
com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(72):
INFO: init LENIENT: false
03:53:44.887 - 30:
com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(73):
INFO: init PARAM_ENCODING: UTF-8
03:53:44.887 - 30:
com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(74):
INFO: init PARAM_LANGUAGE: en
03:53:44.933 - 30:
org.apache.uima.ducc.sampleapps.DuccCasCC.initialize(74): INFO: Outputting
CASes in XmiCas format, zip compressed at level=7
Service:ducc.jd.queue.91 Initialized. Ready To Process Messages From
Queue:ducc.jd.queue.91
03:53:45.97 - 30:
org.apache.uima.aae.controller.PrimitiveAnalysisEngineController_impl.postInitialize:
INFO: ********* Initialized the Controller. ducc.jd.queue.91 Ready To
Process. ********
03:53:45.97 - 1:
org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.doStartListeners:
INFO: Controller: ducc.jd.queue.91 Starting Listener on Endpoint:
queue://ducc.jd.queue.91 Selector: Command=2000 OR Command=2002 Broker:
tcp://greenfairy:61617?wireFormat.maxInactivityDuration=0&closeAsync=false
03:53:45.290 - 1:
org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.doStartListeners:
INFO: Controller: ducc.jd.queue.91 Starting Listener on Endpoint:
queue://ducc.jd.queue.91 Selector: Command=2001 Broker:
tcp://greenfairy:61617?wireFormat.maxInactivityDuration=0&closeAsync=false
13 Nov 2013 15:53:45,299 INFO DUCC.DuccService - boot N/A ...
Component started: managedService
13 Nov 2013 15:53:45,300 INFO DUCC.DuccService - boot N/A Starting
Camel. Use ctrl + c to terminate the JVM.
From: Eddie Epstein <ea...@gmail.com>
To: user@uima.apache.org
Date: 11/14/2013 10:37 AM
Subject: Re: DUCC not leaving Initializing State
On Wed, Nov 13, 2013 at 7:10 PM, Neal R Lewis <nr...@us.ibm.com> wrote:
> I have modeled a CasReader and CasMultiplier based on the RawTextExample
> in the duccbook, and have successfully ran the CR and CM in eclipse with
a
> minimal wrapper.
Did you debug the job using one of the --all_in_one varieties, or some
other
minimal wrapper?
I am using the same CC from the Sample (DuccCasCC)
> I am experiencing a problem where my DUCC job goes through
> WaitingForDriver, WaitingForResources, then Initializing, then doesn't
> change state. After several minutes I just cancel the job.
>
Did you look in the JP logfile?
Eddie
Re: DUCC not leaving Initializing State
Posted by Neal R Lewis <nr...@us.ibm.com>.
The minimal wrapper creates a Cas using uimaFit and loop over the CR,
then to run over the iterator from cm.processAndOutputNewCASes, then
output each cas in the loop like below:
CollectionReader cr = CollectionReaderFactory.createReader
("path.to.my.CasReader");
AnalysisEngine cm = AnalysisEngineFactory.createEngine(
"path.to.my.CasMultiplier");
JCas jcas = JCasFactory.createJCas("
desc.DuccJobFlowControlTS);
while (cr.hasNext()){
cr.getNext(jcas.getCas());
}
CasIterator casIterator =
cm.processAndOutputNewCASes(jcas.getCas());
while (casIterator.hasNext()) {
CAS outCas = casIterator.next();
System.out.println(outCas.getDocumentText());
}
I'll need to debug a full standalone PEAR or Pipeline. But at this point I
was able to at least get the CASes out.
Below is the output of the JP file before the stall and after loading JVM
stuff. It stays like this until shutdown. As far as I can tell the CM
Loads and is waiting.
+ JVM
LIB_PATH:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
+------------------------------------------------------------------
13 Nov 2013 15:53:44,761 INFO DUCC.ThreadPoolTaskExecutor - Initializing
ExecutorService 'pooling_ducc.jd.queue.91_1'
03:53:44.794 - 1:
org.apache.uima.adapter.jms.activemq.JmsInputChannel.setEndpointName:
INFO: top_level_input_queue_service_1 Service Starting - Listening for
Messages
13 Nov 2013 15:53:44,795 INFO DUCC.ThreadPoolTaskExecutor - Initializing
ExecutorService 'pooling_ducc.jd.queue.91_1'
03:53:44.797 - 30:
org.apache.uima.aae.UimaAsThreadFactory$1.UimaAsThreadFactory.run(): INFO:
Controller: ducc.jd.queue.91 Initializing AE instance on Thread Id: 30
03:53:44.799 - 1:
org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.waitForServiceNotification:
INFO: Uima EE Client Blocking - Awaiting Top Level Controller
Initialization Notification
03:53:44.887 - 30:
com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(70):
INFO: Initializing Cas Multiplier Parameters
03:53:44.887 - 30:
com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(71):
INFO: init HOST ADDRESS:
https://thepit.element.almaden.ibm.com/cgi-bin/qm2
03:53:44.887 - 30:
com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(72):
INFO: init LENIENT: false
03:53:44.887 - 30:
com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(73):
INFO: init PARAM_ENCODING: UTF-8
03:53:44.887 - 30:
com.ibm.almaden.disco.quartermaster.SimpleQuarterMasterCasMultiplier.logParameters(74):
INFO: init PARAM_LANGUAGE: en
03:53:44.933 - 30:
org.apache.uima.ducc.sampleapps.DuccCasCC.initialize(74): INFO: Outputting
CASes in XmiCas format, zip compressed at level=7
Service:ducc.jd.queue.91 Initialized. Ready To Process Messages From
Queue:ducc.jd.queue.91
03:53:45.97 - 30:
org.apache.uima.aae.controller.PrimitiveAnalysisEngineController_impl.postInitialize:
INFO: ********* Initialized the Controller. ducc.jd.queue.91 Ready To
Process. ********
03:53:45.97 - 1:
org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.doStartListeners:
INFO: Controller: ducc.jd.queue.91 Starting Listener on Endpoint:
queue://ducc.jd.queue.91 Selector: Command=2000 OR Command=2002 Broker:
tcp://greenfairy:61617?wireFormat.maxInactivityDuration=0&closeAsync=false
03:53:45.290 - 1:
org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.doStartListeners:
INFO: Controller: ducc.jd.queue.91 Starting Listener on Endpoint:
queue://ducc.jd.queue.91 Selector: Command=2001 Broker:
tcp://greenfairy:61617?wireFormat.maxInactivityDuration=0&closeAsync=false
13 Nov 2013 15:53:45,299 INFO DUCC.DuccService - boot N/A ...
Component started: managedService
13 Nov 2013 15:53:45,300 INFO DUCC.DuccService - boot N/A Starting
Camel. Use ctrl + c to terminate the JVM.
From: Eddie Epstein <ea...@gmail.com>
To: user@uima.apache.org
Date: 11/14/2013 10:37 AM
Subject: Re: DUCC not leaving Initializing State
On Wed, Nov 13, 2013 at 7:10 PM, Neal R Lewis <nr...@us.ibm.com> wrote:
> I have modeled a CasReader and CasMultiplier based on the RawTextExample
> in the duccbook, and have successfully ran the CR and CM in eclipse with
a
> minimal wrapper.
Did you debug the job using one of the --all_in_one varieties, or some
other
minimal wrapper?
I am using the same CC from the Sample (DuccCasCC)
> I am experiencing a problem where my DUCC job goes through
> WaitingForDriver, WaitingForResources, then Initializing, then doesn't
> change state. After several minutes I just cancel the job.
>
Did you look in the JP logfile?
Eddie
Re: DUCC not leaving Initializing State
Posted by Eddie Epstein <ea...@gmail.com>.
On Wed, Nov 13, 2013 at 7:10 PM, Neal R Lewis <nr...@us.ibm.com> wrote:
> I have modeled a CasReader and CasMultiplier based on the RawTextExample
> in the duccbook, and have successfully ran the CR and CM in eclipse with a
> minimal wrapper.
Did you debug the job using one of the --all_in_one varieties, or some
other
minimal wrapper?
I am using the same CC from the Sample (DuccCasCC)
> I am experiencing a problem where my DUCC job goes through
> WaitingForDriver, WaitingForResources, then Initializing, then doesn't
> change state. After several minutes I just cancel the job.
>
Did you look in the JP logfile?
Eddie