[jira] [Updated] (OFBIZ-3855) Service engine JobManager polling method has a transaction time-out when the number of jobs is large. Need a way to limit the number of jobs processed during each polling interval.

["Adrian Crum (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/OFBIZ-3855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Adrian Crum updated OFBIZ-3855:
-------------------------------

    Summary: Service engine JobManager polling method has a transaction time-out when the number of jobs is large. Need a way to limit the number of jobs processed during each polling interval.  (was: Service engine JobManager polling method has a transaction time when the number of jobs is large 10k+;  Need a way to configuration transaction time.rge Job series;  There is no way to change transaction time.)
    
> Service engine JobManager polling method has a transaction time-out when the number of jobs is large. Need a way to limit the number of jobs processed during each polling interval.
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: OFBIZ-3855
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-3855
>             Project: OFBiz
>          Issue Type: Improvement
>          Components: ALL COMPONENTS
>    Affects Versions: SVN trunk
>            Reporter: Brett G. Palmer
>            Assignee: Adrian Crum
>             Fix For: SVN trunk
>
>
> Here is an example of when we see the transaction timeout for the polling operation in the JobManager.
> We are running into a problem with the service engine when trying to run an archiving batch service.  Here is how the service runs.
> 1. We run a service that collects a bunch of records for data warehousing.  
> 2. The data warehouse service calls an async service call to process each individual record.
> 3. The service engine them starts working on each individual record.
> Here is the problem:
> There are over 10K records that are submitted as async service calls in step 2 above.  I'm seeing a transaction time out when the service engine tries to process all of these records as it calls an "UPDATE JOB_SANDBOX for job_id=XXX".  I believe it is trying to update all 10k of these jobs individually in a single transaction and times out.  The jobs are then left in a pending status and never changed but the service engine keeps timing out.
> I was able to fix this problem by adding a new poll-transaction-time attribute to the "thread-pool" definition in the serviceengine.xml.  I then looked up this value in the JobManager.poll method and set the transaction timeout at the beginning of the transaction.
> I'll submit a patch for this when I have made it for the latest revision in the trunk.
> Brett

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira