You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@oozie.apache.org by Christian Schneider <cs...@gmail.com> on 2013/06/17 12:48:18 UTC
Oozie org.apache.openjpa.lib.jdbc.ReportingSQLException ~once a week
Hi, our Oozie instance (managed by Cloudera Manager) get a
org.apache.openjpa.lib.jdbc.ReportingSQLException almost once a week.
When this happens, we see this Stacktraces in the log files:
2013-06-17 09:01:20,047 ERROR org.apache.oozie.command.wf.JobsXCommand:
USER[hue] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] Error,
2013-06-17 09:01:53,330 WARN org.apache.oozie.service.JPAService: USER[-]
GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] JPAExecutor
[WorkflowsJobGetJPAExecutor] ended with an active transaction, rolling back
2013-06-17 09:01:53,331 ERROR org.apache.oozie.command.wf.JobsXCommand:
USER[hue] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] XException,
org.apache.oozie.command.CommandException: E0603: SQL error in operation,
Cannot issue commit in a nested connection when there is a pending
operation in the parent connection. {prepstmnt 320277519 SELECT t0.id,
t0.app_name, t0.status, t0.run, t0.user_name, t0.group_name,
t0.created_time, t0.start_time, t0.last_modified_time, t0.end_time,
t0.external_id FROM WF_JOBS t0 WHERE t0.bean_type = ? ORDER BY
t0.created_time DESC [params=?]} [code=30000, state=X0Y66]
at org.apache.oozie.command.wf.JobsXCommand.execute(
JobsXCommand.java:72)
at org.apache.oozie.command.wf.JobsXCommand.execute(
JobsXCommand.java:32)
at org.apache.oozie.command.XCommand.call(XCommand.java:277)
at org.apache.oozie.DagEngine.getJobs(DagEngine.java:439)
at org.apache.oozie.servlet.V1JobsServlet.getWorkflowJobs(
V1JobsServlet.java:314)
at org.apache.oozie.servlet.V1JobsServlet.getJobs(
V1JobsServlet.java:141)
at org.apache.oozie.servlet.BaseJobsServlet.doGet(
BaseJobsServlet.java:121)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
at org.apache.oozie.servlet.JsonRestServlet.service(
JsonRestServlet.java:286)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(
ApplicationFilterChain.java:206)
at org.apache.oozie.servlet.AuthFilter$2.doFilter(
AuthFilter.java:126)
at org.apache.hadoop.security.authentication.server.
AuthenticationFilter.doFilter(AuthenticationFilter.java:384)
at org.apache.oozie.servlet.AuthFilter.doFilter(AuthFilter.java:131)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(
ApplicationFilterChain.java:206)
at org.apache.oozie.servlet.HostnameFilter.doFilter(
HostnameFilter.java:84)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(
ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(
StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(
StandardContextValve.java:191)
at org.apache.catalina.core.StandardHostValve.invoke(
StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(
ErrorReportValve.java:102)
at org.apache.catalina.core.StandardEngineValve.invoke(
StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(
CoyoteAdapter.java:293)
at org.apache.coyote.http11.Http11Processor.process(
Http11Processor.java:859)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.
process(Http11Protocol.java:602)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(
JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:662)
Caused by: <openjpa-2.1.0-r422266:1071316 fatal general error>
org.apache.openjpa.persistence.PersistenceException: Cannot issue commit in
a nested connection when there is a pending operation in the parent
connection. {prepstmnt 320277519 SELECT t0.id, t0.app_name, t0.status,
t0.run, t0.user_name, t0.group_name, t0.created_time, t0.start_time,
t0.last_modified_time, t0.end_time, t0.external_id FROM WF_JOBS t0 WHERE
t0.bean_type = ? ORDER BY t0.created_time DESC [params=?]} [code=30000,
state=X0Y66]
at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(
DBDictionary.java:4869)
at org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(
DBDictionary.java:4829)
at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(
SQLExceptions.java:136)
at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(
SQLExceptions.java:110)
at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(
SQLExceptions.java:62)
at org.apache.openjpa.jdbc.kernel.PreparedSQLStoreQuery$
PreparedSQLExecutor.executeQuery(PreparedSQLStoreQuery.java:139)
at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:1005)
at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:863)
at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:794)
at org.apache.openjpa.kernel.DelegatingQuery.execute(
DelegatingQuery.java:542)
at org.apache.openjpa.persistence.QueryImpl.execute(
QueryImpl.java:305)
at org.apache.openjpa.persistence.QueryImpl.
getResultList(QueryImpl.java:319)
at org.apache.oozie.executor.jpa.WorkflowsJobGetJPAExecutor.execute(
WorkflowsJobGetJPAExecutor.java:247)
at org.apache.oozie.executor.jpa.WorkflowsJobGetJPAExecutor.execute(
WorkflowsJobGetJPAExecutor.java:40)
at org.apache.oozie.service.JPAService.execute(JPAService.java:211)
at org.apache.oozie.command.wf.JobsXCommand.execute(
JobsXCommand.java:61)
... 29 more
Caused by: org.apache.openjpa.lib.jdbc.ReportingSQLException: Cannot issue
commit in a nested connection when there is a pending operation in the
parent connection. {prepstmnt 320277519 SELECT t0.id, t0.app_name,
t0.status, t0.run, t0.user_name, t0.group_name, t0.created_time,
t0.start_time, t0.last_modified_time, t0.end_time, t0.external_id FROM
WF_JOBS t0 WHERE t0.bean_type = ? ORDER BY t0.created_time DESC [params=?]}
[code=30000, state=X0Y66]
at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(
LoggingConnectionDecorator.java:281)
at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(
LoggingConnectionDecorator.java:265)
at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.
access$700(LoggingConnectionDecorator.java:72)
at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator$
LoggingConnection$LoggingPreparedStatement.executeQuery(
LoggingConnectionDecorator.java:1183)
at org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.
executeQuery(DelegatingPreparedStatement.java:284)
at org.apache.openjpa.jdbc.kernel.JDBCStoreManager$
CancelPreparedStatement.executeQuery(JDBCStoreManager.java:1785)
at org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.
executeQuery(DelegatingPreparedStatement.java:274)
at org.apache.openjpa.jdbc.kernel.PreparedSQLStoreQuery$
PreparedSQLExecutor.executeQuery(PreparedSQLStoreQuery.java:118)
... 39 more
2013-06-17 09:01:54,275 WARN org.apache.oozie.servlet.V1JobsServlet:
USER[hue] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] URL[GET
http://ip-10-0-1-149.eu-west-1.compute.internal:11000/oozie/v1/jobs?len=100&doAs=hue&filter=&user.name=hue&timezone=America%2FLos_Angeles&jobtype=wf]
error[E0603], E0603: SQL error in operation, Cannot issue commit in a
nested connection when there is a pending operation in the parent
connection. {prepstmnt 320277519 SELECT t0.id, t0.app_name, t0.status,
t0.run, t0.user_name, t0.group_name, t0.created_time, t0.start_time,
t0.last_modified_time, t0.end_time, t0.external_id FROM WF_JOBS t0 WHERE
t0.bean_type = ? ORDER BY t0.created_time DESC [params=?]} [code=30000,
state=X0Y66]
org.apache.oozie.servlet.XServletException: E0603: SQL error in operation,
Cannot issue commit in a nested connection when there is a pending
operation in the parent connection. {prepstmnt 320277519 SELECTt0.id,
t0.app_name, t0.status, t0.run, t0.user_name, t0.group_name,
t0.created_time, t0.start_time, t0.last_modified_time, t0.end_time,
t0.external_id FROM WF_JOBS t0 WHERE t0.bean_type = ? ORDER BY
t0.created_time DESC [params=?]} [code=30000, state=X0Y66]
at org.apache.oozie.servlet.V1JobsServlet.getWorkflowJobs(
V1JobsServlet.java:323)
at org.apache.oozie.servlet.V1JobsServlet.getJobs(
V1JobsServlet.java:141)
at org.apache.oozie.servlet.BaseJobsServlet.doGet(
BaseJobsServlet.java:121)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
at org.apache.oozie.servlet.JsonRestServlet.service(
JsonRestServlet.java:286)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(
ApplicationFilterChain.java:206)
at org.apache.oozie.servlet.AuthFilter$2.doFilter(
AuthFilter.java:126)
at org.apache.hadoop.security.authentication.server.
AuthenticationFilter.doFilter(AuthenticationFilter.java:384)
at org.apache.oozie.servlet.AuthFilter.doFilter(AuthFilter.java:131)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(
ApplicationFilterChain.java:206)
at org.apache.oozie.servlet.HostnameFilter.doFilter(
HostnameFilter.java:84)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(
ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(
StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(
StandardContextValve.java:191)
at org.apache.catalina.core.StandardHostValve.invoke(
StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(
ErrorReportValve.java:102)
at org.apache.catalina.core.StandardEngineValve.invoke(
StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(
CoyoteAdapter.java:293)
at org.apache.coyote.http11.Http11Processor.process(
Http11Processor.java:859)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.
process(Http11Protocol.java:602)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(
JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.oozie.DagEngineException: E0603: SQL error in
operation, Cannot issue commit in a nested connection when there is a
pending operation in the parent connection. {prepstmnt 320277519 SELECT
t0.id, t0.app_name, t0.status, t0.run, t0.user_name, t0.group_name,
t0.created_time, t0.start_time, t0.last_modified_time, t0.end_time,
t0.external_id FROM WF_JOBS t0 WHERE t0.bean_type = ? ORDER BY
t0.created_time DESC [params=?]} [code=30000, state=X0Y66]
at org.apache.oozie.DagEngine.getJobs(DagEngine.java:442)
at org.apache.oozie.servlet.V1JobsServlet.getWorkflowJobs(
V1JobsServlet.java:314)
... 25 more
Caused by: org.apache.oozie.command.CommandException: E0603: SQL error in
operation, Cannot issue commit in a nested connection when there is a
pending operation in the parent connection. {prepstmnt 320277519 SELECT
t0.id, t0.app_name, t0.status, t0.run, t0.user_name, t0.group_name,
t0.created_time, t0.start_time, t0.last_modified_time, t0.end_time,
t0.external_id FROM WF_JOBS t0 WHERE t0.bean_type = ? ORDER BY
t0.created_time DESC [params=?]} [code=30000, state=X0Y66]
at org.apache.oozie.command.wf.JobsXCommand.execute(
JobsXCommand.java:72)
at org.apache.oozie.command.wf.JobsXCommand.execute(
JobsXCommand.java:32)
at org.apache.oozie.command.XCommand.call(XCommand.java:277)
at org.apache.oozie.DagEngine.getJobs(DagEngine.java:439)
... 26 more
Caused by: <openjpa-2.1.0-r422266:1071316 fatal general error>
org.apache.openjpa.persistence.PersistenceException: Cannot issue commit in
a nested connection when there is a pending operation in the parent
connection. {prepstmnt 320277519 SELECT t0.id, t0.app_name, t0.status,
t0.run, t0.user_name, t0.group_name, t0.created_time, t0.start_time,
t0.last_modified_time, t0.end_time, t0.external_id FROM WF_JOBS t0 WHERE
t0.bean_type = ? ORDER BY t0.created_time DESC [params=?]} [code=30000,
state=X0Y66]
at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(
DBDictionary.java:4869)
at org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(
DBDictionary.java:4829)
at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(
SQLExceptions.java:136)
at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(
SQLExceptions.java:110)
at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(
SQLExceptions.java:62)
at org.apache.openjpa.jdbc.kernel.PreparedSQLStoreQuery$
PreparedSQLExecutor.executeQuery(PreparedSQLStoreQuery.java:139)
at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:1005)
at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:863)
at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:794)
at org.apache.openjpa.kernel.DelegatingQuery.execute(
DelegatingQuery.java:542)
at org.apache.openjpa.persistence.QueryImpl.execute(
QueryImpl.java:305)
at org.apache.openjpa.persistence.QueryImpl.
getResultList(QueryImpl.java:319)
at org.apache.oozie.executor.jpa.WorkflowsJobGetJPAExecutor.execute(
WorkflowsJobGetJPAExecutor.java:247)
at org.apache.oozie.executor.jpa.WorkflowsJobGetJPAExecutor.execute(
WorkflowsJobGetJPAExecutor.java:40)
at org.apache.oozie.service.JPAService.execute(JPAService.java:211)
at org.apache.oozie.command.wf.JobsXCommand.execute(
JobsXCommand.java:61)
... 29 more
Caused by: org.apache.openjpa.lib.jdbc.ReportingSQLException: Cannot issue
commit in a nested connection when there is a pending operation in the
parent connection. {prepstmnt 320277519 SELECT t0.id, t0.app_name,
t0.status, t0.run, t0.user_name, t0.group_name, t0.created_time,
t0.start_time, t0.last_modified_time, t0.end_time, t0.external_id FROM
WF_JOBS t0 WHERE t0.bean_type = ? ORDER BY t0.created_time DESC [params=?]}
[code=30000, state=X0Y66]
at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(
LoggingConnectionDecorator.java:281)
at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(
LoggingConnectionDecorator.java:265)
at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.
access$700(LoggingConnectionDecorator.java:72)
at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator$
LoggingConnection$LoggingPreparedStatement.executeQuery(
LoggingConnectionDecorator.java:1183)
at org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.
executeQuery(DelegatingPreparedStatement.java:284)
at org.apache.openjpa.jdbc.kernel.JDBCStoreManager$
CancelPreparedStatement.executeQuery(JDBCStoreManager.java:1785)
at org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.
executeQuery(DelegatingPreparedStatement.java:274)
at org.apache.openjpa.jdbc.kernel.PreparedSQLStoreQuery$
PreparedSQLExecutor.executeQuery(PreparedSQLStoreQuery.java:118)
... 39 more
After resting the service, the systems will run fine.
I have no idea where to debug or how to solve this.
Best Regards,
Christian.
Re: Oozie org.apache.openjpa.lib.jdbc.ReportingSQLException ~once a
week
Posted by Mona Chitnis <ch...@yahoo-inc.com>.
Also try increasing the Oozie configuration value of
oozie.service.JPAService.pool.max.active.conn to something higher, since
this seems to be hitting the issue of concurrent database operations not
able to execute. Depending on size of staging/production environment, this
value can go from 220 to 370 ish.
--
Mona
On 6/28/13 1:28 AM, "Harsh J" <ha...@cloudera.com> wrote:
>Hi Christian,
>
>What version of Oozie are you using here? And what DB is it running
>against?
>
>On Mon, Jun 17, 2013 at 4:18 PM, Christian Schneider
><cs...@gmail.com> wrote:
>> Hi, our Oozie instance (managed by Cloudera Manager) get a
>> org.apache.openjpa.lib.jdbc.ReportingSQLException almost once a week.
>>
>> When this happens, we see this Stacktraces in the log files:
>>
>> 2013-06-17 09:01:20,047 ERROR org.apache.oozie.command.wf.JobsXCommand:
>> USER[hue] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] Error,
>> 2013-06-17 09:01:53,330 WARN org.apache.oozie.service.JPAService:
>>USER[-]
>> GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] JPAExecutor
>> [WorkflowsJobGetJPAExecutor] ended with an active transaction, rolling
>>back
>> 2013-06-17 09:01:53,331 ERROR org.apache.oozie.command.wf.JobsXCommand:
>> USER[hue] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] XException,
>> org.apache.oozie.command.CommandException: E0603: SQL error in
>>operation,
>> Cannot issue commit in a nested connection when there is a pending
>> operation in the parent connection. {prepstmnt 320277519 SELECT t0.id,
>> t0.app_name, t0.status, t0.run, t0.user_name, t0.group_name,
>> t0.created_time, t0.start_time, t0.last_modified_time, t0.end_time,
>> t0.external_id FROM WF_JOBS t0 WHERE t0.bean_type = ? ORDER BY
>> t0.created_time DESC [params=?]} [code=30000, state=X0Y66]
>> at org.apache.oozie.command.wf.JobsXCommand.execute(
>> JobsXCommand.java:72)
>> at org.apache.oozie.command.wf.JobsXCommand.execute(
>> JobsXCommand.java:32)
>> at org.apache.oozie.command.XCommand.call(XCommand.java:277)
>> at org.apache.oozie.DagEngine.getJobs(DagEngine.java:439)
>> at org.apache.oozie.servlet.V1JobsServlet.getWorkflowJobs(
>> V1JobsServlet.java:314)
>> at org.apache.oozie.servlet.V1JobsServlet.getJobs(
>> V1JobsServlet.java:141)
>> at org.apache.oozie.servlet.BaseJobsServlet.doGet(
>> BaseJobsServlet.java:121)
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
>> at org.apache.oozie.servlet.JsonRestServlet.service(
>> JsonRestServlet.java:286)
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
>> at
>>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
>> ApplicationFilterChain.java:290)
>> at org.apache.catalina.core.ApplicationFilterChain.doFilter(
>> ApplicationFilterChain.java:206)
>> at org.apache.oozie.servlet.AuthFilter$2.doFilter(
>> AuthFilter.java:126)
>> at org.apache.hadoop.security.authentication.server.
>> AuthenticationFilter.doFilter(AuthenticationFilter.java:384)
>> at
>>org.apache.oozie.servlet.AuthFilter.doFilter(AuthFilter.java:131)
>> at
>>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
>> ApplicationFilterChain.java:235)
>> at org.apache.catalina.core.ApplicationFilterChain.doFilter(
>> ApplicationFilterChain.java:206)
>> at org.apache.oozie.servlet.HostnameFilter.doFilter(
>> HostnameFilter.java:84)
>> at
>>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
>> ApplicationFilterChain.java:235)
>> at org.apache.catalina.core.ApplicationFilterChain.doFilter(
>> ApplicationFilterChain.java:206)
>> at org.apache.catalina.core.StandardWrapperValve.invoke(
>> StandardWrapperValve.java:233)
>> at org.apache.catalina.core.StandardContextValve.invoke(
>> StandardContextValve.java:191)
>> at org.apache.catalina.core.StandardHostValve.invoke(
>> StandardHostValve.java:127)
>> at org.apache.catalina.valves.ErrorReportValve.invoke(
>> ErrorReportValve.java:102)
>> at org.apache.catalina.core.StandardEngineValve.invoke(
>> StandardEngineValve.java:109)
>> at org.apache.catalina.connector.CoyoteAdapter.service(
>> CoyoteAdapter.java:293)
>> at org.apache.coyote.http11.Http11Processor.process(
>> Http11Processor.java:859)
>> at
>>org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.
>> process(Http11Protocol.java:602)
>> at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(
>> JIoEndpoint.java:489)
>> at java.lang.Thread.run(Thread.java:662)
>> Caused by: <openjpa-2.1.0-r422266:1071316 fatal general error>
>> org.apache.openjpa.persistence.PersistenceException: Cannot issue
>>commit in
>> a nested connection when there is a pending operation in the parent
>> connection. {prepstmnt 320277519 SELECT t0.id, t0.app_name, t0.status,
>> t0.run, t0.user_name, t0.group_name, t0.created_time, t0.start_time,
>> t0.last_modified_time, t0.end_time, t0.external_id FROM WF_JOBS t0 WHERE
>> t0.bean_type = ? ORDER BY t0.created_time DESC [params=?]} [code=30000,
>> state=X0Y66]
>> at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(
>> DBDictionary.java:4869)
>> at org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(
>> DBDictionary.java:4829)
>> at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(
>> SQLExceptions.java:136)
>> at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(
>> SQLExceptions.java:110)
>> at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(
>> SQLExceptions.java:62)
>> at org.apache.openjpa.jdbc.kernel.PreparedSQLStoreQuery$
>> PreparedSQLExecutor.executeQuery(PreparedSQLStoreQuery.java:139)
>> at
>>org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:1005)
>> at
>>org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:863)
>> at
>>org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:794)
>> at org.apache.openjpa.kernel.DelegatingQuery.execute(
>> DelegatingQuery.java:542)
>> at org.apache.openjpa.persistence.QueryImpl.execute(
>> QueryImpl.java:305)
>> at org.apache.openjpa.persistence.QueryImpl.
>> getResultList(QueryImpl.java:319)
>> at
>>org.apache.oozie.executor.jpa.WorkflowsJobGetJPAExecutor.execute(
>> WorkflowsJobGetJPAExecutor.java:247)
>> at
>>org.apache.oozie.executor.jpa.WorkflowsJobGetJPAExecutor.execute(
>> WorkflowsJobGetJPAExecutor.java:40)
>> at
>>org.apache.oozie.service.JPAService.execute(JPAService.java:211)
>> at org.apache.oozie.command.wf.JobsXCommand.execute(
>> JobsXCommand.java:61)
>> ... 29 more
>> Caused by: org.apache.openjpa.lib.jdbc.ReportingSQLException: Cannot
>>issue
>> commit in a nested connection when there is a pending operation in the
>> parent connection. {prepstmnt 320277519 SELECT t0.id, t0.app_name,
>> t0.status, t0.run, t0.user_name, t0.group_name, t0.created_time,
>> t0.start_time, t0.last_modified_time, t0.end_time, t0.external_id FROM
>> WF_JOBS t0 WHERE t0.bean_type = ? ORDER BY t0.created_time DESC
>>[params=?]}
>> [code=30000, state=X0Y66]
>> at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(
>> LoggingConnectionDecorator.java:281)
>> at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(
>> LoggingConnectionDecorator.java:265)
>> at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.
>> access$700(LoggingConnectionDecorator.java:72)
>> at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator$
>> LoggingConnection$LoggingPreparedStatement.executeQuery(
>> LoggingConnectionDecorator.java:1183)
>> at org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.
>> executeQuery(DelegatingPreparedStatement.java:284)
>> at org.apache.openjpa.jdbc.kernel.JDBCStoreManager$
>> CancelPreparedStatement.executeQuery(JDBCStoreManager.java:1785)
>> at org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.
>> executeQuery(DelegatingPreparedStatement.java:274)
>> at org.apache.openjpa.jdbc.kernel.PreparedSQLStoreQuery$
>> PreparedSQLExecutor.executeQuery(PreparedSQLStoreQuery.java:118)
>> ... 39 more
>> 2013-06-17 09:01:54,275 WARN org.apache.oozie.servlet.V1JobsServlet:
>> USER[hue] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] URL[GET
>>
>>http://ip-10-0-1-149.eu-west-1.compute.internal:11000/oozie/v1/jobs?len=1
>>00&doAs=hue&filter=&user.name=hue&timezone=America%2FLos_Angeles&jobtype=
>>wf]
>> error[E0603], E0603: SQL error in operation, Cannot issue commit in a
>> nested connection when there is a pending operation in the parent
>> connection. {prepstmnt 320277519 SELECT t0.id, t0.app_name, t0.status,
>> t0.run, t0.user_name, t0.group_name, t0.created_time, t0.start_time,
>> t0.last_modified_time, t0.end_time, t0.external_id FROM WF_JOBS t0 WHERE
>> t0.bean_type = ? ORDER BY t0.created_time DESC [params=?]} [code=30000,
>> state=X0Y66]
>> org.apache.oozie.servlet.XServletException: E0603: SQL error in
>>operation,
>> Cannot issue commit in a nested connection when there is a pending
>> operation in the parent connection. {prepstmnt 320277519 SELECTt0.id,
>> t0.app_name, t0.status, t0.run, t0.user_name, t0.group_name,
>> t0.created_time, t0.start_time, t0.last_modified_time, t0.end_time,
>> t0.external_id FROM WF_JOBS t0 WHERE t0.bean_type = ? ORDER BY
>> t0.created_time DESC [params=?]} [code=30000, state=X0Y66]
>> at org.apache.oozie.servlet.V1JobsServlet.getWorkflowJobs(
>> V1JobsServlet.java:323)
>> at org.apache.oozie.servlet.V1JobsServlet.getJobs(
>> V1JobsServlet.java:141)
>> at org.apache.oozie.servlet.BaseJobsServlet.doGet(
>> BaseJobsServlet.java:121)
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
>> at org.apache.oozie.servlet.JsonRestServlet.service(
>> JsonRestServlet.java:286)
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
>> at
>>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
>> ApplicationFilterChain.java:290)
>> at org.apache.catalina.core.ApplicationFilterChain.doFilter(
>> ApplicationFilterChain.java:206)
>> at org.apache.oozie.servlet.AuthFilter$2.doFilter(
>> AuthFilter.java:126)
>> at org.apache.hadoop.security.authentication.server.
>> AuthenticationFilter.doFilter(AuthenticationFilter.java:384)
>> at
>>org.apache.oozie.servlet.AuthFilter.doFilter(AuthFilter.java:131)
>> at
>>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
>> ApplicationFilterChain.java:235)
>> at org.apache.catalina.core.ApplicationFilterChain.doFilter(
>> ApplicationFilterChain.java:206)
>> at org.apache.oozie.servlet.HostnameFilter.doFilter(
>> HostnameFilter.java:84)
>> at
>>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
>> ApplicationFilterChain.java:235)
>> at org.apache.catalina.core.ApplicationFilterChain.doFilter(
>> ApplicationFilterChain.java:206)
>> at org.apache.catalina.core.StandardWrapperValve.invoke(
>> StandardWrapperValve.java:233)
>> at org.apache.catalina.core.StandardContextValve.invoke(
>> StandardContextValve.java:191)
>> at org.apache.catalina.core.StandardHostValve.invoke(
>> StandardHostValve.java:127)
>> at org.apache.catalina.valves.ErrorReportValve.invoke(
>> ErrorReportValve.java:102)
>> at org.apache.catalina.core.StandardEngineValve.invoke(
>> StandardEngineValve.java:109)
>> at org.apache.catalina.connector.CoyoteAdapter.service(
>> CoyoteAdapter.java:293)
>> at org.apache.coyote.http11.Http11Processor.process(
>> Http11Processor.java:859)
>> at
>>org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.
>> process(Http11Protocol.java:602)
>> at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(
>> JIoEndpoint.java:489)
>> at java.lang.Thread.run(Thread.java:662)
>> Caused by: org.apache.oozie.DagEngineException: E0603: SQL error in
>> operation, Cannot issue commit in a nested connection when there is a
>> pending operation in the parent connection. {prepstmnt 320277519 SELECT
>> t0.id, t0.app_name, t0.status, t0.run, t0.user_name, t0.group_name,
>> t0.created_time, t0.start_time, t0.last_modified_time, t0.end_time,
>> t0.external_id FROM WF_JOBS t0 WHERE t0.bean_type = ? ORDER BY
>> t0.created_time DESC [params=?]} [code=30000, state=X0Y66]
>> at org.apache.oozie.DagEngine.getJobs(DagEngine.java:442)
>> at org.apache.oozie.servlet.V1JobsServlet.getWorkflowJobs(
>> V1JobsServlet.java:314)
>> ... 25 more
>> Caused by: org.apache.oozie.command.CommandException: E0603: SQL error
>>in
>> operation, Cannot issue commit in a nested connection when there is a
>> pending operation in the parent connection. {prepstmnt 320277519 SELECT
>> t0.id, t0.app_name, t0.status, t0.run, t0.user_name, t0.group_name,
>> t0.created_time, t0.start_time, t0.last_modified_time, t0.end_time,
>> t0.external_id FROM WF_JOBS t0 WHERE t0.bean_type = ? ORDER BY
>> t0.created_time DESC [params=?]} [code=30000, state=X0Y66]
>> at org.apache.oozie.command.wf.JobsXCommand.execute(
>> JobsXCommand.java:72)
>> at org.apache.oozie.command.wf.JobsXCommand.execute(
>> JobsXCommand.java:32)
>> at org.apache.oozie.command.XCommand.call(XCommand.java:277)
>> at org.apache.oozie.DagEngine.getJobs(DagEngine.java:439)
>> ... 26 more
>> Caused by: <openjpa-2.1.0-r422266:1071316 fatal general error>
>> org.apache.openjpa.persistence.PersistenceException: Cannot issue
>>commit in
>> a nested connection when there is a pending operation in the parent
>> connection. {prepstmnt 320277519 SELECT t0.id, t0.app_name, t0.status,
>> t0.run, t0.user_name, t0.group_name, t0.created_time, t0.start_time,
>> t0.last_modified_time, t0.end_time, t0.external_id FROM WF_JOBS t0 WHERE
>> t0.bean_type = ? ORDER BY t0.created_time DESC [params=?]} [code=30000,
>> state=X0Y66]
>> at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(
>> DBDictionary.java:4869)
>> at org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(
>> DBDictionary.java:4829)
>> at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(
>> SQLExceptions.java:136)
>> at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(
>> SQLExceptions.java:110)
>> at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(
>> SQLExceptions.java:62)
>> at org.apache.openjpa.jdbc.kernel.PreparedSQLStoreQuery$
>> PreparedSQLExecutor.executeQuery(PreparedSQLStoreQuery.java:139)
>> at
>>org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:1005)
>> at
>>org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:863)
>> at
>>org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:794)
>> at org.apache.openjpa.kernel.DelegatingQuery.execute(
>> DelegatingQuery.java:542)
>> at org.apache.openjpa.persistence.QueryImpl.execute(
>> QueryImpl.java:305)
>> at org.apache.openjpa.persistence.QueryImpl.
>> getResultList(QueryImpl.java:319)
>> at
>>org.apache.oozie.executor.jpa.WorkflowsJobGetJPAExecutor.execute(
>> WorkflowsJobGetJPAExecutor.java:247)
>> at
>>org.apache.oozie.executor.jpa.WorkflowsJobGetJPAExecutor.execute(
>> WorkflowsJobGetJPAExecutor.java:40)
>> at
>>org.apache.oozie.service.JPAService.execute(JPAService.java:211)
>> at org.apache.oozie.command.wf.JobsXCommand.execute(
>> JobsXCommand.java:61)
>> ... 29 more
>> Caused by: org.apache.openjpa.lib.jdbc.ReportingSQLException: Cannot
>>issue
>> commit in a nested connection when there is a pending operation in the
>> parent connection. {prepstmnt 320277519 SELECT t0.id, t0.app_name,
>> t0.status, t0.run, t0.user_name, t0.group_name, t0.created_time,
>> t0.start_time, t0.last_modified_time, t0.end_time, t0.external_id FROM
>> WF_JOBS t0 WHERE t0.bean_type = ? ORDER BY t0.created_time DESC
>>[params=?]}
>> [code=30000, state=X0Y66]
>> at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(
>> LoggingConnectionDecorator.java:281)
>> at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(
>> LoggingConnectionDecorator.java:265)
>> at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.
>> access$700(LoggingConnectionDecorator.java:72)
>> at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator$
>> LoggingConnection$LoggingPreparedStatement.executeQuery(
>> LoggingConnectionDecorator.java:1183)
>> at org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.
>> executeQuery(DelegatingPreparedStatement.java:284)
>> at org.apache.openjpa.jdbc.kernel.JDBCStoreManager$
>> CancelPreparedStatement.executeQuery(JDBCStoreManager.java:1785)
>> at org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.
>> executeQuery(DelegatingPreparedStatement.java:274)
>> at org.apache.openjpa.jdbc.kernel.PreparedSQLStoreQuery$
>> PreparedSQLExecutor.executeQuery(PreparedSQLStoreQuery.java:118)
>> ... 39 more
>>
>> After resting the service, the systems will run fine.
>>
>> I have no idea where to debug or how to solve this.
>>
>>
>> Best Regards,
>> Christian.
>
>
>
>--
>Harsh J
Re: Oozie org.apache.openjpa.lib.jdbc.ReportingSQLException ~once a week
Posted by Harsh J <ha...@cloudera.com>.
Hi Christian,
What version of Oozie are you using here? And what DB is it running against?
On Mon, Jun 17, 2013 at 4:18 PM, Christian Schneider
<cs...@gmail.com> wrote:
> Hi, our Oozie instance (managed by Cloudera Manager) get a
> org.apache.openjpa.lib.jdbc.ReportingSQLException almost once a week.
>
> When this happens, we see this Stacktraces in the log files:
>
> 2013-06-17 09:01:20,047 ERROR org.apache.oozie.command.wf.JobsXCommand:
> USER[hue] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] Error,
> 2013-06-17 09:01:53,330 WARN org.apache.oozie.service.JPAService: USER[-]
> GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] JPAExecutor
> [WorkflowsJobGetJPAExecutor] ended with an active transaction, rolling back
> 2013-06-17 09:01:53,331 ERROR org.apache.oozie.command.wf.JobsXCommand:
> USER[hue] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] XException,
> org.apache.oozie.command.CommandException: E0603: SQL error in operation,
> Cannot issue commit in a nested connection when there is a pending
> operation in the parent connection. {prepstmnt 320277519 SELECT t0.id,
> t0.app_name, t0.status, t0.run, t0.user_name, t0.group_name,
> t0.created_time, t0.start_time, t0.last_modified_time, t0.end_time,
> t0.external_id FROM WF_JOBS t0 WHERE t0.bean_type = ? ORDER BY
> t0.created_time DESC [params=?]} [code=30000, state=X0Y66]
> at org.apache.oozie.command.wf.JobsXCommand.execute(
> JobsXCommand.java:72)
> at org.apache.oozie.command.wf.JobsXCommand.execute(
> JobsXCommand.java:32)
> at org.apache.oozie.command.XCommand.call(XCommand.java:277)
> at org.apache.oozie.DagEngine.getJobs(DagEngine.java:439)
> at org.apache.oozie.servlet.V1JobsServlet.getWorkflowJobs(
> V1JobsServlet.java:314)
> at org.apache.oozie.servlet.V1JobsServlet.getJobs(
> V1JobsServlet.java:141)
> at org.apache.oozie.servlet.BaseJobsServlet.doGet(
> BaseJobsServlet.java:121)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
> at org.apache.oozie.servlet.JsonRestServlet.service(
> JsonRestServlet.java:286)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
> ApplicationFilterChain.java:290)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(
> ApplicationFilterChain.java:206)
> at org.apache.oozie.servlet.AuthFilter$2.doFilter(
> AuthFilter.java:126)
> at org.apache.hadoop.security.authentication.server.
> AuthenticationFilter.doFilter(AuthenticationFilter.java:384)
> at org.apache.oozie.servlet.AuthFilter.doFilter(AuthFilter.java:131)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
> ApplicationFilterChain.java:235)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(
> ApplicationFilterChain.java:206)
> at org.apache.oozie.servlet.HostnameFilter.doFilter(
> HostnameFilter.java:84)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
> ApplicationFilterChain.java:235)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(
> ApplicationFilterChain.java:206)
> at org.apache.catalina.core.StandardWrapperValve.invoke(
> StandardWrapperValve.java:233)
> at org.apache.catalina.core.StandardContextValve.invoke(
> StandardContextValve.java:191)
> at org.apache.catalina.core.StandardHostValve.invoke(
> StandardHostValve.java:127)
> at org.apache.catalina.valves.ErrorReportValve.invoke(
> ErrorReportValve.java:102)
> at org.apache.catalina.core.StandardEngineValve.invoke(
> StandardEngineValve.java:109)
> at org.apache.catalina.connector.CoyoteAdapter.service(
> CoyoteAdapter.java:293)
> at org.apache.coyote.http11.Http11Processor.process(
> Http11Processor.java:859)
> at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.
> process(Http11Protocol.java:602)
> at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(
> JIoEndpoint.java:489)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: <openjpa-2.1.0-r422266:1071316 fatal general error>
> org.apache.openjpa.persistence.PersistenceException: Cannot issue commit in
> a nested connection when there is a pending operation in the parent
> connection. {prepstmnt 320277519 SELECT t0.id, t0.app_name, t0.status,
> t0.run, t0.user_name, t0.group_name, t0.created_time, t0.start_time,
> t0.last_modified_time, t0.end_time, t0.external_id FROM WF_JOBS t0 WHERE
> t0.bean_type = ? ORDER BY t0.created_time DESC [params=?]} [code=30000,
> state=X0Y66]
> at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(
> DBDictionary.java:4869)
> at org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(
> DBDictionary.java:4829)
> at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(
> SQLExceptions.java:136)
> at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(
> SQLExceptions.java:110)
> at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(
> SQLExceptions.java:62)
> at org.apache.openjpa.jdbc.kernel.PreparedSQLStoreQuery$
> PreparedSQLExecutor.executeQuery(PreparedSQLStoreQuery.java:139)
> at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:1005)
> at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:863)
> at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:794)
> at org.apache.openjpa.kernel.DelegatingQuery.execute(
> DelegatingQuery.java:542)
> at org.apache.openjpa.persistence.QueryImpl.execute(
> QueryImpl.java:305)
> at org.apache.openjpa.persistence.QueryImpl.
> getResultList(QueryImpl.java:319)
> at org.apache.oozie.executor.jpa.WorkflowsJobGetJPAExecutor.execute(
> WorkflowsJobGetJPAExecutor.java:247)
> at org.apache.oozie.executor.jpa.WorkflowsJobGetJPAExecutor.execute(
> WorkflowsJobGetJPAExecutor.java:40)
> at org.apache.oozie.service.JPAService.execute(JPAService.java:211)
> at org.apache.oozie.command.wf.JobsXCommand.execute(
> JobsXCommand.java:61)
> ... 29 more
> Caused by: org.apache.openjpa.lib.jdbc.ReportingSQLException: Cannot issue
> commit in a nested connection when there is a pending operation in the
> parent connection. {prepstmnt 320277519 SELECT t0.id, t0.app_name,
> t0.status, t0.run, t0.user_name, t0.group_name, t0.created_time,
> t0.start_time, t0.last_modified_time, t0.end_time, t0.external_id FROM
> WF_JOBS t0 WHERE t0.bean_type = ? ORDER BY t0.created_time DESC [params=?]}
> [code=30000, state=X0Y66]
> at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(
> LoggingConnectionDecorator.java:281)
> at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(
> LoggingConnectionDecorator.java:265)
> at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.
> access$700(LoggingConnectionDecorator.java:72)
> at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator$
> LoggingConnection$LoggingPreparedStatement.executeQuery(
> LoggingConnectionDecorator.java:1183)
> at org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.
> executeQuery(DelegatingPreparedStatement.java:284)
> at org.apache.openjpa.jdbc.kernel.JDBCStoreManager$
> CancelPreparedStatement.executeQuery(JDBCStoreManager.java:1785)
> at org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.
> executeQuery(DelegatingPreparedStatement.java:274)
> at org.apache.openjpa.jdbc.kernel.PreparedSQLStoreQuery$
> PreparedSQLExecutor.executeQuery(PreparedSQLStoreQuery.java:118)
> ... 39 more
> 2013-06-17 09:01:54,275 WARN org.apache.oozie.servlet.V1JobsServlet:
> USER[hue] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] URL[GET
> http://ip-10-0-1-149.eu-west-1.compute.internal:11000/oozie/v1/jobs?len=100&doAs=hue&filter=&user.name=hue&timezone=America%2FLos_Angeles&jobtype=wf]
> error[E0603], E0603: SQL error in operation, Cannot issue commit in a
> nested connection when there is a pending operation in the parent
> connection. {prepstmnt 320277519 SELECT t0.id, t0.app_name, t0.status,
> t0.run, t0.user_name, t0.group_name, t0.created_time, t0.start_time,
> t0.last_modified_time, t0.end_time, t0.external_id FROM WF_JOBS t0 WHERE
> t0.bean_type = ? ORDER BY t0.created_time DESC [params=?]} [code=30000,
> state=X0Y66]
> org.apache.oozie.servlet.XServletException: E0603: SQL error in operation,
> Cannot issue commit in a nested connection when there is a pending
> operation in the parent connection. {prepstmnt 320277519 SELECTt0.id,
> t0.app_name, t0.status, t0.run, t0.user_name, t0.group_name,
> t0.created_time, t0.start_time, t0.last_modified_time, t0.end_time,
> t0.external_id FROM WF_JOBS t0 WHERE t0.bean_type = ? ORDER BY
> t0.created_time DESC [params=?]} [code=30000, state=X0Y66]
> at org.apache.oozie.servlet.V1JobsServlet.getWorkflowJobs(
> V1JobsServlet.java:323)
> at org.apache.oozie.servlet.V1JobsServlet.getJobs(
> V1JobsServlet.java:141)
> at org.apache.oozie.servlet.BaseJobsServlet.doGet(
> BaseJobsServlet.java:121)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
> at org.apache.oozie.servlet.JsonRestServlet.service(
> JsonRestServlet.java:286)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
> ApplicationFilterChain.java:290)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(
> ApplicationFilterChain.java:206)
> at org.apache.oozie.servlet.AuthFilter$2.doFilter(
> AuthFilter.java:126)
> at org.apache.hadoop.security.authentication.server.
> AuthenticationFilter.doFilter(AuthenticationFilter.java:384)
> at org.apache.oozie.servlet.AuthFilter.doFilter(AuthFilter.java:131)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
> ApplicationFilterChain.java:235)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(
> ApplicationFilterChain.java:206)
> at org.apache.oozie.servlet.HostnameFilter.doFilter(
> HostnameFilter.java:84)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
> ApplicationFilterChain.java:235)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(
> ApplicationFilterChain.java:206)
> at org.apache.catalina.core.StandardWrapperValve.invoke(
> StandardWrapperValve.java:233)
> at org.apache.catalina.core.StandardContextValve.invoke(
> StandardContextValve.java:191)
> at org.apache.catalina.core.StandardHostValve.invoke(
> StandardHostValve.java:127)
> at org.apache.catalina.valves.ErrorReportValve.invoke(
> ErrorReportValve.java:102)
> at org.apache.catalina.core.StandardEngineValve.invoke(
> StandardEngineValve.java:109)
> at org.apache.catalina.connector.CoyoteAdapter.service(
> CoyoteAdapter.java:293)
> at org.apache.coyote.http11.Http11Processor.process(
> Http11Processor.java:859)
> at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.
> process(Http11Protocol.java:602)
> at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(
> JIoEndpoint.java:489)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: org.apache.oozie.DagEngineException: E0603: SQL error in
> operation, Cannot issue commit in a nested connection when there is a
> pending operation in the parent connection. {prepstmnt 320277519 SELECT
> t0.id, t0.app_name, t0.status, t0.run, t0.user_name, t0.group_name,
> t0.created_time, t0.start_time, t0.last_modified_time, t0.end_time,
> t0.external_id FROM WF_JOBS t0 WHERE t0.bean_type = ? ORDER BY
> t0.created_time DESC [params=?]} [code=30000, state=X0Y66]
> at org.apache.oozie.DagEngine.getJobs(DagEngine.java:442)
> at org.apache.oozie.servlet.V1JobsServlet.getWorkflowJobs(
> V1JobsServlet.java:314)
> ... 25 more
> Caused by: org.apache.oozie.command.CommandException: E0603: SQL error in
> operation, Cannot issue commit in a nested connection when there is a
> pending operation in the parent connection. {prepstmnt 320277519 SELECT
> t0.id, t0.app_name, t0.status, t0.run, t0.user_name, t0.group_name,
> t0.created_time, t0.start_time, t0.last_modified_time, t0.end_time,
> t0.external_id FROM WF_JOBS t0 WHERE t0.bean_type = ? ORDER BY
> t0.created_time DESC [params=?]} [code=30000, state=X0Y66]
> at org.apache.oozie.command.wf.JobsXCommand.execute(
> JobsXCommand.java:72)
> at org.apache.oozie.command.wf.JobsXCommand.execute(
> JobsXCommand.java:32)
> at org.apache.oozie.command.XCommand.call(XCommand.java:277)
> at org.apache.oozie.DagEngine.getJobs(DagEngine.java:439)
> ... 26 more
> Caused by: <openjpa-2.1.0-r422266:1071316 fatal general error>
> org.apache.openjpa.persistence.PersistenceException: Cannot issue commit in
> a nested connection when there is a pending operation in the parent
> connection. {prepstmnt 320277519 SELECT t0.id, t0.app_name, t0.status,
> t0.run, t0.user_name, t0.group_name, t0.created_time, t0.start_time,
> t0.last_modified_time, t0.end_time, t0.external_id FROM WF_JOBS t0 WHERE
> t0.bean_type = ? ORDER BY t0.created_time DESC [params=?]} [code=30000,
> state=X0Y66]
> at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(
> DBDictionary.java:4869)
> at org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(
> DBDictionary.java:4829)
> at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(
> SQLExceptions.java:136)
> at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(
> SQLExceptions.java:110)
> at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(
> SQLExceptions.java:62)
> at org.apache.openjpa.jdbc.kernel.PreparedSQLStoreQuery$
> PreparedSQLExecutor.executeQuery(PreparedSQLStoreQuery.java:139)
> at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:1005)
> at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:863)
> at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:794)
> at org.apache.openjpa.kernel.DelegatingQuery.execute(
> DelegatingQuery.java:542)
> at org.apache.openjpa.persistence.QueryImpl.execute(
> QueryImpl.java:305)
> at org.apache.openjpa.persistence.QueryImpl.
> getResultList(QueryImpl.java:319)
> at org.apache.oozie.executor.jpa.WorkflowsJobGetJPAExecutor.execute(
> WorkflowsJobGetJPAExecutor.java:247)
> at org.apache.oozie.executor.jpa.WorkflowsJobGetJPAExecutor.execute(
> WorkflowsJobGetJPAExecutor.java:40)
> at org.apache.oozie.service.JPAService.execute(JPAService.java:211)
> at org.apache.oozie.command.wf.JobsXCommand.execute(
> JobsXCommand.java:61)
> ... 29 more
> Caused by: org.apache.openjpa.lib.jdbc.ReportingSQLException: Cannot issue
> commit in a nested connection when there is a pending operation in the
> parent connection. {prepstmnt 320277519 SELECT t0.id, t0.app_name,
> t0.status, t0.run, t0.user_name, t0.group_name, t0.created_time,
> t0.start_time, t0.last_modified_time, t0.end_time, t0.external_id FROM
> WF_JOBS t0 WHERE t0.bean_type = ? ORDER BY t0.created_time DESC [params=?]}
> [code=30000, state=X0Y66]
> at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(
> LoggingConnectionDecorator.java:281)
> at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(
> LoggingConnectionDecorator.java:265)
> at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.
> access$700(LoggingConnectionDecorator.java:72)
> at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator$
> LoggingConnection$LoggingPreparedStatement.executeQuery(
> LoggingConnectionDecorator.java:1183)
> at org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.
> executeQuery(DelegatingPreparedStatement.java:284)
> at org.apache.openjpa.jdbc.kernel.JDBCStoreManager$
> CancelPreparedStatement.executeQuery(JDBCStoreManager.java:1785)
> at org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.
> executeQuery(DelegatingPreparedStatement.java:274)
> at org.apache.openjpa.jdbc.kernel.PreparedSQLStoreQuery$
> PreparedSQLExecutor.executeQuery(PreparedSQLStoreQuery.java:118)
> ... 39 more
>
> After resting the service, the systems will run fine.
>
> I have no idea where to debug or how to solve this.
>
>
> Best Regards,
> Christian.
--
Harsh J