You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@oozie.apache.org by Jaboy Mathai <ja...@flytxt.com> on 2018/05/31 12:45:13 UTC
Reg: Oozie 4.1.0 having jdbc error
Hi Team,
Need help.
Have oozie 4.1.0 installed in RHEL 7.5 in node1. In the same network, we
have mariadb 5.5.41 (mariadb.x86_64) in node5(earlier we had version 5.5.56
but we changed that, assuming compatibility issues). Oozie gets started
normally , but when I try to start to run the workflow, I get jdbc error in
oozie.log
*Logs below:*
Launcher ERROR, reason: Main class
[org.apache.oozie.action.hadoop.JavaMain], main() threw exception,
org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get
JDBC Connection; nested exception is
org.apache.commons.dbcp.SQLNestedException: Cannot create
PoolableConnectionFactory (Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The
driver has not received any packets from the server.)
2018-05-31 08:12:23,641 WARN JavaActionExecutor:544 - SERVER[
svdt5neonhadoop01.safaricom.net] USER[hadoop] GROUP[-] TOKEN[]
APP[SAFARICOM_KENYA_CBS_RECHARGE] JOB[0000001-180531081106985-oozie-hado-W]
ACTION[0000001-180531081106985-oozie-hado-W@GetProcessedFileDetails_Recharge_infile]
Launcher exception:
org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get
JDBC Connection; nested exception is
org.apache.commons.dbcp.SQLNestedException: Cannot create
PoolableConnectionFactory (Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The
driver has not received any packets from the server.)
at org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:58)
at
org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:39)
at org.apache.oozie.action.hadoop.JavaMain.main(JavaMain.java:36)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:226)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could
not get JDBC Connection; nested exception is
org.apache.commons.dbcp.SQLNestedException: Cannot create
PoolableConnectionFactory (Communications link failure
We have checked the port 3306 for mariadb using ‘nmap’ in linux and have
checked the connection hops from node1 to node5 (using traceroute)and found
that , there is only a single hop, that is, node1 is directly accessing
node5 and vice-versa in the network(no switch IP shown intraceroute output
or switch firewalls expected).
Checked for iptables and firewalls – no firewalls are running in the
cluster.
*Please note:* Same job (workflow.xml) is running fine in one of our other
cluster with below versions:
Oozie 4.1.0 , RHEL 7.3 and database in mariadb 5.5.41.
*Attaching the logs having oozie startup and the job run log having the
error, with this e-mail.*
*Is there any issue with oozie 4.1.0 working in RHEL 7.5 ?*
Tried to install oozie 4.3.0 but I am stuck as per below screenshot, when I
try to run below command:
# ./mkdistro.sh –DskipTests
I am able to wget the apache-17.pom from the linux shell prompt but the
build process fails. We have added the required proxy to access the
internet. I don’t know if any proxy to be added so that the build process
comes to know about the internet availability through proxy.
Please suggest a solution with the shortest approach possible to solve
this. Will appreciate if someone have the oozie 4.3.0 for RHEL 7.5 readily
available and could share.
Below is the oozie related settings:
WARN: Use of this script is deprecated; use 'oozied.sh stop' instead
Setting OOZIE_HOME: /usr/local/oozie
Setting OOZIE_CONFIG: /usr/local/oozie/conf
Sourcing: /usr/local/oozie/conf/oozie-env.sh
setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
setting OOZIE_BASE_URL="http://
${OOZIE_HTTP_HOSTNAME}:${OOZIE_HTTP_PORT}/oozie"
Setting OOZIE_CONFIG_FILE: oozie-site.xml
Setting OOZIE_DATA: /usr/local/oozie/data
Setting OOZIE_LOG: /usr/local/oozie/logs
Setting OOZIE_LOG4J_FILE: oozie-log4j.properties
Setting OOZIE_LOG4J_RELOAD: 10
Setting OOZIE_HTTP_HOSTNAME: svdt5neonhadoop01.safaricom.net
Setting OOZIE_HTTP_PORT: 11000
Setting OOZIE_ADMIN_PORT: 11001
Setting OOZIE_HTTPS_PORT: 11443
Using OOZIE_BASE_URL: http://:/oozie
Setting CATALINA_BASE: /usr/local/oozie/oozie-server
Setting OOZIE_HTTPS_KEYSTORE_FILE: /home/hadoop/.keystore
Setting OOZIE_HTTPS_KEYSTORE_PASS: password
Setting OOZIE_INSTANCE_ID: svdt5neonhadoop01.safaricom.net
Setting CATALINA_OUT: /usr/local/oozie/logs/catalina.out
Setting CATALINA_PID: /usr/local/oozie/oozie-server/temp/oozie.pid
Using CATALINA_OPTS: -Xmx1024m
-Dderby.stream.error.file=/usr/local/oozie/logs/derby.log
Adding to CATALINA_OPTS: -Doozie.home.dir=/usr/local/oozie
-Doozie.config.dir=/usr/local/oozie/conf
-Doozie.log.dir=/usr/local/oozie/logs
-Doozie.data.dir=/usr/local/oozie/data -Doozie.instance.id=
svdt5neonhadoop01.safaricom.net -Doozie.config.file=oozie-site.xml
-Doozie.log4j.file=oozie-log4j.properties -Doozie.log4j.reload=10
-Doozie.http.hostname=svdt5neonhadoop01.safaricom.net
-Doozie.admin.port=11001 -Doozie.http.port=11000 -Doozie.https.port=11443
-Doozie.base.url=http://:/oozie
-Doozie.https.keystore.file=/home/hadoop/.keystore
-Doozie.https.keystore.pass=password -Djava.library.path=
Using CATALINA_BASE: /usr/local/oozie/oozie-server
Using CATALINA_HOME: /usr/local/oozie/oozie-server
Using CATALINA_TMPDIR: /usr/local/oozie/oozie-server/temp
Using JRE_HOME: /usr/java/jdk1.7.0_67
Using CLASSPATH: /usr/local/oozie/oozie-server/bin/bootstrap.jar
Using CATALINA_PID: /usr/local/oozie/oozie-server/temp/oozie.pid
Thanks & Regards,
*Jaboy Mathai*
RE: Reg: Oozie 4.1.0 having jdbc error
Posted by Jaboy Mathai <ja...@flytxt.com>.
Dear Peter,
There are 3 folders for the jar files in oozie home folder : 'lib' ,
'libext' and 'libtools'. When I build oozie , I had Hadoop jar version
'2.4.0' which I saw in 'lib' folder, but in 'libext' I had '2.6.0' version.
Then I changed the jars in 'lib' folder to '2.6.0' to make the versions look
similar in the oozie classpath. But that did not solve the issue. Hence the
'No such method Error' issue remains.
Oozie has been compiled for RHEL 7.5 and Hadoop 2.6.0 .
Best Regards
Jaboy
-----Original Message-----
From: Peter Cseh [mailto:gezapeti@cloudera.com]
Sent: 11 June 2018 20:20
To: user@oozie.apache.org
Cc: Saurabh Kumar; Balakrishnan Nagiah; Vinod Rajasekharan; Shivam Garg
Subject: Re: Reg: Oozie 4.1.0 having jdbc error
Hey!
Oozie 4.3.1 and newer releases are compiling with Hadoop 2.6.0.
Why is the Hadoop jar you're referring is 2.4.0? Also, what Hadoop version
you're compiling Oozie and you're code submitting to Oozie against?
There might be different versions in play on the Oozie classpath and that
can cause issues like this.
gp
On Thu, Jun 7, 2018 at 3:39 PM, Jaboy Mathai <ja...@flytxt.com>
wrote:
> Hi Peter,
>
> Good Day!
>
> I managed to build oozie 4.3.1 locally and tried to run job. Oozie
> starts up fine. But I get below exception now in the oozie logs.
>
> 2018-06-06 00:08:45,862 INFO ActionStartXCommand:520 -
> SERVER[svdt5neonhadoop01.safaricom.net] USER[hadoop] GROUP[-] TOKEN[]
> APP[SAFARICOM_KENYA_CBS_RECHARGE] JOB[0000003-180605235623225-
> oozie-hado-W]
> ACTION[0000003-180605235623225-oozie-hado-W@GetProcessedFileDetails_
> Recharge_infile]
> Start action
> [0000003-180605235623225-oozie-hado-W@GetProcessedFileDetails_
> Recharge_infile]
> with user-retry state : userRetryCount [0], userRetryMax [0],
> userRetryInterval [10]
> 2018-06-06 00:08:45,990 ERROR ActionStartXCommand:517 -
> SERVER[svdt5neonhadoop01.safaricom.net] USER[hadoop] GROUP[-] TOKEN[]
> APP[SAFARICOM_KENYA_CBS_RECHARGE] JOB[0000003-180605235623225-
> oozie-hado-W]
> ACTION[0000003-180605235623225-oozie-hado-W@GetProcessedFileDetails_
> Recharge_infile]
> Error,
> java.lang.NoSuchMethodError:
> org.apache.hadoop.yarn.util.timeline.TimelineUtils.
> buildTimelineTokenService(Lorg/apache/hadoop/conf/
> Configuration;)Lorg/apache/hadoop/io/Text;
> at
> org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.
> serviceInit(YarnClientImpl.java:166)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(
> ResourceMgrDelegate.java:102)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.mapred.ResourceMgrDelegate.<init>(
> ResourceMgrDelegate.java:96)
> at org.apache.hadoop.mapred.YARNRunner.<init>(YARNRunner.java:112)
> at
> org.apache.hadoop.mapred.YarnClientProtocolProvider.create(
> YarnClientProtocolProvider.java:34)
> at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
> at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:82)
> at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:75)
>
>
> I have checked for the jar file having this method and found that the
> class ' org.apache.hadoop.yarn.util.timeline.TimelineUtils' is present
> in the jar file ' hadoop-yarn-common-2.4.0.jar' but could find the jar
> for '
> org.apache.hadoop.yarn.util.timeline.TimelineUtils.
> buildTimelineTokenService'
> , hence don’t know that which would be the dependent jar file here.
>
> [hadoop@svdt5neonhadoop01 lib]$
> [hadoop@svdt5neonhadoop01 lib]$ egrep -i
> org.apache.hadoop.yarn.util.timeline.TimelineUtils * Binary file
> hadoop-yarn-common-2.4.0.jar matches
> [hadoop@svdt5neonhadoop01 lib]$
> [hadoop@svdt5neonhadoop01 lib]$ egrep -i
> org.apache.hadoop.yarn.util.timeline.TimelineUtils.
> buildTimelineTokenService
> *
> [hadoop@svdt5neonhadoop01 lib]$
> [hadoop@svdt5neonhadoop01 lib]$ ls |wc -l
> 160
> [hadoop@svdt5neonhadoop01 lib]$ pwd
> /usr/local/oozie/lib
> [hadoop@svdt5neonhadoop01 lib]$
>
>
> Please help !
>
> Thanks & Regards,
> Jaboy Mathai
>
>
>
>
> -----Original Message-----
> From: Peter Cseh [mailto:gezapeti@cloudera.com]
> Sent: 01 June 2018 15:48
> To: user@oozie.apache.org
> Cc: Saurabh Kumar; Balakrishnan Nagiah; Vinod Rajasekharan; Shivam
> Garg
> Subject: Re: Reg: Oozie 4.1.0 having jdbc error
>
> I'm not aware of a binary distribution of Apache Oozie available to
> download.
> There are modified versions here and there (e.g. CDH's Oozie can be
> installed via rpm:
> https://www.cloudera.com/documentation/enterprise/5-14-
> x/topics/cdh_ig_cdh5_install.html
> , but there are others available as well).
>
> Peter
>
> On Fri, Jun 1, 2018 at 11:53 AM, Jaboy Mathai
> <ja...@flytxt.com>
> wrote:
>
> > Dear Peter,
> >
> > Thanks for your reply. I am using the same Maven 3.5.3 but the java
> > version is '1.7.0_67'. Will try to build version 4.3.1 , but I am
> > not sure because we don’t have local environment(having RHEL) with
> > internet connection as you pointed. That’s why, I requested for a
> > build, that may be readily available.
> > Let me try building v.4.3.1 and will let you know.
> >
> > Best Regards
> > Jaboy
> >
> > -----Original Message-----
> > From: Peter Cseh [mailto:gezapeti@cloudera.com]
> > Sent: 01 June 2018 14:49
> > To: user@oozie.apache.org
> > Cc: Saurabh Kumar; Balakrishnan Nagiah; Vinod Rajasekharan; Shivam
> > Garg
> > Subject: Re: Reg: Oozie 4.1.0 having jdbc error
> >
> > Hi!
> > Have you tried building 4.3.1?
> > I've managed to build releases 4.3.0, 4.1.0 and 4.3.1. I've even
> > cleaned my maven cache in between.
> > I'm using Maven 3.5.3 with jdk1.8.
> > To resolve the build issue you may compile Oozie locally and copy
> > the distribution up to the server to see if it works that way. I
> > suggest using Oozie 4.3.1 or 5.0.0.
> >
> > Based on the error messages during build (timeout) and Oozie startup
> > (did not receive any packages) it looks like you're having issues
> > with your network.
> >
> > hope it helps
> > gp
> >
> > On Fri, Jun 1, 2018 at 7:16 AM, Jaboy Mathai
> > <ja...@flytxt.com>
> > wrote:
> >
> > > Hi,
> > >
> > >
> > >
> > > This is a bit urgent requirement for me, if someone has some idea
> > > , please help !
> > >
> > >
> > >
> > > BR,
> > >
> > > Jaboy
> > >
> > >
> > >
> > > *From:* Jaboy Mathai [mailto:jaboy.mathai@flytxt.com]
> > > *Sent:* 01 June 2018 10:37
> > > *To:* 'user@oozie.apache.org'
> > > *Cc:* Saurabh Kumar; Balakrishnan Nagiah; Vinod Rajasekharan;
> > > Shivam Garg
> > > *Subject:* RE: Reg: Oozie 4.1.0 having jdbc error
> > >
> > >
> > >
> > > + Looping Shivam.
> > >
> > >
> > >
> > > *From:* Jaboy Mathai [mailto:jaboy.mathai@flytxt.com
> > > <ja...@flytxt.com>]
> > > *Sent:* 31 May 2018 18:15
> > > *To:* 'user@oozie.apache.org'
> > > *Cc:* Saurabh Kumar; Balakrishnan Nagiah; Vinod Rajasekharan
> > > *Subject:* Reg: Oozie 4.1.0 having jdbc error
> > >
> > >
> > >
> > > Hi Team,
> > >
> > >
> > >
> > > Need help.
> > >
> > >
> > >
> > > Have oozie 4.1.0 installed in RHEL 7.5 in node1. In the same
> > > network, we have mariadb 5.5.41 (mariadb.x86_64) in node5(earlier
> > > we had version 5.5.56 but we changed that, assuming compatibility
> > > issues).
> > > Oozie gets started normally , but when I try to start to run the
> > > workflow, I get jdbc error in oozie.log
> > >
> > >
> > >
> > > *Logs below:*
> > >
> > >
> > >
> > > Launcher ERROR, reason: Main class
> > > [org.apache.oozie.action.hadoop.JavaMain],
> > > main() threw exception, org.springframework.jdbc.
> > > CannotGetJdbcConnectionException: Could not get JDBC Connection;
> > > nested exception is org.apache.commons.dbcp.SQLNestedException:
> > > Cannot create PoolableConnectionFactory (Communications link
> > > failure
> > >
> > >
> > >
> > > The last packet sent successfully to the server was 0 milliseconds
> > > ago.
> > > The driver has not received any packets from the server.)
> > >
> > > 2018-05-31 08:12:23,641 WARN JavaActionExecutor:544 - SERVER[
> > > svdt5neonhadoop01.safaricom.net] USER[hadoop] GROUP[-] TOKEN[]
> > > APP[SAFARICOM_KENYA_CBS_RECHARGE]
> > > JOB[0000001-180531081106985-oozie-hado-W]
> > > ACTION[0000001-180531081106985-oozie-hado-W@GetProcessedFileDetail
> > > s_ Re charge_infile] Launcher exception: org.springframework.jdbc.
> > > CannotGetJdbcConnectionException: Could not get JDBC Connection;
> > > nested exception is org.apache.commons.dbcp.SQLNestedException:
> > > Cannot create PoolableConnectionFactory (Communications link
> > > failure
> > >
> > >
> > >
> > > The last packet sent successfully to the server was 0 milliseconds
> > > ago.
> > > The driver has not received any packets from the server.)
> > >
> > > at
> > > org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:58)
> > >
> > > at org.apache.oozie.action.hadoop.LauncherMain.run(
> > > LauncherMain.java:39)
> > >
> > > at
> > > org.apache.oozie.action.hadoop.JavaMain.main(JavaMain.java:36)
> > >
> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> > > Method)
> > >
> > > at sun.reflect.NativeMethodAccessorImpl.invoke(
> > > NativeMethodAccessorImpl.java:57)
> > >
> > > at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> > > DelegatingMethodAccessorImpl.java:43)
> > >
> > > at java.lang.reflect.Method.invoke(Method.java:606)
> > >
> > > at org.apache.oozie.action.hadoop.LauncherMapper.map(
> > > LauncherMapper.java:226)
> > >
> > > at
> > > org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> > >
> > > at
> > > org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
> > >
> > > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> > >
> > > at
> > > org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> > >
> > > at java.security.AccessController.doPrivileged(Native
> > > Method)
> > >
> > > at javax.security.auth.Subject.doAs(Subject.java:415)
> > >
> > > at org.apache.hadoop.security.UserGroupInformation.doAs(
> > > UserGroupInformation.java:1628)
> > >
> > > at
> > > org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> > >
> > > Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException:
> > > Could not get JDBC Connection; nested exception is
> > > org.apache.commons.dbcp.SQLNestedException:
> > > Cannot create PoolableConnectionFactory (Communications link
> > > failure
> > >
> > >
> > >
> > >
> > >
> > > We have checked the port 3306 for mariadb using ‘nmap’ in linux
> > > and have checked the connection hops from node1 to node5 (using
> > > traceroute)and found that , there is only a single hop, that is,
> > > node1 is directly accessing
> > > node5 and vice-versa in the network(no switch IP shown
> > > intraceroute output or switch firewalls expected).
> > >
> > >
> > >
> > > Checked for iptables and firewalls – no firewalls are running in
> > > the cluster.
> > >
> > >
> > >
> > > *Please note:* Same job (workflow.xml) is running fine in one of
> > > our other cluster with below versions:
> > >
> > >
> > >
> > > Oozie 4.1.0 , RHEL 7.3 and database in mariadb 5.5.41.
> > >
> > >
> > >
> > >
> > >
> > > *Attaching the logs having oozie startup and the job run log
> > > having the error, with this e-mail.*
> > >
> > >
> > >
> > > *Is there any issue with oozie 4.1.0 working in RHEL 7.5 ?*
> > >
> > >
> > >
> > > Tried to install oozie 4.3.0 but I am stuck as per below
> > > screenshot, when I try to run below command:
> > >
> > >
> > >
> > > # ./mkdistro.sh –DskipTests
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > I am able to wget the apache-17.pom from the linux shell prompt
> > > but the build process fails. We have added the required proxy to
> > > access the internet. I don’t know if any proxy to be added so that
> > > the build process comes to know about the internet availability
> > > through proxy.
> > >
> > >
> > >
> > > Please suggest a solution with the shortest approach possible to
> > > solve this. Will appreciate if someone have the oozie 4.3.0 for
> > > RHEL
> > > 7.5 readily available and could share.
> > >
> > >
> > >
> > > Below is the oozie related settings:
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > WARN: Use of this script is deprecated; use 'oozied.sh stop'
> > > instead
> > >
> > >
> > >
> > > Setting OOZIE_HOME: /usr/local/oozie
> > >
> > > Setting OOZIE_CONFIG: /usr/local/oozie/conf
> > >
> > > Sourcing: /usr/local/oozie/conf/oozie-env.sh
> > >
> > > setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
> > >
> > > setting OOZIE_BASE_URL="http://${OOZIE_HTTP_HOSTNAME}:${OOZIE_
> > > HTTP_PORT}/oozie"
> > >
> > > Setting OOZIE_CONFIG_FILE: oozie-site.xml
> > >
> > > Setting OOZIE_DATA: /usr/local/oozie/data
> > >
> > > Setting OOZIE_LOG: /usr/local/oozie/logs
> > >
> > > Setting OOZIE_LOG4J_FILE: oozie-log4j.properties
> > >
> > > Setting OOZIE_LOG4J_RELOAD: 10
> > >
> > > Setting OOZIE_HTTP_HOSTNAME: svdt5neonhadoop01.safaricom.net
> > >
> > > Setting OOZIE_HTTP_PORT: 11000
> > >
> > > Setting OOZIE_ADMIN_PORT: 11001
> > >
> > > Setting OOZIE_HTTPS_PORT: 11443
> > >
> > > Using OOZIE_BASE_URL: http://:/oozie
> > >
> > > Setting CATALINA_BASE: /usr/local/oozie/oozie-server
> > >
> > > Setting OOZIE_HTTPS_KEYSTORE_FILE: /home/hadoop/.keystore
> > >
> > > Setting OOZIE_HTTPS_KEYSTORE_PASS: password
> > >
> > > Setting OOZIE_INSTANCE_ID: svdt5neonhadoop01.safaricom.net
> > >
> > > Setting CATALINA_OUT: /usr/local/oozie/logs/catalina.out
> > >
> > > Setting CATALINA_PID: /usr/local/oozie/oozie-server/
> > temp/oozie.pid
> > >
> > >
> > >
> > > Using CATALINA_OPTS: -Xmx1024m -Dderby.stream.error.file=/
> > > usr/local/oozie/logs/derby.log
> > >
> > > Adding to CATALINA_OPTS: -Doozie.home.dir=/usr/local/oozie
> > > -Doozie.config.dir=/usr/local/oozie/conf
> > > -Doozie.log.dir=/usr/local/oozie/logs
> > > -Doozie.data.dir=/usr/local/oozie/data
> > > -Doozie.instance.id=svdt5neonh adoop01.safaricom.net
> > > -Doozie.config.file=oozie-site.xml
> > > -Doozie.log4j.file=oozie-log4j.properties -Doozie.log4j.reload=10
> > > -Doozie.http.hostname=svdt5neonhadoop01.safaricom.net
> > > -Doozie.admin.port=11001 -Doozie.http.port=11000
> > > -Doozie.https.port=11443 -Doozie.base.url=http://:/oozie
> > > -Doozie.https.keystore.file=/home/hadoop/.keystore
> > > -Doozie.https.keystore.pass=password -Djava.library.path=
> > >
> > > Using CATALINA_BASE: /usr/local/oozie/oozie-server
> > >
> > > Using CATALINA_HOME: /usr/local/oozie/oozie-server
> > >
> > > Using CATALINA_TMPDIR: /usr/local/oozie/oozie-server/temp
> > >
> > > Using JRE_HOME: /usr/java/jdk1.7.0_67
> > >
> > > Using CLASSPATH: /usr/local/oozie/oozie-server/bin/bootstrap.jar
> > >
> > > Using CATALINA_PID: /usr/local/oozie/oozie-server/temp/oozie.pid
> > >
> > >
> > >
> > >
> > >
> > > Thanks & Regards,
> > >
> > > *Jaboy Mathai*
> > >
> > >
> > >
> >
> >
> >
> > --
> > *Peter Cseh *| Software Engineer
> > cloudera.com <https://www.cloudera.com>
> >
> > [image: Cloudera] <https://www.cloudera.com/>
> >
> > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> > Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
> > ------------------------------
> >
>
>
>
> --
> *Peter Cseh *| Software Engineer
> cloudera.com <https://www.cloudera.com>
>
> [image: Cloudera] <https://www.cloudera.com/>
>
> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
> ------------------------------
>
--
*Peter Cseh *| Software Engineer
cloudera.com <https://www.cloudera.com>
[image: Cloudera] <https://www.cloudera.com/>
[image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
on LinkedIn] <https://www.linkedin.com/company/cloudera>
------------------------------
Re: Reg: Oozie 4.1.0 having jdbc error
Posted by Peter Cseh <ge...@cloudera.com>.
Hey!
Oozie 4.3.1 and newer releases are compiling with Hadoop 2.6.0.
Why is the Hadoop jar you're referring is 2.4.0? Also, what Hadoop version
you're compiling Oozie and you're code submitting to Oozie against?
There might be different versions in play on the Oozie classpath and that
can cause issues like this.
gp
On Thu, Jun 7, 2018 at 3:39 PM, Jaboy Mathai <ja...@flytxt.com>
wrote:
> Hi Peter,
>
> Good Day!
>
> I managed to build oozie 4.3.1 locally and tried to run job. Oozie starts
> up
> fine. But I get below exception now in the oozie logs.
>
> 2018-06-06 00:08:45,862 INFO ActionStartXCommand:520 -
> SERVER[svdt5neonhadoop01.safaricom.net] USER[hadoop] GROUP[-] TOKEN[]
> APP[SAFARICOM_KENYA_CBS_RECHARGE] JOB[0000003-180605235623225-
> oozie-hado-W]
> ACTION[0000003-180605235623225-oozie-hado-W@GetProcessedFileDetails_
> Recharge_infile]
> Start action
> [0000003-180605235623225-oozie-hado-W@GetProcessedFileDetails_
> Recharge_infile]
> with user-retry state : userRetryCount [0], userRetryMax [0],
> userRetryInterval [10]
> 2018-06-06 00:08:45,990 ERROR ActionStartXCommand:517 -
> SERVER[svdt5neonhadoop01.safaricom.net] USER[hadoop] GROUP[-] TOKEN[]
> APP[SAFARICOM_KENYA_CBS_RECHARGE] JOB[0000003-180605235623225-
> oozie-hado-W]
> ACTION[0000003-180605235623225-oozie-hado-W@GetProcessedFileDetails_
> Recharge_infile]
> Error,
> java.lang.NoSuchMethodError:
> org.apache.hadoop.yarn.util.timeline.TimelineUtils.
> buildTimelineTokenService(Lorg/apache/hadoop/conf/
> Configuration;)Lorg/apache/hadoop/io/Text;
> at
> org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.
> serviceInit(YarnClientImpl.java:166)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(
> ResourceMgrDelegate.java:102)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.mapred.ResourceMgrDelegate.<init>(
> ResourceMgrDelegate.java:96)
> at org.apache.hadoop.mapred.YARNRunner.<init>(YARNRunner.java:112)
> at
> org.apache.hadoop.mapred.YarnClientProtocolProvider.create(
> YarnClientProtocolProvider.java:34)
> at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
> at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:82)
> at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:75)
>
>
> I have checked for the jar file having this method and found that the
> class
> ' org.apache.hadoop.yarn.util.timeline.TimelineUtils' is present in the
> jar
> file ' hadoop-yarn-common-2.4.0.jar' but could find the jar for '
> org.apache.hadoop.yarn.util.timeline.TimelineUtils.
> buildTimelineTokenService'
> , hence don’t know that which would be the dependent jar file here.
>
> [hadoop@svdt5neonhadoop01 lib]$
> [hadoop@svdt5neonhadoop01 lib]$ egrep -i
> org.apache.hadoop.yarn.util.timeline.TimelineUtils *
> Binary file hadoop-yarn-common-2.4.0.jar matches
> [hadoop@svdt5neonhadoop01 lib]$
> [hadoop@svdt5neonhadoop01 lib]$ egrep -i
> org.apache.hadoop.yarn.util.timeline.TimelineUtils.
> buildTimelineTokenService
> *
> [hadoop@svdt5neonhadoop01 lib]$
> [hadoop@svdt5neonhadoop01 lib]$ ls |wc -l
> 160
> [hadoop@svdt5neonhadoop01 lib]$ pwd
> /usr/local/oozie/lib
> [hadoop@svdt5neonhadoop01 lib]$
>
>
> Please help !
>
> Thanks & Regards,
> Jaboy Mathai
>
>
>
>
> -----Original Message-----
> From: Peter Cseh [mailto:gezapeti@cloudera.com]
> Sent: 01 June 2018 15:48
> To: user@oozie.apache.org
> Cc: Saurabh Kumar; Balakrishnan Nagiah; Vinod Rajasekharan; Shivam Garg
> Subject: Re: Reg: Oozie 4.1.0 having jdbc error
>
> I'm not aware of a binary distribution of Apache Oozie available to
> download.
> There are modified versions here and there (e.g. CDH's Oozie can be
> installed via rpm:
> https://www.cloudera.com/documentation/enterprise/5-14-
> x/topics/cdh_ig_cdh5_install.html
> , but there are others available as well).
>
> Peter
>
> On Fri, Jun 1, 2018 at 11:53 AM, Jaboy Mathai <ja...@flytxt.com>
> wrote:
>
> > Dear Peter,
> >
> > Thanks for your reply. I am using the same Maven 3.5.3 but the java
> > version is '1.7.0_67'. Will try to build version 4.3.1 , but I am not
> > sure because we don’t have local environment(having RHEL) with
> > internet connection as you pointed. That’s why, I requested for a
> > build, that may be readily available.
> > Let me try building v.4.3.1 and will let you know.
> >
> > Best Regards
> > Jaboy
> >
> > -----Original Message-----
> > From: Peter Cseh [mailto:gezapeti@cloudera.com]
> > Sent: 01 June 2018 14:49
> > To: user@oozie.apache.org
> > Cc: Saurabh Kumar; Balakrishnan Nagiah; Vinod Rajasekharan; Shivam
> > Garg
> > Subject: Re: Reg: Oozie 4.1.0 having jdbc error
> >
> > Hi!
> > Have you tried building 4.3.1?
> > I've managed to build releases 4.3.0, 4.1.0 and 4.3.1. I've even
> > cleaned my maven cache in between.
> > I'm using Maven 3.5.3 with jdk1.8.
> > To resolve the build issue you may compile Oozie locally and copy the
> > distribution up to the server to see if it works that way. I suggest
> > using Oozie 4.3.1 or 5.0.0.
> >
> > Based on the error messages during build (timeout) and Oozie startup
> > (did not receive any packages) it looks like you're having issues with
> > your network.
> >
> > hope it helps
> > gp
> >
> > On Fri, Jun 1, 2018 at 7:16 AM, Jaboy Mathai <ja...@flytxt.com>
> > wrote:
> >
> > > Hi,
> > >
> > >
> > >
> > > This is a bit urgent requirement for me, if someone has some idea ,
> > > please help !
> > >
> > >
> > >
> > > BR,
> > >
> > > Jaboy
> > >
> > >
> > >
> > > *From:* Jaboy Mathai [mailto:jaboy.mathai@flytxt.com]
> > > *Sent:* 01 June 2018 10:37
> > > *To:* 'user@oozie.apache.org'
> > > *Cc:* Saurabh Kumar; Balakrishnan Nagiah; Vinod Rajasekharan; Shivam
> > > Garg
> > > *Subject:* RE: Reg: Oozie 4.1.0 having jdbc error
> > >
> > >
> > >
> > > + Looping Shivam.
> > >
> > >
> > >
> > > *From:* Jaboy Mathai [mailto:jaboy.mathai@flytxt.com
> > > <ja...@flytxt.com>]
> > > *Sent:* 31 May 2018 18:15
> > > *To:* 'user@oozie.apache.org'
> > > *Cc:* Saurabh Kumar; Balakrishnan Nagiah; Vinod Rajasekharan
> > > *Subject:* Reg: Oozie 4.1.0 having jdbc error
> > >
> > >
> > >
> > > Hi Team,
> > >
> > >
> > >
> > > Need help.
> > >
> > >
> > >
> > > Have oozie 4.1.0 installed in RHEL 7.5 in node1. In the same
> > > network, we have mariadb 5.5.41 (mariadb.x86_64) in node5(earlier we
> > > had version 5.5.56 but we changed that, assuming compatibility issues).
> > > Oozie gets started normally , but when I try to start to run the
> > > workflow, I get jdbc error in oozie.log
> > >
> > >
> > >
> > > *Logs below:*
> > >
> > >
> > >
> > > Launcher ERROR, reason: Main class
> > > [org.apache.oozie.action.hadoop.JavaMain],
> > > main() threw exception, org.springframework.jdbc.
> > > CannotGetJdbcConnectionException: Could not get JDBC Connection;
> > > nested exception is org.apache.commons.dbcp.SQLNestedException:
> > > Cannot create PoolableConnectionFactory (Communications link failure
> > >
> > >
> > >
> > > The last packet sent successfully to the server was 0 milliseconds ago.
> > > The driver has not received any packets from the server.)
> > >
> > > 2018-05-31 08:12:23,641 WARN JavaActionExecutor:544 - SERVER[
> > > svdt5neonhadoop01.safaricom.net] USER[hadoop] GROUP[-] TOKEN[]
> > > APP[SAFARICOM_KENYA_CBS_RECHARGE]
> > > JOB[0000001-180531081106985-oozie-hado-W]
> > > ACTION[0000001-180531081106985-oozie-hado-W@GetProcessedFileDetails_
> > > Re charge_infile] Launcher exception: org.springframework.jdbc.
> > > CannotGetJdbcConnectionException: Could not get JDBC Connection;
> > > nested exception is org.apache.commons.dbcp.SQLNestedException:
> > > Cannot create PoolableConnectionFactory (Communications link failure
> > >
> > >
> > >
> > > The last packet sent successfully to the server was 0 milliseconds ago.
> > > The driver has not received any packets from the server.)
> > >
> > > at
> > > org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:58)
> > >
> > > at org.apache.oozie.action.hadoop.LauncherMain.run(
> > > LauncherMain.java:39)
> > >
> > > at
> > > org.apache.oozie.action.hadoop.JavaMain.main(JavaMain.java:36)
> > >
> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> > > Method)
> > >
> > > at sun.reflect.NativeMethodAccessorImpl.invoke(
> > > NativeMethodAccessorImpl.java:57)
> > >
> > > at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> > > DelegatingMethodAccessorImpl.java:43)
> > >
> > > at java.lang.reflect.Method.invoke(Method.java:606)
> > >
> > > at org.apache.oozie.action.hadoop.LauncherMapper.map(
> > > LauncherMapper.java:226)
> > >
> > > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> > >
> > > at
> > > org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
> > >
> > > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> > >
> > > at
> > > org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> > >
> > > at java.security.AccessController.doPrivileged(Native
> > > Method)
> > >
> > > at javax.security.auth.Subject.doAs(Subject.java:415)
> > >
> > > at org.apache.hadoop.security.UserGroupInformation.doAs(
> > > UserGroupInformation.java:1628)
> > >
> > > at
> > > org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> > >
> > > Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException:
> > > Could not get JDBC Connection; nested exception is
> > > org.apache.commons.dbcp.SQLNestedException:
> > > Cannot create PoolableConnectionFactory (Communications link failure
> > >
> > >
> > >
> > >
> > >
> > > We have checked the port 3306 for mariadb using ‘nmap’ in linux and
> > > have checked the connection hops from node1 to node5 (using
> > > traceroute)and found that , there is only a single hop, that is,
> > > node1 is directly accessing
> > > node5 and vice-versa in the network(no switch IP shown intraceroute
> > > output or switch firewalls expected).
> > >
> > >
> > >
> > > Checked for iptables and firewalls – no firewalls are running in the
> > > cluster.
> > >
> > >
> > >
> > > *Please note:* Same job (workflow.xml) is running fine in one of our
> > > other cluster with below versions:
> > >
> > >
> > >
> > > Oozie 4.1.0 , RHEL 7.3 and database in mariadb 5.5.41.
> > >
> > >
> > >
> > >
> > >
> > > *Attaching the logs having oozie startup and the job run log having
> > > the error, with this e-mail.*
> > >
> > >
> > >
> > > *Is there any issue with oozie 4.1.0 working in RHEL 7.5 ?*
> > >
> > >
> > >
> > > Tried to install oozie 4.3.0 but I am stuck as per below screenshot,
> > > when I try to run below command:
> > >
> > >
> > >
> > > # ./mkdistro.sh –DskipTests
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > I am able to wget the apache-17.pom from the linux shell prompt but
> > > the build process fails. We have added the required proxy to access
> > > the internet. I don’t know if any proxy to be added so that the
> > > build process comes to know about the internet availability through
> > > proxy.
> > >
> > >
> > >
> > > Please suggest a solution with the shortest approach possible to
> > > solve this. Will appreciate if someone have the oozie 4.3.0 for RHEL
> > > 7.5 readily available and could share.
> > >
> > >
> > >
> > > Below is the oozie related settings:
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > WARN: Use of this script is deprecated; use 'oozied.sh stop' instead
> > >
> > >
> > >
> > > Setting OOZIE_HOME: /usr/local/oozie
> > >
> > > Setting OOZIE_CONFIG: /usr/local/oozie/conf
> > >
> > > Sourcing: /usr/local/oozie/conf/oozie-env.sh
> > >
> > > setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
> > >
> > > setting OOZIE_BASE_URL="http://${OOZIE_HTTP_HOSTNAME}:${OOZIE_
> > > HTTP_PORT}/oozie"
> > >
> > > Setting OOZIE_CONFIG_FILE: oozie-site.xml
> > >
> > > Setting OOZIE_DATA: /usr/local/oozie/data
> > >
> > > Setting OOZIE_LOG: /usr/local/oozie/logs
> > >
> > > Setting OOZIE_LOG4J_FILE: oozie-log4j.properties
> > >
> > > Setting OOZIE_LOG4J_RELOAD: 10
> > >
> > > Setting OOZIE_HTTP_HOSTNAME: svdt5neonhadoop01.safaricom.net
> > >
> > > Setting OOZIE_HTTP_PORT: 11000
> > >
> > > Setting OOZIE_ADMIN_PORT: 11001
> > >
> > > Setting OOZIE_HTTPS_PORT: 11443
> > >
> > > Using OOZIE_BASE_URL: http://:/oozie
> > >
> > > Setting CATALINA_BASE: /usr/local/oozie/oozie-server
> > >
> > > Setting OOZIE_HTTPS_KEYSTORE_FILE: /home/hadoop/.keystore
> > >
> > > Setting OOZIE_HTTPS_KEYSTORE_PASS: password
> > >
> > > Setting OOZIE_INSTANCE_ID: svdt5neonhadoop01.safaricom.net
> > >
> > > Setting CATALINA_OUT: /usr/local/oozie/logs/catalina.out
> > >
> > > Setting CATALINA_PID: /usr/local/oozie/oozie-server/
> > temp/oozie.pid
> > >
> > >
> > >
> > > Using CATALINA_OPTS: -Xmx1024m -Dderby.stream.error.file=/
> > > usr/local/oozie/logs/derby.log
> > >
> > > Adding to CATALINA_OPTS: -Doozie.home.dir=/usr/local/oozie
> > > -Doozie.config.dir=/usr/local/oozie/conf
> > > -Doozie.log.dir=/usr/local/oozie/logs
> > > -Doozie.data.dir=/usr/local/oozie/data
> > > -Doozie.instance.id=svdt5neonh adoop01.safaricom.net
> > > -Doozie.config.file=oozie-site.xml
> > > -Doozie.log4j.file=oozie-log4j.properties -Doozie.log4j.reload=10
> > > -Doozie.http.hostname=svdt5neonhadoop01.safaricom.net
> > > -Doozie.admin.port=11001 -Doozie.http.port=11000
> > > -Doozie.https.port=11443 -Doozie.base.url=http://:/oozie
> > > -Doozie.https.keystore.file=/home/hadoop/.keystore
> > > -Doozie.https.keystore.pass=password -Djava.library.path=
> > >
> > > Using CATALINA_BASE: /usr/local/oozie/oozie-server
> > >
> > > Using CATALINA_HOME: /usr/local/oozie/oozie-server
> > >
> > > Using CATALINA_TMPDIR: /usr/local/oozie/oozie-server/temp
> > >
> > > Using JRE_HOME: /usr/java/jdk1.7.0_67
> > >
> > > Using CLASSPATH: /usr/local/oozie/oozie-server/bin/bootstrap.jar
> > >
> > > Using CATALINA_PID: /usr/local/oozie/oozie-server/temp/oozie.pid
> > >
> > >
> > >
> > >
> > >
> > > Thanks & Regards,
> > >
> > > *Jaboy Mathai*
> > >
> > >
> > >
> >
> >
> >
> > --
> > *Peter Cseh *| Software Engineer
> > cloudera.com <https://www.cloudera.com>
> >
> > [image: Cloudera] <https://www.cloudera.com/>
> >
> > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> > Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
> > ------------------------------
> >
>
>
>
> --
> *Peter Cseh *| Software Engineer
> cloudera.com <https://www.cloudera.com>
>
> [image: Cloudera] <https://www.cloudera.com/>
>
> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
> on LinkedIn] <https://www.linkedin.com/company/cloudera>
> ------------------------------
>
--
*Peter Cseh *| Software Engineer
cloudera.com <https://www.cloudera.com>
[image: Cloudera] <https://www.cloudera.com/>
[image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
on LinkedIn] <https://www.linkedin.com/company/cloudera>
------------------------------
RE: Reg: Oozie 4.1.0 having jdbc error
Posted by Jaboy Mathai <ja...@flytxt.com>.
Hi Peter,
Good Day!
I managed to build oozie 4.3.1 locally and tried to run job. Oozie starts up
fine. But I get below exception now in the oozie logs.
2018-06-06 00:08:45,862 INFO ActionStartXCommand:520 -
SERVER[svdt5neonhadoop01.safaricom.net] USER[hadoop] GROUP[-] TOKEN[]
APP[SAFARICOM_KENYA_CBS_RECHARGE] JOB[0000003-180605235623225-oozie-hado-W]
ACTION[0000003-180605235623225-oozie-hado-W@GetProcessedFileDetails_Recharge_infile]
Start action
[0000003-180605235623225-oozie-hado-W@GetProcessedFileDetails_Recharge_infile]
with user-retry state : userRetryCount [0], userRetryMax [0],
userRetryInterval [10]
2018-06-06 00:08:45,990 ERROR ActionStartXCommand:517 -
SERVER[svdt5neonhadoop01.safaricom.net] USER[hadoop] GROUP[-] TOKEN[]
APP[SAFARICOM_KENYA_CBS_RECHARGE] JOB[0000003-180605235623225-oozie-hado-W]
ACTION[0000003-180605235623225-oozie-hado-W@GetProcessedFileDetails_Recharge_infile]
Error,
java.lang.NoSuchMethodError:
org.apache.hadoop.yarn.util.timeline.TimelineUtils.buildTimelineTokenService(Lorg/apache/hadoop/conf/Configuration;)Lorg/apache/hadoop/io/Text;
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.<init>(ResourceMgrDelegate.java:96)
at org.apache.hadoop.mapred.YARNRunner.<init>(YARNRunner.java:112)
at
org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:75)
I have checked for the jar file having this method and found that the class
' org.apache.hadoop.yarn.util.timeline.TimelineUtils' is present in the jar
file ' hadoop-yarn-common-2.4.0.jar' but could find the jar for '
org.apache.hadoop.yarn.util.timeline.TimelineUtils.buildTimelineTokenService'
, hence don’t know that which would be the dependent jar file here.
[hadoop@svdt5neonhadoop01 lib]$
[hadoop@svdt5neonhadoop01 lib]$ egrep -i
org.apache.hadoop.yarn.util.timeline.TimelineUtils *
Binary file hadoop-yarn-common-2.4.0.jar matches
[hadoop@svdt5neonhadoop01 lib]$
[hadoop@svdt5neonhadoop01 lib]$ egrep -i
org.apache.hadoop.yarn.util.timeline.TimelineUtils.buildTimelineTokenService
*
[hadoop@svdt5neonhadoop01 lib]$
[hadoop@svdt5neonhadoop01 lib]$ ls |wc -l
160
[hadoop@svdt5neonhadoop01 lib]$ pwd
/usr/local/oozie/lib
[hadoop@svdt5neonhadoop01 lib]$
Please help !
Thanks & Regards,
Jaboy Mathai
-----Original Message-----
From: Peter Cseh [mailto:gezapeti@cloudera.com]
Sent: 01 June 2018 15:48
To: user@oozie.apache.org
Cc: Saurabh Kumar; Balakrishnan Nagiah; Vinod Rajasekharan; Shivam Garg
Subject: Re: Reg: Oozie 4.1.0 having jdbc error
I'm not aware of a binary distribution of Apache Oozie available to
download.
There are modified versions here and there (e.g. CDH's Oozie can be
installed via rpm:
https://www.cloudera.com/documentation/enterprise/5-14-x/topics/cdh_ig_cdh5_install.html
, but there are others available as well).
Peter
On Fri, Jun 1, 2018 at 11:53 AM, Jaboy Mathai <ja...@flytxt.com>
wrote:
> Dear Peter,
>
> Thanks for your reply. I am using the same Maven 3.5.3 but the java
> version is '1.7.0_67'. Will try to build version 4.3.1 , but I am not
> sure because we don’t have local environment(having RHEL) with
> internet connection as you pointed. That’s why, I requested for a
> build, that may be readily available.
> Let me try building v.4.3.1 and will let you know.
>
> Best Regards
> Jaboy
>
> -----Original Message-----
> From: Peter Cseh [mailto:gezapeti@cloudera.com]
> Sent: 01 June 2018 14:49
> To: user@oozie.apache.org
> Cc: Saurabh Kumar; Balakrishnan Nagiah; Vinod Rajasekharan; Shivam
> Garg
> Subject: Re: Reg: Oozie 4.1.0 having jdbc error
>
> Hi!
> Have you tried building 4.3.1?
> I've managed to build releases 4.3.0, 4.1.0 and 4.3.1. I've even
> cleaned my maven cache in between.
> I'm using Maven 3.5.3 with jdk1.8.
> To resolve the build issue you may compile Oozie locally and copy the
> distribution up to the server to see if it works that way. I suggest
> using Oozie 4.3.1 or 5.0.0.
>
> Based on the error messages during build (timeout) and Oozie startup
> (did not receive any packages) it looks like you're having issues with
> your network.
>
> hope it helps
> gp
>
> On Fri, Jun 1, 2018 at 7:16 AM, Jaboy Mathai <ja...@flytxt.com>
> wrote:
>
> > Hi,
> >
> >
> >
> > This is a bit urgent requirement for me, if someone has some idea ,
> > please help !
> >
> >
> >
> > BR,
> >
> > Jaboy
> >
> >
> >
> > *From:* Jaboy Mathai [mailto:jaboy.mathai@flytxt.com]
> > *Sent:* 01 June 2018 10:37
> > *To:* 'user@oozie.apache.org'
> > *Cc:* Saurabh Kumar; Balakrishnan Nagiah; Vinod Rajasekharan; Shivam
> > Garg
> > *Subject:* RE: Reg: Oozie 4.1.0 having jdbc error
> >
> >
> >
> > + Looping Shivam.
> >
> >
> >
> > *From:* Jaboy Mathai [mailto:jaboy.mathai@flytxt.com
> > <ja...@flytxt.com>]
> > *Sent:* 31 May 2018 18:15
> > *To:* 'user@oozie.apache.org'
> > *Cc:* Saurabh Kumar; Balakrishnan Nagiah; Vinod Rajasekharan
> > *Subject:* Reg: Oozie 4.1.0 having jdbc error
> >
> >
> >
> > Hi Team,
> >
> >
> >
> > Need help.
> >
> >
> >
> > Have oozie 4.1.0 installed in RHEL 7.5 in node1. In the same
> > network, we have mariadb 5.5.41 (mariadb.x86_64) in node5(earlier we
> > had version 5.5.56 but we changed that, assuming compatibility issues).
> > Oozie gets started normally , but when I try to start to run the
> > workflow, I get jdbc error in oozie.log
> >
> >
> >
> > *Logs below:*
> >
> >
> >
> > Launcher ERROR, reason: Main class
> > [org.apache.oozie.action.hadoop.JavaMain],
> > main() threw exception, org.springframework.jdbc.
> > CannotGetJdbcConnectionException: Could not get JDBC Connection;
> > nested exception is org.apache.commons.dbcp.SQLNestedException:
> > Cannot create PoolableConnectionFactory (Communications link failure
> >
> >
> >
> > The last packet sent successfully to the server was 0 milliseconds ago.
> > The driver has not received any packets from the server.)
> >
> > 2018-05-31 08:12:23,641 WARN JavaActionExecutor:544 - SERVER[
> > svdt5neonhadoop01.safaricom.net] USER[hadoop] GROUP[-] TOKEN[]
> > APP[SAFARICOM_KENYA_CBS_RECHARGE]
> > JOB[0000001-180531081106985-oozie-hado-W]
> > ACTION[0000001-180531081106985-oozie-hado-W@GetProcessedFileDetails_
> > Re charge_infile] Launcher exception: org.springframework.jdbc.
> > CannotGetJdbcConnectionException: Could not get JDBC Connection;
> > nested exception is org.apache.commons.dbcp.SQLNestedException:
> > Cannot create PoolableConnectionFactory (Communications link failure
> >
> >
> >
> > The last packet sent successfully to the server was 0 milliseconds ago.
> > The driver has not received any packets from the server.)
> >
> > at
> > org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:58)
> >
> > at org.apache.oozie.action.hadoop.LauncherMain.run(
> > LauncherMain.java:39)
> >
> > at
> > org.apache.oozie.action.hadoop.JavaMain.main(JavaMain.java:36)
> >
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> > Method)
> >
> > at sun.reflect.NativeMethodAccessorImpl.invoke(
> > NativeMethodAccessorImpl.java:57)
> >
> > at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> > DelegatingMethodAccessorImpl.java:43)
> >
> > at java.lang.reflect.Method.invoke(Method.java:606)
> >
> > at org.apache.oozie.action.hadoop.LauncherMapper.map(
> > LauncherMapper.java:226)
> >
> > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> >
> > at
> > org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
> >
> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> >
> > at
> > org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> >
> > at java.security.AccessController.doPrivileged(Native
> > Method)
> >
> > at javax.security.auth.Subject.doAs(Subject.java:415)
> >
> > at org.apache.hadoop.security.UserGroupInformation.doAs(
> > UserGroupInformation.java:1628)
> >
> > at
> > org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> >
> > Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException:
> > Could not get JDBC Connection; nested exception is
> > org.apache.commons.dbcp.SQLNestedException:
> > Cannot create PoolableConnectionFactory (Communications link failure
> >
> >
> >
> >
> >
> > We have checked the port 3306 for mariadb using ‘nmap’ in linux and
> > have checked the connection hops from node1 to node5 (using
> > traceroute)and found that , there is only a single hop, that is,
> > node1 is directly accessing
> > node5 and vice-versa in the network(no switch IP shown intraceroute
> > output or switch firewalls expected).
> >
> >
> >
> > Checked for iptables and firewalls – no firewalls are running in the
> > cluster.
> >
> >
> >
> > *Please note:* Same job (workflow.xml) is running fine in one of our
> > other cluster with below versions:
> >
> >
> >
> > Oozie 4.1.0 , RHEL 7.3 and database in mariadb 5.5.41.
> >
> >
> >
> >
> >
> > *Attaching the logs having oozie startup and the job run log having
> > the error, with this e-mail.*
> >
> >
> >
> > *Is there any issue with oozie 4.1.0 working in RHEL 7.5 ?*
> >
> >
> >
> > Tried to install oozie 4.3.0 but I am stuck as per below screenshot,
> > when I try to run below command:
> >
> >
> >
> > # ./mkdistro.sh –DskipTests
> >
> >
> >
> >
> >
> >
> >
> > I am able to wget the apache-17.pom from the linux shell prompt but
> > the build process fails. We have added the required proxy to access
> > the internet. I don’t know if any proxy to be added so that the
> > build process comes to know about the internet availability through
> > proxy.
> >
> >
> >
> > Please suggest a solution with the shortest approach possible to
> > solve this. Will appreciate if someone have the oozie 4.3.0 for RHEL
> > 7.5 readily available and could share.
> >
> >
> >
> > Below is the oozie related settings:
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > WARN: Use of this script is deprecated; use 'oozied.sh stop' instead
> >
> >
> >
> > Setting OOZIE_HOME: /usr/local/oozie
> >
> > Setting OOZIE_CONFIG: /usr/local/oozie/conf
> >
> > Sourcing: /usr/local/oozie/conf/oozie-env.sh
> >
> > setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
> >
> > setting OOZIE_BASE_URL="http://${OOZIE_HTTP_HOSTNAME}:${OOZIE_
> > HTTP_PORT}/oozie"
> >
> > Setting OOZIE_CONFIG_FILE: oozie-site.xml
> >
> > Setting OOZIE_DATA: /usr/local/oozie/data
> >
> > Setting OOZIE_LOG: /usr/local/oozie/logs
> >
> > Setting OOZIE_LOG4J_FILE: oozie-log4j.properties
> >
> > Setting OOZIE_LOG4J_RELOAD: 10
> >
> > Setting OOZIE_HTTP_HOSTNAME: svdt5neonhadoop01.safaricom.net
> >
> > Setting OOZIE_HTTP_PORT: 11000
> >
> > Setting OOZIE_ADMIN_PORT: 11001
> >
> > Setting OOZIE_HTTPS_PORT: 11443
> >
> > Using OOZIE_BASE_URL: http://:/oozie
> >
> > Setting CATALINA_BASE: /usr/local/oozie/oozie-server
> >
> > Setting OOZIE_HTTPS_KEYSTORE_FILE: /home/hadoop/.keystore
> >
> > Setting OOZIE_HTTPS_KEYSTORE_PASS: password
> >
> > Setting OOZIE_INSTANCE_ID: svdt5neonhadoop01.safaricom.net
> >
> > Setting CATALINA_OUT: /usr/local/oozie/logs/catalina.out
> >
> > Setting CATALINA_PID: /usr/local/oozie/oozie-server/
> temp/oozie.pid
> >
> >
> >
> > Using CATALINA_OPTS: -Xmx1024m -Dderby.stream.error.file=/
> > usr/local/oozie/logs/derby.log
> >
> > Adding to CATALINA_OPTS: -Doozie.home.dir=/usr/local/oozie
> > -Doozie.config.dir=/usr/local/oozie/conf
> > -Doozie.log.dir=/usr/local/oozie/logs
> > -Doozie.data.dir=/usr/local/oozie/data
> > -Doozie.instance.id=svdt5neonh adoop01.safaricom.net
> > -Doozie.config.file=oozie-site.xml
> > -Doozie.log4j.file=oozie-log4j.properties -Doozie.log4j.reload=10
> > -Doozie.http.hostname=svdt5neonhadoop01.safaricom.net
> > -Doozie.admin.port=11001 -Doozie.http.port=11000
> > -Doozie.https.port=11443 -Doozie.base.url=http://:/oozie
> > -Doozie.https.keystore.file=/home/hadoop/.keystore
> > -Doozie.https.keystore.pass=password -Djava.library.path=
> >
> > Using CATALINA_BASE: /usr/local/oozie/oozie-server
> >
> > Using CATALINA_HOME: /usr/local/oozie/oozie-server
> >
> > Using CATALINA_TMPDIR: /usr/local/oozie/oozie-server/temp
> >
> > Using JRE_HOME: /usr/java/jdk1.7.0_67
> >
> > Using CLASSPATH: /usr/local/oozie/oozie-server/bin/bootstrap.jar
> >
> > Using CATALINA_PID: /usr/local/oozie/oozie-server/temp/oozie.pid
> >
> >
> >
> >
> >
> > Thanks & Regards,
> >
> > *Jaboy Mathai*
> >
> >
> >
>
>
>
> --
> *Peter Cseh *| Software Engineer
> cloudera.com <https://www.cloudera.com>
>
> [image: Cloudera] <https://www.cloudera.com/>
>
> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera>
> ------------------------------
>
--
*Peter Cseh *| Software Engineer
cloudera.com <https://www.cloudera.com>
[image: Cloudera] <https://www.cloudera.com/>
[image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
on LinkedIn] <https://www.linkedin.com/company/cloudera>
------------------------------
Re: Reg: Oozie 4.1.0 having jdbc error
Posted by Peter Cseh <ge...@cloudera.com>.
I'm not aware of a binary distribution of Apache Oozie available to
download.
There are modified versions here and there (e.g. CDH's Oozie can be
installed via rpm:
https://www.cloudera.com/documentation/enterprise/5-14-x/topics/cdh_ig_cdh5_install.html
, but there are others available as well).
Peter
On Fri, Jun 1, 2018 at 11:53 AM, Jaboy Mathai <ja...@flytxt.com>
wrote:
> Dear Peter,
>
> Thanks for your reply. I am using the same Maven 3.5.3 but the java version
> is '1.7.0_67'. Will try to build version 4.3.1 , but I am not sure because
> we don’t have local environment(having RHEL) with internet connection as
> you
> pointed. That’s why, I requested for a build, that may be readily
> available.
> Let me try building v.4.3.1 and will let you know.
>
> Best Regards
> Jaboy
>
> -----Original Message-----
> From: Peter Cseh [mailto:gezapeti@cloudera.com]
> Sent: 01 June 2018 14:49
> To: user@oozie.apache.org
> Cc: Saurabh Kumar; Balakrishnan Nagiah; Vinod Rajasekharan; Shivam Garg
> Subject: Re: Reg: Oozie 4.1.0 having jdbc error
>
> Hi!
> Have you tried building 4.3.1?
> I've managed to build releases 4.3.0, 4.1.0 and 4.3.1. I've even cleaned my
> maven cache in between.
> I'm using Maven 3.5.3 with jdk1.8.
> To resolve the build issue you may compile Oozie locally and copy the
> distribution up to the server to see if it works that way. I suggest using
> Oozie 4.3.1 or 5.0.0.
>
> Based on the error messages during build (timeout) and Oozie startup (did
> not receive any packages) it looks like you're having issues with your
> network.
>
> hope it helps
> gp
>
> On Fri, Jun 1, 2018 at 7:16 AM, Jaboy Mathai <ja...@flytxt.com>
> wrote:
>
> > Hi,
> >
> >
> >
> > This is a bit urgent requirement for me, if someone has some idea ,
> > please help !
> >
> >
> >
> > BR,
> >
> > Jaboy
> >
> >
> >
> > *From:* Jaboy Mathai [mailto:jaboy.mathai@flytxt.com]
> > *Sent:* 01 June 2018 10:37
> > *To:* 'user@oozie.apache.org'
> > *Cc:* Saurabh Kumar; Balakrishnan Nagiah; Vinod Rajasekharan; Shivam
> > Garg
> > *Subject:* RE: Reg: Oozie 4.1.0 having jdbc error
> >
> >
> >
> > + Looping Shivam.
> >
> >
> >
> > *From:* Jaboy Mathai [mailto:jaboy.mathai@flytxt.com
> > <ja...@flytxt.com>]
> > *Sent:* 31 May 2018 18:15
> > *To:* 'user@oozie.apache.org'
> > *Cc:* Saurabh Kumar; Balakrishnan Nagiah; Vinod Rajasekharan
> > *Subject:* Reg: Oozie 4.1.0 having jdbc error
> >
> >
> >
> > Hi Team,
> >
> >
> >
> > Need help.
> >
> >
> >
> > Have oozie 4.1.0 installed in RHEL 7.5 in node1. In the same network,
> > we have mariadb 5.5.41 (mariadb.x86_64) in node5(earlier we had
> > version 5.5.56 but we changed that, assuming compatibility issues).
> > Oozie gets started normally , but when I try to start to run the
> > workflow, I get jdbc error in oozie.log
> >
> >
> >
> > *Logs below:*
> >
> >
> >
> > Launcher ERROR, reason: Main class
> > [org.apache.oozie.action.hadoop.JavaMain],
> > main() threw exception, org.springframework.jdbc.
> > CannotGetJdbcConnectionException: Could not get JDBC Connection;
> > nested exception is org.apache.commons.dbcp.SQLNestedException: Cannot
> > create PoolableConnectionFactory (Communications link failure
> >
> >
> >
> > The last packet sent successfully to the server was 0 milliseconds ago.
> > The driver has not received any packets from the server.)
> >
> > 2018-05-31 08:12:23,641 WARN JavaActionExecutor:544 - SERVER[
> > svdt5neonhadoop01.safaricom.net] USER[hadoop] GROUP[-] TOKEN[]
> > APP[SAFARICOM_KENYA_CBS_RECHARGE]
> > JOB[0000001-180531081106985-oozie-hado-W]
> > ACTION[0000001-180531081106985-oozie-hado-W@GetProcessedFileDetails_Re
> > charge_infile] Launcher exception: org.springframework.jdbc.
> > CannotGetJdbcConnectionException: Could not get JDBC Connection;
> > nested exception is org.apache.commons.dbcp.SQLNestedException: Cannot
> > create PoolableConnectionFactory (Communications link failure
> >
> >
> >
> > The last packet sent successfully to the server was 0 milliseconds ago.
> > The driver has not received any packets from the server.)
> >
> > at
> > org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:58)
> >
> > at org.apache.oozie.action.hadoop.LauncherMain.run(
> > LauncherMain.java:39)
> >
> > at
> > org.apache.oozie.action.hadoop.JavaMain.main(JavaMain.java:36)
> >
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >
> > at sun.reflect.NativeMethodAccessorImpl.invoke(
> > NativeMethodAccessorImpl.java:57)
> >
> > at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> > DelegatingMethodAccessorImpl.java:43)
> >
> > at java.lang.reflect.Method.invoke(Method.java:606)
> >
> > at org.apache.oozie.action.hadoop.LauncherMapper.map(
> > LauncherMapper.java:226)
> >
> > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> >
> > at
> > org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
> >
> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> >
> > at
> > org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> >
> > at java.security.AccessController.doPrivileged(Native Method)
> >
> > at javax.security.auth.Subject.doAs(Subject.java:415)
> >
> > at org.apache.hadoop.security.UserGroupInformation.doAs(
> > UserGroupInformation.java:1628)
> >
> > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> >
> > Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException:
> > Could not get JDBC Connection; nested exception is
> > org.apache.commons.dbcp.SQLNestedException:
> > Cannot create PoolableConnectionFactory (Communications link failure
> >
> >
> >
> >
> >
> > We have checked the port 3306 for mariadb using ‘nmap’ in linux and
> > have checked the connection hops from node1 to node5 (using
> > traceroute)and found that , there is only a single hop, that is, node1
> > is directly accessing
> > node5 and vice-versa in the network(no switch IP shown intraceroute
> > output or switch firewalls expected).
> >
> >
> >
> > Checked for iptables and firewalls – no firewalls are running in the
> > cluster.
> >
> >
> >
> > *Please note:* Same job (workflow.xml) is running fine in one of our
> > other cluster with below versions:
> >
> >
> >
> > Oozie 4.1.0 , RHEL 7.3 and database in mariadb 5.5.41.
> >
> >
> >
> >
> >
> > *Attaching the logs having oozie startup and the job run log having
> > the error, with this e-mail.*
> >
> >
> >
> > *Is there any issue with oozie 4.1.0 working in RHEL 7.5 ?*
> >
> >
> >
> > Tried to install oozie 4.3.0 but I am stuck as per below screenshot,
> > when I try to run below command:
> >
> >
> >
> > # ./mkdistro.sh –DskipTests
> >
> >
> >
> >
> >
> >
> >
> > I am able to wget the apache-17.pom from the linux shell prompt but
> > the build process fails. We have added the required proxy to access
> > the internet. I don’t know if any proxy to be added so that the build
> > process comes to know about the internet availability through proxy.
> >
> >
> >
> > Please suggest a solution with the shortest approach possible to solve
> > this. Will appreciate if someone have the oozie 4.3.0 for RHEL 7.5
> > readily available and could share.
> >
> >
> >
> > Below is the oozie related settings:
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > WARN: Use of this script is deprecated; use 'oozied.sh stop' instead
> >
> >
> >
> > Setting OOZIE_HOME: /usr/local/oozie
> >
> > Setting OOZIE_CONFIG: /usr/local/oozie/conf
> >
> > Sourcing: /usr/local/oozie/conf/oozie-env.sh
> >
> > setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
> >
> > setting OOZIE_BASE_URL="http://${OOZIE_HTTP_HOSTNAME}:${OOZIE_
> > HTTP_PORT}/oozie"
> >
> > Setting OOZIE_CONFIG_FILE: oozie-site.xml
> >
> > Setting OOZIE_DATA: /usr/local/oozie/data
> >
> > Setting OOZIE_LOG: /usr/local/oozie/logs
> >
> > Setting OOZIE_LOG4J_FILE: oozie-log4j.properties
> >
> > Setting OOZIE_LOG4J_RELOAD: 10
> >
> > Setting OOZIE_HTTP_HOSTNAME: svdt5neonhadoop01.safaricom.net
> >
> > Setting OOZIE_HTTP_PORT: 11000
> >
> > Setting OOZIE_ADMIN_PORT: 11001
> >
> > Setting OOZIE_HTTPS_PORT: 11443
> >
> > Using OOZIE_BASE_URL: http://:/oozie
> >
> > Setting CATALINA_BASE: /usr/local/oozie/oozie-server
> >
> > Setting OOZIE_HTTPS_KEYSTORE_FILE: /home/hadoop/.keystore
> >
> > Setting OOZIE_HTTPS_KEYSTORE_PASS: password
> >
> > Setting OOZIE_INSTANCE_ID: svdt5neonhadoop01.safaricom.net
> >
> > Setting CATALINA_OUT: /usr/local/oozie/logs/catalina.out
> >
> > Setting CATALINA_PID: /usr/local/oozie/oozie-server/
> temp/oozie.pid
> >
> >
> >
> > Using CATALINA_OPTS: -Xmx1024m -Dderby.stream.error.file=/
> > usr/local/oozie/logs/derby.log
> >
> > Adding to CATALINA_OPTS: -Doozie.home.dir=/usr/local/oozie
> > -Doozie.config.dir=/usr/local/oozie/conf
> > -Doozie.log.dir=/usr/local/oozie/logs
> > -Doozie.data.dir=/usr/local/oozie/data -Doozie.instance.id=svdt5neonh
> > adoop01.safaricom.net -Doozie.config.file=oozie-site.xml
> > -Doozie.log4j.file=oozie-log4j.properties -Doozie.log4j.reload=10
> > -Doozie.http.hostname=svdt5neonhadoop01.safaricom.net
> > -Doozie.admin.port=11001 -Doozie.http.port=11000
> > -Doozie.https.port=11443 -Doozie.base.url=http://:/oozie
> > -Doozie.https.keystore.file=/home/hadoop/.keystore
> > -Doozie.https.keystore.pass=password -Djava.library.path=
> >
> > Using CATALINA_BASE: /usr/local/oozie/oozie-server
> >
> > Using CATALINA_HOME: /usr/local/oozie/oozie-server
> >
> > Using CATALINA_TMPDIR: /usr/local/oozie/oozie-server/temp
> >
> > Using JRE_HOME: /usr/java/jdk1.7.0_67
> >
> > Using CLASSPATH: /usr/local/oozie/oozie-server/bin/bootstrap.jar
> >
> > Using CATALINA_PID: /usr/local/oozie/oozie-server/temp/oozie.pid
> >
> >
> >
> >
> >
> > Thanks & Regards,
> >
> > *Jaboy Mathai*
> >
> >
> >
>
>
>
> --
> *Peter Cseh *| Software Engineer
> cloudera.com <https://www.cloudera.com>
>
> [image: Cloudera] <https://www.cloudera.com/>
>
> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
> on LinkedIn] <https://www.linkedin.com/company/cloudera>
> ------------------------------
>
--
*Peter Cseh *| Software Engineer
cloudera.com <https://www.cloudera.com>
[image: Cloudera] <https://www.cloudera.com/>
[image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
on LinkedIn] <https://www.linkedin.com/company/cloudera>
------------------------------
RE: Reg: Oozie 4.1.0 having jdbc error
Posted by Jaboy Mathai <ja...@flytxt.com>.
Dear Peter,
Thanks for your reply. I am using the same Maven 3.5.3 but the java version
is '1.7.0_67'. Will try to build version 4.3.1 , but I am not sure because
we don’t have local environment(having RHEL) with internet connection as you
pointed. That’s why, I requested for a build, that may be readily available.
Let me try building v.4.3.1 and will let you know.
Best Regards
Jaboy
-----Original Message-----
From: Peter Cseh [mailto:gezapeti@cloudera.com]
Sent: 01 June 2018 14:49
To: user@oozie.apache.org
Cc: Saurabh Kumar; Balakrishnan Nagiah; Vinod Rajasekharan; Shivam Garg
Subject: Re: Reg: Oozie 4.1.0 having jdbc error
Hi!
Have you tried building 4.3.1?
I've managed to build releases 4.3.0, 4.1.0 and 4.3.1. I've even cleaned my
maven cache in between.
I'm using Maven 3.5.3 with jdk1.8.
To resolve the build issue you may compile Oozie locally and copy the
distribution up to the server to see if it works that way. I suggest using
Oozie 4.3.1 or 5.0.0.
Based on the error messages during build (timeout) and Oozie startup (did
not receive any packages) it looks like you're having issues with your
network.
hope it helps
gp
On Fri, Jun 1, 2018 at 7:16 AM, Jaboy Mathai <ja...@flytxt.com>
wrote:
> Hi,
>
>
>
> This is a bit urgent requirement for me, if someone has some idea ,
> please help !
>
>
>
> BR,
>
> Jaboy
>
>
>
> *From:* Jaboy Mathai [mailto:jaboy.mathai@flytxt.com]
> *Sent:* 01 June 2018 10:37
> *To:* 'user@oozie.apache.org'
> *Cc:* Saurabh Kumar; Balakrishnan Nagiah; Vinod Rajasekharan; Shivam
> Garg
> *Subject:* RE: Reg: Oozie 4.1.0 having jdbc error
>
>
>
> + Looping Shivam.
>
>
>
> *From:* Jaboy Mathai [mailto:jaboy.mathai@flytxt.com
> <ja...@flytxt.com>]
> *Sent:* 31 May 2018 18:15
> *To:* 'user@oozie.apache.org'
> *Cc:* Saurabh Kumar; Balakrishnan Nagiah; Vinod Rajasekharan
> *Subject:* Reg: Oozie 4.1.0 having jdbc error
>
>
>
> Hi Team,
>
>
>
> Need help.
>
>
>
> Have oozie 4.1.0 installed in RHEL 7.5 in node1. In the same network,
> we have mariadb 5.5.41 (mariadb.x86_64) in node5(earlier we had
> version 5.5.56 but we changed that, assuming compatibility issues).
> Oozie gets started normally , but when I try to start to run the
> workflow, I get jdbc error in oozie.log
>
>
>
> *Logs below:*
>
>
>
> Launcher ERROR, reason: Main class
> [org.apache.oozie.action.hadoop.JavaMain],
> main() threw exception, org.springframework.jdbc.
> CannotGetJdbcConnectionException: Could not get JDBC Connection;
> nested exception is org.apache.commons.dbcp.SQLNestedException: Cannot
> create PoolableConnectionFactory (Communications link failure
>
>
>
> The last packet sent successfully to the server was 0 milliseconds ago.
> The driver has not received any packets from the server.)
>
> 2018-05-31 08:12:23,641 WARN JavaActionExecutor:544 - SERVER[
> svdt5neonhadoop01.safaricom.net] USER[hadoop] GROUP[-] TOKEN[]
> APP[SAFARICOM_KENYA_CBS_RECHARGE]
> JOB[0000001-180531081106985-oozie-hado-W]
> ACTION[0000001-180531081106985-oozie-hado-W@GetProcessedFileDetails_Re
> charge_infile] Launcher exception: org.springframework.jdbc.
> CannotGetJdbcConnectionException: Could not get JDBC Connection;
> nested exception is org.apache.commons.dbcp.SQLNestedException: Cannot
> create PoolableConnectionFactory (Communications link failure
>
>
>
> The last packet sent successfully to the server was 0 milliseconds ago.
> The driver has not received any packets from the server.)
>
> at
> org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:58)
>
> at org.apache.oozie.action.hadoop.LauncherMain.run(
> LauncherMain.java:39)
>
> at
> org.apache.oozie.action.hadoop.JavaMain.main(JavaMain.java:36)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:57)
>
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:606)
>
> at org.apache.oozie.action.hadoop.LauncherMapper.map(
> LauncherMapper.java:226)
>
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
>
> at
> org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
>
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
>
> at
> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
>
> at java.security.AccessController.doPrivileged(Native Method)
>
> at javax.security.auth.Subject.doAs(Subject.java:415)
>
> at org.apache.hadoop.security.UserGroupInformation.doAs(
> UserGroupInformation.java:1628)
>
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
>
> Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException:
> Could not get JDBC Connection; nested exception is
> org.apache.commons.dbcp.SQLNestedException:
> Cannot create PoolableConnectionFactory (Communications link failure
>
>
>
>
>
> We have checked the port 3306 for mariadb using ‘nmap’ in linux and
> have checked the connection hops from node1 to node5 (using
> traceroute)and found that , there is only a single hop, that is, node1
> is directly accessing
> node5 and vice-versa in the network(no switch IP shown intraceroute
> output or switch firewalls expected).
>
>
>
> Checked for iptables and firewalls – no firewalls are running in the
> cluster.
>
>
>
> *Please note:* Same job (workflow.xml) is running fine in one of our
> other cluster with below versions:
>
>
>
> Oozie 4.1.0 , RHEL 7.3 and database in mariadb 5.5.41.
>
>
>
>
>
> *Attaching the logs having oozie startup and the job run log having
> the error, with this e-mail.*
>
>
>
> *Is there any issue with oozie 4.1.0 working in RHEL 7.5 ?*
>
>
>
> Tried to install oozie 4.3.0 but I am stuck as per below screenshot,
> when I try to run below command:
>
>
>
> # ./mkdistro.sh –DskipTests
>
>
>
>
>
>
>
> I am able to wget the apache-17.pom from the linux shell prompt but
> the build process fails. We have added the required proxy to access
> the internet. I don’t know if any proxy to be added so that the build
> process comes to know about the internet availability through proxy.
>
>
>
> Please suggest a solution with the shortest approach possible to solve
> this. Will appreciate if someone have the oozie 4.3.0 for RHEL 7.5
> readily available and could share.
>
>
>
> Below is the oozie related settings:
>
>
>
>
>
>
>
>
>
> WARN: Use of this script is deprecated; use 'oozied.sh stop' instead
>
>
>
> Setting OOZIE_HOME: /usr/local/oozie
>
> Setting OOZIE_CONFIG: /usr/local/oozie/conf
>
> Sourcing: /usr/local/oozie/conf/oozie-env.sh
>
> setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
>
> setting OOZIE_BASE_URL="http://${OOZIE_HTTP_HOSTNAME}:${OOZIE_
> HTTP_PORT}/oozie"
>
> Setting OOZIE_CONFIG_FILE: oozie-site.xml
>
> Setting OOZIE_DATA: /usr/local/oozie/data
>
> Setting OOZIE_LOG: /usr/local/oozie/logs
>
> Setting OOZIE_LOG4J_FILE: oozie-log4j.properties
>
> Setting OOZIE_LOG4J_RELOAD: 10
>
> Setting OOZIE_HTTP_HOSTNAME: svdt5neonhadoop01.safaricom.net
>
> Setting OOZIE_HTTP_PORT: 11000
>
> Setting OOZIE_ADMIN_PORT: 11001
>
> Setting OOZIE_HTTPS_PORT: 11443
>
> Using OOZIE_BASE_URL: http://:/oozie
>
> Setting CATALINA_BASE: /usr/local/oozie/oozie-server
>
> Setting OOZIE_HTTPS_KEYSTORE_FILE: /home/hadoop/.keystore
>
> Setting OOZIE_HTTPS_KEYSTORE_PASS: password
>
> Setting OOZIE_INSTANCE_ID: svdt5neonhadoop01.safaricom.net
>
> Setting CATALINA_OUT: /usr/local/oozie/logs/catalina.out
>
> Setting CATALINA_PID: /usr/local/oozie/oozie-server/temp/oozie.pid
>
>
>
> Using CATALINA_OPTS: -Xmx1024m -Dderby.stream.error.file=/
> usr/local/oozie/logs/derby.log
>
> Adding to CATALINA_OPTS: -Doozie.home.dir=/usr/local/oozie
> -Doozie.config.dir=/usr/local/oozie/conf
> -Doozie.log.dir=/usr/local/oozie/logs
> -Doozie.data.dir=/usr/local/oozie/data -Doozie.instance.id=svdt5neonh
> adoop01.safaricom.net -Doozie.config.file=oozie-site.xml
> -Doozie.log4j.file=oozie-log4j.properties -Doozie.log4j.reload=10
> -Doozie.http.hostname=svdt5neonhadoop01.safaricom.net
> -Doozie.admin.port=11001 -Doozie.http.port=11000
> -Doozie.https.port=11443 -Doozie.base.url=http://:/oozie
> -Doozie.https.keystore.file=/home/hadoop/.keystore
> -Doozie.https.keystore.pass=password -Djava.library.path=
>
> Using CATALINA_BASE: /usr/local/oozie/oozie-server
>
> Using CATALINA_HOME: /usr/local/oozie/oozie-server
>
> Using CATALINA_TMPDIR: /usr/local/oozie/oozie-server/temp
>
> Using JRE_HOME: /usr/java/jdk1.7.0_67
>
> Using CLASSPATH: /usr/local/oozie/oozie-server/bin/bootstrap.jar
>
> Using CATALINA_PID: /usr/local/oozie/oozie-server/temp/oozie.pid
>
>
>
>
>
> Thanks & Regards,
>
> *Jaboy Mathai*
>
>
>
--
*Peter Cseh *| Software Engineer
cloudera.com <https://www.cloudera.com>
[image: Cloudera] <https://www.cloudera.com/>
[image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
on LinkedIn] <https://www.linkedin.com/company/cloudera>
------------------------------
Re: Reg: Oozie 4.1.0 having jdbc error
Posted by Peter Cseh <ge...@cloudera.com>.
Hi!
Have you tried building 4.3.1?
I've managed to build releases 4.3.0, 4.1.0 and 4.3.1. I've even cleaned my
maven cache in between.
I'm using Maven 3.5.3 with jdk1.8.
To resolve the build issue you may compile Oozie locally and copy the
distribution up to the server to see if it works that way. I suggest using
Oozie 4.3.1 or 5.0.0.
Based on the error messages during build (timeout) and Oozie startup (did
not receive any packages) it looks like you're having issues with your
network.
hope it helps
gp
On Fri, Jun 1, 2018 at 7:16 AM, Jaboy Mathai <ja...@flytxt.com>
wrote:
> Hi,
>
>
>
> This is a bit urgent requirement for me, if someone has some idea , please
> help !
>
>
>
> BR,
>
> Jaboy
>
>
>
> *From:* Jaboy Mathai [mailto:jaboy.mathai@flytxt.com]
> *Sent:* 01 June 2018 10:37
> *To:* 'user@oozie.apache.org'
> *Cc:* Saurabh Kumar; Balakrishnan Nagiah; Vinod Rajasekharan; Shivam Garg
> *Subject:* RE: Reg: Oozie 4.1.0 having jdbc error
>
>
>
> + Looping Shivam.
>
>
>
> *From:* Jaboy Mathai [mailto:jaboy.mathai@flytxt.com
> <ja...@flytxt.com>]
> *Sent:* 31 May 2018 18:15
> *To:* 'user@oozie.apache.org'
> *Cc:* Saurabh Kumar; Balakrishnan Nagiah; Vinod Rajasekharan
> *Subject:* Reg: Oozie 4.1.0 having jdbc error
>
>
>
> Hi Team,
>
>
>
> Need help.
>
>
>
> Have oozie 4.1.0 installed in RHEL 7.5 in node1. In the same network, we
> have mariadb 5.5.41 (mariadb.x86_64) in node5(earlier we had version 5.5.56
> but we changed that, assuming compatibility issues). Oozie gets started
> normally , but when I try to start to run the workflow, I get jdbc error in
> oozie.log
>
>
>
> *Logs below:*
>
>
>
> Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.JavaMain],
> main() threw exception, org.springframework.jdbc.
> CannotGetJdbcConnectionException: Could not get JDBC Connection; nested
> exception is org.apache.commons.dbcp.SQLNestedException: Cannot create
> PoolableConnectionFactory (Communications link failure
>
>
>
> The last packet sent successfully to the server was 0 milliseconds ago.
> The driver has not received any packets from the server.)
>
> 2018-05-31 08:12:23,641 WARN JavaActionExecutor:544 - SERVER[
> svdt5neonhadoop01.safaricom.net] USER[hadoop] GROUP[-] TOKEN[]
> APP[SAFARICOM_KENYA_CBS_RECHARGE] JOB[0000001-180531081106985-oozie-hado-W]
> ACTION[0000001-180531081106985-oozie-hado-W@GetProcessedFileDetails_Recharge_infile]
> Launcher exception: org.springframework.jdbc.
> CannotGetJdbcConnectionException: Could not get JDBC Connection; nested
> exception is org.apache.commons.dbcp.SQLNestedException: Cannot create
> PoolableConnectionFactory (Communications link failure
>
>
>
> The last packet sent successfully to the server was 0 milliseconds ago.
> The driver has not received any packets from the server.)
>
> at org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:58)
>
> at org.apache.oozie.action.hadoop.LauncherMain.run(
> LauncherMain.java:39)
>
> at org.apache.oozie.action.hadoop.JavaMain.main(JavaMain.java:36)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:57)
>
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:606)
>
> at org.apache.oozie.action.hadoop.LauncherMapper.map(
> LauncherMapper.java:226)
>
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
>
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
>
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
>
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
>
> at java.security.AccessController.doPrivileged(Native Method)
>
> at javax.security.auth.Subject.doAs(Subject.java:415)
>
> at org.apache.hadoop.security.UserGroupInformation.doAs(
> UserGroupInformation.java:1628)
>
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
>
> Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException:
> Could not get JDBC Connection; nested exception is org.apache.commons.dbcp.SQLNestedException:
> Cannot create PoolableConnectionFactory (Communications link failure
>
>
>
>
>
> We have checked the port 3306 for mariadb using ‘nmap’ in linux and have
> checked the connection hops from node1 to node5 (using traceroute)and found
> that , there is only a single hop, that is, node1 is directly accessing
> node5 and vice-versa in the network(no switch IP shown intraceroute output
> or switch firewalls expected).
>
>
>
> Checked for iptables and firewalls – no firewalls are running in the
> cluster.
>
>
>
> *Please note:* Same job (workflow.xml) is running fine in one of our
> other cluster with below versions:
>
>
>
> Oozie 4.1.0 , RHEL 7.3 and database in mariadb 5.5.41.
>
>
>
>
>
> *Attaching the logs having oozie startup and the job run log having the
> error, with this e-mail.*
>
>
>
> *Is there any issue with oozie 4.1.0 working in RHEL 7.5 ?*
>
>
>
> Tried to install oozie 4.3.0 but I am stuck as per below screenshot, when
> I try to run below command:
>
>
>
> # ./mkdistro.sh –DskipTests
>
>
>
>
>
>
>
> I am able to wget the apache-17.pom from the linux shell prompt but the
> build process fails. We have added the required proxy to access the
> internet. I don’t know if any proxy to be added so that the build process
> comes to know about the internet availability through proxy.
>
>
>
> Please suggest a solution with the shortest approach possible to solve
> this. Will appreciate if someone have the oozie 4.3.0 for RHEL 7.5 readily
> available and could share.
>
>
>
> Below is the oozie related settings:
>
>
>
>
>
>
>
>
>
> WARN: Use of this script is deprecated; use 'oozied.sh stop' instead
>
>
>
> Setting OOZIE_HOME: /usr/local/oozie
>
> Setting OOZIE_CONFIG: /usr/local/oozie/conf
>
> Sourcing: /usr/local/oozie/conf/oozie-env.sh
>
> setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
>
> setting OOZIE_BASE_URL="http://${OOZIE_HTTP_HOSTNAME}:${OOZIE_
> HTTP_PORT}/oozie"
>
> Setting OOZIE_CONFIG_FILE: oozie-site.xml
>
> Setting OOZIE_DATA: /usr/local/oozie/data
>
> Setting OOZIE_LOG: /usr/local/oozie/logs
>
> Setting OOZIE_LOG4J_FILE: oozie-log4j.properties
>
> Setting OOZIE_LOG4J_RELOAD: 10
>
> Setting OOZIE_HTTP_HOSTNAME: svdt5neonhadoop01.safaricom.net
>
> Setting OOZIE_HTTP_PORT: 11000
>
> Setting OOZIE_ADMIN_PORT: 11001
>
> Setting OOZIE_HTTPS_PORT: 11443
>
> Using OOZIE_BASE_URL: http://:/oozie
>
> Setting CATALINA_BASE: /usr/local/oozie/oozie-server
>
> Setting OOZIE_HTTPS_KEYSTORE_FILE: /home/hadoop/.keystore
>
> Setting OOZIE_HTTPS_KEYSTORE_PASS: password
>
> Setting OOZIE_INSTANCE_ID: svdt5neonhadoop01.safaricom.net
>
> Setting CATALINA_OUT: /usr/local/oozie/logs/catalina.out
>
> Setting CATALINA_PID: /usr/local/oozie/oozie-server/temp/oozie.pid
>
>
>
> Using CATALINA_OPTS: -Xmx1024m -Dderby.stream.error.file=/
> usr/local/oozie/logs/derby.log
>
> Adding to CATALINA_OPTS: -Doozie.home.dir=/usr/local/oozie
> -Doozie.config.dir=/usr/local/oozie/conf -Doozie.log.dir=/usr/local/oozie/logs
> -Doozie.data.dir=/usr/local/oozie/data -Doozie.instance.id=svdt5neonh
> adoop01.safaricom.net -Doozie.config.file=oozie-site.xml
> -Doozie.log4j.file=oozie-log4j.properties -Doozie.log4j.reload=10
> -Doozie.http.hostname=svdt5neonhadoop01.safaricom.net
> -Doozie.admin.port=11001 -Doozie.http.port=11000 -Doozie.https.port=11443
> -Doozie.base.url=http://:/oozie -Doozie.https.keystore.file=/home/hadoop/.keystore
> -Doozie.https.keystore.pass=password -Djava.library.path=
>
> Using CATALINA_BASE: /usr/local/oozie/oozie-server
>
> Using CATALINA_HOME: /usr/local/oozie/oozie-server
>
> Using CATALINA_TMPDIR: /usr/local/oozie/oozie-server/temp
>
> Using JRE_HOME: /usr/java/jdk1.7.0_67
>
> Using CLASSPATH: /usr/local/oozie/oozie-server/bin/bootstrap.jar
>
> Using CATALINA_PID: /usr/local/oozie/oozie-server/temp/oozie.pid
>
>
>
>
>
> Thanks & Regards,
>
> *Jaboy Mathai*
>
>
>
--
*Peter Cseh *| Software Engineer
cloudera.com <https://www.cloudera.com>
[image: Cloudera] <https://www.cloudera.com/>
[image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
on LinkedIn] <https://www.linkedin.com/company/cloudera>
------------------------------
RE: Reg: Oozie 4.1.0 having jdbc error
Posted by Jaboy Mathai <ja...@flytxt.com>.
Hi,
This is a bit urgent requirement for me, if someone has some idea , please
help !
BR,
Jaboy
*From:* Jaboy Mathai [mailto:jaboy.mathai@flytxt.com]
*Sent:* 01 June 2018 10:37
*To:* 'user@oozie.apache.org'
*Cc:* Saurabh Kumar; Balakrishnan Nagiah; Vinod Rajasekharan; Shivam Garg
*Subject:* RE: Reg: Oozie 4.1.0 having jdbc error
+ Looping Shivam.
*From:* Jaboy Mathai [mailto:jaboy.mathai@flytxt.com
<ja...@flytxt.com>]
*Sent:* 31 May 2018 18:15
*To:* 'user@oozie.apache.org'
*Cc:* Saurabh Kumar; Balakrishnan Nagiah; Vinod Rajasekharan
*Subject:* Reg: Oozie 4.1.0 having jdbc error
Hi Team,
Need help.
Have oozie 4.1.0 installed in RHEL 7.5 in node1. In the same network, we
have mariadb 5.5.41 (mariadb.x86_64) in node5(earlier we had version 5.5.56
but we changed that, assuming compatibility issues). Oozie gets started
normally , but when I try to start to run the workflow, I get jdbc error in
oozie.log
*Logs below:*
Launcher ERROR, reason: Main class
[org.apache.oozie.action.hadoop.JavaMain], main() threw exception,
org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get
JDBC Connection; nested exception is
org.apache.commons.dbcp.SQLNestedException: Cannot create
PoolableConnectionFactory (Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The
driver has not received any packets from the server.)
2018-05-31 08:12:23,641 WARN JavaActionExecutor:544 - SERVER[
svdt5neonhadoop01.safaricom.net] USER[hadoop] GROUP[-] TOKEN[]
APP[SAFARICOM_KENYA_CBS_RECHARGE] JOB[0000001-180531081106985-oozie-hado-W]
ACTION[0000001-180531081106985-oozie-hado-W@GetProcessedFileDetails_Recharge_infile]
Launcher exception:
org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get
JDBC Connection; nested exception is
org.apache.commons.dbcp.SQLNestedException: Cannot create
PoolableConnectionFactory (Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The
driver has not received any packets from the server.)
at org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:58)
at
org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:39)
at org.apache.oozie.action.hadoop.JavaMain.main(JavaMain.java:36)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:226)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could
not get JDBC Connection; nested exception is
org.apache.commons.dbcp.SQLNestedException: Cannot create
PoolableConnectionFactory (Communications link failure
We have checked the port 3306 for mariadb using ‘nmap’ in linux and have
checked the connection hops from node1 to node5 (using traceroute)and found
that , there is only a single hop, that is, node1 is directly accessing
node5 and vice-versa in the network(no switch IP shown intraceroute output
or switch firewalls expected).
Checked for iptables and firewalls – no firewalls are running in the
cluster.
*Please note:* Same job (workflow.xml) is running fine in one of our other
cluster with below versions:
Oozie 4.1.0 , RHEL 7.3 and database in mariadb 5.5.41.
*Attaching the logs having oozie startup and the job run log having the
error, with this e-mail.*
*Is there any issue with oozie 4.1.0 working in RHEL 7.5 ?*
Tried to install oozie 4.3.0 but I am stuck as per below screenshot, when I
try to run below command:
# ./mkdistro.sh –DskipTests
I am able to wget the apache-17.pom from the linux shell prompt but the
build process fails. We have added the required proxy to access the
internet. I don’t know if any proxy to be added so that the build process
comes to know about the internet availability through proxy.
Please suggest a solution with the shortest approach possible to solve
this. Will appreciate if someone have the oozie 4.3.0 for RHEL 7.5 readily
available and could share.
Below is the oozie related settings:
WARN: Use of this script is deprecated; use 'oozied.sh stop' instead
Setting OOZIE_HOME: /usr/local/oozie
Setting OOZIE_CONFIG: /usr/local/oozie/conf
Sourcing: /usr/local/oozie/conf/oozie-env.sh
setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
setting OOZIE_BASE_URL="
http://${OOZIE_HTTP_HOSTNAME}:${OOZIE_HTTP_PORT}/oozie"
Setting OOZIE_CONFIG_FILE: oozie-site.xml
Setting OOZIE_DATA: /usr/local/oozie/data
Setting OOZIE_LOG: /usr/local/oozie/logs
Setting OOZIE_LOG4J_FILE: oozie-log4j.properties
Setting OOZIE_LOG4J_RELOAD: 10
Setting OOZIE_HTTP_HOSTNAME: svdt5neonhadoop01.safaricom.net
Setting OOZIE_HTTP_PORT: 11000
Setting OOZIE_ADMIN_PORT: 11001
Setting OOZIE_HTTPS_PORT: 11443
Using OOZIE_BASE_URL: http://:/oozie
Setting CATALINA_BASE: /usr/local/oozie/oozie-server
Setting OOZIE_HTTPS_KEYSTORE_FILE: /home/hadoop/.keystore
Setting OOZIE_HTTPS_KEYSTORE_PASS: password
Setting OOZIE_INSTANCE_ID: svdt5neonhadoop01.safaricom.net
Setting CATALINA_OUT: /usr/local/oozie/logs/catalina.out
Setting CATALINA_PID: /usr/local/oozie/oozie-server/temp/oozie.pid
Using CATALINA_OPTS: -Xmx1024m
-Dderby.stream.error.file=/usr/local/oozie/logs/derby.log
Adding to CATALINA_OPTS: -Doozie.home.dir=/usr/local/oozie
-Doozie.config.dir=/usr/local/oozie/conf
-Doozie.log.dir=/usr/local/oozie/logs
-Doozie.data.dir=/usr/local/oozie/data -Doozie.instance.id=
svdt5neonhadoop01.safaricom.net -Doozie.config.file=oozie-site.xml
-Doozie.log4j.file=oozie-log4j.properties -Doozie.log4j.reload=10
-Doozie.http.hostname=svdt5neonhadoop01.safaricom.net
-Doozie.admin.port=11001 -Doozie.http.port=11000 -Doozie.https.port=11443
-Doozie.base.url=http://:/oozie
-Doozie.https.keystore.file=/home/hadoop/.keystore
-Doozie.https.keystore.pass=password -Djava.library.path=
Using CATALINA_BASE: /usr/local/oozie/oozie-server
Using CATALINA_HOME: /usr/local/oozie/oozie-server
Using CATALINA_TMPDIR: /usr/local/oozie/oozie-server/temp
Using JRE_HOME: /usr/java/jdk1.7.0_67
Using CLASSPATH: /usr/local/oozie/oozie-server/bin/bootstrap.jar
Using CATALINA_PID: /usr/local/oozie/oozie-server/temp/oozie.pid
Thanks & Regards,
*Jaboy Mathai*
RE: Reg: Oozie 4.1.0 having jdbc error
Posted by Jaboy Mathai <ja...@flytxt.com>.
+ Looping Shivam.
*From:* Jaboy Mathai [mailto:jaboy.mathai@flytxt.com]
*Sent:* 31 May 2018 18:15
*To:* 'user@oozie.apache.org'
*Cc:* Saurabh Kumar; Balakrishnan Nagiah; Vinod Rajasekharan
*Subject:* Reg: Oozie 4.1.0 having jdbc error
Hi Team,
Need help.
Have oozie 4.1.0 installed in RHEL 7.5 in node1. In the same network, we
have mariadb 5.5.41 (mariadb.x86_64) in node5(earlier we had version 5.5.56
but we changed that, assuming compatibility issues). Oozie gets started
normally , but when I try to start to run the workflow, I get jdbc error in
oozie.log
*Logs below:*
Launcher ERROR, reason: Main class
[org.apache.oozie.action.hadoop.JavaMain], main() threw exception,
org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get
JDBC Connection; nested exception is
org.apache.commons.dbcp.SQLNestedException: Cannot create
PoolableConnectionFactory (Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The
driver has not received any packets from the server.)
2018-05-31 08:12:23,641 WARN JavaActionExecutor:544 - SERVER[
svdt5neonhadoop01.safaricom.net] USER[hadoop] GROUP[-] TOKEN[]
APP[SAFARICOM_KENYA_CBS_RECHARGE] JOB[0000001-180531081106985-oozie-hado-W]
ACTION[0000001-180531081106985-oozie-hado-W@GetProcessedFileDetails_Recharge_infile]
Launcher exception:
org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get
JDBC Connection; nested exception is
org.apache.commons.dbcp.SQLNestedException: Cannot create
PoolableConnectionFactory (Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The
driver has not received any packets from the server.)
at org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:58)
at
org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:39)
at org.apache.oozie.action.hadoop.JavaMain.main(JavaMain.java:36)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:226)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could
not get JDBC Connection; nested exception is
org.apache.commons.dbcp.SQLNestedException: Cannot create
PoolableConnectionFactory (Communications link failure
We have checked the port 3306 for mariadb using ‘nmap’ in linux and have
checked the connection hops from node1 to node5 (using traceroute)and found
that , there is only a single hop, that is, node1 is directly accessing
node5 and vice-versa in the network(no switch IP shown intraceroute output
or switch firewalls expected).
Checked for iptables and firewalls – no firewalls are running in the
cluster.
*Please note:* Same job (workflow.xml) is running fine in one of our other
cluster with below versions:
Oozie 4.1.0 , RHEL 7.3 and database in mariadb 5.5.41.
*Attaching the logs having oozie startup and the job run log having the
error, with this e-mail.*
*Is there any issue with oozie 4.1.0 working in RHEL 7.5 ?*
Tried to install oozie 4.3.0 but I am stuck as per below screenshot, when I
try to run below command:
# ./mkdistro.sh –DskipTests
I am able to wget the apache-17.pom from the linux shell prompt but the
build process fails. We have added the required proxy to access the
internet. I don’t know if any proxy to be added so that the build process
comes to know about the internet availability through proxy.
Please suggest a solution with the shortest approach possible to solve
this. Will appreciate if someone have the oozie 4.3.0 for RHEL 7.5 readily
available and could share.
Below is the oozie related settings:
WARN: Use of this script is deprecated; use 'oozied.sh stop' instead
Setting OOZIE_HOME: /usr/local/oozie
Setting OOZIE_CONFIG: /usr/local/oozie/conf
Sourcing: /usr/local/oozie/conf/oozie-env.sh
setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
setting OOZIE_BASE_URL="
http://${OOZIE_HTTP_HOSTNAME}:${OOZIE_HTTP_PORT}/oozie"
Setting OOZIE_CONFIG_FILE: oozie-site.xml
Setting OOZIE_DATA: /usr/local/oozie/data
Setting OOZIE_LOG: /usr/local/oozie/logs
Setting OOZIE_LOG4J_FILE: oozie-log4j.properties
Setting OOZIE_LOG4J_RELOAD: 10
Setting OOZIE_HTTP_HOSTNAME: svdt5neonhadoop01.safaricom.net
Setting OOZIE_HTTP_PORT: 11000
Setting OOZIE_ADMIN_PORT: 11001
Setting OOZIE_HTTPS_PORT: 11443
Using OOZIE_BASE_URL: http://:/oozie
Setting CATALINA_BASE: /usr/local/oozie/oozie-server
Setting OOZIE_HTTPS_KEYSTORE_FILE: /home/hadoop/.keystore
Setting OOZIE_HTTPS_KEYSTORE_PASS: password
Setting OOZIE_INSTANCE_ID: svdt5neonhadoop01.safaricom.net
Setting CATALINA_OUT: /usr/local/oozie/logs/catalina.out
Setting CATALINA_PID: /usr/local/oozie/oozie-server/temp/oozie.pid
Using CATALINA_OPTS: -Xmx1024m
-Dderby.stream.error.file=/usr/local/oozie/logs/derby.log
Adding to CATALINA_OPTS: -Doozie.home.dir=/usr/local/oozie
-Doozie.config.dir=/usr/local/oozie/conf
-Doozie.log.dir=/usr/local/oozie/logs
-Doozie.data.dir=/usr/local/oozie/data -Doozie.instance.id=
svdt5neonhadoop01.safaricom.net -Doozie.config.file=oozie-site.xml
-Doozie.log4j.file=oozie-log4j.properties -Doozie.log4j.reload=10
-Doozie.http.hostname=svdt5neonhadoop01.safaricom.net
-Doozie.admin.port=11001 -Doozie.http.port=11000 -Doozie.https.port=11443
-Doozie.base.url=http://:/oozie
-Doozie.https.keystore.file=/home/hadoop/.keystore
-Doozie.https.keystore.pass=password -Djava.library.path=
Using CATALINA_BASE: /usr/local/oozie/oozie-server
Using CATALINA_HOME: /usr/local/oozie/oozie-server
Using CATALINA_TMPDIR: /usr/local/oozie/oozie-server/temp
Using JRE_HOME: /usr/java/jdk1.7.0_67
Using CLASSPATH: /usr/local/oozie/oozie-server/bin/bootstrap.jar
Using CATALINA_PID: /usr/local/oozie/oozie-server/temp/oozie.pid
Thanks & Regards,
*Jaboy Mathai*