You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@oozie.apache.org by Jose Fernandez <jf...@sdl.com> on 2013/12/19 09:23:27 UTC
Java-Opts are completely ignored
Hi folks,
I searched through the archives and googled my brains out and I'm flat out stuck. I'm hoping someone can help me here :) We're getting pretty desperate.
CDH 4.1.3
Oozie v3.2.0
Using MR2 (YARN)
We have a Java action configured in workflow.xml thusly:
<action name="convert-file">
<java>
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<configuration>
<property>
<name>mapred.job.queue.name</name>
<value>${queueName}</value>
</property>
</configuration>
<main-class>com.sdl.hlab.infrastructure.SM2ApiResultsFileToHdfsConverter</main-class>
<java-opts>-Djava.library.path=${sapNativePath} -Dcom.sdl.sap.conf.directory="." -Xmx1900m -Dlogback.configurationFile="conf/${SAP_ENV}/log4j.xml" -Dlog4j.configuration="conf/${SAP_ENV}/log4j.xml"</java-opts>
<arg>${filePath}</arg>
<file>files/sap.properties</file>
</java>
<ok to="cleanup"/>
<error to="convert-file-Email"/>
</action>
This action works perfectly on two different environments (all same CDH and Oozie version), but not on our new third environment. In the first two environments, the java-opts block is respected and I can see it in the Job History logs in stdout under "Java System Properties". In the third environment (which is new) none of the java-opts appear except for java.library.path, which points to the wrong libraries.
I checked everywhere for entries for mapred.child.java.opts and there appears to be one entry buried in mapred-defaults.xml and that's it.
I also tried to move the java opts to a property entry, no luck.
The job log doesn't have anything interesting:
2013-12-19 00:41:05,419 INFO ActionStartXCommand:539 - USER[hdfs] GROUP[-] TOKEN[] APP[hlab-oozie-workflow] JOB[0000012-131218165312793-oozie-oozi-W] ACTION[0000012-131218165312793-oozie-oozi-W@:start:] Start action [0000012-131218165312793-oozie-oozi-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2013-12-19 00:41:05,419 WARN ActionStartXCommand:542 - USER[hdfs] GROUP[-] TOKEN[] APP[hlab-oozie-workflow] JOB[0000012-131218165312793-oozie-oozi-W] ACTION[0000012-131218165312793-oozie-oozi-W@:start:] [***0000012-131218165312793-oozie-oozi-W@:start:***]Action status=DONE
2013-12-19 00:41:05,419 WARN ActionStartXCommand:542 - USER[hdfs] GROUP[-] TOKEN[] APP[hlab-oozie-workflow] JOB[0000012-131218165312793-oozie-oozi-W] ACTION[0000012-131218165312793-oozie-oozi-W@:start:] [***0000012-131218165312793-oozie-oozi-W@:start:***]Action updated in DB!
2013-12-19 00:41:05,561 INFO ActionStartXCommand:539 - USER[hdfs] GROUP[-] TOKEN[] APP[hlab-oozie-workflow] JOB[0000012-131218165312793-oozie-oozi-W] ACTION[0000012-131218165312793-oozie-oozi-W@convert-file] Start action [0000012-131218165312793-oozie-oozi-W@convert-file] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2013-12-19 00:41:07,226 WARN JavaActionExecutor:542 - USER[hdfs] GROUP[-] TOKEN[] APP[hlab-oozie-workflow] JOB[0000012-131218165312793-oozie-oozi-W] ACTION[0000012-131218165312793-oozie-oozi-W@convert-file] credentials is null for the action
2013-12-19 00:41:09,812 INFO JavaActionExecutor:539 - USER[hdfs] GROUP[-] TOKEN[] APP[hlab-oozie-workflow] JOB[0000012-131218165312793-oozie-oozi-W] ACTION[0000012-131218165312793-oozie-oozi-W@convert-file] checking action, external ID [job_1387408789132_0015] status [RUNNING]
2013-12-19 00:41:09,815 WARN ActionStartXCommand:542 - USER[hdfs] GROUP[-] TOKEN[] APP[hlab-oozie-workflow] JOB[0000012-131218165312793-oozie-oozi-W] ACTION[0000012-131218165312793-oozie-oozi-W@convert-file] [***0000012-131218165312793-oozie-oozi-W@convert-file***]Action status=RUNNING
2013-12-19 00:41:09,815 WARN ActionStartXCommand:542 - USER[hdfs] GROUP[-] TOKEN[] APP[hlab-oozie-workflow] JOB[0000012-131218165312793-oozie-oozi-W] ACTION[0000012-131218165312793-oozie-oozi-W@convert-file] [***0000012-131218165312793-oozie-oozi-W@convert-file***]Action updated in DB!
2013-12-19 00:41:28,470 INFO CallbackServlet:539 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000012-131218165312793-oozie-oozi-W] ACTION[0000012-131218165312793-oozie-oozi-W@convert-file] callback for action [0000012-131218165312793-oozie-oozi-W@convert-file]
2013-12-19 00:41:28,755 INFO JavaActionExecutor:539 - USER[hdfs] GROUP[-] TOKEN[] APP[hlab-oozie-workflow] JOB[0000012-131218165312793-oozie-oozi-W] ACTION[0000012-131218165312793-oozie-oozi-W@convert-file] action completed, external ID [job_1387408789132_0015]
2013-12-19 00:41:28,787 WARN JavaActionExecutor:542 - USER[hdfs] GROUP[-] TOKEN[] APP[hlab-oozie-workflow] JOB[0000012-131218165312793-oozie-oozi-W] ACTION[0000012-131218165312793-oozie-oozi-W@convert-file] Launcher ERROR, reason: Main class [our main class], main() threw exception, conf/qa/our.properties (No such file or directory)
2013-12-19 00:41:28,787 WARN JavaActionExecutor:542 - USER[hdfs] GROUP[-] TOKEN[] APP[hlab-oozie-workflow] JOB[0000012-131218165312793-oozie-oozi-W] ACTION[0000012-131218165312793-oozie-oozi-W@convert-file] Launcher exception: conf/qa/our.properties (No such file or directory)
java.io.FileNotFoundException: conf/qa/our.properties (No such file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:120)
at java.io.FileInputStream.<init>(FileInputStream.java:79)
at com.sdl.sap.util.BaseConfiguration.configurationFile(Configuration.scala:78)
at com.sdl.sap.util.BaseConfiguration.loadProperties(Configuration.scala:88)
at com.sdl.sap.util.BaseConfiguration.properties$lzycompute(Configuration.scala:38)
at com.sdl.sap.util.BaseConfiguration.properties(Configuration.scala:38)
at com.sdl.sap.util.BaseConfiguration.stringValue(Configuration.scala:49)
at com.sdl.hlab.infrastructure.SM2ApiResultsFileToHdfsConverter$.buildFileInfo(SM2ApiResultsFileToHdfsConverter.scala:74)
at com.sdl.hlab.infrastructure.SM2ApiResultsFileToHdfsConverter$.main(SM2ApiResultsFileToHdfsConverter.scala:68)
at com.sdl.hlab.infrastructure.SM2ApiResultsFileToHdfsConverter.main(SM2ApiResultsFileToHdfsConverter.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:472)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:399)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:334)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
2013-12-19 00:41:28,864 INFO ActionEndXCommand:539 - USER[hdfs] GROUP[-] TOKEN[] APP[hlab-oozie-workflow] JOB[0000012-131218165312793-oozie-oozi-W] ACTION[0000012-131218165312793-oozie-oozi-W@convert-file] ERROR is considered as FAILED for SLA
2013-12-19 00:41:29,021 INFO ActionStartXCommand:539 - USER[hdfs] GROUP[-] TOKEN[] APP[hlab-oozie-workflow] JOB[0000012-131218165312793-oozie-oozi-W] ACTION[0000012-131218165312793-oozie-oozi-W@convert-file-Email] Start action [0000012-131218165312793-oozie-oozi-W@convert-file-Email] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2013-12-19 00:41:29,112 WARN ActionStartXCommand:542 - USER[hdfs] GROUP[-] TOKEN[] APP[hlab-oozie-workflow] JOB[0000012-131218165312793-oozie-oozi-W] ACTION[0000012-131218165312793-oozie-oozi-W@convert-file-Email] [***0000012-131218165312793-oozie-oozi-W@convert-file-Email***]Action status=DONE
2013-12-19 00:41:29,113 WARN ActionStartXCommand:542 - USER[hdfs] GROUP[-] TOKEN[] APP[hlab-oozie-workflow] JOB[0000012-131218165312793-oozie-oozi-W] ACTION[0000012-131218165312793-oozie-oozi-W@convert-file-Email] [***0000012-131218165312793-oozie-oozi-W@convert-file-Email***]Action updated in DB!
2013-12-19 00:41:29,259 INFO ActionStartXCommand:539 - USER[hdfs] GROUP[-] TOKEN[] APP[hlab-oozie-workflow] JOB[0000012-131218165312793-oozie-oozi-W] ACTION[0000012-131218165312793-oozie-oozi-W@convert-file-fail] Start action [0000012-131218165312793-oozie-oozi-W@convert-file-fail] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2013-12-19 00:41:29,260 WARN ActionStartXCommand:542 - USER[hdfs] GROUP[-] TOKEN[] APP[hlab-oozie-workflow] JOB[0000012-131218165312793-oozie-oozi-W] ACTION[0000012-131218165312793-oozie-oozi-W@convert-file-fail] [***0000012-131218165312793-oozie-oozi-W@convert-file-fail***]Action status=DONE
2013-12-19 00:41:29,260 WARN ActionStartXCommand:542 - USER[hdfs] GROUP[-] TOKEN[] APP[hlab-oozie-workflow] JOB[0000012-131218165312793-oozie-oozi-W] ACTION[0000012-131218165312793-oozie-oozi-W@convert-file-fail] [***0000012-131218165312793-oozie-oozi-W@convert-file-fail***]Action updated in DB!
2013-12-19 00:41:29,440 WARN CoordActionUpdateXCommand:542 - USER[hdfs] GROUP[-] TOKEN[] APP[hlab-oozie-workflow] JOB[0000012-131218165312793-oozie-oozi-W] ACTION[-] E1100: Command precondition does not hold before execution, [, coord action is null], Error Code: E1100
The FIleNotFoundException is the result of the property com.sdl.sap.conf.directory not being set.
Any help would be really appreciated!
</pre><font face="arial" size="2" color="#736F6E">
<a href="http://www.sdl.com/?utm_source=Email&utm_medium=Email%2BSignature&utm_campaign=SDL%2BStandard%2BEmail%2BSignature">
<img src="http://www.sdl.com/Content/themes/common/images/SDL_logo_strapline_GCEM_EmailSig_150x68px.jpg" border=0><br><br>www.sdl.com
</a><br><br>
<font face="arial" size="1" color="#736F6E">
<b>SDL PLC confidential, all rights reserved.</b>
If you are not the intended recipient of this mail SDL requests and requires that you delete it without acting upon or copying any of its contents,
and we further request that you advise us.<BR>
SDL PLC is a public limited company registered in England and Wales.
Registered number: 02675207.<BR>
Registered address: Globe House, Clivemont Road, Maidenhead, Berkshire SL6 7DY, UK.
</font>
This message has been scanned for malware by Websense. www.websense.com
Re: Java-Opts are completely ignored
Posted by Harsh J <ha...@cloudera.com>.
What Robert said. It can especially happen if you use Cloudera Manager
in your environment, and the CM -> MapReduce configs have "Client
Overrides" set for things like heap sizes of tasks. This
administrative enforcement settings could unfortunately cause any
overrides of the mapred.child.java.opts (and others) passed by clients
to be ignored by the TaskTrackers.
On Fri, Dec 20, 2013 at 6:28 AM, Robert Kanter <rk...@cloudera.com> wrote:
> I remember seeing something similar with java-opts where they were being
> set somewhere else (mapred-site?) and declared as final, so when Oozie
> tried to set them, it was ignored. Perhaps you’re running into the same
> problem?
>
>
> On Thu, Dec 19, 2013 at 12:23 AM, Jose Fernandez <jf...@sdl.com> wrote:
>
>> Hi folks,
>>
>> I searched through the archives and googled my brains out and I'm flat out
>> stuck. I'm hoping someone can help me here :) We're getting pretty
>> desperate.
>>
>> CDH 4.1.3
>> Oozie v3.2.0
>> Using MR2 (YARN)
>>
>> We have a Java action configured in workflow.xml thusly:
>>
>> <action name="convert-file">
>> <java>
>> <job-tracker>${jobTracker}</job-tracker>
>> <name-node>${nameNode}</name-node>
>> <configuration>
>> <property>
>> <name>mapred.job.queue.name</name>
>> <value>${queueName}</value>
>> </property>
>> </configuration>
>>
>> <main-class>com.sdl.hlab.infrastructure.SM2ApiResultsFileToHdfsConverter</main-class>
>> <java-opts>-Djava.library.path=${sapNativePath}
>> -Dcom.sdl.sap.conf.directory="." -Xmx1900m
>> -Dlogback.configurationFile="conf/${SAP_ENV}/log4j.xml"
>> -Dlog4j.configuration="conf/${SAP_ENV}/log4j.xml"</java-opts>
>> <arg>${filePath}</arg>
>> <file>files/sap.properties</file>
>> </java>
>> <ok to="cleanup"/>
>> <error to="convert-file-Email"/>
>> </action>
>>
>> This action works perfectly on two different environments (all same CDH
>> and Oozie version), but not on our new third environment. In the first two
>> environments, the java-opts block is respected and I can see it in the Job
>> History logs in stdout under "Java System Properties". In the third
>> environment (which is new) none of the java-opts appear except for
>> java.library.path, which points to the wrong libraries.
>>
>> I checked everywhere for entries for mapred.child.java.opts and there
>> appears to be one entry buried in mapred-defaults.xml and that's it.
>>
>> I also tried to move the java opts to a property entry, no luck.
>>
>> The job log doesn't have anything interesting:
>>
>> 2013-12-19 00:41:05,419 INFO ActionStartXCommand:539 - USER[hdfs]
>> GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
>> JOB[0000012-131218165312793-oozie-oozi-W]
>> ACTION[0000012-131218165312793-oozie-oozi-W@:start:] Start action
>> [0000012-131218165312793-oozie-oozi-W@:start:] with user-retry state :
>> userRetryCount [0], userRetryMax [0], userRetryInterval [10]
>> 2013-12-19 00:41:05,419 WARN ActionStartXCommand:542 - USER[hdfs]
>> GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
>> JOB[0000012-131218165312793-oozie-oozi-W]
>> ACTION[0000012-131218165312793-oozie-oozi-W@:start:]
>> [***0000012-131218165312793-oozie-oozi-W@:start:***]Action status=DONE
>> 2013-12-19 00:41:05,419 WARN ActionStartXCommand:542 - USER[hdfs]
>> GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
>> JOB[0000012-131218165312793-oozie-oozi-W]
>> ACTION[0000012-131218165312793-oozie-oozi-W@:start:]
>> [***0000012-131218165312793-oozie-oozi-W@:start:***]Action updated in DB!
>> 2013-12-19 00:41:05,561 INFO ActionStartXCommand:539 - USER[hdfs]
>> GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
>> JOB[0000012-131218165312793-oozie-oozi-W]
>> ACTION[0000012-131218165312793-oozie-oozi-W@convert-file] Start action
>> [0000012-131218165312793-oozie-oozi-W@convert-file] with user-retry state
>> : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
>> 2013-12-19 00:41:07,226 WARN JavaActionExecutor:542 - USER[hdfs] GROUP[-]
>> TOKEN[] APP[hlab-oozie-workflow] JOB[0000012-131218165312793-oozie-oozi-W]
>> ACTION[0000012-131218165312793-oozie-oozi-W@convert-file] credentials is
>> null for the action
>> 2013-12-19 00:41:09,812 INFO JavaActionExecutor:539 - USER[hdfs] GROUP[-]
>> TOKEN[] APP[hlab-oozie-workflow] JOB[0000012-131218165312793-oozie-oozi-W]
>> ACTION[0000012-131218165312793-oozie-oozi-W@convert-file] checking
>> action, external ID [job_1387408789132_0015] status [RUNNING]
>> 2013-12-19 00:41:09,815 WARN ActionStartXCommand:542 - USER[hdfs]
>> GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
>> JOB[0000012-131218165312793-oozie-oozi-W]
>> ACTION[0000012-131218165312793-oozie-oozi-W@convert-file]
>> [***0000012-131218165312793-oozie-oozi-W@convert-file***]Action
>> status=RUNNING
>> 2013-12-19 00:41:09,815 WARN ActionStartXCommand:542 - USER[hdfs]
>> GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
>> JOB[0000012-131218165312793-oozie-oozi-W]
>> ACTION[0000012-131218165312793-oozie-oozi-W@convert-file]
>> [***0000012-131218165312793-oozie-oozi-W@convert-file***]Action updated
>> in DB!
>> 2013-12-19 00:41:28,470 INFO CallbackServlet:539 - USER[-] GROUP[-]
>> TOKEN[-] APP[-] JOB[0000012-131218165312793-oozie-oozi-W]
>> ACTION[0000012-131218165312793-oozie-oozi-W@convert-file] callback for
>> action [0000012-131218165312793-oozie-oozi-W@convert-file]
>> 2013-12-19 00:41:28,755 INFO JavaActionExecutor:539 - USER[hdfs] GROUP[-]
>> TOKEN[] APP[hlab-oozie-workflow] JOB[0000012-131218165312793-oozie-oozi-W]
>> ACTION[0000012-131218165312793-oozie-oozi-W@convert-file] action
>> completed, external ID [job_1387408789132_0015]
>> 2013-12-19 00:41:28,787 WARN JavaActionExecutor:542 - USER[hdfs] GROUP[-]
>> TOKEN[] APP[hlab-oozie-workflow] JOB[0000012-131218165312793-oozie-oozi-W]
>> ACTION[0000012-131218165312793-oozie-oozi-W@convert-file] Launcher ERROR,
>> reason: Main class [our main class], main() threw exception,
>> conf/qa/our.properties (No such file or directory)
>> 2013-12-19 00:41:28,787 WARN JavaActionExecutor:542 - USER[hdfs] GROUP[-]
>> TOKEN[] APP[hlab-oozie-workflow] JOB[0000012-131218165312793-oozie-oozi-W]
>> ACTION[0000012-131218165312793-oozie-oozi-W@convert-file] Launcher
>> exception: conf/qa/our.properties (No such file or directory)
>> java.io.FileNotFoundException: conf/qa/our.properties (No such file or
>> directory)
>> at java.io.FileInputStream.open(Native Method)
>> at java.io.FileInputStream.<init>(FileInputStream.java:120)
>> at java.io.FileInputStream.<init>(FileInputStream.java:79)
>> at
>> com.sdl.sap.util.BaseConfiguration.configurationFile(Configuration.scala:78)
>> at
>> com.sdl.sap.util.BaseConfiguration.loadProperties(Configuration.scala:88)
>> at
>> com.sdl.sap.util.BaseConfiguration.properties$lzycompute(Configuration.scala:38)
>> at
>> com.sdl.sap.util.BaseConfiguration.properties(Configuration.scala:38)
>> at
>> com.sdl.sap.util.BaseConfiguration.stringValue(Configuration.scala:49)
>> at
>> com.sdl.hlab.infrastructure.SM2ApiResultsFileToHdfsConverter$.buildFileInfo(SM2ApiResultsFileToHdfsConverter.scala:74)
>> at
>> com.sdl.hlab.infrastructure.SM2ApiResultsFileToHdfsConverter$.main(SM2ApiResultsFileToHdfsConverter.scala:68)
>> at
>> com.sdl.hlab.infrastructure.SM2ApiResultsFileToHdfsConverter.main(SM2ApiResultsFileToHdfsConverter.scala)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>> Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:472)
>> at
>> org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
>> at
>> org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:399)
>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:334)
>> at
>> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
>> at java.security.AccessController.doPrivileged(Native
>> Method)
>> at javax.security.auth.Subject.doAs(Subject.java:396)
>> at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332)
>> at
>> org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
>>
>> 2013-12-19 00:41:28,864 INFO ActionEndXCommand:539 - USER[hdfs] GROUP[-]
>> TOKEN[] APP[hlab-oozie-workflow] JOB[0000012-131218165312793-oozie-oozi-W]
>> ACTION[0000012-131218165312793-oozie-oozi-W@convert-file] ERROR is
>> considered as FAILED for SLA
>> 2013-12-19 00:41:29,021 INFO ActionStartXCommand:539 - USER[hdfs]
>> GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
>> JOB[0000012-131218165312793-oozie-oozi-W]
>> ACTION[0000012-131218165312793-oozie-oozi-W@convert-file-Email] Start
>> action [0000012-131218165312793-oozie-oozi-W@convert-file-Email] with
>> user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval
>> [10]
>> 2013-12-19 00:41:29,112 WARN ActionStartXCommand:542 - USER[hdfs]
>> GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
>> JOB[0000012-131218165312793-oozie-oozi-W]
>> ACTION[0000012-131218165312793-oozie-oozi-W@convert-file-Email]
>> [***0000012-131218165312793-oozie-oozi-W@convert-file-Email***]Action
>> status=DONE
>> 2013-12-19 00:41:29,113 WARN ActionStartXCommand:542 - USER[hdfs]
>> GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
>> JOB[0000012-131218165312793-oozie-oozi-W]
>> ACTION[0000012-131218165312793-oozie-oozi-W@convert-file-Email]
>> [***0000012-131218165312793-oozie-oozi-W@convert-file-Email***]Action
>> updated in DB!
>> 2013-12-19 00:41:29,259 INFO ActionStartXCommand:539 - USER[hdfs]
>> GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
>> JOB[0000012-131218165312793-oozie-oozi-W]
>> ACTION[0000012-131218165312793-oozie-oozi-W@convert-file-fail] Start
>> action [0000012-131218165312793-oozie-oozi-W@convert-file-fail] with
>> user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval
>> [10]
>> 2013-12-19 00:41:29,260 WARN ActionStartXCommand:542 - USER[hdfs]
>> GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
>> JOB[0000012-131218165312793-oozie-oozi-W]
>> ACTION[0000012-131218165312793-oozie-oozi-W@convert-file-fail]
>> [***0000012-131218165312793-oozie-oozi-W@convert-file-fail***]Action
>> status=DONE
>> 2013-12-19 00:41:29,260 WARN ActionStartXCommand:542 - USER[hdfs]
>> GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
>> JOB[0000012-131218165312793-oozie-oozi-W]
>> ACTION[0000012-131218165312793-oozie-oozi-W@convert-file-fail]
>> [***0000012-131218165312793-oozie-oozi-W@convert-file-fail***]Action
>> updated in DB!
>> 2013-12-19 00:41:29,440 WARN CoordActionUpdateXCommand:542 - USER[hdfs]
>> GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
>> JOB[0000012-131218165312793-oozie-oozi-W] ACTION[-] E1100: Command
>> precondition does not hold before execution, [, coord action is null],
>> Error Code: E1100
>>
>> The FIleNotFoundException is the result of the property
>> com.sdl.sap.conf.directory not being set.
>>
>> Any help would be really appreciated!
>> </pre><font face="arial" size="2" color="#736F6E">
>>
>>
>>
>> <a href="
>> http://www.sdl.com/?utm_source=Email&utm_medium=Email%2BSignature&utm_campaign=SDL%2BStandard%2BEmail%2BSignature
>> ">
>> <img src="
>> http://www.sdl.com/Content/themes/common/images/SDL_logo_strapline_GCEM_EmailSig_150x68px.jpg"
>> border=0><br><br>www.sdl.com
>> </a><br><br>
>>
>> <font face="arial" size="1" color="#736F6E">
>>
>> <b>SDL PLC confidential, all rights reserved.</b>
>>
>> If you are not the intended recipient of this mail SDL requests and
>> requires that you delete it without acting upon or copying any of its
>> contents,
>> and we further request that you advise us.<BR>
>> SDL PLC is a public limited company registered in England and Wales.
>> Registered number: 02675207.<BR>
>> Registered address: Globe House, Clivemont Road, Maidenhead, Berkshire SL6
>> 7DY, UK.
>> </font>
>>
>>
>> This message has been scanned for malware by Websense. www.websense.com
>>
--
Harsh J
Re: Java-Opts are completely ignored
Posted by Serega Sheypak <se...@gmail.com>.
Did you try this one:
<action>
..
<property>
<name>*oozie.launcher.mapred.child.java.opts*</name>
<value>${oozieLauncherJVMOpts}</value>
</property>
...
</action>
https://issues.apache.org/jira/browse/OOZIE-619
2013/12/20 Robert Kanter <rk...@cloudera.com>
> I remember seeing something similar with java-opts where they were being
> set somewhere else (mapred-site?) and declared as final, so when Oozie
> tried to set them, it was ignored. Perhaps you’re running into the same
> problem?
>
>
> On Thu, Dec 19, 2013 at 12:23 AM, Jose Fernandez <jf...@sdl.com>
> wrote:
>
> > Hi folks,
> >
> > I searched through the archives and googled my brains out and I'm flat
> out
> > stuck. I'm hoping someone can help me here :) We're getting pretty
> > desperate.
> >
> > CDH 4.1.3
> > Oozie v3.2.0
> > Using MR2 (YARN)
> >
> > We have a Java action configured in workflow.xml thusly:
> >
> > <action name="convert-file">
> > <java>
> > <job-tracker>${jobTracker}</job-tracker>
> > <name-node>${nameNode}</name-node>
> > <configuration>
> > <property>
> > <name>mapred.job.queue.name</name>
> > <value>${queueName}</value>
> > </property>
> > </configuration>
> >
> >
> <main-class>com.sdl.hlab.infrastructure.SM2ApiResultsFileToHdfsConverter</main-class>
> > <java-opts>-Djava.library.path=${sapNativePath}
> > -Dcom.sdl.sap.conf.directory="." -Xmx1900m
> > -Dlogback.configurationFile="conf/${SAP_ENV}/log4j.xml"
> > -Dlog4j.configuration="conf/${SAP_ENV}/log4j.xml"</java-opts>
> > <arg>${filePath}</arg>
> > <file>files/sap.properties</file>
> > </java>
> > <ok to="cleanup"/>
> > <error to="convert-file-Email"/>
> > </action>
> >
> > This action works perfectly on two different environments (all same CDH
> > and Oozie version), but not on our new third environment. In the first
> two
> > environments, the java-opts block is respected and I can see it in the
> Job
> > History logs in stdout under "Java System Properties". In the third
> > environment (which is new) none of the java-opts appear except for
> > java.library.path, which points to the wrong libraries.
> >
> > I checked everywhere for entries for mapred.child.java.opts and there
> > appears to be one entry buried in mapred-defaults.xml and that's it.
> >
> > I also tried to move the java opts to a property entry, no luck.
> >
> > The job log doesn't have anything interesting:
> >
> > 2013-12-19 00:41:05,419 INFO ActionStartXCommand:539 - USER[hdfs]
> > GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
> > JOB[0000012-131218165312793-oozie-oozi-W]
> > ACTION[0000012-131218165312793-oozie-oozi-W@:start:] Start action
> > [0000012-131218165312793-oozie-oozi-W@:start:] with user-retry state :
> > userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> > 2013-12-19 00:41:05,419 WARN ActionStartXCommand:542 - USER[hdfs]
> > GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
> > JOB[0000012-131218165312793-oozie-oozi-W]
> > ACTION[0000012-131218165312793-oozie-oozi-W@:start:]
> > [***0000012-131218165312793-oozie-oozi-W@:start:***]Action status=DONE
> > 2013-12-19 00:41:05,419 WARN ActionStartXCommand:542 - USER[hdfs]
> > GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
> > JOB[0000012-131218165312793-oozie-oozi-W]
> > ACTION[0000012-131218165312793-oozie-oozi-W@:start:]
> > [***0000012-131218165312793-oozie-oozi-W@:start:***]Action updated in
> DB!
> > 2013-12-19 00:41:05,561 INFO ActionStartXCommand:539 - USER[hdfs]
> > GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
> > JOB[0000012-131218165312793-oozie-oozi-W]
> > ACTION[0000012-131218165312793-oozie-oozi-W@convert-file] Start action
> > [0000012-131218165312793-oozie-oozi-W@convert-file] with user-retry
> state
> > : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> > 2013-12-19 00:41:07,226 WARN JavaActionExecutor:542 - USER[hdfs]
> GROUP[-]
> > TOKEN[] APP[hlab-oozie-workflow]
> JOB[0000012-131218165312793-oozie-oozi-W]
> > ACTION[0000012-131218165312793-oozie-oozi-W@convert-file] credentials is
> > null for the action
> > 2013-12-19 00:41:09,812 INFO JavaActionExecutor:539 - USER[hdfs]
> GROUP[-]
> > TOKEN[] APP[hlab-oozie-workflow]
> JOB[0000012-131218165312793-oozie-oozi-W]
> > ACTION[0000012-131218165312793-oozie-oozi-W@convert-file] checking
> > action, external ID [job_1387408789132_0015] status [RUNNING]
> > 2013-12-19 00:41:09,815 WARN ActionStartXCommand:542 - USER[hdfs]
> > GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
> > JOB[0000012-131218165312793-oozie-oozi-W]
> > ACTION[0000012-131218165312793-oozie-oozi-W@convert-file]
> > [***0000012-131218165312793-oozie-oozi-W@convert-file***]Action
> > status=RUNNING
> > 2013-12-19 00:41:09,815 WARN ActionStartXCommand:542 - USER[hdfs]
> > GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
> > JOB[0000012-131218165312793-oozie-oozi-W]
> > ACTION[0000012-131218165312793-oozie-oozi-W@convert-file]
> > [***0000012-131218165312793-oozie-oozi-W@convert-file***]Action updated
> > in DB!
> > 2013-12-19 00:41:28,470 INFO CallbackServlet:539 - USER[-] GROUP[-]
> > TOKEN[-] APP[-] JOB[0000012-131218165312793-oozie-oozi-W]
> > ACTION[0000012-131218165312793-oozie-oozi-W@convert-file] callback for
> > action [0000012-131218165312793-oozie-oozi-W@convert-file]
> > 2013-12-19 00:41:28,755 INFO JavaActionExecutor:539 - USER[hdfs]
> GROUP[-]
> > TOKEN[] APP[hlab-oozie-workflow]
> JOB[0000012-131218165312793-oozie-oozi-W]
> > ACTION[0000012-131218165312793-oozie-oozi-W@convert-file] action
> > completed, external ID [job_1387408789132_0015]
> > 2013-12-19 00:41:28,787 WARN JavaActionExecutor:542 - USER[hdfs]
> GROUP[-]
> > TOKEN[] APP[hlab-oozie-workflow]
> JOB[0000012-131218165312793-oozie-oozi-W]
> > ACTION[0000012-131218165312793-oozie-oozi-W@convert-file] Launcher
> ERROR,
> > reason: Main class [our main class], main() threw exception,
> > conf/qa/our.properties (No such file or directory)
> > 2013-12-19 00:41:28,787 WARN JavaActionExecutor:542 - USER[hdfs]
> GROUP[-]
> > TOKEN[] APP[hlab-oozie-workflow]
> JOB[0000012-131218165312793-oozie-oozi-W]
> > ACTION[0000012-131218165312793-oozie-oozi-W@convert-file] Launcher
> > exception: conf/qa/our.properties (No such file or directory)
> > java.io.FileNotFoundException: conf/qa/our.properties (No such file or
> > directory)
> > at java.io.FileInputStream.open(Native Method)
> > at
> java.io.FileInputStream.<init>(FileInputStream.java:120)
> > at
> java.io.FileInputStream.<init>(FileInputStream.java:79)
> > at
> >
> com.sdl.sap.util.BaseConfiguration.configurationFile(Configuration.scala:78)
> > at
> > com.sdl.sap.util.BaseConfiguration.loadProperties(Configuration.scala:88)
> > at
> >
> com.sdl.sap.util.BaseConfiguration.properties$lzycompute(Configuration.scala:38)
> > at
> > com.sdl.sap.util.BaseConfiguration.properties(Configuration.scala:38)
> > at
> > com.sdl.sap.util.BaseConfiguration.stringValue(Configuration.scala:49)
> > at
> >
> com.sdl.hlab.infrastructure.SM2ApiResultsFileToHdfsConverter$.buildFileInfo(SM2ApiResultsFileToHdfsConverter.scala:74)
> > at
> >
> com.sdl.hlab.infrastructure.SM2ApiResultsFileToHdfsConverter$.main(SM2ApiResultsFileToHdfsConverter.scala:68)
> > at
> >
> com.sdl.hlab.infrastructure.SM2ApiResultsFileToHdfsConverter.main(SM2ApiResultsFileToHdfsConverter.scala)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> > Method)
> > at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > at java.lang.reflect.Method.invoke(Method.java:597)
> > at
> >
> org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:472)
> > at
> > org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> > at
> > org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:399)
> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:334)
> > at
> > org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
> > at java.security.AccessController.doPrivileged(Native
> > Method)
> > at javax.security.auth.Subject.doAs(Subject.java:396)
> > at
> >
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332)
> > at
> > org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
> >
> > 2013-12-19 00:41:28,864 INFO ActionEndXCommand:539 - USER[hdfs] GROUP[-]
> > TOKEN[] APP[hlab-oozie-workflow]
> JOB[0000012-131218165312793-oozie-oozi-W]
> > ACTION[0000012-131218165312793-oozie-oozi-W@convert-file] ERROR is
> > considered as FAILED for SLA
> > 2013-12-19 00:41:29,021 INFO ActionStartXCommand:539 - USER[hdfs]
> > GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
> > JOB[0000012-131218165312793-oozie-oozi-W]
> > ACTION[0000012-131218165312793-oozie-oozi-W@convert-file-Email] Start
> > action [0000012-131218165312793-oozie-oozi-W@convert-file-Email] with
> > user-retry state : userRetryCount [0], userRetryMax [0],
> userRetryInterval
> > [10]
> > 2013-12-19 00:41:29,112 WARN ActionStartXCommand:542 - USER[hdfs]
> > GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
> > JOB[0000012-131218165312793-oozie-oozi-W]
> > ACTION[0000012-131218165312793-oozie-oozi-W@convert-file-Email]
> > [***0000012-131218165312793-oozie-oozi-W@convert-file-Email***]Action
> > status=DONE
> > 2013-12-19 00:41:29,113 WARN ActionStartXCommand:542 - USER[hdfs]
> > GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
> > JOB[0000012-131218165312793-oozie-oozi-W]
> > ACTION[0000012-131218165312793-oozie-oozi-W@convert-file-Email]
> > [***0000012-131218165312793-oozie-oozi-W@convert-file-Email***]Action
> > updated in DB!
> > 2013-12-19 00:41:29,259 INFO ActionStartXCommand:539 - USER[hdfs]
> > GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
> > JOB[0000012-131218165312793-oozie-oozi-W]
> > ACTION[0000012-131218165312793-oozie-oozi-W@convert-file-fail] Start
> > action [0000012-131218165312793-oozie-oozi-W@convert-file-fail] with
> > user-retry state : userRetryCount [0], userRetryMax [0],
> userRetryInterval
> > [10]
> > 2013-12-19 00:41:29,260 WARN ActionStartXCommand:542 - USER[hdfs]
> > GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
> > JOB[0000012-131218165312793-oozie-oozi-W]
> > ACTION[0000012-131218165312793-oozie-oozi-W@convert-file-fail]
> > [***0000012-131218165312793-oozie-oozi-W@convert-file-fail***]Action
> > status=DONE
> > 2013-12-19 00:41:29,260 WARN ActionStartXCommand:542 - USER[hdfs]
> > GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
> > JOB[0000012-131218165312793-oozie-oozi-W]
> > ACTION[0000012-131218165312793-oozie-oozi-W@convert-file-fail]
> > [***0000012-131218165312793-oozie-oozi-W@convert-file-fail***]Action
> > updated in DB!
> > 2013-12-19 00:41:29,440 WARN CoordActionUpdateXCommand:542 - USER[hdfs]
> > GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
> > JOB[0000012-131218165312793-oozie-oozi-W] ACTION[-] E1100: Command
> > precondition does not hold before execution, [, coord action is null],
> > Error Code: E1100
> >
> > The FIleNotFoundException is the result of the property
> > com.sdl.sap.conf.directory not being set.
> >
> > Any help would be really appreciated!
> > </pre><font face="arial" size="2" color="#736F6E">
> >
> >
> >
> > <a href="
> >
> http://www.sdl.com/?utm_source=Email&utm_medium=Email%2BSignature&utm_campaign=SDL%2BStandard%2BEmail%2BSignature
> > ">
> > <img src="
> >
> http://www.sdl.com/Content/themes/common/images/SDL_logo_strapline_GCEM_EmailSig_150x68px.jpg
> "
> > border=0><br><br>www.sdl.com
> > </a><br><br>
> >
> > <font face="arial" size="1" color="#736F6E">
> >
> > <b>SDL PLC confidential, all rights reserved.</b>
> >
> > If you are not the intended recipient of this mail SDL requests and
> > requires that you delete it without acting upon or copying any of its
> > contents,
> > and we further request that you advise us.<BR>
> > SDL PLC is a public limited company registered in England and Wales.
> > Registered number: 02675207.<BR>
> > Registered address: Globe House, Clivemont Road, Maidenhead, Berkshire
> SL6
> > 7DY, UK.
> > </font>
> >
> >
> > This message has been scanned for malware by Websense. www.websense.com
> >
>
Re: Java-Opts are completely ignored
Posted by Robert Kanter <rk...@cloudera.com>.
I remember seeing something similar with java-opts where they were being
set somewhere else (mapred-site?) and declared as final, so when Oozie
tried to set them, it was ignored. Perhaps you’re running into the same
problem?
On Thu, Dec 19, 2013 at 12:23 AM, Jose Fernandez <jf...@sdl.com> wrote:
> Hi folks,
>
> I searched through the archives and googled my brains out and I'm flat out
> stuck. I'm hoping someone can help me here :) We're getting pretty
> desperate.
>
> CDH 4.1.3
> Oozie v3.2.0
> Using MR2 (YARN)
>
> We have a Java action configured in workflow.xml thusly:
>
> <action name="convert-file">
> <java>
> <job-tracker>${jobTracker}</job-tracker>
> <name-node>${nameNode}</name-node>
> <configuration>
> <property>
> <name>mapred.job.queue.name</name>
> <value>${queueName}</value>
> </property>
> </configuration>
>
> <main-class>com.sdl.hlab.infrastructure.SM2ApiResultsFileToHdfsConverter</main-class>
> <java-opts>-Djava.library.path=${sapNativePath}
> -Dcom.sdl.sap.conf.directory="." -Xmx1900m
> -Dlogback.configurationFile="conf/${SAP_ENV}/log4j.xml"
> -Dlog4j.configuration="conf/${SAP_ENV}/log4j.xml"</java-opts>
> <arg>${filePath}</arg>
> <file>files/sap.properties</file>
> </java>
> <ok to="cleanup"/>
> <error to="convert-file-Email"/>
> </action>
>
> This action works perfectly on two different environments (all same CDH
> and Oozie version), but not on our new third environment. In the first two
> environments, the java-opts block is respected and I can see it in the Job
> History logs in stdout under "Java System Properties". In the third
> environment (which is new) none of the java-opts appear except for
> java.library.path, which points to the wrong libraries.
>
> I checked everywhere for entries for mapred.child.java.opts and there
> appears to be one entry buried in mapred-defaults.xml and that's it.
>
> I also tried to move the java opts to a property entry, no luck.
>
> The job log doesn't have anything interesting:
>
> 2013-12-19 00:41:05,419 INFO ActionStartXCommand:539 - USER[hdfs]
> GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
> JOB[0000012-131218165312793-oozie-oozi-W]
> ACTION[0000012-131218165312793-oozie-oozi-W@:start:] Start action
> [0000012-131218165312793-oozie-oozi-W@:start:] with user-retry state :
> userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> 2013-12-19 00:41:05,419 WARN ActionStartXCommand:542 - USER[hdfs]
> GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
> JOB[0000012-131218165312793-oozie-oozi-W]
> ACTION[0000012-131218165312793-oozie-oozi-W@:start:]
> [***0000012-131218165312793-oozie-oozi-W@:start:***]Action status=DONE
> 2013-12-19 00:41:05,419 WARN ActionStartXCommand:542 - USER[hdfs]
> GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
> JOB[0000012-131218165312793-oozie-oozi-W]
> ACTION[0000012-131218165312793-oozie-oozi-W@:start:]
> [***0000012-131218165312793-oozie-oozi-W@:start:***]Action updated in DB!
> 2013-12-19 00:41:05,561 INFO ActionStartXCommand:539 - USER[hdfs]
> GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
> JOB[0000012-131218165312793-oozie-oozi-W]
> ACTION[0000012-131218165312793-oozie-oozi-W@convert-file] Start action
> [0000012-131218165312793-oozie-oozi-W@convert-file] with user-retry state
> : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> 2013-12-19 00:41:07,226 WARN JavaActionExecutor:542 - USER[hdfs] GROUP[-]
> TOKEN[] APP[hlab-oozie-workflow] JOB[0000012-131218165312793-oozie-oozi-W]
> ACTION[0000012-131218165312793-oozie-oozi-W@convert-file] credentials is
> null for the action
> 2013-12-19 00:41:09,812 INFO JavaActionExecutor:539 - USER[hdfs] GROUP[-]
> TOKEN[] APP[hlab-oozie-workflow] JOB[0000012-131218165312793-oozie-oozi-W]
> ACTION[0000012-131218165312793-oozie-oozi-W@convert-file] checking
> action, external ID [job_1387408789132_0015] status [RUNNING]
> 2013-12-19 00:41:09,815 WARN ActionStartXCommand:542 - USER[hdfs]
> GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
> JOB[0000012-131218165312793-oozie-oozi-W]
> ACTION[0000012-131218165312793-oozie-oozi-W@convert-file]
> [***0000012-131218165312793-oozie-oozi-W@convert-file***]Action
> status=RUNNING
> 2013-12-19 00:41:09,815 WARN ActionStartXCommand:542 - USER[hdfs]
> GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
> JOB[0000012-131218165312793-oozie-oozi-W]
> ACTION[0000012-131218165312793-oozie-oozi-W@convert-file]
> [***0000012-131218165312793-oozie-oozi-W@convert-file***]Action updated
> in DB!
> 2013-12-19 00:41:28,470 INFO CallbackServlet:539 - USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[0000012-131218165312793-oozie-oozi-W]
> ACTION[0000012-131218165312793-oozie-oozi-W@convert-file] callback for
> action [0000012-131218165312793-oozie-oozi-W@convert-file]
> 2013-12-19 00:41:28,755 INFO JavaActionExecutor:539 - USER[hdfs] GROUP[-]
> TOKEN[] APP[hlab-oozie-workflow] JOB[0000012-131218165312793-oozie-oozi-W]
> ACTION[0000012-131218165312793-oozie-oozi-W@convert-file] action
> completed, external ID [job_1387408789132_0015]
> 2013-12-19 00:41:28,787 WARN JavaActionExecutor:542 - USER[hdfs] GROUP[-]
> TOKEN[] APP[hlab-oozie-workflow] JOB[0000012-131218165312793-oozie-oozi-W]
> ACTION[0000012-131218165312793-oozie-oozi-W@convert-file] Launcher ERROR,
> reason: Main class [our main class], main() threw exception,
> conf/qa/our.properties (No such file or directory)
> 2013-12-19 00:41:28,787 WARN JavaActionExecutor:542 - USER[hdfs] GROUP[-]
> TOKEN[] APP[hlab-oozie-workflow] JOB[0000012-131218165312793-oozie-oozi-W]
> ACTION[0000012-131218165312793-oozie-oozi-W@convert-file] Launcher
> exception: conf/qa/our.properties (No such file or directory)
> java.io.FileNotFoundException: conf/qa/our.properties (No such file or
> directory)
> at java.io.FileInputStream.open(Native Method)
> at java.io.FileInputStream.<init>(FileInputStream.java:120)
> at java.io.FileInputStream.<init>(FileInputStream.java:79)
> at
> com.sdl.sap.util.BaseConfiguration.configurationFile(Configuration.scala:78)
> at
> com.sdl.sap.util.BaseConfiguration.loadProperties(Configuration.scala:88)
> at
> com.sdl.sap.util.BaseConfiguration.properties$lzycompute(Configuration.scala:38)
> at
> com.sdl.sap.util.BaseConfiguration.properties(Configuration.scala:38)
> at
> com.sdl.sap.util.BaseConfiguration.stringValue(Configuration.scala:49)
> at
> com.sdl.hlab.infrastructure.SM2ApiResultsFileToHdfsConverter$.buildFileInfo(SM2ApiResultsFileToHdfsConverter.scala:74)
> at
> com.sdl.hlab.infrastructure.SM2ApiResultsFileToHdfsConverter$.main(SM2ApiResultsFileToHdfsConverter.scala:68)
> at
> com.sdl.hlab.infrastructure.SM2ApiResultsFileToHdfsConverter.main(SM2ApiResultsFileToHdfsConverter.scala)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:472)
> at
> org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> at
> org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:399)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:334)
> at
> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
> at java.security.AccessController.doPrivileged(Native
> Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332)
> at
> org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
>
> 2013-12-19 00:41:28,864 INFO ActionEndXCommand:539 - USER[hdfs] GROUP[-]
> TOKEN[] APP[hlab-oozie-workflow] JOB[0000012-131218165312793-oozie-oozi-W]
> ACTION[0000012-131218165312793-oozie-oozi-W@convert-file] ERROR is
> considered as FAILED for SLA
> 2013-12-19 00:41:29,021 INFO ActionStartXCommand:539 - USER[hdfs]
> GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
> JOB[0000012-131218165312793-oozie-oozi-W]
> ACTION[0000012-131218165312793-oozie-oozi-W@convert-file-Email] Start
> action [0000012-131218165312793-oozie-oozi-W@convert-file-Email] with
> user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval
> [10]
> 2013-12-19 00:41:29,112 WARN ActionStartXCommand:542 - USER[hdfs]
> GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
> JOB[0000012-131218165312793-oozie-oozi-W]
> ACTION[0000012-131218165312793-oozie-oozi-W@convert-file-Email]
> [***0000012-131218165312793-oozie-oozi-W@convert-file-Email***]Action
> status=DONE
> 2013-12-19 00:41:29,113 WARN ActionStartXCommand:542 - USER[hdfs]
> GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
> JOB[0000012-131218165312793-oozie-oozi-W]
> ACTION[0000012-131218165312793-oozie-oozi-W@convert-file-Email]
> [***0000012-131218165312793-oozie-oozi-W@convert-file-Email***]Action
> updated in DB!
> 2013-12-19 00:41:29,259 INFO ActionStartXCommand:539 - USER[hdfs]
> GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
> JOB[0000012-131218165312793-oozie-oozi-W]
> ACTION[0000012-131218165312793-oozie-oozi-W@convert-file-fail] Start
> action [0000012-131218165312793-oozie-oozi-W@convert-file-fail] with
> user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval
> [10]
> 2013-12-19 00:41:29,260 WARN ActionStartXCommand:542 - USER[hdfs]
> GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
> JOB[0000012-131218165312793-oozie-oozi-W]
> ACTION[0000012-131218165312793-oozie-oozi-W@convert-file-fail]
> [***0000012-131218165312793-oozie-oozi-W@convert-file-fail***]Action
> status=DONE
> 2013-12-19 00:41:29,260 WARN ActionStartXCommand:542 - USER[hdfs]
> GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
> JOB[0000012-131218165312793-oozie-oozi-W]
> ACTION[0000012-131218165312793-oozie-oozi-W@convert-file-fail]
> [***0000012-131218165312793-oozie-oozi-W@convert-file-fail***]Action
> updated in DB!
> 2013-12-19 00:41:29,440 WARN CoordActionUpdateXCommand:542 - USER[hdfs]
> GROUP[-] TOKEN[] APP[hlab-oozie-workflow]
> JOB[0000012-131218165312793-oozie-oozi-W] ACTION[-] E1100: Command
> precondition does not hold before execution, [, coord action is null],
> Error Code: E1100
>
> The FIleNotFoundException is the result of the property
> com.sdl.sap.conf.directory not being set.
>
> Any help would be really appreciated!
> </pre><font face="arial" size="2" color="#736F6E">
>
>
>
> <a href="
> http://www.sdl.com/?utm_source=Email&utm_medium=Email%2BSignature&utm_campaign=SDL%2BStandard%2BEmail%2BSignature
> ">
> <img src="
> http://www.sdl.com/Content/themes/common/images/SDL_logo_strapline_GCEM_EmailSig_150x68px.jpg"
> border=0><br><br>www.sdl.com
> </a><br><br>
>
> <font face="arial" size="1" color="#736F6E">
>
> <b>SDL PLC confidential, all rights reserved.</b>
>
> If you are not the intended recipient of this mail SDL requests and
> requires that you delete it without acting upon or copying any of its
> contents,
> and we further request that you advise us.<BR>
> SDL PLC is a public limited company registered in England and Wales.
> Registered number: 02675207.<BR>
> Registered address: Globe House, Clivemont Road, Maidenhead, Berkshire SL6
> 7DY, UK.
> </font>
>
>
> This message has been scanned for malware by Websense. www.websense.com
>