You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@knox.apache.org by "Kuldeep Kulkarni (JIRA)" <ji...@apache.org> on 2017/01/19 12:25:26 UTC
[jira] [Commented] (KNOX-857) Incorrect formation of
causes oozie mapreduce action failure
[ https://issues.apache.org/jira/browse/KNOX-857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15829820#comment-15829820 ]
Kuldeep Kulkarni commented on KNOX-857:
---------------------------------------
If you want to reproduce this issue:
1. Setup HDP cluster with HDP-2.4.0.0 and Ambari 2.2.1.1 (Ambari version doesn't really matter here)
2. Add Oozie and Knox services
3. Run Oozie service check
4. Copy Job configuration and save it as job.xml
5. Configure Knox for Oozie service and verify if its working fine by running below command
{code}
[ambari-qa@oozieknox1 ~]$ curl -i -k -u ambari-qa:hadoop https://oozieknox2.openstacklocal:8443/gateway/mytopology/oozie/v1/admin/build-version
HTTP/1.1 200 OK
Set-Cookie: JSESSIONID=u264gbundtvz1v9lhlp2q2qr7;Path=/gateway/mytopology;Secure;HttpOnly
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Server: Apache-Coyote/1.1
Date: Thu, 19 Jan 2017 12:22:18 GMT
Content-Type: application/json;charset=UTF-8
Content-Length: 36
{"buildVersion":"4.2.0.2.4.0.0-169"}
{code}
Note - Make sure to replace your topogy name in about command along with correct username and password.
6. Once you verify that, above command is working fine, submit an Oozie workflow with Knox
{code}
[ambari-qa@oozieknox1 ~]$ curl -i -k -u ambari-qa:hadoop -H Content-Type:application/xml -T job.xml -X POST 'https://oozieknox3.openstacklocal:8443/gateway/mytopology/oozie/v1/jobs?action=start'
HTTP/1.1 100 Continue
HTTP/1.1 201 Created
Set-Cookie: JSESSIONID=1o5r8jtu485pa1m2glm7fuipy1;Path=/gateway/mytopology;Secure;HttpOnly
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Server: Apache-Coyote/1.1
Date: Thu, 19 Jan 2017 10:42:41 GMT
Content-Type: application/json;charset=UTF-8
Content-Length: 45
{"id":"0000016-170118140025343-oozie-oozi-W"}
{code}
Note - Make sure to change hostnames and job.xml file name according to your environment.
> Incorrect formation of <delete path> causes oozie mapreduce action failure
> --------------------------------------------------------------------------
>
> Key: KNOX-857
> URL: https://issues.apache.org/jira/browse/KNOX-857
> Project: Apache Knox
> Issue Type: Bug
> Components: KnoxCLI
> Affects Versions: 0.6.0
> Environment: HDP-2.4.0.0
> Reporter: Kuldeep Kulkarni
> Priority: Critical
> Attachments: job.xml.bug, knox.conf.tar.gz, knox.log.tz
>
>
> When Oozie workflow is submitted via Knox containing mapreduce-action. it always gets failed with below error:
> {code}
> Heart beat
> {"properties":[{"key":"oozie.launcher.job.id","value":"job_1484748086731_0022","isFinal":false,"resource":"programatically"},{"key":"oozie.job.id","value":"0000017-170118140025343-oozie-oozi-W","isFinal":false,"resource":"programatically"},{"key":"oozie.action.id","value":"0000017-170118140025343-oozie-oozi-W@mr-node","isFinal":false,"resource":"programatically"},{"key":"mapreduce.job.tags","value":"oozie-20b9b1ecd11f08843b078e701ce5c1ad","isFinal":false,"resource":"programatically"}]}Starting the execution of prepare actions
> Prepare execution in the Launcher Mapper has failed
> Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.MapReduceMain], exception invoking main(), java.net.URISyntaxException: Expected scheme-specific part at index 5: hdfs:
> org.apache.oozie.action.hadoop.LauncherException: java.net.URISyntaxException: Expected scheme-specific part at index 5: hdfs:
> at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:193)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
> Caused by: org.apache.oozie.action.hadoop.LauncherException: java.net.URISyntaxException: Expected scheme-specific part at index 5: hdfs:
> at org.apache.oozie.action.hadoop.PrepareActionsDriver.doOperations(PrepareActionsDriver.java:77)
> at org.apache.oozie.action.hadoop.LauncherMapper.executePrepare(LauncherMapper.java:494)
> at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:189)
> ... 8 more
> Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException: Expected scheme-specific part at index 5: hdfs:
> at org.apache.hadoop.fs.Path.initialize(Path.java:205)
> at org.apache.hadoop.fs.Path.<init>(Path.java:171)
> at org.apache.hadoop.fs.Path.<init>(Path.java:93)
> at org.apache.hadoop.fs.Globber.glob(Globber.java:211)
> at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1655)
> at org.apache.oozie.action.hadoop.FSLauncherURIHandler.delete(FSLauncherURIHandler.java:59)
> at org.apache.oozie.action.hadoop.PrepareActionsDriver.execute(PrepareActionsDriver.java:90)
> at org.apache.oozie.action.hadoop.PrepareActionsDriver.doOperations(PrepareActionsDriver.java:68)
> ... 10 more
> Caused by: java.net.URISyntaxException: Expected scheme-specific part at index 5: hdfs:
> at java.net.URI$Parser.fail(URI.java:2848)
> at java.net.URI$Parser.failExpecting(URI.java:2854)
> at java.net.URI$Parser.parse(URI.java:3057)
> at java.net.URI.<init>(URI.java:746)
> at org.apache.hadoop.fs.Path.initialize(Path.java:202)
> ... 17 more
> Oozie Launcher failed, finishing Hadoop job gracefully
> {code}
> This happens because, <delete path> in the action configuration gets generated incorrectly.
> {code}
> <delete path="hdfs://oozieknox2.openstacklocal:8020/user/ambari-qa/examples/output-data/hdfs://oozieknox2.openstacklocal:8020/map-reduce" />
> {code}
> Expected value is:
> {code}
> <delete path="hdfs://oozieknox2.openstacklocal:8020/user/ambari-qa/examples/output-data/map-reduce" />
> {code}
> Command used to submit oozie job is:
> {code}
> [ambari-qa@oozieknox1 ~]$ curl -i -k -u ambari-qa:hadoop -H Content-Type:application/xml -T job.xml.bug -X POST 'https://oozieknox3.openstacklocal:8443/gateway/mytopology/oozie/v1/jobs?action=start'
> HTTP/1.1 100 Continue
> HTTP/1.1 201 Created
> Set-Cookie: JSESSIONID=13d8gzukx9aq019rdkb06li3pj;Path=/gateway/mytopology;Secure;HttpOnly
> Expires: Thu, 01 Jan 1970 00:00:00 GMT
> Server: Apache-Coyote/1.1
> Date: Thu, 19 Jan 2017 12:06:39 GMT
> Content-Type: application/json;charset=UTF-8
> Content-Length: 45
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)