You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by "Artem Ervits (JIRA)" <ji...@apache.org> on 2018/05/23 15:56:00 UTC
[jira] [Commented] (OOZIE-1516) Usability issues submitting jobs
[ https://issues.apache.org/jira/browse/OOZIE-1516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16487515#comment-16487515 ]
Artem Ervits commented on OOZIE-1516:
-------------------------------------
[~andras.piros] invalid?
> Usability issues submitting jobs
> --------------------------------
>
> Key: OOZIE-1516
> URL: https://issues.apache.org/jira/browse/OOZIE-1516
> Project: Oozie
> Issue Type: Improvement
> Components: examples
> Environment: Cloudera CDH4 latest on Centos
> Reporter: Brad Cox
> Priority: Critical
>
> J Harsh suggested filing a usability issue concerning a problem he helped me resolve via email. Problem seems to boil down to differences between localhost and localhost.localdomain that I don't understand well enough to explain here. I've enclosed the email thread as documentation.
> J Harsh...
> I believe we seem to be doing a strict hostname check in Apache Oozie
> and not a resolving check. Could you report an improvement on the
> Apache Oozie JIRA (https://issues.apache.org/jira/browse/OOZIE)
> upstream to perhaps improve this for usability?
> On Sun, Sep 1, 2013 at 2:41 AM, Brad Cox <br...@gmail.com> wrote:
> Copied examples to /user/cloudera/examples and changed nameNode to
> hdfs://localhost.localdomain in job.properties. Results look like a good
> sign.
> [cloudera@localhost Desktop]$ oozie job -oozie http://localhost:11000/oozie
> -config examples/apps/map-reduce/job.properties -run
> job: 0000000-130831130415538-oozie-oozi-W
> But in oozie console, got a job fail because jobTracker's not in whitelist
> so changed that to just localhost.localdomain by analogy.
> [cloudera@localhost Desktop]$ oozie job -oozie http://localhost:11000/oozie
> -config examples/apps/map-reduce/job.properties -run
> job: 0000001-130831130415538-oozie-oozi-W
> Console says that try succeeded. So thanks for the help!!
> All I'm missing is an understanding of why cloudera comes outOfBox with some
> services configured as localhost and others localhost.localdomain. I always
> thought they were equivalent. But these are huge differences; success vs
> fail.
> On Aug 31, 2013, at 4:58 PM, Brad Cox <br...@gmail.com> wrote:
> You mean use localhost.localdomain instead of localhost? Or should I define
> a unique name with hostname cmd, restart all, and use that instead?
> [cloudera@localhost Desktop]$ hostname
> localhost.localdomain
> On Aug 31, 2013, at 4:54 PM, Harsh J <ha...@cloudera.com> wrote:
> The whitelist message is close to the solution. Do not use 'localhost'
> in your job.properties, but replace it with whatever the command
> 'hostname' produces on your terminal, and I believe it should do the
> trick.
> On Sat, Aug 31, 2013 at 11:11 PM, Brad Cox <br...@gmail.com> wrote:
> That's what I tried in previous attempts. That causes this
> oozie job -oozie http://localhost:11000/oozie -config
> examples/apps/map-reduce/job.properties -run
> Error: HTTP error code: 500 : Internal Server Error
> /var/log/lochost.date.log contains
> Caused by: java.net.URISyntaxException: Relative path in absolute URI:
> localhost:8020/user/cloudera/examples/apps/map-reduce
> at java.net.URI.checkPath(URI.java:1788)
> examples is where I put it as you said
> [cloudera@localhost Desktop]$ hdfs dfs -ls /
> Found 8 items
> drwxr-xr-x - hdfs supergroup 0 2013-08-30 15:28 /accumulo
> drwxr-xr-x - oozie supergroup 0 2013-08-31 04:52 /examples
> drwxr-xr-x - hbase hbase 0 2013-08-31 07:22 /hbase
> drwxr-xr-x - hdfs supergroup 0 2013-08-30 16:38 /share
> drwxr-xr-x - solr solr 0 2013-07-17 00:03 /solr
> drwxrwxrwx - hdfs supergroup 0 2013-08-30 16:32 /tmp
> drwxr-xr-x - hdfs supergroup 0 2013-08-30 16:32 /user
> drwxr-xr-x - hdfs supergroup 0 2013-07-17 00:03 /var
> All this is after hacking nameNode=localhost:8020 (was hdfs://...) in
> job.properties based on the Running the Examples section of the Oozie docs.
> On reverting that change, the message changed to the problem I had before:
> oozie job -oozie http://localhost:11000/oozie -config
> examples/apps/map-reduce/job.properties -run
> Error: E0901 : E0901: Namenode [localhost:8020] not allowed, not in Oozies
> whitelist
> Based on that msg, I'd already changed
> oozie.service.HadoopAccessorService.nameNode.whitelist to localhost:8020 in
> /etc/oozie/conf/oozie-site.xml and just made a similar change to
> jobTracker.whitelist to localhost:8021. Best I've been able to do is switch
> mystifying messages.
> On Aug 31, 2013, at 12:22 PM, Harsh J <ha...@cloudera.com> wrote:
> Hi Brad,
> In Oozie, an simple "application" consists of a directory on HDFS that
> carries workflow.xml, and associated libraries/files.
> To invoke an instance of such an application, we use a local
> job.properties file that carries any variable parameter values, to
> begin the application with.
> Typically, to run the examples out of the box on a standalone cluster,
> you can run the following set of commands (assuming you do have the
> oozie-examples tarball):
> $ tar xf oozie-examples.tar.gz
> $ hadoop fs -put examples examples
> $ oozie job -oozie http://localhost:11000/oozie -config
> examples/apps/map-reduce/job.properties -run
> Can you tell me what error you face if you try this out?
> On Sat, Aug 31, 2013 at 8:22 PM, Brad Cox <br...@gmail.com> wrote:
> I'm trying the ozzie examples in CDH4.1 and the error messages make no
> sense. Got through the others by editing the properties file but this one
> really stumps me.
> I've copied the ozzie examples folder to /home/cloudera/Desktop and have
> verified that job.properties path is right.
> [cloudera@localhost Desktop]$ whoami
> cloudera
> [cloudera@localhost Desktop]$ ls
> /home/cloudera/Desktop/examples/apps/map-reduce/job.properties
> Run as cloudera user and trying various permutations on the path
> (with/without file:// prefix) I keep getting this.
> oozie job -oozie http://localhost:11000/oozie -config
> file:///home/cloudera/Desktop/examples/apps/map-reduce/job.properties -run
> java.io.IOException: configuration file
> [file:///home/cloudera/Desktop/examples/apps/map-reduce/job.properties] not
> found
> at org.apache.oozie.cli.OozieCLI.getConfiguration(OozieCLI.java:641)
> at org.apache.oozie.cli.OozieCLI.jobCommand(OozieCLI.java:795)
> at org.apache.oozie.cli.OozieCLI.processCommand(OozieCLI.java:512)
> at org.apache.oozie.cli.OozieCLI.run(OozieCLI.java:485)
> at org.apache.oozie.cli.OozieCLI.main(OozieCLI.java:182)
> configuration file
> [file:///home/cloudera/Desktop/examples/apps/map-reduce/job.properties] not
> found
> I also tried putting the examples in hdfs as the instructions said but that
> doesn't help. In fact that's entirely the wrong idea according to some
> emails I've seen.
> Dr. Brad J. Cox Cell: 703-594-1883 Blog: http://bradjcox.blogspot.com
> http://virtualschool.edu
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)