You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@falcon.apache.org by "Venkatesh Seetharam (JIRA)" <ji...@apache.org> on 2014/09/08 23:09:28 UTC
[jira] [Commented] (FALCON-673) feed schedule trying to create
staging path as ACL owner
[ https://issues.apache.org/jira/browse/FALCON-673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126099#comment-14126099 ]
Venkatesh Seetharam commented on FALCON-673:
--------------------------------------------
I don't think thats the behavior. I coded this as part of FALCON-11 for staging and working dirs MUST be configured on HDFS to be owned by user running falcon. The generated workflow definitions for oozie are serialized to the staging dir as falcon but world readable so oozie can read these workflow definitions.
{code}
org.apache.falcon.entity.parser.ClusterEntityParser#checkPathOwner
Checks for path being owned by user who started falcon
{code}
However, I think this needs to be thought through before working on a solution. I have had brief discussions with [~sriksun].
The issue that staging/working dirs are owned by falcon but world readable is NOT acceptable to certain users for security purposes. Ideally, the dir for a particular entity in staging must be owned by the user but the group can be falcon.
We need to think more on a cleaner solution to this problem.
> feed schedule trying to create staging path as ACL owner
> --------------------------------------------------------
>
> Key: FALCON-673
> URL: https://issues.apache.org/jira/browse/FALCON-673
> Project: Falcon
> Issue Type: Bug
> Components: feed
> Reporter: Samarth Gupta
> Assignee: Suhas Vasu
> Priority: Blocker
>
> while scheduling the feed, falcon is trying to create the workflow xml on hdfs as the user who is mentioned in ACL tag in the feed. However it is not necessary that ACL owner has the write permissions to the cluster workflow location. in such a case feed schedule fails. Logs below :
> {code}
> 2014-09-04 06:20:06,901 DEBUG - [864244733@qtp-1633673452-2:samarth.gupta:POST//entities/schedule/feed/raaw-logs16-55cc9994 1dd9ea6e-822d-4a99-8225-79cbaeb7acd0] ~ Writing definition to /projects/ivory/staging/falcon/workflows/feed/raaw-logs16-55cc9994/92fc5fd4476e4a6977dedf9f1f3a632d_1409811606326/RETENTION/workflow.xml on cluster corp-91e54ac3 (OozieEntityBuilder:139)
> {code}
> {code}
> 2014-09-04 06:20:06,975 ERROR - [864244733@qtp-1633673452-2:samarth.gupta:POST//entities/schedule/feed/raaw-logs16-55cc9994 1dd9ea6e-822d-4a99-8225-79cbaeb7acd0] ~ Action failed: Bad Request
> Error: org.apache.falcon.FalconException: Unable to marshall app object
> at org.apache.falcon.oozie.OozieEntityBuilder.marshal(OozieEntityBuilder.java:155)
> at org.apache.falcon.oozie.OozieOrchestrationWorkflowBuilder.marshal(OozieOrchestrationWorkflowBuilder.java:184)
> at org.apache.falcon.oozie.feed.FeedRetentionWorkflowBuilder.build(FeedRetentionWorkflowBuilder.java:70)
> at org.apache.falcon.oozie.feed.FeedRetentionCoordinatorBuilder.buildCoords(FeedRetentionCoordinatorBuilder.java:98)
> at org.apache.falcon.oozie.feed.FeedBundleBuilder.buildCoords(FeedBundleBuilder.java:43)
> at org.apache.falcon.oozie.OozieBundleBuilder.build(OozieBundleBuilder.java:71)
> at org.apache.falcon.workflow.engine.OozieWorkflowEngine.schedule(OozieWorkflowEngine.java:150)
> at org.apache.falcon.resource.AbstractSchedulableEntityManager.scheduleInternal(AbstractSchedulableEntityManager.java:69)
> at org.apache.falcon.resource.AbstractSchedulableEntityManager.schedule(AbstractSchedulableEntityManager.java:58)
> at org.apache.falcon.resource.SchedulableEntityManager.schedule(SchedulableEntityManager.java:85)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:622)
> at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
> at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
> at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
> at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
> at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
> at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
> at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537)
> at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
> at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
> at org.apache.falcon.security.BasicAuthFilter$2.doFilter(BasicAuthFilter.java:184)
> at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:384) at org.apache.falcon.security.BasicAuthFilter.doFilter(BasicAuthFilter.java:222)
> at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:945)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
> at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> Caused by: org.apache.hadoop.security.AccessControlException: Permission denied: user=dataqa, access=WRITE, inode="/":hdfs:supergroup:drwxr-xr-x
> at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:224)
> at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:204)
> at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:149)
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:4716)
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:4698)
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkAncestorAccess(FSNamesystem.java:4672)
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:1839)
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInt(FSNamesystem.java:1771)
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:1747)
> at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.create(NameNodeRpcServer.java:418)
> at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.create(ClientNamenodeProtocolServerSideTranslatorPB.java:207)
> at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:44942)
> at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:453)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1002)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1701)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1697)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:416)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1695)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)