You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by "LuGuangMing (JIRA)" <ji...@apache.org> on 2018/11/19 03:16:00 UTC
[jira] [Updated] (OOZIE-3385) The situation multi user submit
workflows , occasionally, occur the HDFS visitor user become another one
[ https://issues.apache.org/jira/browse/OOZIE-3385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
LuGuangMing updated OOZIE-3385:
-------------------------------
Description:
The situation multi user submit workflows , occasionally, occur the HDFS visitor user become another one . for example, I need submit a workflow by proxy user "{color:#ff0000}platform{color}" via user oozie (kerberos) , an error occur in oozie source code WorkflowAppService.readDefinition read workflow.xml.
*2018-11-14 00:00:00,497 ERROR [CallableQueue-42]org.apache.oozie.command.wf.SubmitXCommand(517) {color:#ff0000}USER[platform]{color} GROUP[-] TOKEN[] APP[myBulkload-Scheduler-CS_TTL-1539689446] JOB[0002497-180928143722290-oozie-root-C] ACTION[0002497-180928143722290-oozie-root-C@1354] XException, org.apache.oozie.command.CommandException: E0710: Could not read the workflow definition, Permission denied: user={color:#ff0000}dbzq04{color}, access=READ, inode="/phoebus/_fileservice/users/nsplatform/platform/workflows/DataLoadWF-1427-1129/workflow.xml":{color:#ff0000}platform{color}:supergroup:-rw-------*
note: user "{color:#ff0000}dbzq04{color}" also submit some workflow at before, but current submit the workflow of user is user {color:#ff0000}platform. for prove is current user is platform , I insert some logs at oozie source code {color}
{code:java}
/** org.apache.oozie.service.HadoopAccessorService */
public FileSystem createFileSystem(String user, final URI uri, final Configuration conf) throws HadoopAccessorException {
//.........omit......
try {
UserGroupInformation ugi = getUGI(user);
LOG.info("current user="+ugi); //------ my insert log, to print proxy ugi info
return ugi.doAs(new PrivilegedExceptionAction<FileSystem>() {
public FileSystem run() throws Exception {
FileSystem fs = FileSystem.get(uri, conf);
//------ my insert log, to print fs inner ugi info
if(fs instanceof DistributedFileSystem){
LOG.info("hdfs client user, "+((DistributedFileSystem)fs).getClient().toString());
}
return fs;
}
});
}catch (InterruptedException ex) {
throw new HadoopAccessorException(ErrorCode.E0902, ex.getMessage(), ex);
}catch (IOException ex) {
throw new HadoopAccessorException(ErrorCode.E0902, ex.getMessage(), ex);
}
}{code}
*my log print result follows:*
2018-11-14 00:00:00,492 INFO [CallableQueue-42]org.apache.oozie.service.HadoopAccessorService(520) USER[platform] GROUP[-] TOKEN[] APP[myBulkload-Scheduler-CS_TTL-1539689446] JOB[0002497-180928143722290-oozie-root-C] ACTION[0002497-180928143722290-oozie-root-C@1354] *current user=platform (auth:PROXY) via oozie/nsplatformhm@DC1.FH.COM (auth:KERBEROS)*
2018-11-14 00:00:00,493 INFO [CallableQueue-42]org.apache.oozie.service.HadoopAccessorService(520) USER[platform] GROUP[-] TOKEN[] APP[myBulkload-Scheduler-CS_TTL-1539689446] JOB[0002497-180928143722290-oozie-root-C] ACTION[0002497-180928143722290-oozie-root-C@1354] {color:#d04437}*hdfs client user, DFSClient[clientName=DFSClient_NONMAPREDUCE_-515910437_325, ugi=platform (auth:PROXY) via oozie/nsplatformhm@DC1.FH.COM (auth:KERBEROS)]*{color}
*over above the proves Indicate at visited HDFS path of user has been altered, where was user "**{color:#ff0000}dbzq04" {color:#333333} came in? please help me check this problem{color}{color}*{color:#333333}*{color}*
was:
The situation multi user submit workflows , occasionally, occur the HDFS visitor user become another one . for example, I need submit a workflow by proxy user "{color:#FF0000}platform{color}" via user oozie (kerberos) , an error occur in oozie source code WorkflowAppService.readDefinition read workflow.xml.
*2018-11-14 00:00:00,497 ERROR [CallableQueue-42]org.apache.oozie.command.wf.SubmitXCommand(517) {color:#FF0000}USER[platform]{color} GROUP[-] TOKEN[] APP[myBulkload-Scheduler-CS_TTL-1539689446] JOB[0002497-180928143722290-oozie-root-C] ACTION[0002497-180928143722290-oozie-root-C@1354] XException, org.apache.oozie.command.CommandException: E0710: Could not read the workflow definition, Permission denied: user={color:#FF0000}dbzq04{color}, access=READ, inode="/phoebus/_fileservice/users/nsplatform/platform/workflows/DataLoadWF-1427-1129/workflow.xml":{color:#FF0000}platform{color}:supergroup:-rw-------*
note: user "{color:#FF0000}dbzq04{color}" also submit some workflow at before, but current submit the workflow of user is user {color:#FF0000}platform{color:#333333}. for prove is current user is platform , I insert some logs at oozie source code {color}{color}
{code:java}
/** org.apache.oozie.service.HadoopAccessorService */
public FileSystem createFileSystem(String user, final URI uri, final Configuration conf) throws HadoopAccessorException {
//.........omit......
try {
UserGroupInformation ugi = getUGI(user);
LOG.info("current user="+ugi); //------ my insert log, to print proxy ugi info
return ugi.doAs(new PrivilegedExceptionAction<FileSystem>() {
public FileSystem run() throws Exception {
FileSystem fs = FileSystem.get(uri, conf);
//------ my insert log, to print fs inner ugi info
if(fs instanceof DistributedFileSystem){
LOG.info("hdfs client user, "+((DistributedFileSystem)fs).getClient().toString());
}
return fs;
}
});
}catch (InterruptedException ex) {
throw new HadoopAccessorException(ErrorCode.E0902, ex.getMessage(), ex);
}catch (IOException ex) {
throw new HadoopAccessorException(ErrorCode.E0902, ex.getMessage(), ex);
}
}{code}
*my log print result follows:*
2018-11-14 00:00:00,492 INFO [CallableQueue-42]org.apache.oozie.service.HadoopAccessorService(520) USER[platform] GROUP[-] TOKEN[] APP[myBulkload-Scheduler-CS_TTL-1539689446] JOB[0002497-180928143722290-oozie-root-C] ACTION[0002497-180928143722290-oozie-root-C@1354] *{color:#d04437}current user=platform (auth:PROXY) via oozie/nsplatformhm@DC1.FH.COM (auth:KERBEROS){color}*
2018-11-14 00:00:00,493 INFO [CallableQueue-42]org.apache.oozie.service.HadoopAccessorService(520) USER[platform] GROUP[-] TOKEN[] APP[myBulkload-Scheduler-CS_TTL-1539689446] JOB[0002497-180928143722290-oozie-root-C] ACTION[0002497-180928143722290-oozie-root-C@1354] {color:#d04437}*hdfs client user, DFSClient[clientName=DFSClient_NONMAPREDUCE_-515910437_325, ugi=platform (auth:PROXY) via oozie/nsplatformhm@DC1.FH.COM (auth:KERBEROS)]*{color}
*over above the proves Indicate at visited HDFS path of user has been altered, where was user "**{color:#FF0000}dbzq04" {color:#333333} came in? please help me check this problem{color}{color}***
> The situation multi user submit workflows , occasionally, occur the HDFS visitor user become another one
> ---------------------------------------------------------------------------------------------------------
>
> Key: OOZIE-3385
> URL: https://issues.apache.org/jira/browse/OOZIE-3385
> Project: Oozie
> Issue Type: Bug
> Components: core
> Affects Versions: 4.3.1
> Reporter: LuGuangMing
> Priority: Major
>
> The situation multi user submit workflows , occasionally, occur the HDFS visitor user become another one . for example, I need submit a workflow by proxy user "{color:#ff0000}platform{color}" via user oozie (kerberos) , an error occur in oozie source code WorkflowAppService.readDefinition read workflow.xml.
> *2018-11-14 00:00:00,497 ERROR [CallableQueue-42]org.apache.oozie.command.wf.SubmitXCommand(517) {color:#ff0000}USER[platform]{color} GROUP[-] TOKEN[] APP[myBulkload-Scheduler-CS_TTL-1539689446] JOB[0002497-180928143722290-oozie-root-C] ACTION[0002497-180928143722290-oozie-root-C@1354] XException, org.apache.oozie.command.CommandException: E0710: Could not read the workflow definition, Permission denied: user={color:#ff0000}dbzq04{color}, access=READ, inode="/phoebus/_fileservice/users/nsplatform/platform/workflows/DataLoadWF-1427-1129/workflow.xml":{color:#ff0000}platform{color}:supergroup:-rw-------*
> note: user "{color:#ff0000}dbzq04{color}" also submit some workflow at before, but current submit the workflow of user is user {color:#ff0000}platform. for prove is current user is platform , I insert some logs at oozie source code {color}
>
>
> {code:java}
> /** org.apache.oozie.service.HadoopAccessorService */
> public FileSystem createFileSystem(String user, final URI uri, final Configuration conf) throws HadoopAccessorException {
> //.........omit......
> try {
> UserGroupInformation ugi = getUGI(user);
> LOG.info("current user="+ugi); //------ my insert log, to print proxy ugi info
> return ugi.doAs(new PrivilegedExceptionAction<FileSystem>() {
> public FileSystem run() throws Exception {
> FileSystem fs = FileSystem.get(uri, conf);
> //------ my insert log, to print fs inner ugi info
> if(fs instanceof DistributedFileSystem){
> LOG.info("hdfs client user, "+((DistributedFileSystem)fs).getClient().toString());
> }
> return fs;
> }
> });
> }catch (InterruptedException ex) {
> throw new HadoopAccessorException(ErrorCode.E0902, ex.getMessage(), ex);
> }catch (IOException ex) {
> throw new HadoopAccessorException(ErrorCode.E0902, ex.getMessage(), ex);
> }
> }{code}
> *my log print result follows:*
>
> 2018-11-14 00:00:00,492 INFO [CallableQueue-42]org.apache.oozie.service.HadoopAccessorService(520) USER[platform] GROUP[-] TOKEN[] APP[myBulkload-Scheduler-CS_TTL-1539689446] JOB[0002497-180928143722290-oozie-root-C] ACTION[0002497-180928143722290-oozie-root-C@1354] *current user=platform (auth:PROXY) via oozie/nsplatformhm@DC1.FH.COM (auth:KERBEROS)*
> 2018-11-14 00:00:00,493 INFO [CallableQueue-42]org.apache.oozie.service.HadoopAccessorService(520) USER[platform] GROUP[-] TOKEN[] APP[myBulkload-Scheduler-CS_TTL-1539689446] JOB[0002497-180928143722290-oozie-root-C] ACTION[0002497-180928143722290-oozie-root-C@1354] {color:#d04437}*hdfs client user, DFSClient[clientName=DFSClient_NONMAPREDUCE_-515910437_325, ugi=platform (auth:PROXY) via oozie/nsplatformhm@DC1.FH.COM (auth:KERBEROS)]*{color}
> *over above the proves Indicate at visited HDFS path of user has been altered, where was user "**{color:#ff0000}dbzq04" {color:#333333} came in? please help me check this problem{color}{color}*{color:#333333}*{color}*
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)