You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by "Ryota Egashira (JIRA)" <ji...@apache.org> on 2012/11/06 23:46:13 UTC
[jira] [Created] (OOZIE-1052)
HadoopAccessorService.createFileSystem throws exception in map-reduce
action, failing workflow.
Ryota Egashira created OOZIE-1052:
-------------------------------------
Summary: HadoopAccessorService.createFileSystem throws exception in map-reduce action, failing workflow.
Key: OOZIE-1052
URL: https://issues.apache.org/jira/browse/OOZIE-1052
Project: Oozie
Issue Type: Bug
Components: core
Affects Versions: 3.2.0
Reporter: Ryota Egashira
Priority: Blocker
HadoopAccessorService.createFileSystem checks OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED in jobconf, (which is passed as argument), and if false or non-exist, throws exception.
------
if (!conf.getBoolean(OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED, false)) {
throw new HadoopAccessorException(ErrorCode.E0903);
}
------
this is troublesome when it's called from LauncherMapper.hasIdSwap, since it passes new conf as argument, which doesn't include OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED, so always hitting exception mentioned above
-------
FileSystem fs = Services.get().get(HadoopAccessorService.class).createFileSystem(user, p.toUri(),new Configuration());
--------
this problem becomes visible to user as workflow failure, when in map-reduce action, oozie fails to get external jobID from counters, then accessing to HDFS to get it.
the above-mentioned change in HadoopAccessorService.createFileSystem is introduced in 3.2, it is not present in 3.1.4.
this needs to be fixed. also question is what was original intention of the change.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (OOZIE-1052)
HadoopAccessorService.createFileSystem throws exception in map-reduce
action, failing workflow.
Posted by "Ryota Egashira (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ryota Egashira updated OOZIE-1052:
----------------------------------
Fix Version/s: trunk
3.3.0
> HadoopAccessorService.createFileSystem throws exception in map-reduce action, failing workflow.
> -----------------------------------------------------------------------------------------------
>
> Key: OOZIE-1052
> URL: https://issues.apache.org/jira/browse/OOZIE-1052
> Project: Oozie
> Issue Type: Bug
> Components: core
> Affects Versions: 3.3.0, trunk
> Reporter: Ryota Egashira
> Assignee: Ryota Egashira
> Priority: Blocker
> Fix For: 3.3.0, trunk
>
> Attachments: OOZIE-1052.patch
>
>
> HadoopAccessorService.createFileSystem checks OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED in jobconf, (which is passed as argument), and if false or non-exist, throws exception.
> ------
> if (!conf.getBoolean(OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED, false)) {
> throw new HadoopAccessorException(ErrorCode.E0903);
> }
> ------
> this is troublesome when it's called from LauncherMapper.hasIdSwap, since it passes new conf as argument, which doesn't include OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED, so always hitting exception mentioned above
> -------
> FileSystem fs = Services.get().get(HadoopAccessorService.class).createFileSystem(user, p.toUri(),new Configuration());
> --------
> this problem becomes visible to user as workflow failure, when in map-reduce action, oozie fails to get external jobID from counters, then accessing to HDFS to get it.
> the above-mentioned change in HadoopAccessorService.createFileSystem is introduced in 3.2, it is not present in 3.1.4.
> this needs to be fixed. also question is what was original intention of the change.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (OOZIE-1052)
HadoopAccessorService.createFileSystem throws exception in map-reduce
action, failing workflow.
Posted by "Ryota Egashira (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ryota Egashira updated OOZIE-1052:
----------------------------------
Attachment: OOZIE-1052.patch
> HadoopAccessorService.createFileSystem throws exception in map-reduce action, failing workflow.
> -----------------------------------------------------------------------------------------------
>
> Key: OOZIE-1052
> URL: https://issues.apache.org/jira/browse/OOZIE-1052
> Project: Oozie
> Issue Type: Bug
> Components: core
> Affects Versions: 3.2.0
> Reporter: Ryota Egashira
> Assignee: Ryota Egashira
> Priority: Blocker
> Attachments: OOZIE-1052.patch
>
>
> HadoopAccessorService.createFileSystem checks OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED in jobconf, (which is passed as argument), and if false or non-exist, throws exception.
> ------
> if (!conf.getBoolean(OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED, false)) {
> throw new HadoopAccessorException(ErrorCode.E0903);
> }
> ------
> this is troublesome when it's called from LauncherMapper.hasIdSwap, since it passes new conf as argument, which doesn't include OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED, so always hitting exception mentioned above
> -------
> FileSystem fs = Services.get().get(HadoopAccessorService.class).createFileSystem(user, p.toUri(),new Configuration());
> --------
> this problem becomes visible to user as workflow failure, when in map-reduce action, oozie fails to get external jobID from counters, then accessing to HDFS to get it.
> the above-mentioned change in HadoopAccessorService.createFileSystem is introduced in 3.2, it is not present in 3.1.4.
> this needs to be fixed. also question is what was original intention of the change.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (OOZIE-1052)
HadoopAccessorService.createFileSystem throws exception in map-reduce
action, failing workflow.
Posted by "Ryota Egashira (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ryota Egashira updated OOZIE-1052:
----------------------------------
Affects Version/s: (was: 3.2.0)
trunk
3.3.0
> HadoopAccessorService.createFileSystem throws exception in map-reduce action, failing workflow.
> -----------------------------------------------------------------------------------------------
>
> Key: OOZIE-1052
> URL: https://issues.apache.org/jira/browse/OOZIE-1052
> Project: Oozie
> Issue Type: Bug
> Components: core
> Affects Versions: 3.3.0, trunk
> Reporter: Ryota Egashira
> Assignee: Ryota Egashira
> Priority: Blocker
> Attachments: OOZIE-1052.patch
>
>
> HadoopAccessorService.createFileSystem checks OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED in jobconf, (which is passed as argument), and if false or non-exist, throws exception.
> ------
> if (!conf.getBoolean(OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED, false)) {
> throw new HadoopAccessorException(ErrorCode.E0903);
> }
> ------
> this is troublesome when it's called from LauncherMapper.hasIdSwap, since it passes new conf as argument, which doesn't include OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED, so always hitting exception mentioned above
> -------
> FileSystem fs = Services.get().get(HadoopAccessorService.class).createFileSystem(user, p.toUri(),new Configuration());
> --------
> this problem becomes visible to user as workflow failure, when in map-reduce action, oozie fails to get external jobID from counters, then accessing to HDFS to get it.
> the above-mentioned change in HadoopAccessorService.createFileSystem is introduced in 3.2, it is not present in 3.1.4.
> this needs to be fixed. also question is what was original intention of the change.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OOZIE-1052)
HadoopAccessorService.createFileSystem throws exception in map-reduce
action, failing workflow.
Posted by "Mohammad Kamrul Islam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13496734#comment-13496734 ]
Mohammad Kamrul Islam commented on OOZIE-1052:
----------------------------------------------
Committed to trunk, 3.3 and hcat-intre.
> HadoopAccessorService.createFileSystem throws exception in map-reduce action, failing workflow.
> -----------------------------------------------------------------------------------------------
>
> Key: OOZIE-1052
> URL: https://issues.apache.org/jira/browse/OOZIE-1052
> Project: Oozie
> Issue Type: Bug
> Components: core
> Affects Versions: 3.3.0, trunk
> Reporter: Ryota Egashira
> Assignee: Ryota Egashira
> Priority: Blocker
> Fix For: 3.3.0, trunk
>
> Attachments: OOZIE-1052.patch
>
>
> HadoopAccessorService.createFileSystem checks OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED in jobconf, (which is passed as argument), and if false or non-exist, throws exception.
> ------
> if (!conf.getBoolean(OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED, false)) {
> throw new HadoopAccessorException(ErrorCode.E0903);
> }
> ------
> this is troublesome when it's called from LauncherMapper.hasIdSwap, since it passes new conf as argument, which doesn't include OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED, so always hitting exception mentioned above
> -------
> FileSystem fs = Services.get().get(HadoopAccessorService.class).createFileSystem(user, p.toUri(),new Configuration());
> --------
> this problem becomes visible to user as workflow failure, when in map-reduce action, oozie fails to get external jobID from counters, then accessing to HDFS to get it.
> the above-mentioned change in HadoopAccessorService.createFileSystem is introduced in 3.2, it is not present in 3.1.4.
> this needs to be fixed. also question is what was original intention of the change.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OOZIE-1052)
HadoopAccessorService.createFileSystem throws exception in map-reduce
action, failing workflow.
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13493594#comment-13493594 ]
Alejandro Abdelnur commented on OOZIE-1052:
-------------------------------------------
+1 pending jenkins, please attach PATCH to JIRA for jenkins to run.
> HadoopAccessorService.createFileSystem throws exception in map-reduce action, failing workflow.
> -----------------------------------------------------------------------------------------------
>
> Key: OOZIE-1052
> URL: https://issues.apache.org/jira/browse/OOZIE-1052
> Project: Oozie
> Issue Type: Bug
> Components: core
> Affects Versions: 3.2.0
> Reporter: Ryota Egashira
> Assignee: Ryota Egashira
> Priority: Blocker
>
> HadoopAccessorService.createFileSystem checks OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED in jobconf, (which is passed as argument), and if false or non-exist, throws exception.
> ------
> if (!conf.getBoolean(OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED, false)) {
> throw new HadoopAccessorException(ErrorCode.E0903);
> }
> ------
> this is troublesome when it's called from LauncherMapper.hasIdSwap, since it passes new conf as argument, which doesn't include OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED, so always hitting exception mentioned above
> -------
> FileSystem fs = Services.get().get(HadoopAccessorService.class).createFileSystem(user, p.toUri(),new Configuration());
> --------
> this problem becomes visible to user as workflow failure, when in map-reduce action, oozie fails to get external jobID from counters, then accessing to HDFS to get it.
> the above-mentioned change in HadoopAccessorService.createFileSystem is introduced in 3.2, it is not present in 3.1.4.
> this needs to be fixed. also question is what was original intention of the change.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OOZIE-1052)
HadoopAccessorService.createFileSystem throws exception in map-reduce
action, failing workflow.
Posted by "Ryota Egashira (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13493665#comment-13493665 ]
Ryota Egashira commented on OOZIE-1052:
---------------------------------------
thanks for review, Alejandro
> HadoopAccessorService.createFileSystem throws exception in map-reduce action, failing workflow.
> -----------------------------------------------------------------------------------------------
>
> Key: OOZIE-1052
> URL: https://issues.apache.org/jira/browse/OOZIE-1052
> Project: Oozie
> Issue Type: Bug
> Components: core
> Affects Versions: 3.2.0
> Reporter: Ryota Egashira
> Assignee: Ryota Egashira
> Priority: Blocker
>
> HadoopAccessorService.createFileSystem checks OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED in jobconf, (which is passed as argument), and if false or non-exist, throws exception.
> ------
> if (!conf.getBoolean(OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED, false)) {
> throw new HadoopAccessorException(ErrorCode.E0903);
> }
> ------
> this is troublesome when it's called from LauncherMapper.hasIdSwap, since it passes new conf as argument, which doesn't include OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED, so always hitting exception mentioned above
> -------
> FileSystem fs = Services.get().get(HadoopAccessorService.class).createFileSystem(user, p.toUri(),new Configuration());
> --------
> this problem becomes visible to user as workflow failure, when in map-reduce action, oozie fails to get external jobID from counters, then accessing to HDFS to get it.
> the above-mentioned change in HadoopAccessorService.createFileSystem is introduced in 3.2, it is not present in 3.1.4.
> this needs to be fixed. also question is what was original intention of the change.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OOZIE-1052)
HadoopAccessorService.createFileSystem throws exception in map-reduce
action, failing workflow.
Posted by "Ryota Egashira (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13505885#comment-13505885 ]
Ryota Egashira commented on OOZIE-1052:
---------------------------------------
did additional test where I commented out following part in LauncherMapper.hasIdSwap and ran workflow with map-reduce action, which worked.
====
if (counters != null) {
Counters.Group counterGroup = counters.getGroup(COUNTER_GROUP);
if (counterGroup != null) {
swap = counterGroup.getCounter(COUNTER_DO_ID_SWAP) == 1;
}
=====
> HadoopAccessorService.createFileSystem throws exception in map-reduce action, failing workflow.
> -----------------------------------------------------------------------------------------------
>
> Key: OOZIE-1052
> URL: https://issues.apache.org/jira/browse/OOZIE-1052
> Project: Oozie
> Issue Type: Bug
> Components: core
> Affects Versions: 3.3.0, trunk
> Reporter: Ryota Egashira
> Assignee: Ryota Egashira
> Priority: Blocker
> Fix For: 3.3.0, trunk
>
> Attachments: OOZIE-1052.patch
>
>
> HadoopAccessorService.createFileSystem checks OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED in jobconf, (which is passed as argument), and if false or non-exist, throws exception.
> ------
> if (!conf.getBoolean(OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED, false)) {
> throw new HadoopAccessorException(ErrorCode.E0903);
> }
> ------
> this is troublesome when it's called from LauncherMapper.hasIdSwap, since it passes new conf as argument, which doesn't include OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED, so always hitting exception mentioned above
> -------
> FileSystem fs = Services.get().get(HadoopAccessorService.class).createFileSystem(user, p.toUri(),new Configuration());
> --------
> this problem becomes visible to user as workflow failure, when in map-reduce action, oozie fails to get external jobID from counters, then accessing to HDFS to get it.
> the above-mentioned change in HadoopAccessorService.createFileSystem is introduced in 3.2, it is not present in 3.1.4.
> this needs to be fixed. also question is what was original intention of the change.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (OOZIE-1052)
HadoopAccessorService.createFileSystem throws exception in map-reduce
action, failing workflow.
Posted by "Ryota Egashira (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ryota Egashira updated OOZIE-1052:
----------------------------------
Assignee: Ryota Egashira
> HadoopAccessorService.createFileSystem throws exception in map-reduce action, failing workflow.
> -----------------------------------------------------------------------------------------------
>
> Key: OOZIE-1052
> URL: https://issues.apache.org/jira/browse/OOZIE-1052
> Project: Oozie
> Issue Type: Bug
> Components: core
> Affects Versions: 3.2.0
> Reporter: Ryota Egashira
> Assignee: Ryota Egashira
> Priority: Blocker
>
> HadoopAccessorService.createFileSystem checks OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED in jobconf, (which is passed as argument), and if false or non-exist, throws exception.
> ------
> if (!conf.getBoolean(OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED, false)) {
> throw new HadoopAccessorException(ErrorCode.E0903);
> }
> ------
> this is troublesome when it's called from LauncherMapper.hasIdSwap, since it passes new conf as argument, which doesn't include OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED, so always hitting exception mentioned above
> -------
> FileSystem fs = Services.get().get(HadoopAccessorService.class).createFileSystem(user, p.toUri(),new Configuration());
> --------
> this problem becomes visible to user as workflow failure, when in map-reduce action, oozie fails to get external jobID from counters, then accessing to HDFS to get it.
> the above-mentioned change in HadoopAccessorService.createFileSystem is introduced in 3.2, it is not present in 3.1.4.
> this needs to be fixed. also question is what was original intention of the change.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OOZIE-1052)
HadoopAccessorService.createFileSystem throws exception in map-reduce
action, failing workflow.
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13494485#comment-13494485 ]
Hadoop QA commented on OOZIE-1052:
----------------------------------
Testing JIRA OOZIE-1052
Cleaning local svn workspace
----------------------------
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
. {color:green}+1{color} the patch does not introduce any @author tags
. {color:green}+1{color} the patch does not introduce any tabs
. {color:green}+1{color} the patch does not introduce any trailing spaces
. {color:green}+1{color} the patch does not introduce any line longer than 132
. {color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
. {color:green}+1{color} the patch does not seem to introduce new RAT warnings
{color:green}+1 JAVADOC{color}
. {color:green}+1{color} the patch does not seem to introduce new Javadoc warnings
{color:green}+1 COMPILE{color}
. {color:green}+1{color} HEAD compiles
. {color:green}+1{color} patch compiles
. {color:green}+1{color} the patch does not seem to introduce new javac warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
. {color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations
. {color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
. Tests run: 916
{color:green}+1 DISTRO{color}
. {color:green}+1{color} distro tarball builds with the patch
----------------------------
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/169/
> HadoopAccessorService.createFileSystem throws exception in map-reduce action, failing workflow.
> -----------------------------------------------------------------------------------------------
>
> Key: OOZIE-1052
> URL: https://issues.apache.org/jira/browse/OOZIE-1052
> Project: Oozie
> Issue Type: Bug
> Components: core
> Affects Versions: 3.3.0, trunk
> Reporter: Ryota Egashira
> Assignee: Ryota Egashira
> Priority: Blocker
> Fix For: 3.3.0, trunk
>
> Attachments: OOZIE-1052.patch
>
>
> HadoopAccessorService.createFileSystem checks OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED in jobconf, (which is passed as argument), and if false or non-exist, throws exception.
> ------
> if (!conf.getBoolean(OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED, false)) {
> throw new HadoopAccessorException(ErrorCode.E0903);
> }
> ------
> this is troublesome when it's called from LauncherMapper.hasIdSwap, since it passes new conf as argument, which doesn't include OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED, so always hitting exception mentioned above
> -------
> FileSystem fs = Services.get().get(HadoopAccessorService.class).createFileSystem(user, p.toUri(),new Configuration());
> --------
> this problem becomes visible to user as workflow failure, when in map-reduce action, oozie fails to get external jobID from counters, then accessing to HDFS to get it.
> the above-mentioned change in HadoopAccessorService.createFileSystem is introduced in 3.2, it is not present in 3.1.4.
> this needs to be fixed. also question is what was original intention of the change.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OOZIE-1052)
HadoopAccessorService.createFileSystem throws exception in map-reduce
action, failing workflow.
Posted by "Ryota Egashira (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492785#comment-13492785 ]
Ryota Egashira commented on OOZIE-1052:
---------------------------------------
patch uploaded on
https://reviews.apache.org/r/7938/
> HadoopAccessorService.createFileSystem throws exception in map-reduce action, failing workflow.
> -----------------------------------------------------------------------------------------------
>
> Key: OOZIE-1052
> URL: https://issues.apache.org/jira/browse/OOZIE-1052
> Project: Oozie
> Issue Type: Bug
> Components: core
> Affects Versions: 3.2.0
> Reporter: Ryota Egashira
> Assignee: Ryota Egashira
> Priority: Blocker
>
> HadoopAccessorService.createFileSystem checks OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED in jobconf, (which is passed as argument), and if false or non-exist, throws exception.
> ------
> if (!conf.getBoolean(OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED, false)) {
> throw new HadoopAccessorException(ErrorCode.E0903);
> }
> ------
> this is troublesome when it's called from LauncherMapper.hasIdSwap, since it passes new conf as argument, which doesn't include OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED, so always hitting exception mentioned above
> -------
> FileSystem fs = Services.get().get(HadoopAccessorService.class).createFileSystem(user, p.toUri(),new Configuration());
> --------
> this problem becomes visible to user as workflow failure, when in map-reduce action, oozie fails to get external jobID from counters, then accessing to HDFS to get it.
> the above-mentioned change in HadoopAccessorService.createFileSystem is introduced in 3.2, it is not present in 3.1.4.
> this needs to be fixed. also question is what was original intention of the change.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira