You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@oozie.apache.org by "Hadoop QA (JIRA)" <ji...@apache.org> on 2011/09/08 06:41:09 UTC
[jira] [Created] (OOZIE-135) GH-109: Refactor Hadoop Accessor to do
command pattern
GH-109: Refactor Hadoop Accessor to do command pattern
------------------------------------------------------
Key: OOZIE-135
URL: https://issues.apache.org/jira/browse/OOZIE-135
Project: Oozie
Issue Type: Bug
Reporter: Hadoop QA
All Hadoop JobClient?/FileSystem access should be done using the following:
public interface HadoopFSCommand {
public void call(FileSystem fs) throws HadoopCommandException;
}
public interface HadoopJobCommand {
public void call(JobClient jobClient, FileSystem fs) throws HadoopCommandException;
}
//This replaces current StoreService
public class HadoopAccessorService implements Service {
...
public void execute(final HadoopFSCommand fsCommand) throws HadoopCommandException {
//all instrumentation is missing here
UGI.doAs(
FileSystem fs = GET FS
try {
fsCommand.call(fs);
}
finally {
fs.close();
}
);
}
public void execute(final HadoopJobCommand fsCommand) throws HadoopCommandException {
//all instrumentation is missing here
UGI.doAs(
FileSystem fs = GET FS
JobClient jc = GET_JC
try {
fsCommand.call(jc, fs);
}
finally {
try {
jc.close();
}
catch () {
}
try {
fs.close();
}
catch () {
}
}
);
}
}
The accessor should have the USER-UGI second level cache and a a config value should specify if the cache is to be used
or not (if OFF do fs.close() on every call, if ON, do nothing, and cache should do eviction on no use timeout)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (OOZIE-135) GH-109: Refactor Hadoop Accessor to
do command pattern
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hadoop QA resolved OOZIE-135.
-----------------------------
Resolution: Fixed
> GH-109: Refactor Hadoop Accessor to do command pattern
> ------------------------------------------------------
>
> Key: OOZIE-135
> URL: https://issues.apache.org/jira/browse/OOZIE-135
> Project: Oozie
> Issue Type: Bug
> Reporter: Hadoop QA
>
> All Hadoop JobClient?/FileSystem access should be done using the following:
> public interface HadoopFSCommand {
> public void call(FileSystem fs) throws HadoopCommandException;
> }
> public interface HadoopJobCommand {
> public void call(JobClient jobClient, FileSystem fs) throws HadoopCommandException;
> }
> //This replaces current StoreService
> public class HadoopAccessorService implements Service {
> ...
> public void execute(final HadoopFSCommand fsCommand) throws HadoopCommandException {
> //all instrumentation is missing here
> UGI.doAs(
> FileSystem fs = GET FS
> try {
> fsCommand.call(fs);
> }
> finally {
> fs.close();
> }
> );
> }
> public void execute(final HadoopJobCommand fsCommand) throws HadoopCommandException {
> //all instrumentation is missing here
> UGI.doAs(
> FileSystem fs = GET FS
> JobClient jc = GET_JC
> try {
> fsCommand.call(jc, fs);
> }
> finally {
> try {
> jc.close();
> }
> catch () {
> }
> try {
> fs.close();
> }
> catch () {
> }
> }
> );
> }
> }
> The accessor should have the USER-UGI second level cache and a a config value should specify if the cache is to be used
> or not (if OFF do fs.close() on every call, if ON, do nothing, and cache should do eviction on no use timeout)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (OOZIE-135) GH-109: Refactor Hadoop Accessor to
do command pattern
Posted by "Roman Shaposhnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OOZIE-135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Roman Shaposhnik reopened OOZIE-135:
------------------------------------
> GH-109: Refactor Hadoop Accessor to do command pattern
> ------------------------------------------------------
>
> Key: OOZIE-135
> URL: https://issues.apache.org/jira/browse/OOZIE-135
> Project: Oozie
> Issue Type: Bug
> Reporter: Hadoop QA
>
> All Hadoop JobClient?/FileSystem access should be done using the following:
> public interface HadoopFSCommand {
> public void call(FileSystem fs) throws HadoopCommandException;
> }
> public interface HadoopJobCommand {
> public void call(JobClient jobClient, FileSystem fs) throws HadoopCommandException;
> }
> //This replaces current StoreService
> public class HadoopAccessorService implements Service {
> ...
> public void execute(final HadoopFSCommand fsCommand) throws HadoopCommandException {
> //all instrumentation is missing here
> UGI.doAs(
> FileSystem fs = GET FS
> try {
> fsCommand.call(fs);
> }
> finally {
> fs.close();
> }
> );
> }
> public void execute(final HadoopJobCommand fsCommand) throws HadoopCommandException {
> //all instrumentation is missing here
> UGI.doAs(
> FileSystem fs = GET FS
> JobClient jc = GET_JC
> try {
> fsCommand.call(jc, fs);
> }
> finally {
> try {
> jc.close();
> }
> catch () {
> }
> try {
> fs.close();
> }
> catch () {
> }
> }
> );
> }
> }
> The accessor should have the USER-UGI second level cache and a a config value should specify if the cache is to be used
> or not (if OFF do fs.close() on every call, if ON, do nothing, and cache should do eviction on no use timeout)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira