could we run a "tar" command in oozie and put the result back to HDFS

[Derwin Chiang <de...@yahoo-inc.com>]

Hi, All

   Oozie provide java action which could run script,
   And I also tried that write a file into HDFS by using HDFS api of java in the java class,
   So, for any script that read a file or string and output the string, this java action would works well

   But I am curious that how about the oozie behavior for command such as "tar"
   Whose output is a file, not a string, how could we use it?

   For example, I run some pig or mapreduce action and generate some data,
   Then I want to use "tar" command to make this data a *.tar file
   We can write a java action which could tar this data, but how can we put this *.tar file back to HDFS
   Could anyone have any experience could share?
   Thanks

Derwin

Re: could we run a "tar" command in oozie and put the result back to HDFS

[Mohammad Islam <mi...@yahoo.com>]

Hi Derwin,
You can execute a script that will do something like these
* tar ....
* hadoop fs -put ....

Then execute this script through a Java action.
The requirements for all compute nodes are:
* 'tar' command should be in the basic path. Otherwise use the full path.
* Hadoop should be installed in every node. The command 'hadoop' should be in the 
path or use the full path.
f your hadoop is secured version, you will need to do one extra task, Please let me know if you indeed need this.

Regards,
Mohammad


________________________________
From: Derwin Chiang <de...@yahoo-inc.com>
To: "oozie-users@incubator.apache.org" <oo...@incubator.apache.org> 
Sent: Sunday, December 18, 2011 11:04 PM
Subject: could we run a "tar" command in oozie and put the result back to HDFS

Hi, All

   Oozie provide java action which could run script,
   And I also tried that write a file into HDFS by using HDFS api of java in the java class,
   So, for any script that read a file or string and output the string, this java action would works well

   But I am curious that how about the oozie behavior for command such as "tar"
   Whose output is a file, not a string, how could we use it?

   For example, I run some pig or mapreduce action and generate some data,
   Then I want to use "tar" command to make this data a *.tar file
   We can write a java action which could tar this data, but how can we put this *.tar file back to HDFS
   Could anyone have any experience could share?
   Thanks

Derwin