Executing hive command from Shell Action and sending output as attachment in an email

[Narayanan K <kn...@gmail.com>]

Hi,

Basically I wanted to run a hive query in Oozie and store the output
in a file after formatting and send it as an attachment to some e-mail
addresses.

I read about sendMail action , but it doesn't seem to be providing
support for attachments.

So here is what I did using a shell script in SHELL ACTION : (I copied
the script.hql and formatter.pl in the same location as workflow.xml)

--------

hive -f script.hql | perl formatter.pl  > /tmp/output.csv

echo "Attached results" | mutt -s "Results" -a /tmp/output.csv --
example@yahoo.com

--------

But this doesn't seem to work.

One is : It is not able to find the hive client in the Oozie launcher.
I tried "which hive" in the above script and it is giving empty. I was
wondering how oozie launchers executed hive action without a hive
client.

Secondly, the mutt command is giving an error message :

"Stdoutput Could not send the message."

I tried to do something like : echo "Hello World" > /tmp/test
and then used mutt to send this file, but still the same error
message. But "which mutt"
gave that it was installed: /usr/bin/mutt...


Is this the approach tested by someone already.? What am I missing here ?

Are there any alternative approach for this requirement ?

Thanks
Narayanan

RE: Executing hive command from Shell Action and sending output as attachment in an email

[Paul Chavez <pc...@ntent.com>]

Hi Narayan,

I have a workflow that accomplishes this.

It is structured in the following manner:

Hive Action (insert overwrite directory...)  -> Shell Action (getMerge, add header row, email)

The hive action is straightforward. I've included sanitized versions of my shell action and the actual shell script, which uses mutt to send the file.

Thanks,
Paul Chavez



    <action name="SendReport">
        <shell xmlns="uri:oozie:shell-action:0.1">
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>
            <exec>${ScriptFile}</exec>
            <argument>${ProcessDateYMD}</argument>
            <file>${ScriptFile}#${ScriptFile}</file>
        </shell>
        <ok to="SuccessEmail"/>
        <error to="FailureEmail"/>
    </action>
    

    
#!/bin/bash
hostname
 file=$( echo "filename_$1" )
 echo $file
 hdfs dfs -getmerge "/HiveExports/${file}" "${file}_noheader.txt"
 echo "field1,field2,field3" > "${file}.csv"
 cat "${file}_noheader.txt" | tr "\001" "," >> "${file}.csv"
 rm -f "${file}_noheader.txt"
 echo "Report for $1 is attached." | mutt -a "${file}.csv" -s "Report $1" -- person1@example.com person2@example.com
 rm -f "${file}.csv"


-----Original Message-----
From: Narayanan K [mailto:knarayanan88@gmail.com] 
Sent: Monday, June 09, 2014 12:05 PM
To: user@oozie.apache.org
Subject: Re: Executing hive command from Shell Action and sending output as attachment in an email

Hi

Just wanted to check again if someone had any suggestions.

Thanks
Narayanan

On Sun, Jun 8, 2014 at 8:07 AM, Narayanan K <kn...@gmail.com> wrote:
> Hi,
>
> Basically I wanted to run a hive query in Oozie and store the output 
> in a file after formatting and send it as an attachment to some e-mail 
> addresses.
>
> I read about sendMail action , but it doesn't seem to be providing 
> support for attachments.
>
> So here is what I did using a shell script in SHELL ACTION : (I copied 
> the script.hql and formatter.pl in the same location as workflow.xml)
>
> --------
>
> hive -f script.hql | perl formatter.pl  > /tmp/output.csv
>
> echo "Attached results" | mutt -s "Results" -a /tmp/output.csv -- 
> example@yahoo.com
>
> --------
>
> But this doesn't seem to work.
>
> One is : It is not able to find the hive client in the Oozie launcher.
> I tried "which hive" in the above script and it is giving empty. I was 
> wondering how oozie launchers executed hive action without a hive 
> client.
>
> Secondly, the mutt command is giving an error message :
>
> "Stdoutput Could not send the message."
>
> I tried to do something like : echo "Hello World" > /tmp/test and then 
> used mutt to send this file, but still the same error message. But 
> "which mutt"
> gave that it was installed: /usr/bin/mutt...
>
>
> Is this the approach tested by someone already.? What am I missing here ?
>
> Are there any alternative approach for this requirement ?
>
> Thanks
> Narayanan

Re: Executing hive command from Shell Action and sending output as attachment in an email

[Narayanan K <kn...@gmail.com>]

Hi

Just wanted to check again if someone had any suggestions.

Thanks
Narayanan

On Sun, Jun 8, 2014 at 8:07 AM, Narayanan K <kn...@gmail.com> wrote:
> Hi,
>
> Basically I wanted to run a hive query in Oozie and store the output
> in a file after formatting and send it as an attachment to some e-mail
> addresses.
>
> I read about sendMail action , but it doesn't seem to be providing
> support for attachments.
>
> So here is what I did using a shell script in SHELL ACTION : (I copied
> the script.hql and formatter.pl in the same location as workflow.xml)
>
> --------
>
> hive -f script.hql | perl formatter.pl  > /tmp/output.csv
>
> echo "Attached results" | mutt -s "Results" -a /tmp/output.csv --
> example@yahoo.com
>
> --------
>
> But this doesn't seem to work.
>
> One is : It is not able to find the hive client in the Oozie launcher.
> I tried "which hive" in the above script and it is giving empty. I was
> wondering how oozie launchers executed hive action without a hive
> client.
>
> Secondly, the mutt command is giving an error message :
>
> "Stdoutput Could not send the message."
>
> I tried to do something like : echo "Hello World" > /tmp/test
> and then used mutt to send this file, but still the same error
> message. But "which mutt"
> gave that it was installed: /usr/bin/mutt...
>
>
> Is this the approach tested by someone already.? What am I missing here ?
>
> Are there any alternative approach for this requirement ?
>
> Thanks
> Narayanan