Re: Fwd: Oozie spark action

[Mohammad Islam <mi...@yahoo.com.INVALID>]

One option is : copy the hive-site.xml from someplace into workflow app directory. More at link.
Another example. 

    On Monday, March 14, 2016 12:06 PM, Sandeep Bajaj <sa...@yahoo.com.INVALID> wrote:
 

 Thanks Pavan for forwarding.With respect to the issue of not seeing data in hive tables, I figured out that oozie spark action is writing to the default /user/hive/warehouse location, but not to the location that I have set in hive-site.xml by setting hive.metastore.warehouse.dir...I tried to set this location  programmatically in the spark job by explicitly doing a sqlContext.setConf("hive.metastore.warehouse.dir","...")
as well as by trying to set a :val options = Map("path" -> ".....")df.write.mode(tableMode).options(options).saveAsTable(outTable)  But nothing worked...How can I have the oozie spark-action write the table to a different warehouse location other than the default ?
thanksSandeep 

    On Monday, March 14, 2016 5:09 AM, pavan kumar Kolamuri <pa...@gmail.com> wrote:
 

 ++ oozie users

If someone experienced this, please help.

---------- Forwarded message ----------
From: Sandeep Bajaj <sa...@yahoo.com>
Date: Sat, Mar 12, 2016 at 1:16 AM
Subject: Oozie spark action
To: "pavan.kolamuri@gmail.com" <pa...@gmail.com>


Pavan

Thanks for adding support for spark action by resolving :
 [OOZIE-1983] Add spark action executor - ASF JIRA
<https://issues.apache.org/jira/browse/OOZIE-1983>






[OOZIE-1983] Add spark action executor - ASF JIRA
<https://issues.apache.org/jira/browse/OOZIE-1983>
https://reviews.apache.org/r/27614/
View on issues.apache.org <https://issues.apache.org/jira/browse/OOZIE-1983>
Preview by Yahoo


However I have been having some issues using it and was hoping if you could
help. One of the errors that I see occasionally is the "TFS filesytem not
supported" , as discussed in this thread as well:
https://community.hortonworks.com/questions/17791/oozie-submit-spark-action-but-get-a-error-not-impl.html

Secondly as I try to write to hive tables from spark, I dont see any
content in the hive table...there is no associated error in the spark/oozie
logs

Both the above issues dont happen when I run spark-submit. My understanding
is that the spark-action is implemented as a spark-submit job...Do u know
what might be potentially different ?

thanks in advance for your help
Sandeep





-- 
Regards
Pavan Kumar Kolamuri