You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Gaurav Agarwal <ga...@gmail.com> on 2017/02/15 17:24:57 UTC
Regarding transformation with dataframe
Hello
I have loaded 3 dataframes with 3 different Static tables. Now i got the
csv file and with the help of Spark i loaded the csv into dataframe and
named it as temporary table as "Employee".
Now i need to enrich the columns in the Employee DF and query any of 3
static table respectively with some business logic
e.x.
If(employee.firstName!=null){
String emp.FristName=employee.FirstName;
Select mgr.Name from Mgr where broker.firstName=emp.FirstName
}
Once I retrieve Mgr.Name from Mgr table then i will set the value in the
employee table/DF in the corresponding column.
Thanks