You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Gaurav Agarwal <ga...@gmail.com> on 2017/02/15 17:24:57 UTC

Regarding transformation with dataframe

Hello

I have loaded 3 dataframes with 3 different Static tables. Now i got the
csv file and with the help of Spark i loaded the csv into dataframe and
named it as temporary table as "Employee".
Now i need to enrich the columns in the Employee DF and query any of 3
static table respectively with some business logic
e.x.

If(employee.firstName!=null){
String emp.FristName=employee.FirstName;

Select mgr.Name from Mgr where broker.firstName=emp.FirstName
}

Once I retrieve Mgr.Name from Mgr table then i will set the value in the
employee table/DF in the corresponding column.

Thanks