You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Gaurav Agarwal <ga...@gmail.com> on 2017/02/15 18:34:02 UTC

Enrichment with static tables

Hello

We want to enrich our spark RDD loaded with multiple Columns and multiple
Rows . This need to be enriched with 3 different tables that i loaded 3
different spark dataframe . Can we write some logic in spark so i can
enrich my spark RDD with different stattic tables.

Thanks

Re: Enrichment with static tables

Posted by Gaurav Agarwal <ga...@gmail.com>.

Thanks That worked for me previously I was using wrong join .that the
reason it did Not worked for me

Tbanks

On Feb 16, 2017 01:20, "Sam Elamin" <hu...@gmail.com> wrote:

> You can do a join or a union to combine all the dataframes to one fat
> dataframe
>
> or do a select on the columns you want to produce your transformed
> dataframe
>
> Not sure if I understand the question though, If the goal is just an end
> state transformed dataframe that can easily be done
>
>
> Regards
> Sam
>
> On Wed, Feb 15, 2017 at 6:34 PM, Gaurav Agarwal <ga...@gmail.com>
> wrote:
>
>> Hello
>>
>> We want to enrich our spark RDD loaded with multiple Columns and multiple
>> Rows . This need to be enriched with 3 different tables that i loaded 3
>> different spark dataframe . Can we write some logic in spark so i can
>> enrich my spark RDD with different stattic tables.
>>
>> Thanks
>>
>>
>

Re: Enrichment with static tables

Posted by Sam Elamin <hu...@gmail.com>.

You can do a join or a union to combine all the dataframes to one fat
dataframe

or do a select on the columns you want to produce your transformed dataframe

Not sure if I understand the question though, If the goal is just an end
state transformed dataframe that can easily be done


Regards
Sam

On Wed, Feb 15, 2017 at 6:34 PM, Gaurav Agarwal <ga...@gmail.com>
wrote:

> Hello
>
> We want to enrich our spark RDD loaded with multiple Columns and multiple
> Rows . This need to be enriched with 3 different tables that i loaded 3
> different spark dataframe . Can we write some logic in spark so i can
> enrich my spark RDD with different stattic tables.
>
> Thanks
>
>