You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by charles li <ch...@gmail.com> on 2016/07/09 03:52:23 UTC

Is there a way to dynamic load files [ parquet or csv ] in the map function?

hi, guys, is there a way to dynamic load files within the map function.

i.e.

Can I code as bellow:


​

thanks a lot.
​


-- 
*___________________*
​          ​
Quant | Engineer | Boy
*___________________*
*blog*:    http://litaotao.github.io
*github*: www.github.com/litaotao

Re: Is there a way to dynamic load files [ parquet or csv ] in the map function?

Posted by Deepak Sharma <de...@gmail.com>.
Yes .You can do something like this :
.map(x=>mapfunction(x))

Thanks
Deepak
On 9 Jul 2016 9:22 am, "charles li" <ch...@gmail.com> wrote:

>
> hi, guys, is there a way to dynamic load files within the map function.
>
> i.e.
>
> Can I code as bellow:
>
>
> ​
>
> thanks a lot.
> ​
>
>
> --
> *___________________*
> ​          ​
> Quant | Engineer | Boy
> *___________________*
> *blog*:    http://litaotao.github.io
> *github*: www.github.com/litaotao
>