Read HDFS file from an executor(closure)

[Udit Mehta <um...@groupon.com.INVALID>]

Hi,

Is there a way to read a text file from inside a spark executor? I need to
do this for an streaming application where we need to read a file(whose
contents would change) from a closure.

I cannot use the "sc.textFile" method since spark context is not
serializable. I also cannot read a file using the Hadoop Api since the
"FileSystem" class is not serializable as well.

Does anyone have any idea on how I can go about this?

Thanks,
Udit