You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Jerry <je...@gmail.com> on 2015/08/26 23:03:27 UTC
Does the driver program always run local to where you submit the job from?
Assuming your submitting the job from terminal; when main() is called, if I
try to open a file locally, can I assume the machine is always the one I
submitted the job from? Currently I'm working off of a single machine, but
I'm wondering if I'll run into issues when I move over to a cluster. The
file I'm opening is purely for the driver program and not something the
worker nodes are going to read from.
Thanks,
Jerry
Re: Does the driver program always run local to where you submit the
job from?
Posted by Jerry <je...@gmail.com>.
Thanks!
On Wed, Aug 26, 2015 at 2:06 PM, Marcelo Vanzin <va...@cloudera.com> wrote:
> On Wed, Aug 26, 2015 at 2:03 PM, Jerry <je...@gmail.com> wrote:
> > Assuming your submitting the job from terminal; when main() is called,
> if I
> > try to open a file locally, can I assume the machine is always the one I
> > submitted the job from?
>
> See the "--deploy-mode" option. "client" works as you describe;
> "cluster" does not.
>
> --
> Marcelo
>
Re: Does the driver program always run local to where you submit the
job from?
Posted by Marcelo Vanzin <va...@cloudera.com>.
On Wed, Aug 26, 2015 at 2:03 PM, Jerry <je...@gmail.com> wrote:
> Assuming your submitting the job from terminal; when main() is called, if I
> try to open a file locally, can I assume the machine is always the one I
> submitted the job from?
See the "--deploy-mode" option. "client" works as you describe;
"cluster" does not.
--
Marcelo
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org