Re: read both local path and HDFS path

[Prannoy <pr...@sigmoidanalytics.com>]

Hi,

The configuration you provide is just to access the HDFS when you give an
HDFS path. When you provide a HDFS path with the HDFS nameservice, like in
your case hmaster155:9000 it goes inside the HDFS to look for the file. For
accessing local file just give the local path of the file. Go to the file
in the local and do a "pwd". This will give you the full path of the file.
Just give that path as your local path for the file and you will do good.

Thanks.

On Fri, Nov 28, 2014 at 8:57 AM, tuyuri [via Apache Spark User List] <
ml-node+s1001560n19990h92@n3.nabble.com> wrote:

>
> I have setup a Spark cluster config with HDFS and I know that default file
> path will be read by Spark all in HDFS example :
>
> /ad-cpc/2014-11-28/ Spark will read in :
> hdfs://hmaster155:9000/ad-cpc/2014-11-28/
> sometimes I wonder how can i force Spark read a file in local without
> reConfig my cluster ( to not use hdfs).
>
> please help me !!!
>
> ------------------------------
>  If you reply to this email, your message will be added to the discussion
> below:
>
> http://apache-spark-user-list.1001560.n3.nabble.com/read-both-local-path-and-HDFS-path-tp19990.html
>  To start a new topic under Apache Spark User List, email
> ml-node+s1001560n1h33@n3.nabble.com
> To unsubscribe from Apache Spark User List, click here
> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=1&code=cHJhbm5veUBzaWdtb2lkYW5hbHl0aWNzLmNvbXwxfC0xNTI2NTg4NjQ2>
> .
> NAML
> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>




--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/read-both-local-path-and-HDFS-path-tp19990p19995.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.