You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Jean Georges Perrin <jp...@lumeris.com> on 2017/11/03 10:48:51 UTC
Re: How to get the data url
I am a little confused by your question… Are you trying to ingest a file from S3?
If so… look for net.jgp.labs.spark on GitHub and look for net.jgp.labs.spark.l000_ingestion.l001_csv_in_progress.S3CsvToDataset
You can modify the file as the keys are yours…
If you want to download first: look at net.jgp.labs.spark.l900_analytics.ListNCSchoolDistricts
jgp
> On Oct 29, 2017, at 22:20, onoke <onokeiji@gmail.com <ma...@gmail.com>> wrote:
>
> Hi,
>
> I am searching a useful API for getting a data URL that is accessed by a
> application on Spark.
> For example, when this URL is in a application
>
> new
> URL("https://s3.amazonaws.com/apache-zeppelin/tutorial/bank/bank.csv <https://s3.amazonaws.com/apache-zeppelin/tutorial/bank/bank.csv>")
>
> How to get this url from using Spark API?
> I looked in org.apach.api.java and org.apache.spark.status.api.v1, but they
> do not provide any URL info.
>
> Any advice are welcome.
>
> -Keiji
>
>
>
> --
> Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ <http://apache-spark-user-list.1001560.n3.nabble.com/>
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: user-unsubscribe@spark.apache.org <ma...@spark.apache.org>
>
Re: How to get the data url
Posted by 小野圭二 <on...@gmail.com>.
Thank you for your reply, jgp.
The URL in my question was just a sample. Whatever it is.
I mean, let's image multi user environment spark.
This is just a case model.
-I am a watcher on a spark system.
-Some users run their applications on my spark and ,I need to know what
URL running on my spark.
-Maybe You say I can see the URL in a log file, but it is troublesome.
-If there is a API that get a access url string, I could make a program to
gather all running URL.
I believe there is the best API in it, but just i do not know yet.
Thx.
-Keiji
2017-11-03 19:48 GMT+09:00 Jean Georges Perrin <jp...@lumeris.com>:
> I am a little confused by your question… Are you trying to ingest a file
> from S3?
>
> If so… look for net.jgp.labs.spark on GitHub and look for
> net.jgp.labs.spark.l000_ingestion.l001_csv_in_progress.S3CsvToDataset
>
> You can modify the file as the keys are yours…
>
> If you want to download first: look at net.jgp.labs.spark.l900_analytics.
> ListNCSchoolDistricts
>
> jgp
>
> On Oct 29, 2017, at 22:20, onoke <on...@gmail.com> wrote:
>
> Hi,
>
> I am searching a useful API for getting a data URL that is accessed by a
> application on Spark.
> For example, when this URL is in a application
>
> new
> URL("https://s3.amazonaws.com/apache-zeppelin/tutorial/bank/bank.csv")
>
> How to get this url from using Spark API?
> I looked in org.apach.api.java and org.apache.spark.status.api.v1, but
> they
> do not provide any URL info.
>
> Any advice are welcome.
>
> -Keiji
>
>
>
> --
> Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>
>
>